Обсуждение: Vertica targeting PostgreSQL users
Hi,
Vertica current marketing is heavily targeting Postgres users:
and features this directly from their homepage. I also know of at least one campaign that has gone out around this.
The good news for our community is that because of all the advances in Postgres and the overall growth to our user base, there are database companies looking to win users over to their platforms. This is also a good learning experience for us too: because we have so many users, we also have to look at user retention to ensure that we have enough features and available knowledge to manage Postgres. I have not read the case study that they proposed, but knowing the group of people who work on Postgres and our software ecosystem, we may have been able to retain those databases on Postgres.
Going forward, we must continue to understand our users’ needs while ensuring that we can provide them as many resources as possible to help them manage Postgres and show them that here is great help available when it is required.
Best,
Jonathan
On Mon, Nov 20, 2017 at 6:43 PM, Jonathan S. Katz <jkatz@postgresql.org> wrote: > Hi, > > Vertica current marketing is heavily targeting Postgres users: > > https://www.vertica.com/postgresql/ > > and features this directly from their homepage. I also know of at least one > campaign that has gone out around this. > > The good news for our community is that because of all the advances in > Postgres and the overall growth to our user base, there are database > companies looking to win users over to their platforms. This is also a good > learning experience for us too: because we have so many users, we also have > to look at user retention to ensure that we have enough features and > available knowledge to manage Postgres. I have not read the case study that > they proposed, but knowing the group of people who work on Postgres and our > software ecosystem, we may have been able to retain those databases on > Postgres. > > Going forward, we must continue to understand our users’ needs while > ensuring that we can provide them as many resources as possible to help them > manage Postgres and show them that here is great help available when it is > required. "Our user's needs" are well formulated by Vertica :) It'd be nice if we agree on our future and combine our quite limited resources. There are several postgres-centric companies, which actually hire almost all major developers. The companies have to compete on database market and meet the needs of their customers. What community needs is the community roadmap to let users know what to expect in the next release, in 3-5 years. > Best, > > Jonathan >
On 11/20/2017 07:43 AM, Jonathan S. Katz wrote: > Hi, > > Vertica current marketing is heavily targeting Postgres users: > > https://www.vertica.com/postgresql/ > > and features this directly from their homepage. I also know of at > least one campaign that has gone out around this. > > The good news for our community is that because of all the advances in > Postgres and the overall growth to our user base, there are database > companies looking to win users over to their platforms. This is also > a good learning experience for us too: because we have so many users, > we also have to look at user retention to ensure that we have enough > features and available knowledge to manage Postgres. I have not read > the case study that they proposed, but knowing the group of people who > work on Postgres and our software ecosystem, we may have been able to > retain those databases on Postgres. > > Going forward, we must continue to understand our users’ needs while > ensuring that we can provide them as many resources as possible to > help them manage Postgres and show them that here is great help > available when it is required. You aren't wrong but this is also all but impossible for PostgreSQL.org. The majority of features now developed are developed by the needs of not PostgreSQL.Org but the needs of 2Q, EDB customers or even Amazon (in a different vein). The good news is we have at least two highly dedicated companies that donate code to the open code base, the bad news is both of these companies are move their clients to their closed platforms as soon as possible: https://www.2ndquadrant.com/en/resources/2ndqpostgres/ https://www.enterprisedb.com/products/edb-postgres-platform Or the users are migrating to something like RDS or Aurora. I am not criticizing 2Q, EDB or Amazon. They are all conducting smart business but if we want more code that is generated by user demands and to increase user retention, then we need to make the barriers of participation lower for users. Right now they are exceedingly high. Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
Josh, all, * Joshua D. Drake (jd@commandprompt.com) wrote: > You aren't wrong but this is also all but impossible for > PostgreSQL.org. I disagree and I don't feel it's useful to downplay what Jonathan is trying to do here. > The majority of features now developed are developed > by the needs of not PostgreSQL.Org but the needs of 2Q, EDB > customers or even Amazon (in a different vein). The good news is we > have at least two highly dedicated companies that donate code to the > open code base, the bad news is both of these companies are move > their clients to their closed platforms as soon as possible: I'm confident that there's quite a few more companies which contribute to PostgreSQL development than the two which you list, but I don't think it's useful or productive to call out specific companies in the manner you have here. I agree with your later comment that we should work to decrease the barrier to entry, but the way to do that is with specific concrete recommendations which we can act on. Let's be positive here instead of negative. Thanks! Stephen
On 11/20/2017 09:45 AM, Stephen Frost wrote: >> PostgreSQL.org. > I disagree and I don't feel it's useful to downplay what Jonathan is > trying to do here. I am not downplaying anything Jonathan wrote. I am also pretty sure he is a big boy and can defend points just fine. I was simply providing my insight as someone who does quite a bit of advocacy for this community. > > I'm confident that there's quite a few more companies which contribute > to PostgreSQL development than the two which you list, but I don't think > it's useful or productive to call out specific companies in the manner > you have here. There are but there is no question that EDB and 2Q are the largest contributors. I am sure others will also increase their portfolio of services offered to the community should their business deem it is a good investment. I am not sure why calling out to very successful companies who are contributing an enormous amount of resources to the community would not be considered productive. We should be thankful and gracious to those contributing companies. > > I agree with your later comment that we should work to decrease the > barrier to entry, but the way to do that is with specific concrete > recommendations which we can act on. Let's be positive here instead of > negative. Stating a fact is not negative. To your point: Let's have a public issue tracker that allows not only proper tracking of bugs but also features and desires of the project so that new and current users can have a simple interface to determine if there is anything they can help with. Thanks! JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
Josh, * Joshua D. Drake (jd@commandprompt.com) wrote: > To your point: Let's have a public > issue tracker that allows not only proper tracking of bugs but also > features and desires of the project so that new and current users > can have a simple interface to determine if there is anything they > can help with. We do have a todo list which is available here: https://wiki.postgresql.org/wiki/Todo There is also the GSoC projects list that's a good place for folks to look at who wish to contribute as well: https://wiki.postgresql.org/wiki/GSoC_2017 I'll be creating one for 2018 soon (most likely starting with a copy of the 2017 effort) as we'll need one in January when we go to apply for GSoC again. There's also the start of a roadmap here: https://wiki.postgresql.org/wiki/PostgreSQL11_Roadmap And I certainly encourage people to add to that. Thanks! Stephen
On 11/20/2017 10:07 AM, Stephen Frost wrote: > We do have a todo list which is available here: > https://wiki.postgresql.org/wiki/Todo > > There is also the GSoC projects list that's a good place for folks to > look at who wish to contribute as well: > > https://wiki.postgresql.org/wiki/GSoC_2017 > > I'll be creating one for 2018 soon (most likely starting with a copy of > the 2017 effort) as we'll need one in January when we go to apply for > GSoC again. > > There's also the start of a roadmap here: > > https://wiki.postgresql.org/wiki/PostgreSQL11_Roadmap > > And I certainly encourage people to add to that. > > Thanks! > > Stephen Stephen, Thank you for responding. You are absolutely correct that those sources are available and I was aware of them. Let me ask you a few questions about them: * How am I as a potential enthusiastic new contributor going to find those pages? They aren't linked from anywhere thatI can easily find. * How do I know the priority of the entries? * How do I know if the entries are official? * How doI know if it is already being worked on? o How do I know the progress? o How do I contribute to discussion of aparticular feature? Is that pgsql-hackers or is there a issue tracker? o Is pgsql-hackers the only way to contributeto development discussion? * Who decides the roadmap? o Can it be altered? o Why is it all but blank?* How can I get mentoring for something I am trying to work on? The list of barriers to providing a welcoming community to new potential contributors continues past this very brief list but I think I have illustrated my point. I am not in any way trying to be negative but I believe there is no question that we as a community could be doing a lot more to decrease the barrier of entry as well as increase the positivity of the experience of potential new comers. Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
On 2017-11-20 13:07:11 -0500, Stephen Frost wrote: > * Joshua D. Drake (jd@commandprompt.com) wrote: > > To your point: Let's have a public > > issue tracker that allows not only proper tracking of bugs but also > > features and desires of the project so that new and current users > > can have a simple interface to determine if there is anything they > > can help with. FWIW, I think a bugtracker to track bugs would be a good thing. But I seriously doubt it does anything meaningful for what we're discussing here. > We do have a todo list which is available here: > > https://wiki.postgresql.org/wiki/Todo Which is basically a list of project you shouldn't even consider doing without at least a flame retardant suit. It has negative value, because it steers people towards things that are likely going to be painful to tackle. Greetings, Andres Freund
On Mon, Nov 20, 2017 at 10:38 AM, Andres Freund <andres@anarazel.de> wrote: >> We do have a todo list which is available here: >> >> https://wiki.postgresql.org/wiki/Todo > > Which is basically a list of project you shouldn't even consider doing > without at least a flame retardant suit. It has negative value, because > it steers people towards things that are likely going to be painful to > tackle. That's not completely fair. Some of the items are actually implemented without too much fanfare, but just never get removed from the Todo list. -- Peter Geoghegan
On Mon, Nov 20, 2017 at 7:38 PM, Andres Freund <andres@anarazel.de> wrote:
-- On 2017-11-20 13:07:11 -0500, Stephen Frost wrote:
> * Joshua D. Drake (jd@commandprompt.com) wrote:
> > To your point: Let's have a public
> > issue tracker that allows not only proper tracking of bugs but also
> > features and desires of the project so that new and current users
> > can have a simple interface to determine if there is anything they
> > can help with.
FWIW, I think a bugtracker to track bugs would be a good thing. But I
seriously doubt it does anything meaningful for what we're discussing
here.
+<as many as I have left in my quota of pluses today> on both those arguments.
On 20 November 2017 at 12:18, Joshua D. Drake <jd@commandprompt.com> wrote: > The good news is we have at least two highly dedicated > companies that donate code to the open code base, the bad news is both of > these companies are move their clients to their closed platforms as soon as > possible: > > https://www.2ndquadrant.com/en/resources/2ndqpostgres/ > https://www.enterprisedb.com/products/edb-postgres-platform > > Or the users are migrating to something like RDS or Aurora. > > I am not criticizing 2Q, EDB or Amazon. They are all conducting smart > business but if we want more code that is generated by user demands and to > increase user retention, then we need to make the barriers of participation > lower for users. Right now they are exceedingly high. Josh, I believe you just made up "fake news" about 2ndQuadrant. Please explain your source of this information, or retract and apologise now. Thank you -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 11/20/2017 01:23 PM, Simon Riggs wrote: > > Josh, > > I believe you just made up "fake news" about 2ndQuadrant. > > Please explain your source of this information, or retract and apologise now. If you read the email I believe the source is clear. If the source is incorrect or I am misunderstanding what 2Q is doing, please feel free to clear it up. Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
On Mon, Nov 20, 2017 at 1:23 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > I believe you just made up "fake news" about 2ndQuadrant. > > Please explain your source of this information, or retract and apologise now. I also think that JD misrepresented 2ndQPostgres, which I gather is a set of backpatches for each stable release (it would be nice if that was open source, but that's beside the point). I think it's reasonable for a small number of users to want to get this or that critical performance feature without upgrading major version. I did this on an ad-hoc basis for 2ndQuadrant several years ago, usually for customers that really needed it. Isn't this kind of customization a big advantage of open source? When a user is screaming for smoother I/O from checkpoints on their enormous mission critical database, for example, are you really going to tell them that they're wrong for wanting something like this? That can amount to real downtime. Some full upgrades require a lot of planning, on account of changes to the catalogs and possibly on disk representation, but a lot of things don't need that. We've made huge progress in the last number of releases on performance, often by adding things that don't care about on-disk representation at all. While we as a community are better at judging risk than users collectively, that isn't necessarily true of individual users. I find it easy to believe that something like 2ndQPostgres could make sense in a small number of individual special cases, where there are already real problems, and the user really knows what they're doing. -- Peter Geoghegan
On 11/20/2017 03:03 PM, Peter Geoghegan wrote: > On Mon, Nov 20, 2017 at 1:23 PM, Simon Riggs <simon@2ndquadrant.com> wrote: >> I believe you just made up "fake news" about 2ndQuadrant. >> >> Please explain your source of this information, or retract and apologise now. > I also think that JD misrepresented 2ndQPostgres, which I gather is a > set of backpatches for each stable release (it would be nice if that > was open source, but that's beside the point). I am certainly not intending to misrepresent anyone but the fact that it isn't open source is exactly the point. """ The majority of features now developed are developed by the needs of not PostgreSQL.Org but the needs of 2Q, EDB customers or even Amazon (in a different vein). """ Nothing wrong with it not being open source but it is clearly driven by a need to build up their subscription model. > > I think it's reasonable for a small number of users to want to get > this or that critical performance feature without upgrading major > version. I did this on an ad-hoc basis for 2ndQuadrant several years > ago, usually for customers that really needed it. Yep, good for them. It is a great business model. > Isn't this kind of customization a big advantage of open source? When > a user is screaming for smoother I/O from checkpoints on their > enormous mission critical database, for example, are you really going > to tell them that they're wrong for wanting something like this? That > can amount to real downtime. You are absolutely correct and I agree with you. > > Some full upgrades require a lot of planning, on account of changes to > the catalogs and possibly on disk representation, but a lot of things > don't need that. We've made huge progress in the last number of > releases on performance, often by adding things that don't care about > on-disk representation at all. While we as a community are better at > judging risk than users collectively, that isn't necessarily true of > individual users. I find it easy to believe that something like > 2ndQPostgres could make sense in a small number of individual special > cases, where there are already real problems, and the user really > knows what they're doing. Nobody is arguing that, my original post (and follow up post) defended what 2Q (and EDB) are doing. I reread my original email and the only point of contention that I can find is here: ... the bad news is both of these companies are move their clients to their closed platforms as soon as possible Which was badly worded on my part but I don't think is inaccurate. We know EDB does this and I have had conversations with 2Q about their closed platform and although they are not doing exactly what EDB is doing you yourself posted that they are developing patches for back branches that are closed, not open. That is a bummer for the community but the reason they would do that is twofold: 1. Customer demand (see original post) 2. Revenue generation (see original post) To use a term from Simon, the only "FakeNews" here is that people are trying to make my post into something it wasn't. Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
On Mon, Nov 20, 2017 at 7:43 AM, Jonathan S. Katz <jkatz@postgresql.org> wrote: > Vertica current marketing is heavily targeting Postgres users: > > https://www.vertica.com/postgresql/ > Going forward, we must continue to understand our users’ needs while > ensuring that we can provide them as many resources as possible to help them > manage Postgres and show them that here is great help available when it is > required. I noticed this quote, which was fairly prominently placed: "PostgreSQL just isn’t designed to do analytic-type queries. Those are big aggregations, and Postgres, even though it is a great relational database, is really tailored for single-record lookup." Isn't the latter sentence really quite fair? It seems unwise to try to compete with a dedicated MPP column store solution like Vertica. Of course it's going to be much better at Postgres for a use-case that is truly within its niche. That said, I definitely think that systems like Vertica could easily get users due to the extremely simplistic, over-confident thinking that many people display around scalability. For some reason, I've met a number of people that believe that using Postgres somehow becomes untenable once you reach 1TB of data. Ideas like this imbed themselves by being simple, and getting repeated without being challenged. People think they need a column store, or something like Cassandra, when in fact they need to do some performance triage using pg_stat_statements, rethink backups, and maybe upgrade hardware. The lesson for us, as people that want to do better advocacy, may be that we need to counter these preposterous rules of thumb with simple counter examples. For example: I can restore the entire stack overflow databases on my laptop; it contains all stack overflow posts, ever, and comes in at approximately 100GB, including basic indexes. The largest table can have an index created on it in a couple of minutes on my machine that weighs less than 2KG, as we see here: https://blog.anayrat.info/en/2017/11/19/postgresql-10--icu--abbreviated-keys/ The actual stack overflow production database runs on a single SQL Server node, and does come in at 2 - 4 TB IIRC, because they have event data too, but the fact remains that you essentially get all of stack overflow in a ~100GB Postgres database. Many users have no idea how far a traditional monolithic relational database can scale without much difficulty, because they measure the wrong thing -- the thing that is easiest to measure. -- Peter Geoghegan
On 2017-11-20 15:20:21 -0800, Joshua D. Drake wrote: > On 11/20/2017 03:03 PM, Peter Geoghegan wrote: > > On Mon, Nov 20, 2017 at 1:23 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > > > I believe you just made up "fake news" about 2ndQuadrant. > > > > > > Please explain your source of this information, or retract and apologise now. > > I also think that JD misrepresented 2ndQPostgres, which I gather is a > > set of backpatches for each stable release (it would be nice if that > > was open source, but that's beside the point). > I am certainly not intending to misrepresent anyone but the fact that it > isn't open source is exactly the point. > > """ > The majority of features now developed are developed by the needs of not > PostgreSQL.Org but the needs of 2Q, EDB customers or even Amazon (in a > different vein). > """ > > Nothing wrong with it not being open source but it is clearly driven by a > need to build up their subscription model. So basically you critize the companies that actually do postgres development. Most of the other companies making money with postgres contribute pretty much no development time back. Greetings, Andres Freund
On 21/11/2017 07:58, Joshua D. Drake wrote: > On 11/20/2017 01:23 PM, Simon Riggs wrote: >> >> Josh, >> >> I believe you just made up "fake news" about 2ndQuadrant. >> >> Please explain your source of this information, or retract and >> apologise now. > > If you read the email I believe the source is clear. If the source is > incorrect or I am misunderstanding what 2Q is doing, please feel free to > clear it up. Josh, I think that 2ndQPostgres isn't as closed as you think it is. It appears to be a custom postgresql build, which has 2Q's new development features they plan to get merged into the main development. So it provides early access to new features that aren't in the community postgresql *yet* While the code doesn't appear to be easily accessible, 2ndqpostgres looks to be a binary build that combines the other projects which (mostly?) have github repos. On the 2ndqpostgres page it states - > No lock-in. 2ndQPostgres allows you the ability to move between community > PostgreSQL and 2ndQPostgres, and back again just by replacing the > binaries and restarting the server from a clean shutdown > Our customers are able to utilize cutting-edge features while the > community acceptance process continues at its own pace. By maintaining a binary compatible data storage, they aren't locking you into there product. Of course any development using new features will need adjusting if you go back to the community edition. This also helps by getting more testing before the features are added to the main development. Simon - is that correct? -- Shane Ambler pgSQL (at) Sheeky (dot) Biz
On 11/20/2017 04:04 PM, Andres Freund wrote: > > So basically you critize the companies that actually do postgres > development. Most of the other companies making money with postgres > contribute pretty much no development time back. You know what I find funny about this whole thread? 1. I defended 2Q 2. I defended EDB 3. I stated they are conducting good business 4. I stated that they are the largest contributors 5. I stated we should be thankful and gracious to those companies People all upset that I didn't 100% agree with how they do things. We apparently have exactly nothing better to do with our lives but pick apart an email and look for any crumb of discontent so we can jump on it like lemmings falling over a cliff. JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
On Mon, Nov 20, 2017 at 3:20 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > I reread my original email and the only point of contention that I can find > is here: > > ... the bad news is both of these companies are move their clients to their > closed platforms as soon as possible > > Which was badly worded on my part but I don't think is inaccurate. We know > EDB does this and I have had conversations with 2Q about their closed > platform and although they are not doing exactly what EDB is doing you > yourself posted that they are developing patches for back branches that are > closed, not open. But I don't see any evidence that they're doing that. The 2ndQuadrant soft fork is expressly about getting select features on earlier Postgres versions. That's not a platform. They don't have a v10 right now, presumably because that will only come when v11 is released, and has features that select users may have a sense of urgency about. > That is a bummer for the community but the reason they > would do that is twofold: > > 1. Customer demand (see original post) > 2. Revenue generation (see original post) > > To use a term from Simon, the only "FakeNews" here is that people are trying > to make my post into something it wasn't. I think that you insinuated plenty. I was the person that caused some kind of profound loss of innocence on your part, according to this blog post of yours: https://www.commandprompt.com/blog/the_fall_of_open_source/ (To be fair, I wasn't identifiable from the blogpost, which is just as well because I am significantly misrepresented in it.) I noticed today that Command Prompt prominently advertises being all about AWS: https://amazon.cioreview.com/vendor/2017/command_prompt,_inc. I really think that you need to give up this Postgres Trotskyite thing. It doesn't help. -- Peter Geoghegan
Hi, > On Nov 20, 2017, at 7:26 PM, Joshua D. Drake <jd@commandprompt.com> wrote: > > On 11/20/2017 04:04 PM, Andres Freund wrote: >> >> So basically you critize the companies that actually do postgres >> development. Most of the other companies making money with postgres >> contribute pretty much no development time back. > > You know what I find funny about this whole thread? > > 1. I defended 2Q > 2. I defended EDB > 3. I stated they are conducting good business > 4. I stated that they are the largest contributors > 5. I stated we should be thankful and gracious to those companies > > People all upset that I didn't 100% agree with how they do things. > > We apparently have exactly nothing better to do with our lives but pick apart an email and look for any crumb of discontentso we can jump on it like lemmings falling over a cliff. So the intent of my original email was such: 1. Demonstrate validation for Postgres being a market leader based on the fact that competitive databases are featuring usprominently in their marketing materials. This is really exciting!2. Show that there is more we can learn about how tohandle various data sets and workloads and see if there are things that exist in the community to already help with those,and if so, how we can make those more visible3. A reminder that it’s good for us to continue to listen to what ourusers need and want so we can continue to build a better product, and also highlight where we have excellent support bothas a community and from our commercial partners that sponsor and support our work.4. Lastly show that one of the newchallenges facing the community is user retention: this is a different problem than we have traditionally faced as weprimarily were focused on user acquisition. That’s all. I’m glad it’s generated some good ideas and discussion around those points, but the intention is to show thatwe as a community can all work together to solve these problems (in part, what Oleg suggested). We all want Postgresto be successful and all the different companies supporting Postgres have different ways of getting there, but atthe end of the day, we all share common goal of helping to improve Postgres and the community. Jonathan
On 20 November 2017 at 16:28, Joshua D. Drake <jd@commandprompt.com> wrote: > On 11/20/2017 01:23 PM, Simon Riggs wrote: >> >> >> Josh, >> >> I believe you just made up "fake news" about 2ndQuadrant. >> >> Please explain your source of this information, or retract and apologise >> now. > > > If you read the email I believe the source is clear. If the source is > incorrect or I am misunderstanding what 2Q is doing, please feel free to > clear it up. You have written > the bad news is both of > these companies are move their clients to their closed platforms as soon as > possible: > > https://www.2ndquadrant.com/en/resources/2ndqpostgres/ Clearly, the web page you cite is not a source for your comments, nor do I believe any such source exists because that is not nor has it ever been a 2ndQuadrant goal/activity. If you do not have any source then it is clear you just made that up. Writing bad news that you just made up is damaging. State your source, or retract. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 11/20/2017 04:35 PM, Peter Geoghegan wrote: > >> That is a bummer for the community but the reason they >> would do that is twofold: >> >> 1. Customer demand (see original post) >> 2. Revenue generation (see original post) >> >> To use a term from Simon, the only "FakeNews" here is that people are trying >> to make my post into something it wasn't. > I think that you insinuated plenty. I didn't, how is that unicorn ride working for you? JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
On 11/20/2017 04:49 PM, Simon Riggs wrote: > On 20 November 2017 at 16:28, Joshua D. Drake <jd@commandprompt.com> wrote: >> On 11/20/2017 01:23 PM, Simon Riggs wrote: >>> >>> Josh, >>> >>> I believe you just made up "fake news" about 2ndQuadrant. >>> >>> Please explain your source of this information, or retract and apologise >>> now. >> >> If you read the email I believe the source is clear. If the source is >> incorrect or I am misunderstanding what 2Q is doing, please feel free to >> clear it up. > You have written > >> the bad news is both of >> these companies are move their clients to their closed platforms as soon as >> possible: >> >> https://www.2ndquadrant.com/en/resources/2ndqpostgres/ > Clearly, the web page you cite is not a source for your comments, nor > do I believe any such source exists because that is not nor has it > ever been a 2ndQuadrant goal/activity. > > If you do not have any source then it is clear you just made that up. > Writing bad news that you just made up is damaging. > > State your source, or retract. Meh. > -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
On 20 November 2017 at 19:07, Shane Ambler <pgsql@sheeky.biz> wrote: > On 21/11/2017 07:58, Joshua D. Drake wrote: >> On 11/20/2017 01:23 PM, Simon Riggs wrote: >>> >>> Josh, >>> >>> I believe you just made up "fake news" about 2ndQuadrant. >>> >>> Please explain your source of this information, or retract and >>> apologise now. >> >> If you read the email I believe the source is clear. If the source is >> incorrect or I am misunderstanding what 2Q is doing, please feel free to >> clear it up. > > Josh, > I think that 2ndQPostgres isn't as closed as you think it is. It appears > to be a custom postgresql build, which has 2Q's new development features > they plan to get merged into the main development. So it provides early > access to new features that aren't in the community postgresql *yet* > > While the code doesn't appear to be easily accessible, 2ndqpostgres > looks to be a binary build that combines the other projects which > (mostly?) have github repos. > > On the 2ndqpostgres page it states - >> No lock-in. 2ndQPostgres allows you the ability to move between community >> PostgreSQL and 2ndQPostgres, and back again just by replacing the >> binaries and restarting the server from a clean shutdown > >> Our customers are able to utilize cutting-edge features while the >> community acceptance process continues at its own pace. > > By maintaining a binary compatible data storage, they aren't locking you > into there product. Of course any development using new features will > need adjusting if you go back to the community edition. > > This also helps by getting more testing before the features are added to > the main development. > > Simon - is that correct? Thanks Shane, that appears correct. And on the main point of contention: 2ndQuadrant is not migrating users to closed versions "as soon as possible", it is simply an extra service that customers may choose to utilize if they wish to do so because of issues they experience in a current supported version. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hah, I notice that nowhere in those marketing materials does it mention the amount of PostgreSQL code which is in Vertica. -- Josh Berkus Containers & Databases Oh My!
On Mon, Nov 20, 2017 at 10:46:57AM -0800, Peter Geoghegan wrote: > On Mon, Nov 20, 2017 at 10:38 AM, Andres Freund <andres@anarazel.de> wrote: > >> We do have a todo list which is available here: > >> > >> https://wiki.postgresql.org/wiki/Todo > > > > Which is basically a list of project you shouldn't even consider doing > > without at least a flame retardant suit. It has negative value, because > > it steers people towards things that are likely going to be painful to > > tackle. > > That's not completely fair. Some of the items are actually implemented > without too much fanfare, but just never get removed from the Todo > list. I go through the TODO list after every major release and remove the completed items I find. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Your email began with You aren't wrong but And then you make a fuss about people calling you “negative”. Yeah, funny. I didn’t miss all this wasted time, hope some of you have fun at least, -- Dimitri Fontaine > Le 20 nov. 2017 à 16:26, Joshua D. Drake <jd@commandprompt.com> a écrit : > > You know what I find funny about this whole thread?
On Mon, Nov 20, 2017 at 6:33 PM, Bruce Momjian <bruce@momjian.us> wrote: >> That's not completely fair. Some of the items are actually implemented >> without too much fanfare, but just never get removed from the Todo >> list. > > I go through the TODO list after every major release and remove the > completed items I find. I was referencing the "sorting" section, which has some obsolete items, like "Consider being smarter about memory and external files used during sorts". To be fair, whether or not the other sorting items are obsolete is more debatable, and it's evident that you've made an effort to maintain it. It's just very hard for anyone to maintain a todo list like this. The idea that I was trying to express was my remark was that the todo list inevitably becomes a thing with projects that are either likely controversial, or inherently very difficult. Because if they weren't, then they would have already been implemented (unless somebody forgot to remove them!). There is a lot of stuff like "Vacuum Gin indexes in physically order rather than logical order". In theory, that should be totally uncontroversial. In practice, it's very controversial, because the reason that it doesn't already work that way is subtle, has a lot to do with complicated concurrency issues. And, I suspect that the existing GIN VACUUM code is buggy. A list like the todo list sounds great, but in practice most of these things are so high context that in practice the best thing to do is seek guidance from experienced contributors. On the one hand, I think that it's good for new contributors to choose their own projects, so that they have a sense of ownership. On the other hand, there are significant risks if they really do that. It takes careful consideration on the part of the more experienced contributor giving advice. I've only managed to give good project advice 3 or 4 times, myself. -- Peter Geoghegan
On Nov 20, 2017, at 6:52 PM, Peter Geoghegan <pg@bowt.ie> wrote:On Mon, Nov 20, 2017 at 7:43 AM, Jonathan S. Katz <jkatz@postgresql.org> wrote:Vertica current marketing is heavily targeting Postgres users:
https://www.vertica.com/postgresql/Going forward, we must continue to understand our users’ needs while
ensuring that we can provide them as many resources as possible to help them
manage Postgres and show them that here is great help available when it is
required.
I noticed this quote, which was fairly prominently placed:
"PostgreSQL just isn’t designed to do analytic-type queries. Those are
big aggregations, and Postgres, even though it is a great relational
database, is really tailored for single-record lookup."
Isn't the latter sentence really quite fair? It seems unwise to try to
compete with a dedicated MPP column store solution like Vertica. Of
course it's going to be much better at Postgres for a use-case that is
truly within its niche.
That said, I definitely think that systems like Vertica could easily
get users due to the extremely simplistic, over-confident thinking
that many people display around scalability. For some reason, I've met
a number of people that believe that using Postgres somehow becomes
untenable once you reach 1TB of data. Ideas like this imbed themselves
by being simple, and getting repeated without being challenged. People
think they need a column store, or something like Cassandra, when in
fact they need to do some performance triage using pg_stat_statements,
rethink backups, and maybe upgrade hardware.
The lesson for us, as people that want to do better advocacy, may be
that we need to counter these preposterous rules of thumb with simple
counter examples.
Agreed 100%! There are a bunch of case studies strewn about the internet that show this too. For now, I think this is where we can leverage Planet PostgreSQL and blog more about these things with the examples. We could probably pick up some traffic around that and get more buzz about how well Postgres vertically scales.
[And to eat my own words, I am working on setting up my own blog after many, many years].
For example: I can restore the entire stack overflow
databases on my laptop; it contains all stack overflow posts, ever,
and comes in at approximately 100GB, including basic indexes. The
largest table can have an index created on it in a couple of minutes
on my machine that weighs less than 2KG, as we see here:
https://blog.anayrat.info/en/2017/11/19/postgresql-10--icu--abbreviated-keys/
The actual stack overflow production database runs on a single SQL
Server node, and does come in at 2 - 4 TB IIRC, because they have
event data too, but the fact remains that you essentially get all of
stack overflow in a ~100GB Postgres database. Many users have no idea
how far a traditional monolithic relational database can scale without
much difficulty, because they measure the wrong thing -- the thing
that is easiest to measure.
I would love to hear more and notice it’s not on http://pgeoghegan.blogspot.com/ ;-) But this would be great to promote and demonstrate how well Postgres performs with larger datasets.
Jonathan
On Mon, Nov 20, 2017 at 07:10:00PM -0800, Peter Geoghegan wrote: > On Mon, Nov 20, 2017 at 6:33 PM, Bruce Momjian <bruce@momjian.us> wrote: > >> That's not completely fair. Some of the items are actually implemented > >> without too much fanfare, but just never get removed from the Todo > >> list. > > > > I go through the TODO list after every major release and remove the > > completed items I find. > > I was referencing the "sorting" section, which has some obsolete > items, like "Consider being smarter about memory and external files > used during sorts". To be fair, whether or not the other sorting items > are obsolete is more debatable, and it's evident that you've made an > effort to maintain it. It's just very hard for anyone to maintain a > todo list like this. > > The idea that I was trying to express was my remark was that the todo > list inevitably becomes a thing with projects that are either likely > controversial, or inherently very difficult. Because if they weren't, > then they would have already been implemented (unless somebody forgot > to remove them!). > > There is a lot of stuff like "Vacuum Gin indexes in physically order > rather than logical order". In theory, that should be totally > uncontroversial. In practice, it's very controversial, because the > reason that it doesn't already work that way is subtle, has a lot to > do with complicated concurrency issues. And, I suspect that the > existing GIN VACUUM code is buggy. > > A list like the todo list sounds great, but in practice most of these > things are so high context that in practice the best thing to do is > seek guidance from experienced contributors. On the one hand, I think > that it's good for new contributors to choose their own projects, so > that they have a sense of ownership. On the other hand, there are > significant risks if they really do that. It takes careful > consideration on the part of the more experienced contributor giving > advice. I've only managed to give good project advice 3 or 4 times, > myself. Agreed. Feel free to improve the TODO list. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Josh, * Joshua D. Drake (jd@commandprompt.com) wrote: > Thank you for responding. You are absolutely correct that those > sources are available and I was aware of them. Let me ask you a few > questions about them: I suspect you know the answer to these questions as well, but since you asked, I'll answer them for you. > * How am I as a potential enthusiastic new contributor going to find > those pages? They aren't linked from anywhere that I can easily find. The wiki main page, at https://wiki.postgresql.org has a specific section on "Contributor Information." In that section is a link to "Development Information" and, after following that link, the very first link is to the ToDo list. The GSoC list is linked from the Google Summer of Code website which is seen by any student interested in working with the PostgreSQL project. We got 8 submissions last year for GSoC projects, leading me to believe that it was a successful approach to making that information available. > * How do I know the priority of the entries? This is an open source project and therefore the priorities are those of the individuals and the companies which contribute to PostgreSQL. I don't believe this is news to anyone who is familiar with Open Source. While I'd certainly applaud efforts to improve the understanding of Open Source in the general public, I do believe that's outside of the scope of what the PostgreSQL needs to be focusing on. > * How do I know if the entries are official? Same as the last question. > * How do I know if it is already being worked on? Following the link from "Development Information" leads to a number of resources, including the "Developer FAQ", which links immediately to pgsql-hackers, and to the commitfest app. Those are clearly the best places to look to see if a given item you're interested in is being worked on. > o How do I know the progress? Same as the last question. > o How do I contribute to discussion of a particular feature? Is > that pgsql-hackers or is there a issue tracker? The Developer FAQ makes this reasonably clear, but I'm sure it could be improved if someone would spend some effort going over it. > o Is pgsql-hackers the only way to contribute to development > discussion? No. > * Who decides the roadmap? > o Can it be altered? > o Why is it all but blank? This goes back to the question about priorities, answered above. > * How can I get mentoring for something I am trying to work on? This is covered in some detail when it comes to GSoC. For non-GSoC efforts, posting to -hackers would be the correct approach. > The list of barriers to providing a welcoming community to new > potential contributors continues past this very brief list but I > think I have illustrated my point. I am not in any way trying to be > negative but I believe there is no question that we as a community > could be doing a lot more to decrease the barrier of entry as well > as increase the positivity of the experience of potential new > comers. You asked a number of questions but didn't provide anything that materially moved us forward, from what I can see. If you feel that having a wiki page with the above questions and subsequent answers would be helpful then I encourage you to add such a page and link to it from appropriate places. The Developer FAQ actually does a reasonable job and isn't difficult to find, at least from my point of view, but I don't believe anyone would complain if further efforts were made to improve it. Thanks! Stephen
Andres, * Andres Freund (andres@anarazel.de) wrote: > On 2017-11-20 13:07:11 -0500, Stephen Frost wrote: > > * Joshua D. Drake (jd@commandprompt.com) wrote: > > > To your point: Let's have a public > > > issue tracker that allows not only proper tracking of bugs but also > > > features and desires of the project so that new and current users > > > can have a simple interface to determine if there is anything they > > > can help with. > > FWIW, I think a bugtracker to track bugs would be a good thing. But I > seriously doubt it does anything meaningful for what we're discussing > here. Yes, we've been around that discussion before and various attempts have been made to make it happen. I agree it's also largely orthogonal to the this discussion. > > We do have a todo list which is available here: > > > > https://wiki.postgresql.org/wiki/Todo > > Which is basically a list of project you shouldn't even consider doing > without at least a flame retardant suit. It has negative value, because > it steers people towards things that are likely going to be painful to > tackle. Perhaps you could provide some specifics about projects that aren't painful to tackle but would be good for newcomers to PostgreSQL to work on? In particular, we need that for GSoC 2018, but anything which isn't picked up as a GSoC project could certainly be picked up by non-GSoC developers. Thanks! Stephen
Gentlemen's,
This thread has followed an opposite direction for what Jonathan original intent. Please open a new thread for any non-vertica related mail.
In particular for the GSoC 2018 planning and the wiki/mailing-list as a development/planing tool vs something else.
Regards,
Jorge Solórzano
On Tue, Nov 21, 2017 at 6:41 AM, Stephen Frost <sfrost@snowman.net> wrote:
Perhaps you could provide some specifics about projects that aren't
painful to tackle but would be good for newcomers to PostgreSQL to work
on? In particular, we need that forGSoC 2018, but anything which isn't
picked up as a GSoC project could certainly be picked up by non-GSoC
developers.
Thanks!
Stephen
On 11/21/2017 04:39 AM, Stephen Frost wrote: > Josh, > > * Joshua D. Drake (jd@commandprompt.com) wrote: >> Thank you for responding. You are absolutely correct that those >> sources are available and I was aware of them. Let me ask you a few >> questions about them: > I suspect you know the answer to these questions as well, but since you > asked, I'll answer them for you. I do, my questions were rhetorical. If you are a new potential contributor these answers don't help either. As Jorge has mentioned this is far off topic for this thread. jD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL Centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://pgconf.org ***** Unless otherwise stated, opinions are my own. *****
On Mon, Nov 20, 2017 at 4:43 PM, Jonathan S. Katz <jkatz@postgresql.org> wrote:
Hi,Vertica current marketing is heavily targeting Postgres users:and features this directly from their homepage. I also know of at least one campaign that has gone out around this.The good news for our community is that because of all the advances in Postgres and the overall growth to our user base, there are database companies looking to win users over to their platforms. This is also a good learning experience for us too: because we have so many users, we also have to look at user retention to ensure that we have enough features and available knowledge to manage Postgres. I have not read the case study that they proposed, but knowing the group of people who work on Postgres and our software ecosystem, we may have been able to retain those databases on Postgres.
To my mind I am not that concerned about who moves away to PostgreSQL. People migrate for various reasons. Some of those are technical. Some of them are political. In the end whether a given user migrates is not, really, in our control. I don't think focusing on "why did we lose these" is helpful. What is helpful instead is understanding how to do the same sorts of things on PostgreSQL as Vertica.
Going forward, we must continue to understand our users’ needs while ensuring that we can provide them as many resources as possible to help them manage Postgres and show them that here is great help available when it is required.
There are other important topics here in analytics workloads. Up until this year most of my experience, for example, was in workflows which were mixed buy oriented primarily towards OLTP engines. So things like ERP, scientific data processing and the like. Now I have stepped into analytics and am working in a very large environment. The big differences in terms of how you handle sparse matrices and other sorts of things crop up. Systems like Cassandra and Vertica solve some analytics problems but they don't solve all problems extremely well. They have their own caveats and learning curves. At least as far as our experience at Adjust we have typically solved these problems ourselves.
The arguments about the development direction of PostgreSQL in my view is similarly problematic. Companies contribute what their users need. And that's good. But almost all the literature published about PostgreSQL focuses on OLTP problems and solutions. What we are actually missing is a series of good solid papers, discussions, and so forth regarding how to do OLAP on PostgreSQL effectively. This is an area where PostgreSQL is rapidly growing but without guidance, people aren't going to know how to think about, model, and approach a lot of these problems.
So I am going to suggest a different approach, namely that we see what we can do to create space and exposure for stuff discussing OLAP etc. on PostgreSQL. Maybe a special category of case studies and maybe contributed white papers or the like. But this is an area where a lot of knowledge sharing could be very helpful.
Best,Jonathan
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in.
Josh, Just a heads up for your info, I got the following yesterday. The following message to <jd@commandprompt.com> was undeliverable. The reason for the problem: 5.3.0 - Other mail system problem 554-'5.7.1 DNS Blacklisted by safe.dnsbl.sorbs.net'