How can we expand PostgreSQL ecosystem?

On 06/03/16 18:29, MauMau wrote:

As I said in the previous greeting mail, I'd like to discuss how to
expand PostgreSQL ecosystem. Here, ecosystem means "interoperability"
-- the software products and cloud services which use/support
PostgreSQL. If pgsql-advocacy or somewhere else is better for this
topic, just tell me so.

For cloud - in particular Openstack (which I am working with ATM), the
biggest thing would be:

- multi-master replication

or failing that:

- self managing single master failover (voting/quorum etc)

so that operators can essentially 'set and forget'. We currently use
Mysql+ Galera (multi master) and Mongodb (self managing single master)
and the convenience and simplicity is just so important (Openstack is a
huge complex collection of services - hand holding of any one service is
pretty much a non starter).

regards

Mark

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 6 March 2016 at 13:29, MauMau <maumau307@gmail.com> wrote:

However, there is a problem. The number of software is very small that
the users can know to interoperate with PostgreSQL.

Yep, even among many big and popular OSS apps.

Many applications might be interoperable through standard interfaces like
JDBC/ODBC, but the case is unknown.

Not tons, in my experience, unless they're doing only trivial SQL. Many
*are* PostgreSQL-compatible via higher level abstractions like
Hibernate/JPA, ActiveRecord, etc, but those are also the systems that are
least interesting from a database PoV since they generally use only the
lowest common denominator of DB features, with the DB treated as not much
more than a dumb row store.

There *is* some cool stuff we could do with ORMs if we'd stop sneering at
them for long enough. I'd love to see the ability to omit repeated values
on a LEFT JOIN for example, where we project the left tuple completely the
first time we see it, i.e. when the key changes, then we emit only the key
when we project it again for subsequent right-hand rows. This would save
ORMs an immense amount of computational effort and network bandwidth... and
you only have to patch each ORM to use it, not each app.

Even more interesting would be to emit JSON as a left-join product, so a
typical chained left join to fetch 'customer' with 'account' and 'contact'
for each 'account' would emit

[
{
customer_id: '42',
...
accounts: [
{
account_id = 1,
...,
contacts: [
{
contact_id = 4,
...
},
{
contact_id = 5,
...
}
]
},
{
account_id = 2,
...
}
]
},
{
customer_id = 43,
...
}
]

i.e. emit the requested data in the form the application actually wants,
without needing to produce a giant relational projection and then process
it in the application.

We could help ORMs solve the N+1 SELECTs problem and help them avoid
transferring vast join projections unnecessarily. That'd make PostgreSQL
pretty compelling for exactly the users we're mostly too busy dismissing to
consider.

When you consider the adoption of PostgreSQL in the Rails/ActiveRecord
crowd I think there's some interesting potential there. What there isn't is
funding AFAIK.

I'd be interested in reaching out to some Hibernate/JPA and ActiveRecord
folks about how the DB could help the ORM and have been meaning to explore
this area for a while, but -ENOTIME. If anyone pursues it I'll be really
interested in hearing how things go.

Besides, in practice, we probably should increase the number of software
interoperable with PostgreSQL. e.g. one customer asked us whether Arcserve
can be used to back up PostgreSQL databases, but unfortunately we had to
answer no. They are using Arcserve to back up Oracle databases and other
resources. "Then, you can use NetVault instead" is not the best answer;
they just want to replace the database.

The "we" here is the problem. It's not likely to be folks focused on
PostgreSQL core dev, but ... who, exactly?

Provide technical assistance to those vendors as an organization so that
they can support PostgreSQL smoothly.

This one is a help. That said, pgsql-general is pretty helpful already...

* Make a directory of software/services that can be used with PostgreSQL
on the community web site (wiki.postgresql.org or www.postgresql.org).
Software/services vendors and PostgreSQL developers/users can edit this
directory.

I thought we had that? Yep.

http://www.postgresql.org/download/product-categories/

It's pretty invisible though, partly due to the postgresql.org landing
page's need for a trim-down and tidy. (Don't even get me started on
http://www.postgresql.org/about/ )

* How/Where can we get the knowledge of expanding the software ecosystem?
Is there any OSS project that we can learn from?

Mongo.

Despite the real world capabilities being ... patchy ... and the magic
autoscaling claims etc being a bit dubious, they've successfully marketed
it as the magic solution for everything.

This leads to a fair bit of disillusionment, but also massive adoption. I
think they take it too far, but try opening these two pages side by side:

http://www.postgresql.org/
https://www.mongodb.org/

... and tell me which you'd look over first if you were evaluating things.

How can we attract software vendors to support PostgreSQL? What words are
convincing to appeal the increasing potential of PostgreSQL as a good
replacement for commercial databases?

Change the name :p

<dives for fireproof clothing>

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Mark Kirkwood
For cloud - in particular Openstack (which I am working with ATM), the
biggest thing would be:

- multi-master replication

or failing that:

- self managing single master failover (voting/quorum etc)

so that operators can essentially 'set and forget'. We currently use
Mysql+ Galera (multi master) and Mongodb (self managing single master)
and the convenience and simplicity is just so important (Openstack is a
huge complex collection of services - hand holding of any one service is
pretty much a non starter).

Yes, I was also asked whether PostgreSQL has any optional functionality like Galera Cluster for MySQL. He was planning a scalable PaaS service which performs heavy reads and writes. Demand exists.

Regards
Takayuki Tsunakawa

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

* Only 24 open source projects are listed as interoperable.
Open Source Projects Using PostgreSQL
https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL

It's pity that some very popular OSS projects do not care about
PostgreSQL. WordPress is one of such OSS projects. Long time ago
someone came up with "PostgreSQL plugin" but it didn't catch up
WordPress releases and now cannot be used with the latest version of
WordPress. In my understanding WordPress fixes security issues only
with the latest version; as a result, we cannot use safe version of
WordPress if we want to use PostgreSQL for its database engine.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 03/07/2016 07:30 AM, Tsunakawa, Takayuki wrote:

From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Mark Kirkwood
For cloud - in particular Openstack (which I am working with ATM), the
biggest thing would be:

- multi-master replication

or failing that:

- self managing single master failover (voting/quorum etc)

Hmm.... consul (optionally in a dockerized setup) ?
https://www.consul.io/

so that operators can essentially 'set and forget'. We currently use
Mysql+ Galera (multi master) and Mongodb (self managing single master)
and the convenience and simplicity is just so important (Openstack is a
huge complex collection of services - hand holding of any one service is
pretty much a non starter).

Yes, I was also asked whether PostgreSQL has any optional functionality like Galera Cluster for MySQL. He was planning a scalable PaaS service which performs heavy reads and writes. Demand exists.

AFAIK, Galera has its own set of drawbacks....

HTH,

/ JL

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 7 March 2016 at 17:06, Tatsuo Ishii <ishii@postgresql.org> wrote:

* Only 24 open source projects are listed as interoperable.
Open Source Projects Using PostgreSQL
https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL

It's pity that some very popular OSS projects do not care about
PostgreSQL. WordPress is one of such OSS projects. Long time ago
someone came up with "PostgreSQL plugin" but it didn't catch up
WordPress releases and now cannot be used with the latest version of
WordPress. In my understanding WordPress fixes security issues only
with the latest version; as a result, we cannot use safe version of
WordPress if we want to use PostgreSQL for its database engine.
<http://www.postgresql.org/mailpref/pgsql-hackers&gt;

The killer there was that the plugin could only alter queries used by
Wordpress core. Nobody just uses Wordpress core. The whole reason
Wordpress became popular is the vast collection of plugins, themes, etc.
90% of which are written by three stoned monkeys who once saw a PHP 4
manual in a mirror, which is part of why it has such an appalling security
history. Plugins can just talk straight to the DB, and are written by
people who have never heard of parametrized queries or, half the time,
transactions.

What makes it popular and successful is also what makes supporting Pg in a
way that'll actually see useful adoption hard.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 6 March 2016 at 13:46, Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
wrote:

For cloud - in particular Openstack (which I am working with ATM), the
biggest thing would be:

- multi-master replication

Working on it ;)

or failing that:

- self managing single master failover (voting/quorum etc)

I don't think that's either/or, frankly, in that you can have multimaster
that needs a lot of babying or multimaster that has magic autoscaling and
autosharding and autofailover (or some approximation of).

That said, I firmly believe that autofailover is a giant foot-cannon unless
*regularly tested in the live environment*. The same is true of most of the
other autoFOOs. Products that claim to offer them often do some creative
bending of the guarantees they claim to offer elsewhere, so I also tend to
take such claims with a grain or ten of salt. Witness the recent
discussions of Galleria.

Pg currently makes testing autofailover pretty painful, since failover is a
major and disruptive event that reduces your redundancy until the old
master has resynced. pg_rewind will help a lot there if it proves robust
enough - hopefully enough that we can start adopting a
ChaosMonkey-flavoured approach to HA.

Eventually, a long way down the track, I'd like to see logical replication
based failover be part of that picture too.

so that operators can essentially 'set and forget'. We currently use

Mysql+ Galera (multi master) and Mongodb (self managing single master)
and the convenience and simplicity is just so important (Openstack is a
huge complex collection of services - hand holding of any one service is
pretty much a non starter).

I rarely see a service where "self managing" doesn't turn into "needs lots
of tuning, tweaking and fixing" at significant scales, but it's nice to
have it be automagic and correct enough of the time for small to medium
deployments.

That's one area where Pg core's focus on getting it right can hold us back.
Not necessarily for the worse. We're not usually big on saying "oh, it
probably won't corrupt your data if you use it at reasonable transaction
rates with reasonable inter-node latencies". Having had a few interesting
surprises with
set-and-forget-until-it-brings-the-whole-network-down-and-you-have-no-idea-how-to-fix-it
systems I tend to favour doing it right myself. Which is why some of the
compromises currently necessary in the logical replication space on Pg irk
me so....

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

From: Craig Ringer
--------------------------------------------------
We could help ORMs solve the N+1 SELECTs problem and help them avoid transferring vast join projections unnecessarily. That'd make PostgreSQL pretty compelling for exactly the users we're mostly too busy dismissing to consider.

I'd be interested in reaching out to some Hibernate/JPA and ActiveRecord folks about how the DB could help the ORM and have been meaning to explore this area for a while, but -ENOTIME. If anyone pursues it I'll be really interested in hearing how things go.
--------------------------------------------------

You have various ideas and experience, don't you?
Are those ideas on ORMs beneficial exclusively to PostgreSQL or to all DBMSs? I don't know the structure of ORMs allows for improvements to be advantageous to a specific DBMS.

From: Craig Ringer
--------------------------------------------------
When you consider the adoption of PostgreSQL in the Rails/ActiveRecord crowd I think there's some interesting potential there. What there isn't is funding AFAIK.
--------------------------------------------------

Yes, this needs investigation. But I'm not sure Ruby ecosystem is the world where PostgreSQL should go. Do many users of commercial databases have assets based on Ruby which they want to continue to use?

Recently, my colleague, a Ruby committer, told me that Ruby may be losing momentum because developers are moving toward JavaScript+Node.js. Addressing JavaScript+Node.js ecosystem may be more beneficial to PostgreSQL popularity.

From: Craig Ringer
--------------------------------------------------
Besides, in practice, we probably should increase the number of software interoperable with PostgreSQL. e.g. one customer asked us whether Arcserve can be used to back up PostgreSQL databases, but unfortunately we had to answer no. They are using Arcserve to back up Oracle databases and other resources. "Then, you can use NetVault instead" is not the best answer; they just want to replace the database.

The "we" here is the problem. It's not likely to be folks focused on PostgreSQL core dev, but ... who, exactly?
--------------------------------------------------

I simply meant PostgreSQL developers and users (of course, including me) who wish PostgreSQL to become more popular and thrive for a long time. Most users use databases combined with applications (not just the DBMS alone), so increasing the number of applications supporting PostgreSQL will attract more users. PostgreSQL developers will gain wider market and business/job oppotunities.

From: Craig Ringer
--------------------------------------------------
Provide technical assistance to those vendors as an organization so that they can support PostgreSQL smoothly.

This one is a help. That said, pgsql-general is pretty helpful already...
--------------------------------------------------

Certainly.

From: Craig Ringer
--------------------------------------------------
* Make a directory of software/services that can be used with PostgreSQL on the community web site (wiki.postgresql.org or www.postgresql.org).
Software/services vendors and PostgreSQL developers/users can edit this directory.

I thought we had that? Yep.

http://www.postgresql.org/download/product-categories/

It's pretty invisible though, partly due to the postgresql.org landing page's need for a trim-down and tidy. (Don't even get me started on http://www.postgresql.org/about/ )
--------------------------------------------------

Thanks, I know the page. It totally slipped off my head. Yes, I'm afraid the path to the page (Home -> Download -> Software Catalog) is not good.

From: Craig Ringer
--------------------------------------------------
* How/Where can we get the knowledge of expanding the software ecosystem? Is there any OSS project that we can learn from?

Mongo.

http://www.postgresql.org/
https://www.mongodb.org/

... and tell me which you'd look over first if you were evaluating things.
--------------------------------------------------

Yes, I want something like a site map. But I couldn't find information about software supporting MongoDB within a few minutes, and I gave up. MongoDB website may not be necessarily nice.

In that respect, MySQL website is good, because "Partners" link is on the home page. PostgreSQL website can also have "Related software" or something like that on the home page.

From: Craig Ringer
--------------------------------------------------
How can we attract software vendors to support PostgreSQL? What words are convincing to appeal the increasing potential of PostgreSQL as a good replacement for commercial databases?

Change the name :p
--------------------------------------------------

Sorry, I couldn't catch the implication. Do you mean changing the name PostgreSQL to something else, or just a joke?

Regards
MauMau

The killer there was that the plugin could only alter queries used by
Wordpress core. Nobody just uses Wordpress core. The whole reason
Wordpress became popular is the vast collection of plugins, themes, etc.
90% of which are written by three stoned monkeys who once saw a PHP 4
manual in a mirror, which is part of why it has such an appalling security
history. Plugins can just talk straight to the DB, and are written by
people who have never heard of parametrized queries or, half the time,
transactions.

Well, I think one of the reasons is they only know MySQL. MySQL had
been lack of transactions and parametrized queries.

What makes it popular and successful is also what makes supporting Pg in a
way that'll actually see useful adoption hard.

Let's enlighten WordPress users by supporting PostgreSQL.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Craig Ringer
--------------------------------------------------
* Make a directory of software/services that can be used with PostgreSQL on the community web site (wiki.postgresql.org or www.postgresql.org).
Software/services vendors and PostgreSQL developers/users can edit this directory.

I thought we had that? Yep.

http://www.postgresql.org/download/product-categories/
--------------------------------------------------

There are 172 software listed here in total. Excluding the following categories which can be considered part of a DBMS, the number is 121. Certainly not much.

•Drivers and interfaces
•PostgreSQL-derived servers
•PostgreSQL extensions
•Procedural languages

Speaking without fear of misunderstanding, there are few software products listed that migrators from commercial databases would want to know... Even famous software like Pentaho, Microstrategy, Tableau, Eclipse and Visual Studio are not listed, which are known to work or should work with PostgreSQL. This is a shame.

Why don't we enrich the catalog? I'd like to hear ideas on how to enrich the catalog efficiently. It's ideal for software vendors and users to voluntarily add to the catalog.

Regards
MauMau

On 03/07/2016 11:31 AM, MauMau wrote:

Why don't we enrich the catalog? I'd like to hear ideas on how to
enrich the catalog efficiently. It's ideal for software vendors and
users to voluntarily add to the catalog.

I think the product/software directory has vastly outlived its purpose.
We are not longer a, "I wonder if Pg works with X". It is more, "Why
doesn't X work with Pg" nowadays.

I could see a page that discusses various tools in general that are
supported but by no means do we need a "list" anymore.

Sincerely,

JD

--
Command Prompt, Inc. http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 07/03/16 11:54, José Luis Tallón wrote:

On 03/07/2016 07:30 AM, Tsunakawa, Takayuki wrote:

From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Mark Kirkwood
For cloud - in particular Openstack (which I am working with ATM), the
biggest thing would be:

- multi-master replication

or failing that:

- self managing single master failover (voting/quorum etc)

Hmm.... consul (optionally in a dockerized setup) ?
https://www.consul.io/

There are already HA solutions based on consensus, like Patroni:
https://github.com/zalando/patroni

so that operators can essentially 'set and forget'. We currently use
Mysql+ Galera (multi master) and Mongodb (self managing single master)
and the convenience and simplicity is just so important (Openstack is a
huge complex collection of services - hand holding of any one
service is
pretty much a non starter).

Yes, I was also asked whether PostgreSQL has any optional
functionality like Galera Cluster for MySQL. He was planning a
scalable PaaS service which performs heavy reads and writes. Demand
exists.

AFAIK, Galera has its own set of drawbacks....

Right, some of them are explained here:
https://aphyr.com/posts/327-jepsen-mariadb-galera-cluster

Regards,

Álvaro

--
Álvaro Hernández Tortosa

-----------
8Kdata

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

MauMau,

Crossing this over to pgsql-advocacy list where it really belongs.
That's what that list is *for*.

Especially since the discussion on -hackers has focused on new
PostgreSQL Features, which while also good don't address the general
question.

On 03/05/2016 09:29 PM, MauMau wrote:

Hello,
As I said in the previous greeting mail, I'd like to discuss how to
expand PostgreSQL ecosystem. Here, ecosystem means "interoperability"
-- the software products and cloud services which use/support
PostgreSQL. If pgsql-advocacy or somewhere else is better for this
topic, just tell me so.
THE BACKGROUND
==================================================
Thanks to the long and hard efforts by the community, PostgreSQL has
been evolving to be a really great software comparable to existing
strong commercial products. Open source databases are gaining more
popularity to influence the database market.
Open source threatens to eat the database market
http://www.infoworld.com/article/2916057/open-source-software/open-source-threatens-to-eat-the-database-market.html
"Though the proprietary RDBMS market grew at a sluggish 5.4 percent in
2014, the open source database market grew 31 percent to hit $562 million."
"As Gartner highlights in a recent research report, open source
databases now consume 25 percent of relational database usage."
Perhaps related to this is that the revenues of Oracle, IBM and
Microsoft have been declining (but I read in an article that SQL Server
is gaining more revenue).
On the other hand, there is a gulf between the two top popular databases
-- Oracle and MySQL -- and PostgreSQL. They are nearly five times more
popular than PostgreSQL.
DB-Engines Ranking
http://db-engines.com/en/ranking
Yes, I understand this ranking doesn't necessarily reflect the actual
use, but I also don't think the ranking is far from the real
popularity. In fact, some surveys show that MySQL has been in more
widespread use even here in Japan than PostgreSQL since around 2010 (IIRC).
What should we do to boost the popularity of PostgreSQL? One challenge
is to increase the number of software which supports PostgreSQL. To take
advantage of the trend of shift from commercial products to open source,
PostgreSQL needs to interoperate with many software that are used
together with the commercial databases.
The easily understandable target is Oracle, because it is anticipated
that more users of Oracle will seek another database to avoid the
expensive Oracle Standard Edition 2 and increasing maintenance costs.
In addition, PostgreSQL has affinity for Oracle.
However, there is a problem. The number of software is very small that
the users can know to interoperate with PostgreSQL. That is, when the
users want to migrate from commercial databases to PostgreSQL, they
can't get information on whether they can continue to use their assets
with PostgreSQL. Many applications might be interoperable through
standard interfaces like JDBC/ODBC, but the case is unknown. For example:
* Only 24 open source projects are listed as interoperable.
Open Source Projects Using PostgreSQL
https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL
* Even EnterpriseDB has only 12 certified application vendors.
http://www.enterprisedb.com/partner-programs/enterprisedb-certified-application-vendors
* PostgreSQL Enterprise Consortium lists only about30 related products
(Japanese only).
https://www.pgecons.org/postgresql-info/business_sw/
* MySQL touts more than 2,000 ISV/OEM/VARs.
http://www.mysql.com/oem/
Besides, in practice, we probably should increase the number of software
interoperable with PostgreSQL. e.g. one customer asked us whether
Arcserve can be used to back up PostgreSQL databases, but unfortunately
we had to answer no. They are using Arcserve to back up Oracle
databases and other resources. "Then, you can use NetVault instead" is
not the best answer; they just want to replace the database.
PROPOSAL
==================================================
Last month, I attended the steering committee of PostgreSQL Enterprise
Consortium (PGECons) for the first time and proposed starting the
following activity. PGECons is a Japanese non-profit organization to
promote PostgreSQL for enterprise use. The members include NTT, SRA OSS
(Tatsuo Ishii runs), NEC, Hitachi, HP, Fujitsu, etc. We concluded that
we need to consult the PostgreSQL community on how to proceed the
activity and work in cooperation with the community.
* Attract and ask product/service vendors to support/use PostgreSQL.
Provide technical assistance to those vendors as an organization so that
they can support PostgreSQL smoothly.
If the vendors aren't proactive, we verify the interoperability with
their software by executing it.
* Make a directory of software/services that can be used with PostgreSQL
on the community web site (wiki.postgresql.org or www.postgresql.org).
Software/services vendors and PostgreSQL developers/users can edit this
directory.
This list not only has the names of software and its vendors, but also
other information such as the level of interoperability (certified by
the vendor, or verified by the community/users) and remarks about
configuration, tuning, and whatever useful for users.
PostgreSQL Enterprise Consortium (PGECons)
https://www.pgecons.org/en
CONSULTATION AND DISCUSSION
==================================================
I'd like to discuss and hear opinions on how can we expand the ecosystem
of PostgreSQL. Example points are:
* How/Where can we get the knowledge of expanding the software
ecosystem? Is there any OSS project that we can learn from?
How can we attract software vendors to support PostgreSQL? What words
are convincing to appeal the increasing potential of PostgreSQL as a
good replacement for commercial databases?
* How can we gain momentum for the activity?
Can we involve globally influential companies like Dell, HPE, and Red Hat?
* Do we need some kind of assistance from a foundation or establish a
foundation?
There should be many, many software to address, so the ecosystem
activity has to be long-lasting. Plus, money and expertise is necessary.
Would we benefit if we join the Linux Foundation Collaborative Projects?
Linux Foundation Collaborative Projects
http://collabprojects.linuxfoundation.org/
* Which software/services in what category should we address preferentially?
What software would many users desire to be interoperable when migrating
from commercial databases?
What is the effective way to absorb user requests for this? Is it
enough to make a questionnaire like the following? What is the popular
questionnaire site which can catch many users (SurveyMonkey?)
https://postgresql.uservoice.com/forums/21853-general
Regards
MauMau

--
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy

On 03/07/2016 01:37 PM, Josh berkus wrote:

MauMau,

Crossing this over to pgsql-advocacy list where it really belongs.
That's what that list is *for*.

Especially since the discussion on -hackers has focused on new
PostgreSQL Features, which while also good don't address the general
question.

On 03/05/2016 09:29 PM, MauMau wrote:

Hello,
As I said in the previous greeting mail, I'd like to discuss how to
expand PostgreSQL ecosystem. Here, ecosystem means "interoperability"
-- the software products and cloud services which use/support
PostgreSQL. If pgsql-advocacy or somewhere else is better for this
topic, just tell me so.
THE BACKGROUND
==================================================
Thanks to the long and hard efforts by the community, PostgreSQL has
been evolving to be a really great software comparable to existing
strong commercial products. Open source databases are gaining more
popularity to influence the database market.
Open source threatens to eat the database market
http://www.infoworld.com/article/2916057/open-source-software/open-source-threatens-to-eat-the-database-market.html
"Though the proprietary RDBMS market grew at a sluggish 5.4 percent in
2014, the open source database market grew 31 percent to hit $562 million."
"As Gartner highlights in a recent research report, open source
databases now consume 25 percent of relational database usage."
Perhaps related to this is that the revenues of Oracle, IBM and
Microsoft have been declining (but I read in an article that SQL Server
is gaining more revenue).
On the other hand, there is a gulf between the two top popular databases
-- Oracle and MySQL -- and PostgreSQL. They are nearly five times more
popular than PostgreSQL.
DB-Engines Ranking
http://db-engines.com/en/ranking
Yes, I understand this ranking doesn't necessarily reflect the actual
use, but I also don't think the ranking is far from the real
popularity. In fact, some surveys show that MySQL has been in more
widespread use even here in Japan than PostgreSQL since around 2010 (IIRC).
What should we do to boost the popularity of PostgreSQL? One challenge
is to increase the number of software which supports PostgreSQL. To take
advantage of the trend of shift from commercial products to open source,
PostgreSQL needs to interoperate with many software that are used
together with the commercial databases.
The easily understandable target is Oracle, because it is anticipated
that more users of Oracle will seek another database to avoid the
expensive Oracle Standard Edition 2 and increasing maintenance costs.
In addition, PostgreSQL has affinity for Oracle.
However, there is a problem. The number of software is very small that
the users can know to interoperate with PostgreSQL. That is, when the
users want to migrate from commercial databases to PostgreSQL, they
can't get information on whether they can continue to use their assets
with PostgreSQL. Many applications might be interoperable through
standard interfaces like JDBC/ODBC, but the case is unknown. For example:
* Only 24 open source projects are listed as interoperable.
Open Source Projects Using PostgreSQL
https://wiki.postgresql.org/wiki/OpenSource_Projects_Using_PostgreSQL
* Even EnterpriseDB has only 12 certified application vendors.
http://www.enterprisedb.com/partner-programs/enterprisedb-certified-application-vendors
* PostgreSQL Enterprise Consortium lists only about30 related products
(Japanese only).
https://www.pgecons.org/postgresql-info/business_sw/
* MySQL touts more than 2,000 ISV/OEM/VARs.
http://www.mysql.com/oem/

Though if you go to the actual list:
http://www.mysql.com/customers/embedded/

There is at a rough count, 300.

This points to the central dilemma, some of which is addressed below,
separating fact from fiction.

Besides, in practice, we probably should increase the number of software
interoperable with PostgreSQL. e.g. one customer asked us whether
Arcserve can be used to back up PostgreSQL databases, but unfortunately
we had to answer no. They are using Arcserve to back up Oracle
databases and other resources. "Then, you can use NetVault instead" is
not the best answer; they just want to replace the database.
PROPOSAL
==================================================
Last month, I attended the steering committee of PostgreSQL Enterprise
Consortium (PGECons) for the first time and proposed starting the
following activity. PGECons is a Japanese non-profit organization to
promote PostgreSQL for enterprise use. The members include NTT, SRA OSS
(Tatsuo Ishii runs), NEC, Hitachi, HP, Fujitsu, etc. We concluded that
we need to consult the PostgreSQL community on how to proceed the
activity and work in cooperation with the community.
* Attract and ask product/service vendors to support/use PostgreSQL.
Provide technical assistance to those vendors as an organization so that
they can support PostgreSQL smoothly.
If the vendors aren't proactive, we verify the interoperability with
their software by executing it.
* Make a directory of software/services that can be used with PostgreSQL
on the community web site (wiki.postgresql.org or www.postgresql.org).
Software/services vendors and PostgreSQL developers/users can edit this
directory.
This list not only has the names of software and its vendors, but also
other information such as the level of interoperability (certified by
the vendor, or verified by the community/users) and remarks about
configuration, tuning, and whatever useful for users.
PostgreSQL Enterprise Consortium (PGECons)
https://www.pgecons.org/en
CONSULTATION AND DISCUSSION
==================================================
I'd like to discuss and hear opinions on how can we expand the ecosystem
of PostgreSQL. Example points are:
* How/Where can we get the knowledge of expanding the software
ecosystem? Is there any OSS project that we can learn from?
How can we attract software vendors to support PostgreSQL? What words
are convincing to appeal the increasing potential of PostgreSQL as a
good replacement for commercial databases?
* How can we gain momentum for the activity?
Can we involve globally influential companies like Dell, HPE, and Red Hat?
* Do we need some kind of assistance from a foundation or establish a
foundation?
There should be many, many software to address, so the ecosystem
activity has to be long-lasting. Plus, money and expertise is necessary.
Would we benefit if we join the Linux Foundation Collaborative Projects?
Linux Foundation Collaborative Projects
http://collabprojects.linuxfoundation.org/
* Which software/services in what category should we address preferentially?
What software would many users desire to be interoperable when migrating
from commercial databases?
What is the effective way to absorb user requests for this? Is it
enough to make a questionnaire like the following? What is the popular
questionnaire site which can catch many users (SurveyMonkey?)
https://postgresql.uservoice.com/forums/21853-general
Regards
MauMau

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy

On Mon, Mar 7, 2016 at 9:25 PM, Craig Ringer wrote:

pg_rewind will help a lot there if it proves robust enough -

FWIW, some of my colleagues are doing a lot of QE/QA on a HA solution
based on pg_rewind, and it is proving to be quite stable for the
moment, they are having a hard time breaking it (that's the 9.4
version, though it is intentionally close enough to what is in core,
I'm careful about that).
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 7 March 2016 at 20:55, MauMau <maumau307@gmail.com> wrote:

From: Craig Ringer
--------------------------------------------------
We could help ORMs solve the N+1 SELECTs problem and help them avoid
transferring vast join projections unnecessarily. That'd make PostgreSQL
pretty compelling for exactly the users we're mostly too busy dismissing to
consider.

I'd be interested in reaching out to some Hibernate/JPA and ActiveRecord
folks about how the DB could help the ORM and have been meaning to explore
this area for a while, but -ENOTIME. If anyone pursues it I'll be really
interested in hearing how things go.
--------------------------------------------------

You have various ideas and experience, don't you?
Are those ideas on ORMs beneficial exclusively to PostgreSQL or to all
DBMSs? I don't know the structure of ORMs allows for improvements to be
advantageous to a specific DBMS.

Most ORMs have dialect layers for query generation, DB-specific
customisations, etc.

Whether they're flexible enough to handle this sort of change - I don't
know. That's part of why I'd like to explore the ideas with ActiveRecord,
Hibernate, JPA WG, etc folks.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Hello, Josh,

From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Josh berkus> Crossing this over to pgsql-advocacy list where it really belongs.
That's what that list is *for*.

Especially since the discussion on -hackers has focused on new PostgreSQL
Features, which while also good don't address the general question.

Thank you for pointing me to the correct place. I wondered which list is better, because I thought this topic whould be better discussed among hackers. I'll post subsequent mails only to pgsql-advocacy.

Regards
Takayuki Tsunakawa

--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy

From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Joshua D. Drake
On 03/07/2016 11:31 AM, MauMau wrote:

Why don't we enrich the catalog? I'd like to hear ideas on how to
enrich the catalog efficiently. It's ideal for software vendors and
users to voluntarily add to the catalog.

I think the product/software directory has vastly outlived its purpose.
We are not longer a, "I wonder if Pg works with X". It is more, "Why doesn't
X work with Pg" nowadays.

Yes, I want to take that attitude and encourage software vendors and OSS projects to support PostgreSQL. But in reality, the questions I hear are "Does PG work with X?" Any ideas to convince software vendors to support PostgreSQL?

I could see a page that discusses various tools in general that are supported
but by no means do we need a "list" anymore.

Could you share your image by taking a good example page of some successful commercial or open source products? I'd like to understand the ideal level of information on the ecosystem page.

Anyway, I think some form of web pages are necessary where people who are considering to migrate from other databases can judge whether they can use their existing software assets, and new database users can choose PostgreSQL with relief. For example, relatively new and successful companies like MongoDB and Cloudera have the pages for ecosystem. Cloudera's website is nice because it has the "Partners" link on the home page.

MongoDB Enterprise Certification Program
https://www.mongodb.com/partners/partner-program/technology/certification

Cloudera: Partners
http://www.cloudera.com/partners.html

It needs discussion whether the items on the page should be individual products or just companies/OSS-organizations.

Regards
Takayuki Tsunakawa

--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy

From: pgsql-advocacy-owner@postgresql.org
[mailto:pgsql-advocacy-owner@postgresql.org] On Behalf Of Adrian Klaver

* MySQL touts more than 2,000 ISV/OEM/VARs.
http://www.mysql.com/oem/

Though if you go to the actual list:
http://www.mysql.com/customers/embedded/

There is at a rough count, 300.

This points to the central dilemma, some of which is addressed below,
separating fact from fiction.

It seems the "Embedded" category shows products which use MySQL internally. Other products/vendors, such as JasperSoft and Talend which "can be used with" MySQL, are probably list elsewhere. e.g. Talend lists MySQL in their ecosystem page:

https://jp.talend.com/ecosystem/company/mysql

The 2,000 ISV/OEM/VARs Oracle touts should be found in the "MySQL specialization" section of Oracle Partner Network (OPN) site, though I couldn't access. I wonder if and where I can get the partner/product listing for Oracle Database. That's what I want.

Regards
Takayuki Tsunakawa

--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy