RFC: Remove contrib entirely

JD,

* Joshua D. Drake (jd@commandprompt.com) wrote:

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in
features that are not part of the core PostgreSQL system, mainly
because they address a limited audience or are too experimental to
be part of the main source tree. This does not preclude their
usefulness."

We certainly seem to have moved on from that definition. If nothing
else, it'd be valuable to clarify what contrib is and then rearrange
things accordingly.

It has also been mentioned many times over the years that contrib is
a holding tank for technology that would hopefully be pushed into
core someday.

I continue to feel this is a good use-case for contrib.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A
few of these are pretty obvious:

pg_stat_statements
citext
postgres_fdw
hstore
pg_crypto
[...]

We don't really have a place in "-core" for them.. There has been
ongoing discussion about that but nothing has changed in that regard, as
far as I'm aware. We have moved a few things out of contrib and into
bin, but that doesn't make sense for the above.

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

I am sure there will be plenty of fun to be had with what should or
shouldn't be merged into core. I think if we argue about the
guidelines of how to analyze what should be in core versus the
merits of any particular module, life will be easier. Here are some
for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

I don't particularly like having a time-based requirement drive what's
in which area, especially one that's double the length of our major
release cycle.

2. Push the rest out into a .Org project called contrib. Let those
who are interested in the technology work on them or use them. This
project since it is outside of core proper can work just like other
extension projects. Alternately, allow the maintainers push them
wherever they like (Landscape, Github, Savannah, git.postgresql.org
...).

That's an interesting idea, but it's unclear how this would be any
different from PGXN..?

Thanks!

Stephen

On 05/28/2015 12:35 PM, Stephen Frost wrote:

JD,

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

I am suggesting that things like citext be made a type proper. No
install required. For things like pg_stat_statements of course there
could be configuration required (need to add it to the preload) but it
is automatically installed with the packages.

I am sure there will be plenty of fun to be had with what should or
shouldn't be merged into core. I think if we argue about the
guidelines of how to analyze what should be in core versus the
merits of any particular module, life will be easier. Here are some
for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

I don't particularly like having a time-based requirement drive what's
in which area, especially one that's double the length of our major
release cycle.

I think there is only one or two contrib modules that don't actually fit
requirement A.

2. Push the rest out into a .Org project called contrib. Let those
who are interested in the technology work on them or use them. This
project since it is outside of core proper can work just like other
extension projects. Alternately, allow the maintainers push them
wherever they like (Landscape, Github, Savannah, git.postgresql.org
...).

That's an interesting idea, but it's unclear how this would be any
different from PGXN..?

PGXN is CPAN not Perl or DBD::Pg.

It is actually a compliment to PGXN because it is still needed and
needed more because that is where you are going to get the "latest and
greatest" of the modules.

Sincerely,

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 04:05 PM, Joshua D. Drake wrote:

On 05/28/2015 12:35 PM, Stephen Frost wrote:

JD,

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

I am suggesting that things like citext be made a type proper. No
install required.

This seems to come up regularly. Maybe we should put it in an FAQ
somewhere. The barriers to making non-core types into core types are
very very high, possibly insurmountable. This is pretty much not an option.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 01:11 PM, Andrew Dunstan wrote:

This seems to come up regularly. Maybe we should put it in an FAQ
somewhere. The barriers to making non-core types into core types are
very very high, possibly insurmountable. This is pretty much not an option.

O.k., then either:

* We install it by default and document that it is available (and how
to enable it)

* Push it out of core and let it be happy wherever Theory wants it to be

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 04:22 PM, Joshua D. Drake wrote:

On 05/28/2015 01:11 PM, Andrew Dunstan wrote:

This seems to come up regularly. Maybe we should put it in an FAQ
somewhere. The barriers to making non-core types into core types are
very very high, possibly insurmountable. This is pretty much not an
option.

O.k., then either:

* We install it by default and document that it is available (and how
to enable it)

* Push it out of core and let it be happy wherever Theory wants it to be

I'd be ok with installing it by default.

But the case that's a lot harder to require to be always installed is
pgcrypto, as has often been discussed in the past.

In any case, we will always want to have some loadable modules, not
least because it's a part of eating our own dog food.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 5/28/15 3:35 PM, Stephen Frost wrote:

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 06:25 PM, Andrew Dunstan wrote:

I'd be ok with installing it by default.

But the case that's a lot harder to require to be always installed is
pgcrypto, as has often been discussed in the past.

It used to be but IIRC we don't have those restrictions anymore. If so,
then we need to pull it out and just call it the "Encryption Extension"
or whatever....

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 06:50 PM, Peter Eisentraut wrote:

On 5/28/15 3:35 PM, Stephen Frost wrote:

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

F.1. adminpack
F.2. auth_delay
F.3. auto_explain
F.4. btree_gin
F.5. btree_gist
F.6. chkpass
F.7. citext
F.8. cube
F.9. dblink
F.10. dict_int
F.11. dict_xsyn
F.12. earthdistance
F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore
F.16. intagg
F.17. intarray
F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect
F.22. passwordcheck
F.23. pg_buffercache
F.24. pgcrypto
F.25. pg_freespacemap
F.26. pg_prewarm
F.27. pgrowlocks
F.28. pg_stat_statements
F.29. pgstattuple
F.30. pg_trgm
F.31. postgres_fdw
F.32. seg
F.33. sepgsql
F.34. spi
F.35. sslinfo
F.36. tablefunc
F.37. tcn
F.38. test_decoding
F.39. tsearch2
F.40. tsm_system_rows
F.41. tsm_system_time
F.42. unaccent
F.43. uuid-ossp
F.44. xml2

Look at these, a lot of them are obvious... just include for goodness sakes:

pg_trgm has been in contrib for a decade of not more. Either rip it out
or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I just don't think this would be that hard if we were willing to put our
minds to it.

Or.. another way:

Nobody has yet to provide an argument as to why we need contrib when we
have a fully functioning extension capability.

Ego... is not a good reason.

Of course the other option:

Freeze contrib. What is in there now, is all there will ever be in there
and the goal is to slowly reduce it to the point that it doesn't matter.

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-05-29 AM 11:14, Joshua D. Drake wrote:

pg_trgm has been in contrib for a decade of not more. Either rip it out or
include it by default.

There are jsonb gin operator class related files in src/backend/utils/adt/.
Perhaps, trgm_gin.c, trgm_gist.c, trgm_op.c could be moved there. Similarly
for other gin/gist operators classes - hstore, intarray, ltree maybe. Or each
can have its own directory (including jsonb).

But the original comment was about having these moved to an 'extensions'
directory if at all, so perhaps this suggestion is moot.

Thanks,
Amit

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

JD,

* Joshua D. Drake (jd@commandprompt.com) wrote:

On 05/28/2015 06:50 PM, Peter Eisentraut wrote:

On 5/28/15 3:35 PM, Stephen Frost wrote:

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

Alright, I'll bite. :)

F.1. adminpack

Need it- pgAdmin can't senibly install it or even include it in some
way, and it'd be *very* painful to not have it for a lot of users.

F.2. auth_delay

Should be a core feature. Having this in a contrib module is silly.

F.3. auto_explain

Move to extension directory in the repo.

F.4. btree_gin
F.5. btree_gist

Both of these should simply be in core.

F.6. chkpass
F.7. citext
F.8. cube

Push out and/or keep it in contrib in repo.

F.9. dblink

Move to extension directory in the repo.

F.10. dict_int
F.11. dict_xsyn

Looks like these are just examples? Maybe move to an 'examples'
directory, or into src/test/modules, or keep in contrib.

F.12. earthdistance

Depends on cube, so, same as whatever happens there. I don't think
extensions-in-repo should depend on contrib modules, as a rule.

F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore

Move to extension directory in the repo.

F.16. intagg

Obsolute, per the docs. Push out and deal with the break, or keep it in
contrib in repo.

F.17. intarray

Move to extension directory in the repo.

F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect

Move to extension directory in the repo.

F.22. passwordcheck

Should be an in-core capability and not shoved off into an extension.

F.23. pg_buffercache

Pull it into core.

F.24. pgcrypto

Move to extension directory in the repo.

F.25. pg_freespacemap

Should be in core.

F.26. pg_prewarm
F.27. pgrowlocks

Should be in core.

F.28. pg_stat_statements

I'd actually prefer that this be in core, but I'd be alright with it
being in extension directory in the repo.

F.29. pgstattuple
F.30. pg_trgm

Should be in core.

F.31. postgres_fdw

Move to extension directory in the repo.
(actually, I'd be fine with both this and file_fdw being included in
core.. I'm just not 100% sure how that'd look)

F.32. seg
F.33. sepgsql

Move to extension directory in the repo.

F.34. spi

Maybe pull some into core.. or maybe all, or move to an extension.

F.35. sslinfo

Should be in core.

F.36. tablefunc

My gut reaction is that it should be in core for crosstab(), but David's
talking about implementing PIVOT, so..

F.37. tcn

Should be in core, imv, but not a position I hold very strongly.

F.38. test_decoding

Should be in src/test/modules, or maybe some 'examples' dir.

F.39. tsearch2

I'm inclined to just drop this.. Or we could keep it in contrib in the
repo.

F.40. tsm_system_rows
F.41. tsm_system_time

These should be in core.

F.42. unaccent

Move to extension directory in the repo.

F.43. uuid-ossp

This one probably deserves its own thread, heh..

F.44. xml2

Push it out, or keep it in contrib in repo.

Look at these, a lot of them are obvious... just include for goodness sakes:

pg_trgm has been in contrib for a decade of not more. Either rip it
out or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

Agreed.

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Not sure that being platform agnostic has to be a requirement of being
in the repo or being an extension in the repo... It does need some work
though and discussion has recently started about if the sepgsql types
defined in the SELinux reference policy should continue to exist or if
they should be changed. I'm following that discussion with interest.

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I'd keep it in the repo as an extension. Pushing it out would just
cause lots of trouble for little gain.

I just don't think this would be that hard if we were willing to put
our minds to it.

Or.. another way:

Nobody has yet to provide an argument as to why we need contrib when
we have a fully functioning extension capability.

pg_stat_statements is perhaps one of the best reasons. Not something
that we necessairly want to force on everyone who installs PG
(presumably the additional shared memory and performance impact is an
issue for some people... though I'm not sure I agree, though it's been
a while since that discussion and perhaps the impact looks like less
from a distance), but a capability that we absolutely want to have.

Not sure if we want the FDWs to be installed by default... I had been
thinking there would be an issue with that at first, but the more I
consider it, the more I'm agreeing with you that we should just pull in
both postgres_fdw and file_fdw.

Still, there are extensions which we'll want to have because they simply
can't be included in core for one reason or another, they're very useful
bits of code, and we're willing and eager to maintain them moving
forward.

One of the big questions which will arise out of this though is- how do
we describe contrib vs. this set of extensions? Do we treat security
issues in the 'new' contrib differently? Do we only leave things which
are examples or are deprecated in the 'new' contrib? I'm presuming that
the items which end up in 'extensions' are more formally considered part
of the project and therefore held to the overall project standard (as
contrib is now being, even though it clearly wasn't always that way).

Freeze contrib. What is in there now, is all there will ever be in
there and the goal is to slowly reduce it to the point that it
doesn't matter.

I don't particularly like this option.

Thanks!

Stephen

On 05/28/2015 08:10 PM, Stephen Frost wrote:

JD,

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

Alright, I'll bite. :)

I knew somebody eventually would ;)

F.1. adminpack

Need it- pgAdmin can't senibly install it or even include it in some
way, and it'd be *very* painful to not have it for a lot of users.

Fair enough, although keep in mind we aren't telling people pgAdmin
isn't useful. We are just pushing it out of core. Who runs from source
except developers? Distributions would take care of this for us.

F.2. auth_delay

Should be a core feature. Having this in a contrib module is silly.

+1

F.3. auto_explain

Move to extension directory in the repo.

+1

F.4. btree_gin
F.5. btree_gist

Both of these should simply be in core.

+1

F.6. chkpass
F.7. citext
F.8. cube

Push out and/or keep it in contrib in repo.

Agreed except citext which I think should install by default.

F.9. dblink

Move to extension directory in the repo.

Agreed.

F.10. dict_int
F.11. dict_xsyn

Looks like these are just examples? Maybe move to an 'examples'
directory, or into src/test/modules, or keep in contrib.

Agreed.

F.12. earthdistance

Depends on cube, so, same as whatever happens there. I don't think
extensions-in-repo should depend on contrib modules, as a rule.

F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore

Move to extension directory in the repo.

Disagree, hstore should be in core. Rest, fine.

F.16. intagg

Obsolute, per the docs. Push out and deal with the break, or keep it in
contrib in repo.

Spelling mistake aside ;) agreed

F.17. intarray

Move to extension directory in the repo.

Agreed

F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect

Move to extension directory in the repo.

Except for maybe pageinspect, agreed.

F.22. passwordcheck

Should be an in-core capability and not shoved off into an extension.

Agreed

F.23. pg_buffercache

Pull it into core.

Agreed

F.24. pgcrypto

Move to extension directory in the repo.

Sure.

F.25. pg_freespacemap

Should be in core.

Agreed.

F.26. pg_prewarm
F.27. pgrowlocks

Should be in core.

With a change to pg_rowlocks, agreed.

F.28. pg_stat_statements

I'd actually prefer that this be in core, but I'd be alright with it
being in extension directory in the repo.

Agreed just not enabled by default.

F.29. pgstattuple
F.30. pg_trgm

Should be in core.

Agreed.

F.31. postgres_fdw

Move to extension directory in the repo.
(actually, I'd be fine with both this and file_fdw being included in
core.. I'm just not 100% sure how that'd look)

I think they should be in core, not all FDWs of course but file and
postgres are kind of obvious to me.

F.32. seg
F.33. sepgsql

Move to extension directory in the repo.

Agreed.

F.34. spi

Maybe pull some into core.. or maybe all, or move to an extension.

No opinion.

F.35. sslinfo

Should be in core.

Agreed.

F.36. tablefunc

My gut reaction is that it should be in core for crosstab(), but David's
talking about implementing PIVOT, so..

Easy... give it 1 more release. If we get PIVOT, then we don't need it,
if we don't... all the better for us.

F.37. tcn

Should be in core, imv, but not a position I hold very strongly.

no opinion

F.38. test_decoding

Should be in src/test/modules, or maybe some 'examples' dir.

agreed

F.39. tsearch2

I'm inclined to just drop this.. Or we could keep it in contrib in the
repo.

Release a "final release" as a pgxn capable extension and rip it out.

F.40. tsm_system_rows
F.41. tsm_system_time

These should be in core.

Agreed

F.42. unaccent

Move to extension directory in the repo.

Agreed

F.43. uuid-ossp

This one probably deserves its own thread, heh..

F.44. xml2

Push it out, or keep it in contrib in repo.

Look at these, a lot of them are obvious... just include for goodness sakes:

Agreed.

pg_trgm has been in contrib for a decade of not more. Either rip it
out or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

Agreed.

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Not sure that being platform agnostic has to be a requirement of being
in the repo or being an extension in the repo... It does need some work
though and discussion has recently started about if the sepgsql types
defined in the SELinux reference policy should continue to exist or if
they should be changed. I'm following that discussion with interest.

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I'd keep it in the repo as an extension. Pushing it out would just
cause lots of trouble for little gain.

I just don't think this would be that hard if we were willing to put
our minds to it.

See... we agree on pretty much all of it in principle.

Seriously guys, this particular argument is an argument of "what?". This
is writing on the wall. If Frost and I are in this much agreement... :P

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Le 29 mai 2015 5:33 AM, "Joshua D. Drake" <jd@commandprompt.com> a écrit :

On 05/28/2015 08:10 PM, Stephen Frost wrote:

JD,

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

Alright, I'll bite. :)

I knew somebody eventually would ;)

F.1. adminpack

Need it- pgAdmin can't senibly install it or even include it in some
way, and it'd be *very* painful to not have it for a lot of users.

Painful? The adminpack allows pgadmin to change the config files remotely
with a UI that doesn't make it easy to say the least. You may well trash
your pg_hba.conf file and not be able to connect again after reloading. It
also allows you to read your log files remotely... if it only contains UTF8
characters (which doesn't happen much with my french customers). And
loading a 1GB log file is definitely painful.

I would be of the idea it doesn't give much (though it doesn't mean I want
it to be dropped), but I'm pretty much telling my customers to drop it
whenever I can.

Fair enough, although keep in mind we aren't telling people pgAdmin isn't

useful. We are just pushing it out of core. Who runs from source except
developers? Distributions would take care of this for us.

Yeah. The way I see this is that distributions would make packages for each
extension. And I don't see the difference between doing a

yum install postgresql94-contrib

And a

yum install postgresql94-adminpack

for example, except I would have to run various "yum install" commands to
install every extensions I need, but this is much better for me than
bloating my system with extensions I never use (earthdistance comes to mind
for example).

FWIW, I don't mind which one we put in core and which one we put out of
core. But I like Joshua's idea of getting rid of contribs and pushing them
out as any other extensions.

F.2. auth_delay

Should be a core feature. Having this in a contrib module is silly.

+1

F.3. auto_explain

Move to extension directory in the repo.

+1

F.4. btree_gin
F.5. btree_gist

Both of these should simply be in core.

+1

F.6. chkpass
F.7. citext
F.8. cube

Push out and/or keep it in contrib in repo.

Agreed except citext which I think should install by default.

F.9. dblink

Move to extension directory in the repo.

Agreed.

F.10. dict_int
F.11. dict_xsyn

Looks like these are just examples? Maybe move to an 'examples'
directory, or into src/test/modules, or keep in contrib.

Agreed.

F.12. earthdistance

Depends on cube, so, same as whatever happens there. I don't think
extensions-in-repo should depend on contrib modules, as a rule.

F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore

Move to extension directory in the repo.

Disagree, hstore should be in core. Rest, fine.

F.16. intagg

Obsolute, per the docs. Push out and deal with the break, or keep it in
contrib in repo.

Spelling mistake aside ;) agreed

F.17. intarray

Move to extension directory in the repo.

Agreed

F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect

Move to extension directory in the repo.

Except for maybe pageinspect, agreed.

F.22. passwordcheck

Should be an in-core capability and not shoved off into an extension.

Agreed

F.23. pg_buffercache

Pull it into core.

Agreed

F.24. pgcrypto

Move to extension directory in the repo.

Sure.

F.25. pg_freespacemap

Should be in core.

Agreed.

F.26. pg_prewarm
F.27. pgrowlocks

Should be in core.

With a change to pg_rowlocks, agreed.

F.28. pg_stat_statements

I'd actually prefer that this be in core, but I'd be alright with it
being in extension directory in the repo.

Agreed just not enabled by default.

F.29. pgstattuple
F.30. pg_trgm

Should be in core.

Agreed.

F.31. postgres_fdw

Move to extension directory in the repo.
(actually, I'd be fine with both this and file_fdw being included in
core.. I'm just not 100% sure how that'd look)

I think they should be in core, not all FDWs of course but file and

postgres are kind of obvious to me.

F.32. seg
F.33. sepgsql

Move to extension directory in the repo.

Agreed.

F.34. spi

Maybe pull some into core.. or maybe all, or move to an extension.

No opinion.

F.35. sslinfo

Should be in core.

Agreed.

F.36. tablefunc

My gut reaction is that it should be in core for crosstab(), but David's
talking about implementing PIVOT, so..

Easy... give it 1 more release. If we get PIVOT, then we don't need it,

if we don't... all the better for us.

F.37. tcn

Should be in core, imv, but not a position I hold very strongly.

no opinion

F.38. test_decoding

Should be in src/test/modules, or maybe some 'examples' dir.

agreed

F.39. tsearch2

I'm inclined to just drop this.. Or we could keep it in contrib in the
repo.

Release a "final release" as a pgxn capable extension and rip it out.

F.40. tsm_system_rows
F.41. tsm_system_time

These should be in core.

Agreed

F.42. unaccent

Move to extension directory in the repo.

Agreed

F.43. uuid-ossp

This one probably deserves its own thread, heh..

F.44. xml2

Push it out, or keep it in contrib in repo.

Look at these, a lot of them are obvious... just include for goodness

sakes:

Agreed.

pg_trgm has been in contrib for a decade of not more. Either rip it
out or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

Agreed.

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Not sure that being platform agnostic has to be a requirement of being
in the repo or being an extension in the repo... It does need some work
though and discussion has recently started about if the sepgsql types
defined in the SELinux reference policy should continue to exist or if
they should be changed. I'm following that discussion with interest.

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I'd keep it in the repo as an extension. Pushing it out would just
cause lots of trouble for little gain.

I just don't think this would be that hard if we were willing to put
our minds to it.

See... we agree on pretty much all of it in principle.

Seriously guys, this particular argument is an argument of "what?". This

is writing on the wall. If Frost and I are in this much agreement... :P

Show quoted text

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

FWIW, I don't mind which one we put in core and which one we put out of
core. But I like Joshua's idea of getting rid of contribs and pushing them
out as any other extensions.

Hmmm.

I like the contrib directory as a living example of "how to do an
extension" directly available in the source tree. It also allows to test
in-tree that the extension mechanism works. So I think it should be kept
at least with a minimum set of dummy examples for this purpose, even if
all current extensions are moved out.

Also, removing a feature is a regression, and someone is always bound to
complain... What is the real benefit? ISTM that it is a solution that
fixes no important problem. Reaching a consensus about what to move here
or there will consume valuable time that could be spent on more important
tasks... Is it worth it?

Also moving things into postgresql main sources makes pg heavier for all
and benefits only to some, so I think that some careful filtering must be
done bevore moving large contribs there. I guess this is part of the
argumentation.

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment - It
doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

Regards

Pavel

2015-05-28 18:19 GMT+02:00 Joshua D. Drake <jd@commandprompt.com>:

Show quoted text

Hello,

This is a topic that has come up in various ways over the years. After the
long thread on pg_audit, I thought it might be time to bring it up again.

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in features
that are not part of the core PostgreSQL system, mainly because they
address a limited audience or are too experimental to be part of the main
source tree. This does not preclude their usefulness."

It has also been mentioned many times over the years that contrib is a
holding tank for technology that would hopefully be pushed into core
someday.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A few of
these are pretty obvious:

pg_stat_statements
citext
postgres_fdw
hstore
pg_crypto
[...]

I am sure there will be plenty of fun to be had with what should or
shouldn't be merged into core. I think if we argue about the guidelines of
how to analyze what should be in core versus the merits of any particular
module, life will be easier. Here are some for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

2. Push the rest out into a .Org project called contrib. Let those who are
interested in the technology work on them or use them. This project since
it is outside of core proper can work just like other extension projects.
Alternately, allow the maintainers push them wherever they like (Landscape,
Github, Savannah, git.postgresql.org ...).

Why I am suggesting this:

1. Less code to maintain in core
2. Eliminates the mysticism of contrib
3. Removal of experimental code from core
4. Most of the distributions package contrib separately anyway
5. Some of core is extremely small use case (sepgsql, tsearch2, lo ...)
6. Finding utilities for PostgreSQL used to be harder. It is rather dumb
simple teenage snapchat user easy now.
8. Isn't this what pgxs is for?
9. Everybody hates cleaning the closet until the end result.
10. Several of these modules would make PostgreSQL look good anyway
(default case insensitive index searching with citext? It is a gimme)
11. Contrib has been getting smaller and smaller. Let's cut the cord.
12. Isn't this the whole point of extensions?

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment - It

doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

Regards

Pavel

2015-05-28 18:19 GMT+02:00 Joshua D. Drake <jd@commandprompt.com>:

Hello,

This is a topic that has come up in various ways over the years. After

the long thread on pg_audit, I thought it might be time to bring it up
again.

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in features

that are not part of the core PostgreSQL system, mainly because they
address a limited audience or are too experimental to be part of the main
source tree. This does not preclude their usefulness."

It has also been mentioned many times over the years that contrib is a

holding tank for technology that would hopefully be pushed into core
someday.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A few

of these are pretty obvious:

pg_stat_statements
citext
postgres_fdw
hstore
pg_crypto
[...]

I am sure there will be plenty of fun to be had with what should or

shouldn't be merged into core. I think if we argue about the guidelines of
how to analyze what should be in core versus the merits of any particular
module, life will be easier. Here are some for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

2. Push the rest out into a .Org project called contrib. Let those who

are interested in the technology work on them or use them. This project
since it is outside of core proper can work just like other extension
projects. Alternately, allow the maintainers push them wherever they like
(Landscape, Github, Savannah, git.postgresql.org ...).

Why I am suggesting this:

1. Less code to maintain in core
2. Eliminates the mysticism of contrib
3. Removal of experimental code from core
4. Most of the distributions package contrib separately anyway
5. Some of core is extremely small use case (sepgsql, tsearch2, lo ...)
6. Finding utilities for PostgreSQL used to be harder. It is rather dumb

simple teenage snapchat user easy now.

8. Isn't this what pgxs is for?
9. Everybody hates cleaning the closet until the end result.
10. Several of these modules would make PostgreSQL look good anyway

(default case insensitive index searching with citext? It is a gimme)

Show quoted text

11. Contrib has been getting smaller and smaller. Let's cut the cord.
12. Isn't this the whole point of extensions?

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Le 29 mai 2015 8:01 AM, "Fabien COELHO" <coelho@cri.ensmp.fr> a écrit :

FWIW, I don't mind which one we put in core and which one we put out of
core. But I like Joshua's idea of getting rid of contribs and pushing

them

out as any other extensions.

Hmmm.

I like the contrib directory as a living example of "how to do an

extension" directly available in the source tree. It also allows to test
in-tree that the extension mechanism works. So I think it should be kept at
least with a minimum set of dummy examples for this purpose, even if all
current extensions are moved out.

Agreed.

Also, removing a feature is a regression, and someone is always bound to

complain... What is the real benefit? ISTM that it is a solution that fixes
no important problem. Reaching a consensus about what to move here or there
will consume valuable time that could be spent on more important tasks...
Is it worth it?

It would be less confusing for users. Contrib modules seem to be first
class extensions, leaving doubt on other extensions. But the fact they
aren't in core make them not fully trusted by some users. Trying to explain
all that in a training is a PITA. It would be much less confusing if they
were either in core or in their own repository.

Also moving things into postgresql main sources makes pg heavier for all

and benefits only to some, so I think that some careful filtering must be
done bevore moving large contribs there. I guess this is part of the
argumentation.

Agreed.

2015-05-29 8:20 GMT+02:00 Guillaume Lelarge <guillaume@lelarge.info>:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a
écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment -

It doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

I afraid so dependency on EDB in this case is wrong - I have respect to EDB
due work, but installation other extension from EDB stack is difficult,
unclean, and nothing what I would to use as new base.

Show quoted text

Regards

Pavel

2015-05-28 18:19 GMT+02:00 Joshua D. Drake <jd@commandprompt.com>:

Hello,

This is a topic that has come up in various ways over the years. After

the long thread on pg_audit, I thought it might be time to bring it up
again.

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in features

that are not part of the core PostgreSQL system, mainly because they
address a limited audience or are too experimental to be part of the main
source tree. This does not preclude their usefulness."

It has also been mentioned many times over the years that contrib is a

holding tank for technology that would hopefully be pushed into core
someday.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A few

of these are pretty obvious:

pg_stat_statements
citext
postgres_fdw
hstore
pg_crypto
[...]

I am sure there will be plenty of fun to be had with what should or

shouldn't be merged into core. I think if we argue about the guidelines of
how to analyze what should be in core versus the merits of any particular
module, life will be easier. Here are some for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

2. Push the rest out into a .Org project called contrib. Let those who

are interested in the technology work on them or use them. This project
since it is outside of core proper can work just like other extension
projects. Alternately, allow the maintainers push them wherever they like
(Landscape, Github, Savannah, git.postgresql.org ...).

Why I am suggesting this:

1. Less code to maintain in core
2. Eliminates the mysticism of contrib
3. Removal of experimental code from core
4. Most of the distributions package contrib separately anyway
5. Some of core is extremely small use case (sepgsql, tsearch2, lo ...)
6. Finding utilities for PostgreSQL used to be harder. It is rather

dumb simple teenage snapchat user easy now.

8. Isn't this what pgxs is for?
9. Everybody hates cleaning the closet until the end result.
10. Several of these modules would make PostgreSQL look good anyway

(default case insensitive index searching with citext? It is a gimme)

11. Contrib has been getting smaller and smaller. Let's cut the cord.
12. Isn't this the whole point of extensions?

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 3:20 PM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment - It
doesn't helps with MS Windows. Installing necessary software for compilation
there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

Speaking with my Windows packager hat on, compiling with MinGW would
not be that terrible I think for extensions, as all the existing
Makefile machinery could be used for this purpose. MSVC stuff is more
complicated though with what we have in-core, but still I think that
we could do something with them if we refactor a bit the code and make
easier the declaration of Project objects and have some proper
documentation in the extension chapter, the idea being that users
should not need to build more than a simple build.pl file linking to
some of .pm files of the in-core perl module scripts, basically with a
switch to src/tools/msvc. You would need to have those modules as well
as the compiled deliverables to compile the extensions, but that's the
same deal as any devel-* package on Linux.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

2015-05-29 8:54 GMT+02:00 Michael Paquier <michael.paquier@gmail.com>:

On Fri, May 29, 2015 at 3:20 PM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a

écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment -

It

doesn't helps with MS Windows. Installing necessary software for

compilation

there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules

in

their windows installer to help users. They already go way further than

any

Linux packages.

Speaking with my Windows packager hat on, compiling with MinGW would
not be that terrible I think for extensions, as all the existing
Makefile machinery could be used for this purpose. MSVC stuff is more
complicated though with what we have in-core, but still I think that
we could do something with them if we refactor a bit the code and make
easier the declaration of Project objects and have some proper
documentation in the extension chapter, the idea being that users
should not need to build more than a simple build.pl file linking to
some of .pm files of the in-core perl module scripts, basically with a
switch to src/tools/msvc. You would need to have those modules as well
as the compiled deliverables to compile the extensions, but that's the
same deal as any devel-* package on Linux.

I had a lot of problems with MSVC installation

Issues:

1. VS requires relatively new MS Windows - problem for people with Ms Win 7
and older

2. After installation you have to find and apply some critical fixes - some
is bad documented.

I have a few customers on MS Win - nobody was able to compile extension
there - and I had lot of trables - (but I worked with MSVC ten years ago)

Regards

Pavel

Show quoted text

--
Michael

On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote:

1. VS requires relatively new MS Windows - problem for people with Ms Win 7
and older

Really, I use Win 2k8 stuff and Win7 quite a lot.

2. After installation you have to find and apply some critical fixes - some
is bad documented.

Example? Perhaps we could improve the docs then.

I have a few customers on MS Win - nobody was able to compile extension
there - and I had lot of trables - (but I worked with MSVC ten years ago)

Well, FWIW, I have some extensions compiled on Windows using MSVC
modules with stuff like perl -I src/tools/msvc build.pl that are out
of the vanilla scripts and it works... The script is rather simple
even if a bit grotty.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

2015-05-29 9:42 GMT+02:00 Michael Paquier <michael.paquier@gmail.com>:

On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote:

1. VS requires relatively new MS Windows - problem for people with Ms

Win 7

and older

Really, I use Win 2k8 stuff and Win7 quite a lot.

On Win 7 you have to search and install now unsupported VS EE 2010.

VS Express 2013 requires Win 8. And finding VS 2010 is not simply - I had
to take install files from some private sources.

2. After installation you have to find and apply some critical fixes -

some

is bad documented.

Example? Perhaps we could improve the docs then.

It was a VS 2010 related issues - not PostgreSQL issues

I have a few customers on MS Win - nobody was able to compile extension
there - and I had lot of trables - (but I worked with MSVC ten years ago)

Well, FWIW, I have some extensions compiled on Windows using MSVC
modules with stuff like perl -I src/tools/msvc build.pl that are out
of the vanilla scripts and it works... The script is rather simple
even if a bit grotty.

see steps from
http://dba.stackexchange.com/questions/68644/problems-while-building-plpgsql-check-on-windows

It is not hard for Windows C++ developer, It is little bit messy for Linux
developer, and it is out of possibility usual Windows Visual Basic (and
similar level) developers.

Show quoted text

--
Michael

On Fri, May 29, 2015 at 7:20 AM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment - It
doesn't helps with MS Windows. Installing necessary software for compilation
there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

The only extra extension we ship is pl/debugger.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:

2015-05-29 8:20 GMT+02:00 Guillaume Lelarge <guillaume@lelarge.info>:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit
:

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment -
It doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

I afraid so dependency on EDB in this case is wrong - I have respect to EDB
due work, but installation other extension from EDB stack is difficult,
unclean, and nothing what I would to use as new base.

The five or six mouse clicks required to install something like Slony
or PostGIS (or both at once) is difficult and unclean?

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 8:50 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:

2015-05-29 9:42 GMT+02:00 Michael Paquier <michael.paquier@gmail.com>:

On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote:

1. VS requires relatively new MS Windows - problem for people with Ms
Win 7
and older

Really, I use Win 2k8 stuff and Win7 quite a lot.

On Win 7 you have to search and install now unsupported VS EE 2010.

I've been running 2013 on Windows 7 since it came out. Works perfectly
well, and didn't require any unusual installation.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 9:35 AM, Dave Page <dpage@pgadmin.org> wrote:

On Fri, May 29, 2015 at 7:20 AM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment - It
doesn't helps with MS Windows. Installing necessary software for compilation
there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

The only extra extension we ship is pl/debugger.

To clarify - that's the only one we ship in the PostgreSQL installer.
There are many more that we ship (both from EDB and other sources),
but users use StackBuilder to choose, download and install them.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

2015-05-29 10:40 GMT+02:00 Dave Page <dpage@pgadmin.org>:

On Fri, May 29, 2015 at 8:50 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2015-05-29 9:42 GMT+02:00 Michael Paquier <michael.paquier@gmail.com>:

On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote:

1. VS requires relatively new MS Windows - problem for people with Ms
Win 7
and older

Really, I use Win 2k8 stuff and Win7 quite a lot.

On Win 7 you have to search and install now unsupported VS EE 2010.

I've been running 2013 on Windows 7 since it came out. Works perfectly
well, and didn't require any unusual installation.

I had different experience - When I did it year ago, I had to searching and
install VS 2010.

Pavel

Show quoted text

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

2015-05-29 10:37 GMT+02:00 Dave Page <dpage@pgadmin.org>:

On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2015-05-29 8:20 GMT+02:00 Guillaume Lelarge <guillaume@lelarge.info>:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a

écrit

:

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment -
It doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules

in

their windows installer to help users. They already go way further than

any

Linux packages.

I afraid so dependency on EDB in this case is wrong - I have respect to

EDB

due work, but installation other extension from EDB stack is difficult,
unclean, and nothing what I would to use as new base.

The five or six mouse clicks required to install something like Slony
or PostGIS (or both at once) is difficult and unclean?

I had a problem with downloading isolated packages with stackbuilder - but
it was specific, because comp where some packages was installed was not
access to internet.

Show quoted text

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, May 29, 2015 at 5:52 PM, Pavel Stehule <pavel.stehule@gmail.com> wrote:

2015-05-29 10:40 GMT+02:00 Dave Page <dpage@pgadmin.org>:

On Fri, May 29, 2015 at 8:50 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2015-05-29 9:42 GMT+02:00 Michael Paquier <michael.paquier@gmail.com>:

On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote:

1. VS requires relatively new MS Windows - problem for people with Ms
Win 7
and older

Really, I use Win 2k8 stuff and Win7 quite a lot.

On Win 7 you have to search and install now unsupported VS EE 2010.

I've been running 2013 on Windows 7 since it came out. Works perfectly
well, and didn't require any unusual installation.

I had different experience - When I did it year ago, I had to searching and
install VS 2010.

And it is not mandatory to use or install visual basic, a simple
Windows SDK with perl installed is enough to build and install PG from
a command prompt.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 9:55 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:

2015-05-29 10:37 GMT+02:00 Dave Page <dpage@pgadmin.org>:

On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2015-05-29 8:20 GMT+02:00 Guillaume Lelarge <guillaume@lelarge.info>:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a
écrit
:

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment
-
It doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

I agree it's hard to compile an extension on Windows, but that's
already
what we have. And I'm sure EDB will put all interesting contrib modules
in
their windows installer to help users. They already go way further than
any
Linux packages.

I afraid so dependency on EDB in this case is wrong - I have respect to
EDB
due work, but installation other extension from EDB stack is difficult,
unclean, and nothing what I would to use as new base.

The five or six mouse clicks required to install something like Slony
or PostGIS (or both at once) is difficult and unclean?

I had a problem with downloading isolated packages with stackbuilder - but
it was specific, because comp where some packages was installed was not
access to internet.

Run SB on a machine that is connected and:

- Select "remote server" on the first step. This will disable any
filtering based on database server version, which is normally applied
if you're installing to the local machine (it will only offer packages
compatible with the database servers you have installed). This is not
required if you have the appropriate DB server installed on the
internet connected machine as well.

- Make a note of the Download Directory on step 3

- Check "Skip installation" on step 4.

Copy the downloaded installed from the Download Directory to your
server machine, and run them.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

2015-05-29 11:02 GMT+02:00 Dave Page <dpage@pgadmin.org>:

On Fri, May 29, 2015 at 9:55 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2015-05-29 10:37 GMT+02:00 Dave Page <dpage@pgadmin.org>:

On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule <pavel.stehule@gmail.com

wrote:

2015-05-29 8:20 GMT+02:00 Guillaume Lelarge <guillaume@lelarge.info>:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a
écrit
:

Hi

I am not sure if PGXN can substitute contrib - mainly due

deployment

-
It doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

I agree it's hard to compile an extension on Windows, but that's
already
what we have. And I'm sure EDB will put all interesting contrib

modules

in
their windows installer to help users. They already go way further

than

any
Linux packages.

I afraid so dependency on EDB in this case is wrong - I have respect

to

EDB
due work, but installation other extension from EDB stack is

difficult,

unclean, and nothing what I would to use as new base.

The five or six mouse clicks required to install something like Slony
or PostGIS (or both at once) is difficult and unclean?

I had a problem with downloading isolated packages with stackbuilder -

but

it was specific, because comp where some packages was installed was not
access to internet.

Run SB on a machine that is connected and:

- Select "remote server" on the first step. This will disable any
filtering based on database server version, which is normally applied
if you're installing to the local machine (it will only offer packages
compatible with the database servers you have installed). This is not
required if you have the appropriate DB server installed on the
internet connected machine as well.

- Make a note of the Download Directory on step 3

- Check "Skip installation" on step 4.

Copy the downloaded installed from the Download Directory to your
server machine, and run them.

Dave, I didn't say so it is difficult task - I solved it relatively simply
- but my customer - long year Oracle DBA, Postgres beginner was messy and
needed help and access to EDB support. He expected and searched a web page
with download, what is much more usual.

Regards

Pavel

Show quoted text

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 05/28/2015 11:01 PM, Fabien COELHO wrote:

Also, removing a feature is a regression, and someone is always bound to
complain...

We aren't removing any features. These are all items that are NOT
installed or functional by default.

Sincerely,

JD

--
The most kicking donkey PostgreSQL Infrastructure company in existence.
The oldest, the most experienced, the consulting company to the stars.
Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 -
24x7 - 365 - Proactive and Managed Professional Services!

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 11:08 PM, Pavel Stehule wrote:

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment -
It doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

Anyone who is building for Windows won't have that problem. They already
have the environment setup. Windows users for the most part are going to
download our stackbuilder distro and go from there. They are not going
to compile modules. What I am suggesting doesn't stop that.

JD

--
The most kicking donkey PostgreSQL Infrastructure company in existence.
The oldest, the most experienced, the consulting company to the stars.
Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 -
24x7 - 365 - Proactive and Managed Professional Services!

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

* Joshua D. Drake (jd@commandprompt.com) wrote:

On 05/28/2015 11:01 PM, Fabien COELHO wrote:

Also, removing a feature is a regression, and someone is always bound to
complain...

We aren't removing any features. These are all items that are NOT
installed or functional by default.

Uh, we've already had reports by Debian users about problems upgrading
when they forgot to install the contrib package for the version they're
moving to, so I'm not sure it's quite that simple. Or, at least, it
won't be for the packagers, who I do believe this would be creating a
fair bit of work for.

That work will be much less if we simply split what's in contrib now
into extension and contrib directories, as it's still all one source
repo to the packagers. If we punt things out (unless they're being
formally deprecated/removed) to another repo somewhere, then the
packagers are going to have to deal with new source repos and related
complexity. I'm not saying that's a horrible thing and it might make
sense in some cases, but generally it's a lot easier to go from one
soruce package to a bunch of binary ones than from lots of tiny source
packages to lots of tiny packages.

The versioning aspect of this does come into play though, as having
everything with one relatively slow versioning cycle (on the order of
months) is actually keeping the load on the packagers down. Lots of
little releases, all at different times, from lots of different source
packages, would increase the workload.

I'm not sure where I ultimately come down on the question about in-repo
vs. out-of-repo. My gut feeling is that if the community is willing to
continue maintaining contrib modules, then that's ultimately a good
thing and many of them are relatively low-maintenance anyway. Having a
high barrier to entry for new modules looks a bit odd, given the
definition of contrib, but would be more understandable with a proper
'extensions' directory. Of course, we'd have to collectivly agree that
we feel comfortable with a lower barrier for contrib that what is being
done now, if the distinction is going to have any meaning.

Thanks!

Stephen

On Thu, May 28, 2015 at 11:01 PM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

FWIW, I don't mind which one we put in core and which one we put out of

core. But I like Joshua's idea of getting rid of contribs and pushing them
out as any other extensions.

Hmmm.

I like the contrib directory as a living example of "how to do an
extension" directly available in the source tree. It also allows to test
in-tree that the extension mechanism works. So I think it should be kept at
least with a minimum set of dummy examples for this purpose, even if all
current extensions are moved out.

It is mostly an example of "How to do an contrib module" rather than "how
to do an extension". There are differences between those things regarding
the the USE_PGXS and some other things. If we want to keep it as an
example of what we want people to do in the future, it needs be a really
good example. And if we want to step new things from going into contrib,
we wouldn't want to provide an example of how to put new things into it.

Also, removing a feature is a regression, and someone is always bound to
complain... What is the real benefit? ISTM that it is a solution that fixes
no important problem. Reaching a consensus about what to move here or there
will consume valuable time that could be spent on more important tasks...
Is it worth it?

Yeah, I don't really see the benefit either. Some could be moved to core,
and some could just be removed, but many of them it just seems like we
would end up inventing a new 'contrib' to which is the same as the old, but
with a different name.

Cheers,

Jeff

On Thu, May 28, 2015 at 11:26 PM, Guillaume Lelarge <guillaume@lelarge.info>
wrote:

Le 29 mai 2015 8:01 AM, "Fabien COELHO" <coelho@cri.ensmp.fr> a écrit :

FWIW, I don't mind which one we put in core and which one we put out of
core. But I like Joshua's idea of getting rid of contribs and pushing

them

out as any other extensions.

Hmmm.

I like the contrib directory as a living example of "how to do an

extension" directly available in the source tree. It also allows to test
in-tree that the extension mechanism works. So I think it should be kept at
least with a minimum set of dummy examples for this purpose, even if all
current extensions are moved out.

Agreed.

Also, removing a feature is a regression, and someone is always bound to

complain... What is the real benefit? ISTM that it is a solution that fixes
no important problem. Reaching a consensus about what to move here or there
will consume valuable time that could be spent on more important tasks...
Is it worth it?

It would be less confusing for users. Contrib modules seem to be first
class extensions, leaving doubt on other extensions.

Presumably there are still going to be some extensions maintained by
-hackers, and some not. I don't think we are going to change that, so the
difference will still need to be explained, regardless of what words are
used. And people *should* have doubts about other extensions. Couldn't
any talented programmer write an extension which gives them a backdoor into
the deployer's system, and then upload it to github?

I would certainly be cautious about installing any old extension I found
some some place on the internet.

But the fact they aren't in core make them not fully trusted by some
users.

Would it help if we called it "base" or "minimal" rather than "core" in the
docs? (And called 'contrib' something different as well? The docs already
do call it "Additional Supplied Modules" and use "contrib" only when
referring the the directory, not the concept.)

Trying to explain all that in a training is a PITA. It would be much less
confusing if they were either in core or in their own repository.

Several of the contrib modules should be kept in tight sync with src or
else testing and debugging would be a disaster. Putting them in different
git repositories wouldn't work well. Unless those would among the ones
moved to "core".

Cheers,

Jeff

All,

So there are currently three kinds of things in contrib:

A. Extra commands and tools which aren't considered general enough, or
reliable enough, to be included by default, e.g. pg_standby, pgbench and
vacuumlo.

B. Developer tools, like spi, start-scripts, and oid2name.

C. "Core Extensions", which fall into three further groups:
C1: encryption extensions we can't include in core
for legal reasons (pg_crypto)
C2: example extensions which show useful things about
how to build an extension
C3: Admin extensions which are not core because they carry
risks (e.g. pgstattuple, auto_explain)
C4: Extensions which are generally useful, used, and
maintained with Postgres (e.g. hstore, citext)

So A and B need to stay somewhere in the source distribution. I would
like to see them go into /admin-tools and /developer-tools directories;
I believe that Greg Smith came up with a patch to do just this at
sometime in the past. C2 could also go into /developer-tools, and C3
are really just more admin-tools. C1 would need to stick around as a
special case.

To move C4, you'd have to solve the problem of "how do we turn a former
external extension into a core feature", which nobody yet has solved.

All of this ignores the critical part of this, which is packaging.
Right now packagers ship a "contrib" package which includes everything
in /contrib. Shifting to having any other arrangement is going to
involve working with them and convincing them that a change to packaging
is worthwhile. And then getting the news to our users.

Given that, there needs to be significant benefit to our users in the
change. So, what's the benefit?

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/29/2015 11:02 AM, Jeff Janes wrote:

Also, removing a feature is a regression, and someone is always
bound to complain... What is the real benefit? ISTM that it is a
solution that fixes no important problem. Reaching a consensus about
what to move here or there will consume valuable time that could be
spent on more important tasks... Is it worth it?

Yeah, I don't really see the benefit either. Some could be moved to
core, and some could just be removed, but many of them it just seems
like we would end up inventing a new 'contrib' to which is the same as
the old, but with a different name.

Name one.

Sincerely,

JD
--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/29/2015 11:27 AM, Jeff Janes wrote:

It would be less confusing for users. Contrib modules seem to be
first class extensions, leaving doubt on other extensions.

Presumably there are still going to be some extensions maintained by
-hackers, and some not. I don't think we are going to change that, so
the difference will still need to be explained, regardless of what words
are used. And people *should* have doubts about other extensions.
Couldn't any talented programmer write an extension which gives them a
backdoor into the deployer's system, and then upload it to github?

Yes, it is called Open Source development.

I would certainly be cautious about installing any old extension I found
some some place on the internet.

But the fact they aren't in core make them not fully trusted by some
users.

No. This is completely wrong thinking. If you are compiling this stuff
from source you are taking that risk on yourself.

Most people are not compiling from source, they are installing from a
distribution (or downloading a distribution package).

Trying to explain all that in a training is a PITA. It would be much
less confusing if they were either in core or in their own repository.

Several of the contrib modules should be kept in tight sync with src or
else testing and debugging would be a disaster. Putting them in
different git repositories wouldn't work well. Unless those would among
the ones moved to "core".

Note: I actually don't care if the current contrib gets pushed into core
proper and is default installed.

I care about this idea that contrib exists. It isn't needed and leads to
a discussion like this one (or the pg_audit), almost every release.

Contrib made sense years ago. It does not any longer. Let's put the old
horse down and raise a new herd of ponies on a new pasture.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

2015-05-29 21:20 GMT+02:00 Joshua D. Drake <jd@commandprompt.com>:

On 05/29/2015 11:27 AM, Jeff Janes wrote:

It would be less confusing for users. Contrib modules seem to be

first class extensions, leaving doubt on other extensions.

Presumably there are still going to be some extensions maintained by
-hackers, and some not. I don't think we are going to change that, so
the difference will still need to be explained, regardless of what words
are used. And people *should* have doubts about other extensions.
Couldn't any talented programmer write an extension which gives them a
backdoor into the deployer's system, and then upload it to github?

Yes, it is called Open Source development.

I would certainly be cautious about installing any old extension I found
some some place on the internet.

But the fact they aren't in core make them not fully trusted by some
users.

No. This is completely wrong thinking. If you are compiling this stuff
from source you are taking that risk on yourself.

Most people are not compiling from source, they are installing from a
distribution (or downloading a distribution package).

Trying to explain all that in a training is a PITA. It would be much
less confusing if they were either in core or in their own repository.

Several of the contrib modules should be kept in tight sync with src or
else testing and debugging would be a disaster. Putting them in
different git repositories wouldn't work well. Unless those would among
the ones moved to "core".

Note: I actually don't care if the current contrib gets pushed into core
proper and is default installed.

I care about this idea that contrib exists. It isn't needed and leads to a
discussion like this one (or the pg_audit), almost every release.

Contrib made sense years ago. It does not any longer. Let's put the old
horse down and raise a new herd of ponies on a new pasture.

Still there is strong sense - it is a referential implementation of our
extension API. We need it to find regressions, changes. I don't believe so
external extensions can do it. Only PostGIS is massively accepted and
developed by more than few people. Personally I am thinking so removing
contrib is not good idea.

Pavel

Show quoted text

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/29/2015 12:30 PM, Pavel Stehule wrote:

Contrib made sense years ago. It does not any longer. Let's put the
old horse down and raise a new herd of ponies on a new pasture.

Still there is strong sense - it is a referential implementation of our
extension API. We need it to find regressions, changes. I don't believe

No, then we need a proper test suite for the extension API.

so external extensions can do it. Only PostGIS is massively accepted and
developed by more than few people. Personally I am thinking so removing
contrib is not good idea.

Is there an extension/contrib module in the last decade that more than
once has shown to help us with that?

Sincerely,

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

2015-05-29 21:59 GMT+02:00 Joshua D. Drake <jd@commandprompt.com>:

On 05/29/2015 12:30 PM, Pavel Stehule wrote:

Contrib made sense years ago. It does not any longer. Let's put the

old horse down and raise a new herd of ponies on a new pasture.

Still there is strong sense - it is a referential implementation of our
extension API. We need it to find regressions, changes. I don't believe

No, then we need a proper test suite for the extension API.

maybe partially, but it is.

so external extensions can do it. Only PostGIS is massively accepted and

developed by more than few people. Personally I am thinking so removing
contrib is not good idea.

Is there an extension/contrib module in the last decade that more than
once has shown to help us with that?

What I know - 9.5 transformations for testing on more platforms.

It is hard to calculate how often the code from contrib helps - but any
feature last four years has not to break contrib test too, so I believe it
enforce better API stability.

It is hard to imagine to design and maintaining any extension API without
platform like contrib. It can be renamed, divided, but some like contrib
must exists in core code base if PostgreSQL should be extensible database.

Show quoted text

Sincerely,

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Fri, May 29, 2015 at 11:47 AM, Josh Berkus <josh@agliodbs.com> wrote:

A. Extra commands and tools which aren't considered general enough, or
reliable enough, to be included by default, e.g. pg_standby, pgbench and
vacuumlo.

B. Developer tools, like spi, start-scripts, and oid2name.

C. "Core Extensions", which fall into three further groups:
C1: encryption extensions we can't include in core
for legal reasons (pg_crypto)
C2: example extensions which show useful things about
how to build an extension
C3: Admin extensions which are not core because they carry
risks (e.g. pgstattuple, auto_explain)
C4: Extensions which are generally useful, used, and
maintained with Postgres (e.g. hstore, citext)

I always liked the idea of organizing contrib along these lines.

I know that I will never be successful in convincing people to remove,
say, contrib/isn, which is total garbage, but the next best thing is
to categorize it in a way that sets expectations very low.

--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/29/2015 02:08 PM, Peter Geoghegan wrote:

On Fri, May 29, 2015 at 11:47 AM, Josh Berkus <josh@agliodbs.com> wrote:

A. Extra commands and tools which aren't considered general enough, or
reliable enough, to be included by default, e.g. pg_standby, pgbench and
vacuumlo.

B. Developer tools, like spi, start-scripts, and oid2name.

C. "Core Extensions", which fall into three further groups:
C1: encryption extensions we can't include in core
for legal reasons (pg_crypto)
C2: example extensions which show useful things about
how to build an extension
C3: Admin extensions which are not core because they carry
risks (e.g. pgstattuple, auto_explain)
C4: Extensions which are generally useful, used, and
maintained with Postgres (e.g. hstore, citext)

I always liked the idea of organizing contrib along these lines.

I know that I will never be successful in convincing people to remove,
say, contrib/isn, which is total garbage, but the next best thing is
to categorize it in a way that sets expectations very low.

Well, contrib/isn is still useful (I use it). But there's no good
reason it couldn't be on pgxn.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Josh Berkus <josh@agliodbs.com> writes:

On 05/29/2015 02:08 PM, Peter Geoghegan wrote:

I always liked the idea of organizing contrib along these lines.

I know that I will never be successful in convincing people to remove,
say, contrib/isn, which is total garbage, but the next best thing is
to categorize it in a way that sets expectations very low.

Well, contrib/isn is still useful (I use it). But there's no good
reason it couldn't be on pgxn.

We already did one round of removal of low-grade contrib items.
Admittedly that was in 2006, and maybe some of the stuff that survived
that cut no longer looks good enough. But I don't think there's all
that much low-hanging fruit there.

But let's get to the point: the real reason for keeping most of these
contrib modules in the core distribution is that they are essential test
cases for core's extensibility features. contrib/isn is actually a good
example of that. It made us realize that extensions that create types
that are physically equivalent to int8 or float8 were broken when we made
those types potentially pass-by-value; we had to add a CREATE TYPE option
to allow that to still work (cf commit 3f936aacc057e4b3). If contrib/isn
had not been around and been getting built by the buildfarm, we would have
found that out only much later and with much more pain.

You could imagine some other way to address that, like generalizing the
buildfarm so that it can pull in extensions from other source repos for
testing purposes. But that's going to be a lot of work and I'm not even
real sure we want it --- it'd increase the trust problem for buildfarm
owners quite a bit, for one thing.

I'm not particularly on board with renaming things just to get rid of the
term "contrib". We have much better things to do with our time.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 2:35 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

It made us realize that extensions that create types
that are physically equivalent to int8 or float8 were broken when we made
those types potentially pass-by-value; we had to add a CREATE TYPE option
to allow that to still work (cf commit 3f936aacc057e4b3). If contrib/isn
had not been around and been getting built by the buildfarm, we would have
found that out only much later and with much more pain.

Interesting.

FWIW, my concerns with contrib/isn are limited to the ISBN type and
related types. These types enforce that ISBNs are within certain
ranges known by the module to be valid. The first patch I reviewed for
Postgres back in 2010 extended this range, and I first raised the
issue then -- how many such patches can we expect in the future?

The problem here is that these ranges are controlled by a
decentralized patchwork of national standards bodies, and the ranges
are always subject to revision. I think that it's egregious that
contrib/isn imagines it can track that with a static array.

Since contrib is a place that example code is supposed to live,
perhaps contrib/isn could be held up as an example of what not to
do...
--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Peter Geoghegan <pg@heroku.com> writes:

The problem here is that these ranges are controlled by a
decentralized patchwork of national standards bodies, and the ranges
are always subject to revision. I think that it's egregious that
contrib/isn imagines it can track that with a static array.

Well, that module has already been rewritten once (which proves that
there's an audience out there for it). Perhaps somebody will rewrite it
again to support a non-hardwired set of ranges. Now that we have the
concept of an extension configuration table, that'd be one possible
way to fix it ...

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 2:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Well, that module has already been rewritten once (which proves that
there's an audience out there for it). Perhaps somebody will rewrite it
again to support a non-hardwired set of ranges. Now that we have the
concept of an extension configuration table, that'd be one possible
way to fix it ...

I wouldn't bother.

ISBNs already have a UPC-style weighted sum check digit that will
catch the vast majority of errors, including all transposition errors.
You'd have to try hard to fatfinger an ISBN in a way that produced
something that accidentally had a valid check digit.

contrib/isn suffers from a bad case of protecting against Machiavelli
rather than Murphy. The enforcement isn't just obviously wrong, it's
also ridiculous in principle.

You're right, though -- we have better things to worry about.
--
Peter Geoghegan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/29/2015 02:54 PM, Tom Lane wrote:

Peter Geoghegan <pg@heroku.com> writes:

The problem here is that these ranges are controlled by a
decentralized patchwork of national standards bodies, and the ranges
are always subject to revision. I think that it's egregious that
contrib/isn imagines it can track that with a static array.

Well, that module has already been rewritten once (which proves that
there's an audience out there for it). Perhaps somebody will rewrite it
again to support a non-hardwired set of ranges. Now that we have the
concept of an extension configuration table, that'd be one possible
way to fix it ...

FWIW, neither of the projects I know of which uses ISBN has had any
issues with the range since the 2010 updates. While the ranges can be
updated in theory, in practice is happens glacially slowly.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 5/29/15 5:35 PM, Tom Lane wrote:

But let's get to the point: the real reason for keeping most of these
contrib modules in the core distribution is that they are essential test
cases for core's extensibility features. contrib/isn is actually a good
example of that. It made us realize that extensions that create types
that are physically equivalent to int8 or float8 were broken when we made
those types potentially pass-by-value; we had to add a CREATE TYPE option
to allow that to still work (cf commit 3f936aacc057e4b3). If contrib/isn
had not been around and been getting built by the buildfarm, we would have
found that out only much later and with much more pain.

But the generalization of that argument is that we need to keep around
all possible code forever because it might teach us something in the future.

But it is a valid point that we'd need to build up more extension API
tests before we start cutting back significantly on the
maybe-example-maybe-real extensions that we ship in contrib. We need to
find a good middle ground.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Peter Eisentraut wrote:

But it is a valid point that we'd need to build up more extension API
tests before we start cutting back significantly on the
maybe-example-maybe-real extensions that we ship in contrib. We need to
find a good middle ground.

Nowadays we can test concurrent behavior with isolationtester, and we
can test APIs with src/test/modules. We can now easily add tests for
lots of stuff that we don't currently test, without having modules that
actually produce for-install programs or libraries.

In any case, +1 on moving useful extensions to src/extensions/. I just
read somewhere (not this thread?) that Debian for their 9.5 packaging is
going to get rid of the postgresql-contrib package, and instead they are
going to ship all that stuff in the main package. Seems they were
prompted to do this because of our inaction in this area ...

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Mon, Jun 01, 2015 at 05:48:16PM -0300, Alvaro Herrera wrote:

Peter Eisentraut wrote:

But it is a valid point that we'd need to build up more extension
API tests before we start cutting back significantly on the
maybe-example-maybe-real extensions that we ship in contrib. We
need to find a good middle ground.

Nowadays we can test concurrent behavior with isolationtester, and
we can test APIs with src/test/modules. We can now easily add tests
for lots of stuff that we don't currently test, without having
modules that actually produce for-install programs or libraries.

In any case, +1 on moving useful extensions to src/extensions/. I
just read somewhere (not this thread?) that Debian for their 9.5
packaging is going to get rid of the postgresql-contrib package, and
instead they are going to ship all that stuff in the main package.
Seems they were prompted to do this because of our inaction in this
area ...

I suspect they may also have realized that anything not shipped in the
main distribution can cause enough bureaucratic pain, as in "write a
justification for the legal department for why we should install this
software," to cause things simply never to get installed.

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 10:30 AM, Stephen Frost <sfrost@snowman.net> wrote:

That work will be much less if we simply split what's in contrib now
into extension and contrib directories, as it's still all one source
repo to the packagers. If we punt things out (unless they're being
formally deprecated/removed) to another repo somewhere, then the
packagers are going to have to deal with new source repos and related
complexity. I'm not saying that's a horrible thing and it might make
sense in some cases, but generally it's a lot easier to go from one
soruce package to a bunch of binary ones than from lots of tiny source
packages to lots of tiny packages.

The versioning aspect of this does come into play though, as having
everything with one relatively slow versioning cycle (on the order of
months) is actually keeping the load on the packagers down. Lots of
little releases, all at different times, from lots of different source
packages, would increase the workload.

These are all good points.

I'm not sure where I ultimately come down on the question about in-repo
vs. out-of-repo. My gut feeling is that if the community is willing to
continue maintaining contrib modules, then that's ultimately a good
thing and many of them are relatively low-maintenance anyway. Having a
high barrier to entry for new modules looks a bit odd, given the
definition of contrib, but would be more understandable with a proper
'extensions' directory. Of course, we'd have to collectivly agree that
we feel comfortable with a lower barrier for contrib that what is being
done now, if the distinction is going to have any meaning.

I don't agree. It may make sense to keep some stuff in contrib that
is not great code or not really that useful just for historical
reasons. But if we accept every extension with the quality of, say,
contrib/isn into the core distribution, or core distribution is going
to be huge and bloated with crap.

I'd like to point out that we've already done some significant cleanup
of contrib - some things got moved to src/test/modules, and other
stuff to src/bin. So what's in that directory is already a lot more
homogenous than it was a few years ago. Pretty much everything in
there is a loadable module, and if it does something SQL-visible, it's
packaged as an extension. The only exceptions are oid2name,
pg_standby, start-scripts, and vacuumlo. For a directory with >60
things in it, that's pretty good.

I don't think it's very practical to talk about getting rid of contrib
when we reliably add to it in every release:

9.1 added auth_delay, dummy_seclabel, file_fdw, pg_test_fsync, and sepgsql
9.2 added pg_test_timing and tcn
9.3 added pg_xlogdump, postgres_fdw, and worker_spi
9.4 added pg_prewarm, test_decoding, and test_shm_mq
9.5 added hstore_plperl, hstore_plpython, ltree_plpython,
tsm_system_rows, and tsm_system_time

I haven't really gotten familiar with the 9.5 stuff, but most of that
earlier stuff was, uh, pretty good stuff. We wouldn't have been
better off rejecting it, and we wouldn't have been better off putting
it into the main tree.

Any individual person who looks at contrib will no doubt see a fairly
high percentage of stuff they don't care about. Given that part of
the charter of contrib is to hold things that we don't want to put in
the core product for one reason or another, that is to be expected.
But every single one of those 18 contrib modules we've added in the
last 5 years was something that someone cared deeply about, many of
them get real use, and we're just sticking our head in the sand if we
think that's not going to keep happening.

For what it's worth, I also don't particularly support renaming
contrib. I don't really see that it buys us enough to justify the
hassle it will cause. One thing that may be worth doing yet is
separating the code that is just intended as a POC (like worker_spi,
auth_delay and test_decoding) from the stuff that you are really
intended to run in production (like tcn and hstore). But that
distinction is fuzzier than you might think, because while auth_delay
was intended as a POC, I've subsequently heard rumors of it being used
in production with satisfactory results. It's very easy to get the
idea that you know "what PostgreSQL users use" but usually that tends
to mean "what I use" and the community is broad enough that those
things are Not The Same.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/04/2015 10:34 AM, Robert Haas wrote:

For what it's worth, I also don't particularly support renaming
contrib. I don't really see that it buys us enough to justify the
hassle it will cause. One thing that may be worth doing yet is
separating the code that is just intended as a POC (like worker_spi,
auth_delay and test_decoding) from the stuff that you are really
intended to run in production (like tcn and hstore). But that
distinction is fuzzier than you might think, because while auth_delay
was intended as a POC, I've subsequently heard rumors of it being used
in production with satisfactory results. It's very easy to get the
idea that you know "what PostgreSQL users use" but usually that tends
to mean "what I use" and the community is broad enough that those
things are Not The Same.

The biggest problem is that packagers tend just to bundle contrib
together in one lump. If we could divide it into two, something like
"standard modules" and "misc", with the former being included with the
server package, I think that would be an advance, although packagers
might reasonably want to treat pgcrypto as a special case.

I'm also not in favor of booting packages wholesale out of core, for all
the good reasons you and others have advanced.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Jun 4, 2015 at 11:22 AM, Andrew Dunstan <andrew@dunslane.net> wrote:

The biggest problem is that packagers tend just to bundle contrib together
in one lump. If we could divide it into two, something like "standard
modules" and "misc", with the former being included with the server package,
I think that would be an advance, although packagers might reasonably want
to treat pgcrypto as a special case.

The problem is that it's very hard to agree on which stuff ought to be
standard and which stuff ought to be misc. There's probably some
stuff that almost everyone would agree is pretty useful (like hstore
and postgres_fdw) but figuring out which stuff isn't useful is a lot
harder. Almost anything you say - that's junk - someone else will say
- no, that thing is great, I use it all the time. For example, I just
offered contrib/isn as a sort of archetypal example of stuff that's
pretty marginal crap and Josh immediately came back and said, hey, I
use that! I don't see any principled way of getting past that
difficulty. Just because a module isn't regularly used by someone who
reads -hackers daily doesn't mean it's not worth keeping.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/04/2015 07:34 AM, Robert Haas wrote:

I don't think it's very practical to talk about getting rid of contrib
when we reliably add to it in every release:

Except, that is kind of the point. Why are we adding to it? Contrib
(AFAICS) is all things that don't need to be in -core. That is the whole
point of having extensions, is it not?

Sincerely,

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 6/4/15 10:30 AM, Robert Haas wrote:

On Thu, Jun 4, 2015 at 11:22 AM, Andrew Dunstan<andrew@dunslane.net> wrote:

The biggest problem is that packagers tend just to bundle contrib together
in one lump. If we could divide it into two, something like "standard
modules" and "misc", with the former being included with the server package,
I think that would be an advance, although packagers might reasonably want
to treat pgcrypto as a special case.

The problem is that it's very hard to agree on which stuff ought to be
standard and which stuff ought to be misc.

What I took away upthread was the idea here was to distinguish things
that were "intended as a POC (like worker_spi,
auth_delay and test_decoding)" from everything else.

I think the real problem here that we're skirting around is this idea of
'blessed extensions', because that's really the only user benefit
contrib brings: the idea that this stuff is formally blessed by the
community. If that's really what we're after then we should just be
explicit about that. Then we can decide if the best way to approach that
is keeping it in the main repo (as opposed to say, publishing a list of
explict PGXN package versions and their checksums).

Personally, I'd rather we publish a list of formally vetted and approved
versions of PGXN modules. There are many benefits to that, and the
downside of not having that stuff as part of make check would be
overcome by the explicit testing we would need to have for approved modules.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Robert Haas <robertmhaas@gmail.com> writes:

On Thu, Jun 4, 2015 at 11:22 AM, Andrew Dunstan <andrew@dunslane.net> wrote:

The biggest problem is that packagers tend just to bundle contrib together
in one lump. If we could divide it into two, something like "standard
modules" and "misc", with the former being included with the server package,
I think that would be an advance, although packagers might reasonably want
to treat pgcrypto as a special case.

As an ex-packager, I agree that would be a good thing. In particular
we should try to avoid people packaging stuff that's meant either for
server testing or as sample-source-code-not-useful-in-itself. We've
made some progress on the former via src/test/modules but I wonder
if we're all the way there; test_decoding surely shouldn't be in
contrib on this measure should it?

BTW, pg_upgrade is also a special case from a packager's viewpoint,
since it needs to be bundled with back-version executables.

The problem is that it's very hard to agree on which stuff ought to be
standard and which stuff ought to be misc. There's probably some
stuff that almost everyone would agree is pretty useful (like hstore
and postgres_fdw) but figuring out which stuff isn't useful is a lot
harder. Almost anything you say - that's junk - someone else will say
- no, that thing is great, I use it all the time. For example, I just
offered contrib/isn as a sort of archetypal example of stuff that's
pretty marginal crap and Josh immediately came back and said, hey, I
use that! I don't see any principled way of getting past that
difficulty. Just because a module isn't regularly used by someone who
reads -hackers daily doesn't mean it's not worth keeping.

Yeah. One possible way of measuring this would be to go through the
commit logs and count commits in contrib/ that either added a new
feature or fixed a field-reported bug (ie, did not arise simply from
core-code-driven housekeeping). Either would be solid evidence that
somebody out there is using it. Gathering the evidence would be
pretty tedious though :-(

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/04/2015 11:49 AM, Joshua D. Drake wrote:

On 06/04/2015 07:34 AM, Robert Haas wrote:

I don't think it's very practical to talk about getting rid of contrib
when we reliably add to it in every release:

Except, that is kind of the point. Why are we adding to it? Contrib
(AFAICS) is all things that don't need to be in -core. That is the
whole point of having extensions, is it not?

No. You keep getting this wrong. The fact that we have extensions
doesn't mean that we want to throw out everything that is an extension.
It's perfectly reasonable for us to maintain some ourselves, not least
as part of eating out own dog food.

And I say that as someone who has created and maintains quite a few
third party extensions.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/04/2015 09:00 AM, Andrew Dunstan wrote:

No. You keep getting this wrong. The fact that we have extensions
doesn't mean that we want to throw out everything that is an extension.
It's perfectly reasonable for us to maintain some ourselves, not least
as part of eating out own dog food.

Yes. I agree with that. Which is why I suggested .Org -contrib project.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/04/2015 08:55 AM, Jim Nasby wrote:

Personally, I'd rather we publish a list of formally vetted and approved
versions of PGXN modules. There are many benefits to that, and the
downside of not having that stuff as part of make check would be
overcome by the explicit testing we would need to have for approved
modules.

I tried to come up with more words but: Exactly.

The benefit to this idea so far exceeds the benefit to having contrib.

Sincerely,

JD

--
The most kicking donkey PostgreSQL Infrastructure company in existence.
The oldest, the most experienced, the consulting company to the stars.
Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 -
24x7 - 365 - Proactive and Managed Professional Services!

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Jun 4, 2015 at 11:49 AM, Joshua D. Drake <jd@commandprompt.com> wrote:

Except, that is kind of the point. Why are we adding to it?

If you don't know the answer to that question already, then you
probably shouldn't be proposing to get rid of the thing.

I think it's because there are some things we want to include in the
core distribution without baking them irrevocably into the server.

Which, IMV, is 100% reasonable.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/04/2015 10:34 AM, Robert Haas wrote:

On Thu, Jun 4, 2015 at 11:49 AM, Joshua D. Drake <jd@commandprompt.com> wrote:

Except, that is kind of the point. Why are we adding to it?

If you don't know the answer to that question already, then you
probably shouldn't be proposing to get rid of the thing.

I know the answer some people are saying. That doesn't mean it is the
correct answer, that I agree with it or that it is a good answer.

I think it's because there are some things we want to include in the
core distribution without baking them irrevocably into the server.

I have mentioned before isn't really what this discussion is about.
Stephen Frost and I even went through the entire list of modules of what
should and should not be included.

Which, IMV, is 100% reasonable.

Nobody is arguing with that.

Sincerely,

Joshua D. Drake

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Jun 4, 2015 at 1:57 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

I think it's because there are some things we want to include in the
core distribution without baking them irrevocably into the server.

I have mentioned before isn't really what this discussion is about. Stephen
Frost and I even went through the entire list of modules of what should and
should not be included.

Which, IMV, is 100% reasonable.

Nobody is arguing with that.

Well then I am confused.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/04/2015 11:01 AM, Robert Haas wrote:

On Thu, Jun 4, 2015 at 1:57 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

I think it's because there are some things we want to include in the
core distribution without baking them irrevocably into the server.

I have mentioned before isn't really what this discussion is about. Stephen
Frost and I even went through the entire list of modules of what should and
should not be included.

Which, IMV, is 100% reasonable.

Nobody is arguing with that.

Well then I am confused.

My argument was (after some preliminary discussion):

1. Review contrib
2. All modules that are "core worthy" install by default
3. Push all other contrib out into the wild

Possibly:

1. Have a contrib project that sat outside of core
2. Push all contrib or some contrib to pgxn (or some other place)

Because:

1. Decrease in code maintenance for core
2. Removal of the idea that contrib is a holding queue for not quite up
to snuff code
3. Most extensions don't need to follow the same development pattern
that core does
4. Eliminate the EGO of saying "I have a contrib module in core"

Derived from:

1. 15 years of the same argument (current source: pg_audit)
2. Helping reduce overall resource need to manage contrib
3. And now (Nasby's excellent argument)

I am not trying to throw a wrench in things. I am trying to help
streamline them. A lot of the arguments presented just don't hold water
(eating your own dogfood) because we aren't stopping anyone from doing
that testing. Whether or not the source (whatever it may be) is in
-contrib doesn't prevent that testing.

Sincerely,

Joshua D. Drake

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-06-04 11:33:47 -0700, Joshua D. Drake wrote:

My argument was (after some preliminary discussion):

1. Review contrib
2. All modules that are "core worthy" install by default
3. Push all other contrib out into the wild

Possibly:

1. Have a contrib project that sat outside of core
2. Push all contrib or some contrib to pgxn (or some other place)

Because:

1. Decrease in code maintenance for core

The modules that aren't going to be in core, hardly cost time, do they?

2. Removal of the idea that contrib is a holding queue for not quite up to
snuff code

I don't think that idea exists widely.

3. Most extensions don't need to follow the same development pattern that
core does

The point is that users want them to follow that.

4. Eliminate the EGO of saying "I have a contrib module in core"

Seriously?

1. 15 years of the same argument (current source: pg_audit)

I don't see getting rid of contrib helping with that. The only change
will then be whether something should be in core.

And there's stuff that's just very hard to develop out of core; but
doesn't necessarily immediately belong into
core. E.g. pg_stat_statements is relatively closely tied to things in
core.

2. Helping reduce overall resource need to manage contrib

This discussion a significant share of that.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-06-04 20:41:46 +0200, Andres Freund wrote:

1. 15 years of the same argument (current source: pg_audit)

I don't see getting rid of contrib helping with that. The only change
will then be whether something should be in core.

And there's stuff that's just very hard to develop out of core; but
doesn't necessarily immediately belong into
core. E.g. pg_stat_statements is relatively closely tied to things in
core.

An even better example is pg_upgrade. It'd not have been possible to
bring it somewhere close to maturity without having it in contrib/. It
requires core cooperation, and we weren't immediately to declare it as
part of core.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Jun 4, 2015, at 10:55 AM, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:

Personally, I'd rather we publish a list of formally vetted and approved versions of PGXN modules. There are many benefits to that, and the downside of not having that stuff as part of make check would be overcome by the explicit testing we would need to have for approved modules.

I have looked at PGXN and would never install anything from it. Why? Because it is impossible to tell, without inside knowledge or a lot of work, what is actively maintained and tested, and what is an abandoned proof-of-concept or idea. There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can be run to prove the module works correctly with a particular version of pg. There are many modules that have not been updated for several years. What is their status? If they break is there still someone around to fix them or even cares about them? If not, then why waste my time.

So adding to Jim’s comment above, anything that vets or approves PGXN modules is, in my opinion, essentially required to make PGXN useful for anything other than a scratchpad. A big help would be to pull in the date of the last git commit in the module overview and ask the authors to edit the readme to add what major version of pg the author last tested or ran on.

When I install from contrib, at least I have some minimal assurance that the code is meant to work with the corresponding version of pg.

Neil

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Jun 4, 2015, at 11:53 AM, Neil Tiffin <neilt@neiltiffin.com> wrote:

I have looked at PGXN and would never install anything from it. Why? Because it is impossible to tell, without inside knowledge or a lot of work, what is actively maintained and tested, and what is an abandoned proof-of-concept or idea.

Well, you can see the last release dates for a basic idea of that sort of thing. Also the release status (stable, unstable, testing).

There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can be run to prove the module works correctly with a particular version of pg.

Yeah, I’ve been meaning to integrate http://pgxn-tester.org/ results for all modules, which would help with that. In the meantime you can hit that site itself. Awesome work by Tomas Vondra.

There are many modules that have not been updated for several years. What is their status? If they break is there still someone around to fix them or even cares about them? If not, then why waste my time.

These are challenges to open-source software in general, and not specific to PGXN.

So adding to Jim’s comment above, anything that vets or approves PGXN modules is, in my opinion, essentially required to make PGXN useful for anything other than a scratchpad.

Most of the distributions on PGXN feature links to their source code repositories.

A big help would be to pull in the date of the last git commit in the module overview and ask the authors to edit the readme to add what major version of pg the author last tested or ran on.

That’s difficult to maintain; I used to do it for pgTAP, was too much work. pgxn-tester.org is a much better idea.

Best,

David

On 6/4/15 3:11 PM, David E. Wheeler wrote:

There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can be run to prove the module works correctly with a particular version of pg.

Yeah, I’ve been meaning to integratehttp://pgxn-tester.org/ results for all modules, which would help with that. In the meantime you can hit that site itself. Awesome work by Tomas Vondra.

A number of modules also run Travis-CI. Might be worth having a way for
a module to provide it's status .png.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Jun 4, 2015 at 2:33 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

My argument was (after some preliminary discussion):

1. Review contrib
2. All modules that are "core worthy" install by default
3. Push all other contrib out into the wild

So above, I said that we keep adding to contrib because "there are
some things we want to include in the core distribution without baking
them irrevocably into the server" and you said that you weren't
arguing with that, but here you're saying you don't want any such
things to exist. That doesn't really make any sense.

postgres_fdw is a good example. It's "core-worthy" in the sense that
it is useful enough to be installed in core, but not everybody may
want their database server to have the ability to make network
connections at the behest of unprivileged users. Today, they can
achieve that by not installing the extension if they don't want users
to have access to it. Putting it into core would require us to come
up with some new way to control whether that functionality is
available or not. That seems like making a lot of work for ourselves
for no real benefit.

This same argument applies to (at least) dblink and adminpack.

1. Decrease in code maintenance for core

contrib requires very little maintenance, and is often very helpful
for judging whether other core changes - e.g. changes to hooks - are
working properly. I see no maintenance benefit to removing it; it
would probably just make it harder to figure out whether other stuff
is broken. And the removal itself would be a ton of work.

2. Removal of the idea that contrib is a holding queue for not quite up to
snuff code

I don't think it's really being used that away any more.

3. Most extensions don't need to follow the same development pattern that
core does

That's not a reason to move things that are already in contrib to
somewhere else. At least not that I can see.

4. Eliminate the EGO of saying "I have a contrib module in core"

I've got multiple major features in core. Any ego I may have about my
PostgreSQL contributions is not based on pg_prewarm.

1. 15 years of the same argument (current source: pg_audit)

The argument about pg_audit has little to do with contrib. It is
primarily about code quality, and secondarily about whether one
committer can go do something unliterally when a long list of other
committers and contributors have expressed doubts about it.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Robert,

* Robert Haas (robertmhaas@gmail.com) wrote:

On Thu, Jun 4, 2015 at 2:33 PM, Joshua D. Drake <jd@commandprompt.com> wrote:

1. 15 years of the same argument (current source: pg_audit)

The argument about pg_audit has little to do with contrib. It is
primarily about code quality, and secondarily about whether one
committer can go do something unliterally when a long list of other
committers and contributors have expressed doubts about it.

I would certainly welcome any further review or comments regarding the
code quality of pg_audit from anyone interested in the capability. I do
not agree that the code quality is significantly below that of other
modules or core. There were design restrictions due to it being an
extension which quite a few people had questions and concerns about,
which I addressed through the discussions on the list.

Further, pg_audit was originally presented by 2ndQ, worked on by
multiple major contributors and committers, and had multiple committers
expressing interest in committing it during the push to close out the
final CF for 9.5. I understand that I've been the one who has primairly
been spending time discussing it on the lists, but I was also one of the
biggest nay-sayers of it over the summer last year. What changed my
opinion of it? The point made by other committers that the upgrade
concerns could be addressed and the strong interest from users in the
capability.

Painting it as the unilateral actions of one committer is uncharitable,
at best.

Thanks!

Stephen

On 29 May 2015 at 02:50, Peter Eisentraut <peter_e@gmx.net> wrote:

On 5/28/15 3:35 PM, Stephen Frost wrote:

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

+1 for Extensions directory for 9.6

This doesn't seem worth delaying the release for.

--
Simon Riggs http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/&gt;
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Fri, Jun 5, 2015 at 8:32 AM, Stephen Frost <sfrost@snowman.net> wrote:

Painting it as the unilateral actions of one committer is uncharitable,
at best.

As I see it, it is just stating the facts. There were several emails
from other committers on the pg_audit thread not that long before it
was committed, and zero of them were supportive. None of them were
dead-set against it, but it is not our typical way of doing business
to press forward with committing something when that many doubts have
been expressed. I believe you will struggle to find comparable
examples involving other patches.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/05/2015 04:56 AM, Robert Haas wrote:
somewhere else. At least not that I can see.

4. Eliminate the EGO of saying "I have a contrib module in core"

I've got multiple major features in core. Any ego I may have about my
PostgreSQL contributions is not based on pg_prewarm.

This was worded badly by me. This isn't about your ego, it is about the
mysticism surrounding the idea that "they have a feature in core". It is
really last on the list and not really important to this discussion.

1. 15 years of the same argument (current source: pg_audit)

The argument about pg_audit has little to do with contrib. It is
primarily about code quality, and secondarily about whether one
committer can go do something unliterally when a long list of other
committers and contributors have expressed doubts about it.

The argument was about whether it should be in contrib, code quality or
not. If contrib didn't exist and we accepted that extensions are an
outside core thing, the argument never would have happened.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 06/05/2015 03:34 AM, Jim Nasby wrote:

On 6/4/15 3:11 PM, David E. Wheeler wrote:

There is no indication of what versions of pg any of PGXN modules
are tested on, or even if there are tests that can be run to prove
the module works correctly with a particular version of pg.

Yeah, I’ve been meaning to integratehttp://pgxn-tester.org/ results
for all modules, which would help with that. In the meantime you can
hit that site itself. Awesome work by Tomas Vondra.

A number of modules also run Travis-CI. Might be worth having a way
for a module to provide it's status .png.

It's also quite possible to test extensions in the buildfarm using an
addon module, which is mostly boilerplate. Of course, that does require
that you're running a buildfarm member. Here's the code for the module
that tests the FileTextArray extension.
<https://github.com/PGBuildFarm/client-code/blob/master/PGBuild/Modules/FileTextArrayFDW.pm&gt;

This and the RedisFDW are tested daily or more often, on machines I control.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Jun 5, 2015 at 02:42:45PM +0100, Simon Riggs wrote:

On 29 May 2015 at 02:50, Peter Eisentraut <peter_e@gmx.net> wrote:

On 5/28/15 3:35 PM, Stephen Frost wrote:

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are.� With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory.� I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me.� It's in line with the recent move from
contrib to bin.� It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?)� Maybe
start by picking the top 5 and sort those out.

+1 for Extensions directory for 9.6

This doesn't seem worth delaying the release for.

I didn't think any of this was for 9.5 consideration.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Jun 04, 2015 at 01:11:21PM -0700, David E. Wheeler wrote:

On Jun 4, 2015, at 11:53 AM, Neil Tiffin <neilt@neiltiffin.com> wrote:

I have looked at PGXN and would never install anything from it.
Why? Because it is impossible to tell, without inside knowledge
or a lot of work, what is actively maintained and tested, and what
is an abandoned proof-of-concept or idea.

Well, you can see the last release dates for a basic idea of that
sort of thing. Also the release status (stable, unstable, testing).

There is no indication of what versions of pg any of PGXN modules
are tested on, or even if there are tests that can be run to prove
the module works correctly with a particular version of pg.

Yeah, I’ve been meaning to integrate http://pgxn-tester.org/ results
for all modules, which would help with that. In the meantime you can
hit that site itself. Awesome work by Tomas Vondra.

Giant +1 for this. I had no idea it existed until this morning. As a
PGXN contributor, this kind of feedback is invaluable to me.

There are many modules that have not been updated for several
years. What is their status? If they break is there still
someone around to fix them or even cares about them? If not, then
why waste my time.

These are challenges to open-source software in general, and not
specific to PGXN.

The pgxn-tester system answers more of that question than a lot of
other projects do. At some point, each organization using software,
free or proprietary, open source or closed, has to do some of their
own vetting.

So adding to Jim’s comment above, anything that vets or approves
PGXN modules is, in my opinion, essentially required to make PGXN
useful for anything other than a scratchpad.

Most of the distributions on PGXN feature links to their source code
repositories.

Should this just be made a hard requirement for PGXN? Lack of a
source code repo is a pretty good sign of abandonment.

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Jun 4, 2015, at 3:11 PM, David E. Wheeler <david@justatheory.com> wrote:

On Jun 4, 2015, at 11:53 AM, Neil Tiffin <neilt@neiltiffin.com> wrote:

I have looked at PGXN and would never install anything from it. Why? Because it is impossible to tell, without inside knowledge or a lot of work, what is actively maintained and tested, and what is an abandoned proof-of-concept or idea.

Well, you can see the last release dates for a basic idea of that sort of thing. Also the release status (stable, unstable, testing).

There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can be run to prove the module works correctly with a particular version of pg.

Yeah, I’ve been meaning to integrate http://pgxn-tester.org/ results for all modules, which would help with that. In the meantime you can hit that site itself. Awesome work by Tomas Vondra.

There are many modules that have not been updated for several years. What is their status? If they break is there still someone around to fix them or even cares about them? If not, then why waste my time.

These are challenges to open-source software in general, and not specific to PGXN.

Of course, but the solution is having tools to easily determine the risk. The fact that the modules pass or fail the tests on pgxn-tester is a significant step. Knowing how long the module has been failing would be even better.

So adding to Jim’s comment above, anything that vets or approves PGXN modules is, in my opinion, essentially required to make PGXN useful for anything other than a scratchpad.

Most of the distributions on PGXN feature links to their source code repositories.

A big help would be to pull in the date of the last git commit in the module overview and ask the authors to edit the readme to add what major version of pg the author last tested or ran on.

That’s difficult to maintain; I used to do it for pgTAP, was too much work. pgxn-tester.org is a much better idea.

Yes it is.

Wow, that is awesome work (pgxn-tester.org). Thanks Tomas Vondra, and David for pointing it out. This improved my opinion of PGXN significantly. It might be helpful to at least put a link on the PGXN home page, beta or not, its awesome and even in beta it shows the future direction.

It would be nice to see the development branch in the tests. One project I am working on now targets 9.5.

It is important to know how long a stable module has been failing for a specific version of pg. This is IMO a critical measure of the level of support a module is receiving.

Neil

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Jun 5, 2015, at 12:34 AM, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:

A number of modules also run Travis-CI. Might be worth having a way for a module to provide it's status .png.

Right. Just stick it in your README.

http://blog.pgxn.org/post/116087351668/badges

David

Andrew Dunstan wrote:

It's also quite possible to test extensions in the buildfarm using an addon
module, which is mostly boilerplate. Of course, that does require that
you're running a buildfarm member. Here's the code for the module that tests
the FileTextArray extension. <https://github.com/PGBuildFarm/client-code/blob/master/PGBuild/Modules/FileTextArrayFDW.pm&gt;

I think important modules should all be tested and reported in the
buildfarm in this way. For instance, I think PostGIS should report
there as well, so that we know when we break it the moment we do.

This and the RedisFDW are tested daily or more often, on machines I control.

Yeah, that's pretty neat.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 6/5/15 10:49 AM, David E. Wheeler wrote:

On Jun 5, 2015, at 12:34 AM, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:

A number of modules also run Travis-CI. Might be worth having a way for a module to provide it's status .png.

Right. Just stick it in your README.

http://blog.pgxn.org/post/116087351668/badges

I meant something more integrated with PGXN itself, such as what you're
proposing for pgxn-tester. That would allow for things like PGXN search
results to show Travis results.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Jun 5, 2015, at 10:47 AM, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:

Right. Just stick it in your README.

http://blog.pgxn.org/post/116087351668/badges

I meant something more integrated with PGXN itself, such as what you're proposing for pgxn-tester. That would allow for things like PGXN search results to show Travis results.

Well, it’s not really possible to tell if there’s a Travis job for a project. I could add something to the META.json, though.

David

On Fri, Jun 5, 2015 at 7:42 AM, Joshua D. Drake <jd@commandprompt.com>
wrote:

On 06/05/2015 04:56 AM, Robert Haas wrote:
somewhere else. At least not that I can see.

4. Eliminate the EGO of saying "I have a contrib module in core"

I've got multiple major features in core. Any ego I may have about my
PostgreSQL contributions is not based on pg_prewarm.

This was worded badly by me. This isn't about your ego, it is about the
mysticism surrounding the idea that "they have a feature in core". It is
really last on the list and not really important to this discussion.

1. 15 years of the same argument (current source: pg_audit)

The argument about pg_audit has little to do with contrib. It is
primarily about code quality, and secondarily about whether one
committer can go do something unliterally when a long list of other
committers and contributors have expressed doubts about it.

The argument was about whether it should be in contrib, code quality or
not. If contrib didn't exist and we accepted that extensions are an outside
core thing, the argument never would have happened.

That is only the case if we also accept that all extensions are just an
amorphous mass. If some extensions are "blessed" or "vetted" or "by
default tested against HEAD by the buildfarm" or whatever, then the debate
would still happen, just with different words. And I don't think that
forestalling debate is such a worthwhile goal in itself. Some debates are
worth having.

Cheers,

Jeff