RFC: Remove contrib entirely

JD,

* Joshua D. Drake (jd@commandprompt.com) wrote:

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in
features that are not part of the core PostgreSQL system, mainly
because they address a limited audience or are too experimental to
be part of the main source tree. This does not preclude their
usefulness."

We certainly seem to have moved on from that definition. If nothing
else, it'd be valuable to clarify what contrib is and then rearrange
things accordingly.

It has also been mentioned many times over the years that contrib is
a holding tank for technology that would hopefully be pushed into
core someday.

I continue to feel this is a good use-case for contrib.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A
few of these are pretty obvious:

pg_stat_statements
citext
postgres_fdw
hstore
pg_crypto
[...]

We don't really have a place in "-core" for them.. There has been
ongoing discussion about that but nothing has changed in that regard, as
far as I'm aware. We have moved a few things out of contrib and into
bin, but that doesn't make sense for the above.

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

I am sure there will be plenty of fun to be had with what should or
shouldn't be merged into core. I think if we argue about the
guidelines of how to analyze what should be in core versus the
merits of any particular module, life will be easier. Here are some
for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

I don't particularly like having a time-based requirement drive what's
in which area, especially one that's double the length of our major
release cycle.

2. Push the rest out into a .Org project called contrib. Let those
who are interested in the technology work on them or use them. This
project since it is outside of core proper can work just like other
extension projects. Alternately, allow the maintainers push them
wherever they like (Landscape, Github, Savannah, git.postgresql.org
...).

That's an interesting idea, but it's unclear how this would be any
different from PGXN..?

Thanks!

Stephen

On 05/28/2015 12:35 PM, Stephen Frost wrote:

JD,

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

I am suggesting that things like citext be made a type proper. No
install required. For things like pg_stat_statements of course there
could be configuration required (need to add it to the preload) but it
is automatically installed with the packages.

I am sure there will be plenty of fun to be had with what should or
shouldn't be merged into core. I think if we argue about the
guidelines of how to analyze what should be in core versus the
merits of any particular module, life will be easier. Here are some
for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

I don't particularly like having a time-based requirement drive what's
in which area, especially one that's double the length of our major
release cycle.

I think there is only one or two contrib modules that don't actually fit
requirement A.

2. Push the rest out into a .Org project called contrib. Let those
who are interested in the technology work on them or use them. This
project since it is outside of core proper can work just like other
extension projects. Alternately, allow the maintainers push them
wherever they like (Landscape, Github, Savannah, git.postgresql.org
...).

That's an interesting idea, but it's unclear how this would be any
different from PGXN..?

PGXN is CPAN not Perl or DBD::Pg.

It is actually a compliment to PGXN because it is still needed and
needed more because that is where you are going to get the "latest and
greatest" of the modules.

Sincerely,

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 04:05 PM, Joshua D. Drake wrote:

On 05/28/2015 12:35 PM, Stephen Frost wrote:

JD,

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

I am suggesting that things like citext be made a type proper. No
install required.

This seems to come up regularly. Maybe we should put it in an FAQ
somewhere. The barriers to making non-core types into core types are
very very high, possibly insurmountable. This is pretty much not an option.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 01:11 PM, Andrew Dunstan wrote:

This seems to come up regularly. Maybe we should put it in an FAQ
somewhere. The barriers to making non-core types into core types are
very very high, possibly insurmountable. This is pretty much not an option.

O.k., then either:

* We install it by default and document that it is available (and how
to enable it)

* Push it out of core and let it be happy wherever Theory wants it to be

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 04:22 PM, Joshua D. Drake wrote:

On 05/28/2015 01:11 PM, Andrew Dunstan wrote:

This seems to come up regularly. Maybe we should put it in an FAQ
somewhere. The barriers to making non-core types into core types are
very very high, possibly insurmountable. This is pretty much not an
option.

O.k., then either:

* We install it by default and document that it is available (and how
to enable it)

* Push it out of core and let it be happy wherever Theory wants it to be

I'd be ok with installing it by default.

But the case that's a lot harder to require to be always installed is
pgcrypto, as has often been discussed in the past.

In any case, we will always want to have some loadable modules, not
least because it's a part of eating our own dog food.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 5/28/15 3:35 PM, Stephen Frost wrote:

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 06:25 PM, Andrew Dunstan wrote:

I'd be ok with installing it by default.

But the case that's a lot harder to require to be always installed is
pgcrypto, as has often been discussed in the past.

It used to be but IIRC we don't have those restrictions anymore. If so,
then we need to pull it out and just call it the "Encryption Extension"
or whatever....

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 05/28/2015 06:50 PM, Peter Eisentraut wrote:

On 5/28/15 3:35 PM, Stephen Frost wrote:

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

F.1. adminpack
F.2. auth_delay
F.3. auto_explain
F.4. btree_gin
F.5. btree_gist
F.6. chkpass
F.7. citext
F.8. cube
F.9. dblink
F.10. dict_int
F.11. dict_xsyn
F.12. earthdistance
F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore
F.16. intagg
F.17. intarray
F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect
F.22. passwordcheck
F.23. pg_buffercache
F.24. pgcrypto
F.25. pg_freespacemap
F.26. pg_prewarm
F.27. pgrowlocks
F.28. pg_stat_statements
F.29. pgstattuple
F.30. pg_trgm
F.31. postgres_fdw
F.32. seg
F.33. sepgsql
F.34. spi
F.35. sslinfo
F.36. tablefunc
F.37. tcn
F.38. test_decoding
F.39. tsearch2
F.40. tsm_system_rows
F.41. tsm_system_time
F.42. unaccent
F.43. uuid-ossp
F.44. xml2

Look at these, a lot of them are obvious... just include for goodness sakes:

pg_trgm has been in contrib for a decade of not more. Either rip it out
or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I just don't think this would be that hard if we were willing to put our
minds to it.

Or.. another way:

Nobody has yet to provide an argument as to why we need contrib when we
have a fully functioning extension capability.

Ego... is not a good reason.

Of course the other option:

Freeze contrib. What is in there now, is all there will ever be in there
and the goal is to slowly reduce it to the point that it doesn't matter.

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-05-29 AM 11:14, Joshua D. Drake wrote:

pg_trgm has been in contrib for a decade of not more. Either rip it out or
include it by default.

There are jsonb gin operator class related files in src/backend/utils/adt/.
Perhaps, trgm_gin.c, trgm_gist.c, trgm_op.c could be moved there. Similarly
for other gin/gist operators classes - hstore, intarray, ltree maybe. Or each
can have its own directory (including jsonb).

But the original comment was about having these moved to an 'extensions'
directory if at all, so perhaps this suggestion is moot.

Thanks,
Amit

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

JD,

* Joshua D. Drake (jd@commandprompt.com) wrote:

On 05/28/2015 06:50 PM, Peter Eisentraut wrote:

On 5/28/15 3:35 PM, Stephen Frost wrote:

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are. With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory. I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

Alright, I'll bite. :)

F.1. adminpack

Need it- pgAdmin can't senibly install it or even include it in some
way, and it'd be *very* painful to not have it for a lot of users.

F.2. auth_delay

Should be a core feature. Having this in a contrib module is silly.

F.3. auto_explain

Move to extension directory in the repo.

F.4. btree_gin
F.5. btree_gist

Both of these should simply be in core.

F.6. chkpass
F.7. citext
F.8. cube

Push out and/or keep it in contrib in repo.

F.9. dblink

Move to extension directory in the repo.

F.10. dict_int
F.11. dict_xsyn

Looks like these are just examples? Maybe move to an 'examples'
directory, or into src/test/modules, or keep in contrib.

F.12. earthdistance

Depends on cube, so, same as whatever happens there. I don't think
extensions-in-repo should depend on contrib modules, as a rule.

F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore

Move to extension directory in the repo.

F.16. intagg

Obsolute, per the docs. Push out and deal with the break, or keep it in
contrib in repo.

F.17. intarray

Move to extension directory in the repo.

F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect

Move to extension directory in the repo.

F.22. passwordcheck

Should be an in-core capability and not shoved off into an extension.

F.23. pg_buffercache

Pull it into core.

F.24. pgcrypto

Move to extension directory in the repo.

F.25. pg_freespacemap

Should be in core.

F.26. pg_prewarm
F.27. pgrowlocks

Should be in core.

F.28. pg_stat_statements

I'd actually prefer that this be in core, but I'd be alright with it
being in extension directory in the repo.

F.29. pgstattuple
F.30. pg_trgm

Should be in core.

F.31. postgres_fdw

Move to extension directory in the repo.
(actually, I'd be fine with both this and file_fdw being included in
core.. I'm just not 100% sure how that'd look)

F.32. seg
F.33. sepgsql

Move to extension directory in the repo.

F.34. spi

Maybe pull some into core.. or maybe all, or move to an extension.

F.35. sslinfo

Should be in core.

F.36. tablefunc

My gut reaction is that it should be in core for crosstab(), but David's
talking about implementing PIVOT, so..

F.37. tcn

Should be in core, imv, but not a position I hold very strongly.

F.38. test_decoding

Should be in src/test/modules, or maybe some 'examples' dir.

F.39. tsearch2

I'm inclined to just drop this.. Or we could keep it in contrib in the
repo.

F.40. tsm_system_rows
F.41. tsm_system_time

These should be in core.

F.42. unaccent

Move to extension directory in the repo.

F.43. uuid-ossp

This one probably deserves its own thread, heh..

F.44. xml2

Push it out, or keep it in contrib in repo.

Look at these, a lot of them are obvious... just include for goodness sakes:

pg_trgm has been in contrib for a decade of not more. Either rip it
out or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

Agreed.

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Not sure that being platform agnostic has to be a requirement of being
in the repo or being an extension in the repo... It does need some work
though and discussion has recently started about if the sepgsql types
defined in the SELinux reference policy should continue to exist or if
they should be changed. I'm following that discussion with interest.

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I'd keep it in the repo as an extension. Pushing it out would just
cause lots of trouble for little gain.

I just don't think this would be that hard if we were willing to put
our minds to it.

Or.. another way:

Nobody has yet to provide an argument as to why we need contrib when
we have a fully functioning extension capability.

pg_stat_statements is perhaps one of the best reasons. Not something
that we necessairly want to force on everyone who installs PG
(presumably the additional shared memory and performance impact is an
issue for some people... though I'm not sure I agree, though it's been
a while since that discussion and perhaps the impact looks like less
from a distance), but a capability that we absolutely want to have.

Not sure if we want the FDWs to be installed by default... I had been
thinking there would be an issue with that at first, but the more I
consider it, the more I'm agreeing with you that we should just pull in
both postgres_fdw and file_fdw.

Still, there are extensions which we'll want to have because they simply
can't be included in core for one reason or another, they're very useful
bits of code, and we're willing and eager to maintain them moving
forward.

One of the big questions which will arise out of this though is- how do
we describe contrib vs. this set of extensions? Do we treat security
issues in the 'new' contrib differently? Do we only leave things which
are examples or are deprecated in the 'new' contrib? I'm presuming that
the items which end up in 'extensions' are more formally considered part
of the project and therefore held to the overall project standard (as
contrib is now being, even though it clearly wasn't always that way).

Freeze contrib. What is in there now, is all there will ever be in
there and the goal is to slowly reduce it to the point that it
doesn't matter.

I don't particularly like this option.

Thanks!

Stephen

On 05/28/2015 08:10 PM, Stephen Frost wrote:

JD,

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

Alright, I'll bite. :)

I knew somebody eventually would ;)

F.1. adminpack

Need it- pgAdmin can't senibly install it or even include it in some
way, and it'd be *very* painful to not have it for a lot of users.

Fair enough, although keep in mind we aren't telling people pgAdmin
isn't useful. We are just pushing it out of core. Who runs from source
except developers? Distributions would take care of this for us.

F.2. auth_delay

Should be a core feature. Having this in a contrib module is silly.

+1

F.3. auto_explain

Move to extension directory in the repo.

+1

F.4. btree_gin
F.5. btree_gist

Both of these should simply be in core.

+1

F.6. chkpass
F.7. citext
F.8. cube

Push out and/or keep it in contrib in repo.

Agreed except citext which I think should install by default.

F.9. dblink

Move to extension directory in the repo.

Agreed.

F.10. dict_int
F.11. dict_xsyn

Looks like these are just examples? Maybe move to an 'examples'
directory, or into src/test/modules, or keep in contrib.

Agreed.

F.12. earthdistance

Depends on cube, so, same as whatever happens there. I don't think
extensions-in-repo should depend on contrib modules, as a rule.

F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore

Move to extension directory in the repo.

Disagree, hstore should be in core. Rest, fine.

F.16. intagg

Obsolute, per the docs. Push out and deal with the break, or keep it in
contrib in repo.

Spelling mistake aside ;) agreed

F.17. intarray

Move to extension directory in the repo.

Agreed

F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect

Move to extension directory in the repo.

Except for maybe pageinspect, agreed.

F.22. passwordcheck

Should be an in-core capability and not shoved off into an extension.

Agreed

F.23. pg_buffercache

Pull it into core.

Agreed

F.24. pgcrypto

Move to extension directory in the repo.

Sure.

F.25. pg_freespacemap

Should be in core.

Agreed.

F.26. pg_prewarm
F.27. pgrowlocks

Should be in core.

With a change to pg_rowlocks, agreed.

F.28. pg_stat_statements

I'd actually prefer that this be in core, but I'd be alright with it
being in extension directory in the repo.

Agreed just not enabled by default.

F.29. pgstattuple
F.30. pg_trgm

Should be in core.

Agreed.

F.31. postgres_fdw

Move to extension directory in the repo.
(actually, I'd be fine with both this and file_fdw being included in
core.. I'm just not 100% sure how that'd look)

I think they should be in core, not all FDWs of course but file and
postgres are kind of obvious to me.

F.32. seg
F.33. sepgsql

Move to extension directory in the repo.

Agreed.

F.34. spi

Maybe pull some into core.. or maybe all, or move to an extension.

No opinion.

F.35. sslinfo

Should be in core.

Agreed.

F.36. tablefunc

My gut reaction is that it should be in core for crosstab(), but David's
talking about implementing PIVOT, so..

Easy... give it 1 more release. If we get PIVOT, then we don't need it,
if we don't... all the better for us.

F.37. tcn

Should be in core, imv, but not a position I hold very strongly.

no opinion

F.38. test_decoding

Should be in src/test/modules, or maybe some 'examples' dir.

agreed

F.39. tsearch2

I'm inclined to just drop this.. Or we could keep it in contrib in the
repo.

Release a "final release" as a pgxn capable extension and rip it out.

F.40. tsm_system_rows
F.41. tsm_system_time

These should be in core.

Agreed

F.42. unaccent

Move to extension directory in the repo.

Agreed

F.43. uuid-ossp

This one probably deserves its own thread, heh..

F.44. xml2

Push it out, or keep it in contrib in repo.

Look at these, a lot of them are obvious... just include for goodness sakes:

Agreed.

pg_trgm has been in contrib for a decade of not more. Either rip it
out or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

Agreed.

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Not sure that being platform agnostic has to be a requirement of being
in the repo or being an extension in the repo... It does need some work
though and discussion has recently started about if the sepgsql types
defined in the SELinux reference policy should continue to exist or if
they should be changed. I'm following that discussion with interest.

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I'd keep it in the repo as an extension. Pushing it out would just
cause lots of trouble for little gain.

I just don't think this would be that hard if we were willing to put
our minds to it.

See... we agree on pretty much all of it in principle.

Seriously guys, this particular argument is an argument of "what?". This
is writing on the wall. If Frost and I are in this much agreement... :P

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Le 29 mai 2015 5:33 AM, "Joshua D. Drake" <jd@commandprompt.com> a écrit :

On 05/28/2015 08:10 PM, Stephen Frost wrote:

JD,

This seems reasonable to me. It's in line with the recent move from
contrib to bin. It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?) Maybe
start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

Alright, I'll bite. :)

I knew somebody eventually would ;)

F.1. adminpack

Need it- pgAdmin can't senibly install it or even include it in some
way, and it'd be *very* painful to not have it for a lot of users.

Painful? The adminpack allows pgadmin to change the config files remotely
with a UI that doesn't make it easy to say the least. You may well trash
your pg_hba.conf file and not be able to connect again after reloading. It
also allows you to read your log files remotely... if it only contains UTF8
characters (which doesn't happen much with my french customers). And
loading a 1GB log file is definitely painful.

I would be of the idea it doesn't give much (though it doesn't mean I want
it to be dropped), but I'm pretty much telling my customers to drop it
whenever I can.

Fair enough, although keep in mind we aren't telling people pgAdmin isn't

useful. We are just pushing it out of core. Who runs from source except
developers? Distributions would take care of this for us.

Yeah. The way I see this is that distributions would make packages for each
extension. And I don't see the difference between doing a

yum install postgresql94-contrib

And a

yum install postgresql94-adminpack

for example, except I would have to run various "yum install" commands to
install every extensions I need, but this is much better for me than
bloating my system with extensions I never use (earthdistance comes to mind
for example).

FWIW, I don't mind which one we put in core and which one we put out of
core. But I like Joshua's idea of getting rid of contribs and pushing them
out as any other extensions.

F.2. auth_delay

Should be a core feature. Having this in a contrib module is silly.

+1

F.3. auto_explain

Move to extension directory in the repo.

+1

F.4. btree_gin
F.5. btree_gist

Both of these should simply be in core.

+1

F.6. chkpass
F.7. citext
F.8. cube

Push out and/or keep it in contrib in repo.

Agreed except citext which I think should install by default.

F.9. dblink

Move to extension directory in the repo.

Agreed.

F.10. dict_int
F.11. dict_xsyn

Looks like these are just examples? Maybe move to an 'examples'
directory, or into src/test/modules, or keep in contrib.

Agreed.

F.12. earthdistance

Depends on cube, so, same as whatever happens there. I don't think
extensions-in-repo should depend on contrib modules, as a rule.

F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore

Move to extension directory in the repo.

Disagree, hstore should be in core. Rest, fine.

F.16. intagg

Obsolute, per the docs. Push out and deal with the break, or keep it in
contrib in repo.

Spelling mistake aside ;) agreed

F.17. intarray

Move to extension directory in the repo.

Agreed

F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect

Move to extension directory in the repo.

Except for maybe pageinspect, agreed.

F.22. passwordcheck

Should be an in-core capability and not shoved off into an extension.

Agreed

F.23. pg_buffercache

Pull it into core.

Agreed

F.24. pgcrypto

Move to extension directory in the repo.

Sure.

F.25. pg_freespacemap

Should be in core.

Agreed.

F.26. pg_prewarm
F.27. pgrowlocks

Should be in core.

With a change to pg_rowlocks, agreed.

F.28. pg_stat_statements

I'd actually prefer that this be in core, but I'd be alright with it
being in extension directory in the repo.

Agreed just not enabled by default.

F.29. pgstattuple
F.30. pg_trgm

Should be in core.

Agreed.

F.31. postgres_fdw

Move to extension directory in the repo.
(actually, I'd be fine with both this and file_fdw being included in
core.. I'm just not 100% sure how that'd look)

I think they should be in core, not all FDWs of course but file and

postgres are kind of obvious to me.

F.32. seg
F.33. sepgsql

Move to extension directory in the repo.

Agreed.

F.34. spi

Maybe pull some into core.. or maybe all, or move to an extension.

No opinion.

F.35. sslinfo

Should be in core.

Agreed.

F.36. tablefunc

My gut reaction is that it should be in core for crosstab(), but David's
talking about implementing PIVOT, so..

Easy... give it 1 more release. If we get PIVOT, then we don't need it,

if we don't... all the better for us.

F.37. tcn

Should be in core, imv, but not a position I hold very strongly.

no opinion

F.38. test_decoding

Should be in src/test/modules, or maybe some 'examples' dir.

agreed

F.39. tsearch2

I'm inclined to just drop this.. Or we could keep it in contrib in the
repo.

Release a "final release" as a pgxn capable extension and rip it out.

F.40. tsm_system_rows
F.41. tsm_system_time

These should be in core.

Agreed

F.42. unaccent

Move to extension directory in the repo.

Agreed

F.43. uuid-ossp

This one probably deserves its own thread, heh..

F.44. xml2

Push it out, or keep it in contrib in repo.

Look at these, a lot of them are obvious... just include for goodness

sakes:

Agreed.

pg_trgm has been in contrib for a decade of not more. Either rip it
out or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

Agreed.

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Not sure that being platform agnostic has to be a requirement of being
in the repo or being an extension in the repo... It does need some work
though and discussion has recently started about if the sepgsql types
defined in the SELinux reference policy should continue to exist or if
they should be changed. I'm following that discussion with interest.

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I'd keep it in the repo as an extension. Pushing it out would just
cause lots of trouble for little gain.

I just don't think this would be that hard if we were willing to put
our minds to it.

See... we agree on pretty much all of it in principle.

Seriously guys, this particular argument is an argument of "what?". This

is writing on the wall. If Frost and I are in this much agreement... :P

Show quoted text

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

FWIW, I don't mind which one we put in core and which one we put out of
core. But I like Joshua's idea of getting rid of contribs and pushing them
out as any other extensions.

Hmmm.

I like the contrib directory as a living example of "how to do an
extension" directly available in the source tree. It also allows to test
in-tree that the extension mechanism works. So I think it should be kept
at least with a minimum set of dummy examples for this purpose, even if
all current extensions are moved out.

Also, removing a feature is a regression, and someone is always bound to
complain... What is the real benefit? ISTM that it is a solution that
fixes no important problem. Reaching a consensus about what to move here
or there will consume valuable time that could be spent on more important
tasks... Is it worth it?

Also moving things into postgresql main sources makes pg heavier for all
and benefits only to some, so I think that some careful filtering must be
done bevore moving large contribs there. I guess this is part of the
argumentation.

--
Fabien.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment - It
doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

Regards

Pavel

2015-05-28 18:19 GMT+02:00 Joshua D. Drake <jd@commandprompt.com>:

Show quoted text

Hello,

This is a topic that has come up in various ways over the years. After the
long thread on pg_audit, I thought it might be time to bring it up again.

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in features
that are not part of the core PostgreSQL system, mainly because they
address a limited audience or are too experimental to be part of the main
source tree. This does not preclude their usefulness."

It has also been mentioned many times over the years that contrib is a
holding tank for technology that would hopefully be pushed into core
someday.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A few of
these are pretty obvious:

pg_stat_statements
citext
postgres_fdw
hstore
pg_crypto
[...]

I am sure there will be plenty of fun to be had with what should or
shouldn't be merged into core. I think if we argue about the guidelines of
how to analyze what should be in core versus the merits of any particular
module, life will be easier. Here are some for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

2. Push the rest out into a .Org project called contrib. Let those who are
interested in the technology work on them or use them. This project since
it is outside of core proper can work just like other extension projects.
Alternately, allow the maintainers push them wherever they like (Landscape,
Github, Savannah, git.postgresql.org ...).

Why I am suggesting this:

1. Less code to maintain in core
2. Eliminates the mysticism of contrib
3. Removal of experimental code from core
4. Most of the distributions package contrib separately anyway
5. Some of core is extremely small use case (sepgsql, tsearch2, lo ...)
6. Finding utilities for PostgreSQL used to be harder. It is rather dumb
simple teenage snapchat user easy now.
8. Isn't this what pgxs is for?
9. Everybody hates cleaning the closet until the end result.
10. Several of these modules would make PostgreSQL look good anyway
(default case insensitive index searching with citext? It is a gimme)
11. Contrib has been getting smaller and smaller. Let's cut the cord.
12. Isn't this the whole point of extensions?

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment - It

doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

Regards

Pavel

2015-05-28 18:19 GMT+02:00 Joshua D. Drake <jd@commandprompt.com>:

Hello,

This is a topic that has come up in various ways over the years. After

the long thread on pg_audit, I thought it might be time to bring it up
again.

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in features

that are not part of the core PostgreSQL system, mainly because they
address a limited audience or are too experimental to be part of the main
source tree. This does not preclude their usefulness."

It has also been mentioned many times over the years that contrib is a

holding tank for technology that would hopefully be pushed into core
someday.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A few

of these are pretty obvious:

pg_stat_statements
citext
postgres_fdw
hstore
pg_crypto
[...]

I am sure there will be plenty of fun to be had with what should or

shouldn't be merged into core. I think if we argue about the guidelines of
how to analyze what should be in core versus the merits of any particular
module, life will be easier. Here are some for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

2. Push the rest out into a .Org project called contrib. Let those who

are interested in the technology work on them or use them. This project
since it is outside of core proper can work just like other extension
projects. Alternately, allow the maintainers push them wherever they like
(Landscape, Github, Savannah, git.postgresql.org ...).

Why I am suggesting this:

1. Less code to maintain in core
2. Eliminates the mysticism of contrib
3. Removal of experimental code from core
4. Most of the distributions package contrib separately anyway
5. Some of core is extremely small use case (sepgsql, tsearch2, lo ...)
6. Finding utilities for PostgreSQL used to be harder. It is rather dumb

simple teenage snapchat user easy now.

8. Isn't this what pgxs is for?
9. Everybody hates cleaning the closet until the end result.
10. Several of these modules would make PostgreSQL look good anyway

(default case insensitive index searching with citext? It is a gimme)

Show quoted text

11. Contrib has been getting smaller and smaller. Let's cut the cord.
12. Isn't this the whole point of extensions?

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Le 29 mai 2015 8:01 AM, "Fabien COELHO" <coelho@cri.ensmp.fr> a écrit :

FWIW, I don't mind which one we put in core and which one we put out of
core. But I like Joshua's idea of getting rid of contribs and pushing

them

out as any other extensions.

Hmmm.

I like the contrib directory as a living example of "how to do an

extension" directly available in the source tree. It also allows to test
in-tree that the extension mechanism works. So I think it should be kept at
least with a minimum set of dummy examples for this purpose, even if all
current extensions are moved out.

Agreed.

Also, removing a feature is a regression, and someone is always bound to

complain... What is the real benefit? ISTM that it is a solution that fixes
no important problem. Reaching a consensus about what to move here or there
will consume valuable time that could be spent on more important tasks...
Is it worth it?

It would be less confusing for users. Contrib modules seem to be first
class extensions, leaving doubt on other extensions. But the fact they
aren't in core make them not fully trusted by some users. Trying to explain
all that in a training is a PITA. It would be much less confusing if they
were either in core or in their own repository.

Also moving things into postgresql main sources makes pg heavier for all

and benefits only to some, so I think that some careful filtering must be
done bevore moving large contribs there. I guess this is part of the
argumentation.

Agreed.

2015-05-29 8:20 GMT+02:00 Guillaume Lelarge <guillaume@lelarge.info>:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a
écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment -

It doesn't helps with MS Windows. Installing necessary software for
compilation there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

I afraid so dependency on EDB in this case is wrong - I have respect to EDB
due work, but installation other extension from EDB stack is difficult,
unclean, and nothing what I would to use as new base.

Show quoted text

Regards

Pavel

2015-05-28 18:19 GMT+02:00 Joshua D. Drake <jd@commandprompt.com>:

Hello,

This is a topic that has come up in various ways over the years. After

the long thread on pg_audit, I thought it might be time to bring it up
again.

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in features

that are not part of the core PostgreSQL system, mainly because they
address a limited audience or are too experimental to be part of the main
source tree. This does not preclude their usefulness."

It has also been mentioned many times over the years that contrib is a

holding tank for technology that would hopefully be pushed into core
someday.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A few

of these are pretty obvious:

pg_stat_statements
citext
postgres_fdw
hstore
pg_crypto
[...]

I am sure there will be plenty of fun to be had with what should or

shouldn't be merged into core. I think if we argue about the guidelines of
how to analyze what should be in core versus the merits of any particular
module, life will be easier. Here are some for a start:

A. Must have been in contrib for at least two releases
B. Must have visible community (and thus use case)

2. Push the rest out into a .Org project called contrib. Let those who

are interested in the technology work on them or use them. This project
since it is outside of core proper can work just like other extension
projects. Alternately, allow the maintainers push them wherever they like
(Landscape, Github, Savannah, git.postgresql.org ...).

Why I am suggesting this:

1. Less code to maintain in core
2. Eliminates the mysticism of contrib
3. Removal of experimental code from core
4. Most of the distributions package contrib separately anyway
5. Some of core is extremely small use case (sepgsql, tsearch2, lo ...)
6. Finding utilities for PostgreSQL used to be harder. It is rather

dumb simple teenage snapchat user easy now.

8. Isn't this what pgxs is for?
9. Everybody hates cleaning the closet until the end result.
10. Several of these modules would make PostgreSQL look good anyway

(default case insensitive index searching with citext? It is a gimme)

11. Contrib has been getting smaller and smaller. Let's cut the cord.
12. Isn't this the whole point of extensions?

Sincerely,

jD

--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, May 29, 2015 at 3:20 PM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment - It
doesn't helps with MS Windows. Installing necessary software for compilation
there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules in
their windows installer to help users. They already go way further than any
Linux packages.

Speaking with my Windows packager hat on, compiling with MinGW would
not be that terrible I think for extensions, as all the existing
Makefile machinery could be used for this purpose. MSVC stuff is more
complicated though with what we have in-core, but still I think that
we could do something with them if we refactor a bit the code and make
easier the declaration of Project objects and have some proper
documentation in the extension chapter, the idea being that users
should not need to build more than a simple build.pl file linking to
some of .pm files of the in-core perl module scripts, basically with a
switch to src/tools/msvc. You would need to have those modules as well
as the compiled deliverables to compile the extensions, but that's the
same deal as any devel-* package on Linux.
--
Michael

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

2015-05-29 8:54 GMT+02:00 Michael Paquier <michael.paquier@gmail.com>:

On Fri, May 29, 2015 at 3:20 PM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:

Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a

écrit :

Hi

I am not sure if PGXN can substitute contrib - mainly due deployment -

It

doesn't helps with MS Windows. Installing necessary software for

compilation

there is terrible.

I agree it's hard to compile an extension on Windows, but that's already
what we have. And I'm sure EDB will put all interesting contrib modules

in

their windows installer to help users. They already go way further than

any

Linux packages.

Speaking with my Windows packager hat on, compiling with MinGW would
not be that terrible I think for extensions, as all the existing
Makefile machinery could be used for this purpose. MSVC stuff is more
complicated though with what we have in-core, but still I think that
we could do something with them if we refactor a bit the code and make
easier the declaration of Project objects and have some proper
documentation in the extension chapter, the idea being that users
should not need to build more than a simple build.pl file linking to
some of .pm files of the in-core perl module scripts, basically with a
switch to src/tools/msvc. You would need to have those modules as well
as the compiled deliverables to compile the extensions, but that's the
same deal as any devel-* package on Linux.

I had a lot of problems with MSVC installation

Issues:

1. VS requires relatively new MS Windows - problem for people with Ms Win 7
and older

2. After installation you have to find and apply some critical fixes - some
is bad documented.

I have a few customers on MS Win - nobody was able to compile extension
there - and I had lot of trables - (but I worked with MSVC ten years ago)

Regards

Pavel

Show quoted text

--
Michael