A thought: should we run pgindent now?

2010/2/18 Tom Lane <tgl@sss.pgh.pa.us>:

In connection with the recent discussion of changing SearchSysCache call
format, Robert espoused the view that right now is the time when there
are a minimal number of outstanding patches that would suffer merge
problems from an invasive change.  That seems correct to me --- although
ideally everyone should be thinking "beta test" for the next few months,
we all know there will be some development going on in people's private
trees.

Which leads me to the thought that rather than postponing running
pgindent until late beta, maybe we should run it *now*, and get the
bulk of its work done for the new code in 9.0.  Then people would have
a solid base to patch against, rather than having to expect a major
merge hassle at the end of beta.

We'd probably still want to run pgindent again at the traditional
point in the cycle, but if we did one now then the final run would
only be fixing sloppiness in beta-period fixes, and it should make
relatively few changes.

I have a personal interest in this because I'm hoping to spend time
over the next few weeks reading all of the HS/SR code in detail, and
it will be nicer to look at if it's formatted to project standards;
which quite a lot of it is not at the moment.

Comments?

I think it's a good idea in general. There are of course people out
there with patches *already* that will have problems with this, but
they'll have the problem eventually anyway. The only real stopper
there is if someone (Simon would be the most likelyi I guess?) has a
big fixup change queued up or so - but if someone does, we can just
postpone until right after that one...

The followup question is of course, what do we do with fixup patches
that land *after* this? Do we run pgindent once more later in the
cycle? That should be a fairly small run in that case, so it might be
worth doing it that way?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:

2010/2/18 Tom Lane <tgl@sss.pgh.pa.us>:

Which leads me to the thought that rather than postponing running
pgindent until late beta, maybe we should run it *now*, and get the
bulk of its work done for the new code in 9.0. Then people would have
a solid base to patch against, rather than having to expect a major
merge hassle at the end of beta.

...
I think it's a good idea in general.

Yep, +1 for running pgindent now.

There are of course people out
there with patches *already* that will have problems with this, but
they'll have the problem eventually anyway. The only real stopper
there is if someone (Simon would be the most likelyi I guess?) has a
big fixup change queued up or so - but if someone does, we can just
postpone until right after that one...

It's worth noting that any patches that bit-rot because of pgindent run
can be fixed with the following procedure:

1. check out the source tree just before pgindent.
2. Apply patch
3. Run pgindent
4. Diff against source tree just after pgindent.

The followup question is of course, what do we do with fixup patches
that land *after* this? Do we run pgindent once more later in the
cycle? That should be a fairly small run in that case, so it might be
worth doing it that way?

Yeah, that was Tom's plan.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

2010/2/18 Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>:

Magnus Hagander wrote:

There are of course people out
there with patches *already* that will have problems with this, but
they'll have the problem eventually anyway. The only real stopper
there is if someone (Simon would be the most likelyi I guess?) has a
big fixup change queued up or so - but if someone does, we can just
postpone until right after that one...

It's worth noting that any patches that bit-rot because of pgindent run
 can be fixed with the following procedure:

1. check out the source tree just before pgindent.
2. Apply patch
3. Run pgindent
4. Diff against source tree just after pgindent.

Doesn't that require that all pgindent runs produce the same output?
Which they generally don't due to different sets of typedefs and
stuff? It's a solvable problem of course, but not quite as simple as
you make it sound :-)

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:

2010/2/18 Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>:

It's worth noting that any patches that bit-rot because of pgindent run
can be fixed with the following procedure:

1. check out the source tree just before pgindent.
2. Apply patch
3. Run pgindent
4. Diff against source tree just after pgindent.

Doesn't that require that all pgindent runs produce the same output?
Which they generally don't due to different sets of typedefs and
stuff? It's a solvable problem of course, but not quite as simple as
you make it sound :-)

True. So everyone will have to send their patches to Bruce for bit-rot
fixing ;-)

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Feb 17, 2010, at 11:34 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Which leads me to the thought that rather than postponing running
pgindent until late beta, maybe we should run it *now*, and get the
bulk of its work done for the new code in 9.0. Then people would have
a solid base to patch against, rather than having to expect a major
merge hassle at the end of beta.

I was going to write an email proposing something along these lines,
too - so +1 from me.

...Robert

Magnus Hagander wrote:

Doesn't that require that all pgindent runs produce the same output?
Which they generally don't due to different sets of typedefs and
stuff? It's a solvable problem of course, but not quite as simple as
you make it sound :-)

The typedef file emitted by the buildfarm is supposed to be rather
static, no?

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Tom Lane wrote:

Which leads me to the thought that rather than postponing running
pgindent until late beta, maybe we should run it *now*, and get the
bulk of its work done for the new code in 9.0. Then people would have
a solid base to patch against, rather than having to expect a major
merge hassle at the end of beta.

(BTW, by "now" I don't mean *today* --- seems better to wait till
alpha4 has been wrapped. But as soon as that's done.)

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and
then get this better whacked into shape for another run at the more
traditional time.

cheers

andrew

Andrew Dunstan wrote:

Tom Lane wrote:

Which leads me to the thought that rather than postponing running
pgindent until late beta, maybe we should run it *now*, and get the
bulk of its work done for the new code in 9.0. Then people would have
a solid base to patch against, rather than having to expect a major
merge hassle at the end of beta.

(BTW, by "now" I don't mean *today* --- seems better to wait till
alpha4 has been wrapped. But as soon as that's done.)

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and
then get this better whacked into shape for another run at the more
traditional time.

I am ready to run pgindent whenever requested.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Alvaro Herrera wrote:

Magnus Hagander wrote:

Doesn't that require that all pgindent runs produce the same output?
Which they generally don't due to different sets of typedefs and
stuff? It's a solvable problem of course, but not quite as simple as
you make it sound :-)

The typedef file emitted by the buildfarm is supposed to be rather
static, no?

Umm, static in what sense? Clearly if we add things to the code that can
involve extra typedefs being found. The buildfarm's list is the union of
all the typedefs found by the contributing members at the time they do
their runs.

cheers

andrew

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Magnus Hagander wrote:

Doesn't that require that all pgindent runs produce the same output?

True. So everyone will have to send their patches to Bruce for bit-rot
fixing ;-)

I think Bruce ought to publish the specific typedef list he uses for
each run (maybe commit it in src/tools/pgindent/). In principle,
at least, that's the only variable that couldn't be reproduced by
someone else.

regards, tom lane

Andrew Dunstan wrote:

Alvaro Herrera wrote:

Magnus Hagander wrote:

Doesn't that require that all pgindent runs produce the same output?
Which they generally don't due to different sets of typedefs and
stuff? It's a solvable problem of course, but not quite as simple as
you make it sound :-)

The typedef file emitted by the buildfarm is supposed to be rather
static, no?

Umm, static in what sense? Clearly if we add things to the code that
can involve extra typedefs being found. The buildfarm's list is the
union of all the typedefs found by the contributing members at the
time they do their runs.

Yeah, but most typedefs are already there; I mean we're not likely to
change even 5% of the current list.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Tom Lane wrote:

Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:

Magnus Hagander wrote:

Doesn't that require that all pgindent runs produce the same output?

True. So everyone will have to send their patches to Bruce for bit-rot
fixing ;-)

I think Bruce ought to publish the specific typedef list he uses for
each run (maybe commit it in src/tools/pgindent/). In principle,
at least, that's the only variable that couldn't be reproduced by
someone else.

OK, we can put it in src/tools/pgindent, I guess.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and
then get this better whacked into shape for another run at the more
traditional time.

I am ready to run pgindent whenever requested.

There is an updated typedefs list at
<http://www.pgbuildfarm.org/cgi-bin/typedefs.pl&gt;

cheers

andrew

On Thu, Feb 18, 2010 at 11:51 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Bruce Momjian wrote:

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and then
get this better whacked into shape for another run at the more traditional
time.

I am ready to run pgindent whenever requested.

There is an updated typedefs list at
<http://www.pgbuildfarm.org/cgi-bin/typedefs.pl&gt;

Bruce, you want to go ahead and do this? The sooner the better, AFAICS.

...Robert

Robert Haas wrote:

On Thu, Feb 18, 2010 at 11:51 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Bruce Momjian wrote:

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and then
get this better whacked into shape for another run at the more traditional
time.

I am ready to run pgindent whenever requested.

There is an updated typedefs list at
<http://www.pgbuildfarm.org/cgi-bin/typedefs.pl&gt;

Bruce, you want to go ahead and do this? The sooner the better, AFAICS.

I am ready.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com
  PG East:  http://www.enterprisedb.com/community/nav-pg-east-2010.do
  + If your life is a hard drive, Christ can be your backup. +

On Thu, Feb 25, 2010 at 3:17 PM, Bruce Momjian <bruce@momjian.us> wrote:

Robert Haas wrote:

On Thu, Feb 18, 2010 at 11:51 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Bruce Momjian wrote:

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and then
get this better whacked into shape for another run at the more traditional
time.

I am ready to run pgindent whenever requested.

There is an updated typedefs list at
<http://www.pgbuildfarm.org/cgi-bin/typedefs.pl&gt;

Bruce, you want to go ahead and do this?  The sooner the better, AFAICS.

I am ready.

Does that mean you're going to do it, or are you waiting for some sort
of OK? I believe everyone who expressed an opinion is in favor.

...Robert

Robert Haas wrote:

On Thu, Feb 25, 2010 at 3:17 PM, Bruce Momjian <bruce@momjian.us> wrote:

Robert Haas wrote:

On Thu, Feb 18, 2010 at 11:51 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Bruce Momjian wrote:

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and then
get this better whacked into shape for another run at the more traditional
time.

I am ready to run pgindent whenever requested.

There is an updated typedefs list at
<http://www.pgbuildfarm.org/cgi-bin/typedefs.pl&gt;

Bruce, you want to go ahead and do this? ?The sooner the better, AFAICS.

I am ready.

Does that mean you're going to do it, or are you waiting for some sort
of OK? I believe everyone who expressed an opinion is in favor.

I was waiting a few hours to get feedback. I will do it at 0100 GMT
(2000 EST).

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com
  PG East:  http://www.enterprisedb.com/community/nav-pg-east-2010.do
  + If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <bruce@momjian.us> writes:

Robert Haas wrote:

Does that mean you're going to do it, or are you waiting for some sort
of OK? I believe everyone who expressed an opinion is in favor.

I was waiting a few hours to get feedback. I will do it at 0100 GMT
(2000 EST).

You can do it now as far as I'm concerned --- my next bit of work is in
the back branches anyway.

regards, tom lane

On Thu, Feb 25, 2010 at 4:48 PM, Bruce Momjian <bruce@momjian.us> wrote:

Robert Haas wrote:

On Thu, Feb 25, 2010 at 3:17 PM, Bruce Momjian <bruce@momjian.us> wrote:

Robert Haas wrote:

On Thu, Feb 18, 2010 at 11:51 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Bruce Momjian wrote:

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and then
get this better whacked into shape for another run at the more traditional
time.

I am ready to run pgindent whenever requested.

There is an updated typedefs list at
<http://www.pgbuildfarm.org/cgi-bin/typedefs.pl&gt;

Bruce, you want to go ahead and do this? ?The sooner the better, AFAICS.

I am ready.

Does that mean you're going to do it, or are you waiting for some sort
of OK?  I believe everyone who expressed an opinion is in favor.

I was waiting a few hours to get feedback.  I will do it at 0100 GMT
(2000 EST).

OK, great! Thanks for the clarification.

...Robert

Robert Haas wrote:

On Thu, Feb 25, 2010 at 4:48 PM, Bruce Momjian <bruce@momjian.us> wrote:

Robert Haas wrote:

On Thu, Feb 25, 2010 at 3:17 PM, Bruce Momjian <bruce@momjian.us> wrote:

Robert Haas wrote:

On Thu, Feb 18, 2010 at 11:51 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

Bruce Momjian wrote:

I have a TODO on fixing some of the typedef finding. But I can generate
an up to date version of the list Bruce last used in a day or two, and then
get this better whacked into shape for another run at the more traditional
time.

I am ready to run pgindent whenever requested.

There is an updated typedefs list at
<http://www.pgbuildfarm.org/cgi-bin/typedefs.pl&gt;

Bruce, you want to go ahead and do this? ?The sooner the better, AFAICS.

I am ready.

Does that mean you're going to do it, or are you waiting for some sort
of OK? ?I believe everyone who expressed an opinion is in favor.

I was waiting a few hours to get feedback. ?I will do it at 0100 GMT
(2000 EST).

OK, great! Thanks for the clarification.

Done. The diff is here:

http://momjian.us/tmp/pgindent.diff

and I checked into CVS a copy of the typedef list I used from Andrew
Dunstan.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com
  PG East:  http://www.enterprisedb.com/community/nav-pg-east-2010.do
  + If your life is a hard drive, Christ can be your backup. +

On Thu, Feb 25, 2010 at 9:03 PM, Bruce Momjian <bruce@momjian.us> wrote:

Done.  The diff is here:

       http://momjian.us/tmp/pgindent.diff

and I checked into CVS a copy of the typedef list I used from Andrew
Dunstan.

Cool, thanks. Let the rebasing (if any) begin.

...Robert

Bruce Momjian wrote:

The diff is here:

http://momjian.us/tmp/pgindent.diff

and I checked into CVS a copy of the typedef list I used from Andrew
Dunstan.

Is it worthwhile to consider writing up a "how to run pgindent like
Bruce does" page on the wiki? I've been scared off of that topic before
because it always seemed like it was perilous and there were missing
pieces. I think we'd get some uptake from contributors doing this
themselves before submitting patches if it were made easy enough for
them to do. I know I'd do it just to get rid of the accidental
tab/space errors that always manage to sneak into non-trivial work, if I
could automate the whole thing as a pass to do just before generating a
final diff to submit.

--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com www.2ndQuadrant.us

Greg Smith wrote:

Bruce Momjian wrote:

The diff is here:

http://momjian.us/tmp/pgindent.diff

and I checked into CVS a copy of the typedef list I used from Andrew
Dunstan.

Is it worthwhile to consider writing up a "how to run pgindent like
Bruce does" page on the wiki? I've been scared off of that topic before
because it always seemed like it was perilous and there were missing
pieces. I think we'd get some uptake from contributors doing this
themselves before submitting patches if it were made easy enough for
them to do. I know I'd do it just to get rid of the accidental
tab/space errors that always manage to sneak into non-trivial work, if I
could automate the whole thing as a pass to do just before generating a
final diff to submit.

Have you read the README in src/tools/pgindent?

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com
  PG East:  http://www.enterprisedb.com/community/nav-pg-east-2010.do
  + If your life is a hard drive, Christ can be your backup. +

I was just trying out 9.0a4 and I noticed. That I can't connect to
pgbouncer with psql from 9.0a4 as a result of the set application_name
changes to psql.

I imagine this is really mostly pgbouncer's problem, but I couldn't
figure out how if it was possible to make psql not set application_name
from the psql man page. If not, at least from a users point of view psql
9.0a4 looks incompatible with existing versions of pgbouncer in at least
in this configuration that works with an older versions of psql.

... or, is that a crazy way to look at it?

Garick

Details are below..

My setup (Solaris 10 / sparc )

[ psql 9.0a4 ] -> [ pgbouncer 1.3.1 ] -> [ postgres 9.0a4 ]

$ /usr/local/var/postgres/9.0a4/bin/psql -h 127.0.0.1 -p 6543
psql: ERROR: Unknown startup parameter

..pgbouncer's log..
2010-02-25 21:56:29.721 6979 WARNING C-71d48: (nodb)/(nouser)@127.0.0.1:54000 unsupported startup parameter: application_name=psql
2010-02-25 21:56:29.721 6979 LOG C-71d48: (nodb)/(nouser)@127.0.0.1:54000 closing because: Unknown startup parameter (age=0)
2010-02-25 21:56:29.722 6979 LOG C-71d48: (nodb)/(nouser)@127.0.0.1:54000 Pooler Error: Unknown startup parameter

On Fri, Feb 26, 2010 at 11:26 AM, Garick Hamlin <ghamlin@isc.upenn.edu> wrote:

I imagine this is really mostly pgbouncer's problem

yes it is, but you can workaround it using "ignore_startup_parameters"
parameter in pgbouncer.ini
http://pgbouncer.projects.postgresql.org/doc/config.html#toc4

--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

Hi Garick,

Add an ignore_startup_parameters to your pgbouncer.ini file
with application_name.

Cheers,
Ken

On Fri, Feb 26, 2010 at 11:38:20AM -0500, Jaime Casanova wrote:

On Fri, Feb 26, 2010 at 11:26 AM, Garick Hamlin <ghamlin@isc.upenn.edu> wrote:

I imagine this is really mostly pgbouncer's problem

yes it is, but you can workaround it using "ignore_startup_parameters"
parameter in pgbouncer.ini
http://pgbouncer.projects.postgresql.org/doc/config.html#toc4

Thanks!
I am new to pgbouncer, sorry if this was obvious.

Garick

On Fri, Feb 26, 2010 at 12:23 PM, Garick Hamlin <ghamlin@isc.upenn.edu> wrote:

I am new to pgbouncer, sorry if this was obvious.

if you never ask you never learn... nothing to be sorry about... well,
besides the fact that you don't read the whole manual before trying
it, but just a few months a ago one postgres hacker was surprised
because i have read the entire postgres documentation

--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157