Postgresql.conf cleanup

This time *with* the attachment.

On Jul 2, 2007, at 6:03 AM, Josh Berkus wrote:

(change requires restart): this phrase appears over 20 times in the
notes. This is enough times to be really repetitive and take up a
lot of scrolling space, while not actually covering all startup-
time parameters. We should either (a) remove all such notes and
rely on docs, or (b) make an annotation symbol (e.g. *R) and mark
100% of them. Votes?

Probably the #1 question I'm asked is "does this mean I need to
restart?". +1 for marking everything.
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Josh Berkus <josh@agliodbs.com> writes:

I'm working on cleaning up postgresql.conf and pg_settings for the
...

seq_scan_cost: this is independant of all of the other _costs.

So? All the other costs are independent of it, too. I don't understand
what problem you have with it.

(change requires restart): this phrase appears over 20 times in the
notes. This is enough times to be really repetitive and take up a lot
of scrolling space, while not actually covering all startup-time
parameters. We should either (a) remove all such notes and rely on
docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them.

That was put in deliberately not long ago, so I doubt (a) will pass.
(b) seems fine to me.

transaction_isolation and transaction_read_only appear more than once in
the pg_settings pseudo_table.

Not for me.

# work_mem = ( RAM * 0.5 ) / max_connections, or less

That seems guaranteed to drive people into swap hell, unless they
execute only trivial queries.

# wal_buffers = 1MB

Is there really evidence in favor of such a high setting for this,
either? (I expect the walwriter in the async-commit patch will change
the landscape here, btw.)

# max_fsm_pages = expected database size * 0.1

This might be too small.

# checkpoint_segments = 8 to 16 if you have the disk space (0.3 to 0.6 GB)

This seems definitely too small --- for write-intensive databases I like
to set it to 30 or so, which should eat about a GB if I did the
arithmetic right.

#explain_pretty_print = on

Putting this under "planner options" is wrong and illogical.

The file seems to be missing the effects of some recently committed
patches, eg, bgwriter_all_percent shouldn't be there anymore.

regards, tom lane

Tom Lane wrote:

(change requires restart): this phrase appears over 20 times in the
notes. This is enough times to be really repetitive and take up a lot
of scrolling space, while not actually covering all startup-time
parameters. We should either (a) remove all such notes and rely on
docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them.

That was put in deliberately not long ago, so I doubt (a) will pass.
(b) seems fine to me.

+1 on (b), -1 on (a)

# work_mem = ( RAM * 0.5 ) / max_connections, or less

That seems guaranteed to drive people into swap hell, unless they
execute only trivial queries.

Maybe he meant .05, which would be semi-reasonable?

# checkpoint_segments = 8 to 16 if you have the disk space (0.3 to 0.6 GB)

This seems definitely too small --- for write-intensive databases I like
to set it to 30 or so, which should eat about a GB if I did the
arithmetic right.

Hmpf, I set it to 30 just to get it out of the way. I would agree that
8-16 is too small.

Sincerely,

Joshua D. Drake

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

On Mon, 2 Jul 2007, Tom Lane wrote:

# wal_buffers = 1MB

Is there really evidence in favor of such a high setting for this,
either?

I noticed consistant improvements in throughput on pgbench results with
lots of clients going from the default to 256KB, flatlining above that; it
seemed sufficiently large for any system I've used. I've taken to using
1MB anyway nowadays because others suggested that number, and it seemed to
be well beyond the useful range and thus never likely to throttle
anything. Is there any downside to it being larger than necessary beyond
what seems like a trivial amount of additional RAM?

# checkpoint_segments = 8 to 16 if you have the disk space (0.3 to 0.6 GB)

This seems definitely too small --- for write-intensive databases I like
to set it to 30 or so, which should eat about a GB if I did the
arithmetic right.

You did--I approximate larger values in my head by saying 1GB at 30
segments and scaling up from there. But don't forget this is impacted by
the LDC change, with the segments expected to be active now

(2 + checkpoint_completion_target) * checkpoint_segments + 1

so with a default install setting the segments to 30 will creep that up to
closer to a 1.2GB footprint.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

Greg Smith <gsmith@gregsmith.com> writes:

On Mon, 2 Jul 2007, Tom Lane wrote:

# wal_buffers = 1MB

Is there really evidence in favor of such a high setting for this,
either?

I noticed consistant improvements in throughput on pgbench results with
lots of clients going from the default to 256KB, flatlining above that; it
seemed sufficiently large for any system I've used. I've taken to using
1MB anyway nowadays because others suggested that number, and it seemed to
be well beyond the useful range and thus never likely to throttle
anything. Is there any downside to it being larger than necessary beyond
what seems like a trivial amount of additional RAM?

There might be some value in keeping wal_buffers small enough to fit in
L2 cache; not sure.

But pgbench is not really the poster child for large wal_buffers,
because it consists exclusively of short transactions. The gain from
enlarging wal_buffers stops the moment it passes your largest
time-between-commits, since a commit has to flush out whatever's in
there.

There's probably not much point in arguing this now, though; once the
async commit patch is in there we will have to re-measure all the
behavior and develop new recommendations (and, quite possibly, a new
default value). The existence of the walwriter will reduce the useful
size of wal_buffers, but the existence of async commit might increase it.

regards, tom lane

Am Montag, 2. Juli 2007 13:03 schrieb Josh Berkus:

(change requires restart): this phrase appears over 20 times in the
notes. �This is enough times to be really repetitive and take up a lot
of scrolling space, while not actually covering all startup-time
parameters.

Which ones are missing?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Josh, is any of this happening for 8.3?

---------------------------------------------------------------------------

Josh Berkus wrote:

All,

I'm working on cleaning up postgresql.conf and pg_settings for the
release. Attached is a sample WIP. It's not in patch form because I'm
not done yet; I've just been editing postgresql.conf and need to fix the
docs and pg_settings to match.

Issues encountered and changes made:

PostgreSQL.conf
----------------

suggestions: added section with the 7 most important obvious settings at
the top and suggestions on how to calculate them. If people like this,
I'll add it to the Tutorial in the docs as well.

seq_scan_cost: this is independant of all of the other _costs. I can't
think of any way in which that doesn't make the whole set of costs
unmanageable. For example, if you want to change seq_scan_cost in order
to make query cost more-or-less match up with ms execution time, you
have to modify all 6 settings. If we do implement per-tablespace
costs, then we'll need per-tablespace random_page_cost as well. Or am I
missing something?

(change requires restart): this phrase appears over 20 times in the
notes. This is enough times to be really repetitive and take up a lot
of scrolling space, while not actually covering all startup-time
parameters. We should either (a) remove all such notes and rely on
docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them.
Votes?

Vacuum: all vacuum & autovacuum parameters put under their own section.

Client Cost Defaults: this section became a "catch-all" for all userset
parameters which people weren't sure what to do with. I've divided it
into logical subsections, and moved some parameters to other sections
where they logically belong (for example, explain_pretty_print belongs
in Query Tuning).

pg_settings issues
--------------------

transaction_isolation and transaction_read_only appear more than once in
the pg_settings pseudo_table. The setting column is supposed to be unique.

Given the amount of cleanup/improvement which I'm seeing as necessary
for the GUCs, I'm wondering if I put this off too long for 8.3.

--Josh

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bruce Momjian wrote:

Josh, is any of this happening for 8.3?

I would be willing to take a stab at this and submit by Monday.

Joshua D. Drake

---------------------------------------------------------------------------

Josh Berkus wrote:

All,

I'm working on cleaning up postgresql.conf and pg_settings for the
release. Attached is a sample WIP. It's not in patch form because I'm
not done yet; I've just been editing postgresql.conf and need to fix the
docs and pg_settings to match.

Issues encountered and changes made:

PostgreSQL.conf
----------------

suggestions: added section with the 7 most important obvious settings at
the top and suggestions on how to calculate them. If people like this,
I'll add it to the Tutorial in the docs as well.

seq_scan_cost: this is independant of all of the other _costs. I can't
think of any way in which that doesn't make the whole set of costs
unmanageable. For example, if you want to change seq_scan_cost in order
to make query cost more-or-less match up with ms execution time, you
have to modify all 6 settings. If we do implement per-tablespace
costs, then we'll need per-tablespace random_page_cost as well. Or am I
missing something?

(change requires restart): this phrase appears over 20 times in the
notes. This is enough times to be really repetitive and take up a lot
of scrolling space, while not actually covering all startup-time
parameters. We should either (a) remove all such notes and rely on
docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them. New tasks
Votes?

Vacuum: all vacuum & autovacuum parameters put under their own section.

Client Cost Defaults: this section became a "catch-all" for all userset
parameters which people weren't sure what to do with. I've divided it
into logical subsections, and moved some parameters to other sections
where they logically belong (for example, explain_pretty_print belongs
in Query Tuning).

pg_settings issues
--------------------

transaction_isolation and transaction_read_only appear more than once in
the pg_settings pseudo_table. The setting column is supposed to be unique.

Given the amount of cleanup/improvement which I'm seeing as necessary
for the GUCs, I'm wondering if I put this off too long for 8.3.

--Josh

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

- --

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997 http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG6WfUATb/zqfZUUQRAqK5AJ46wEGl3MklaS1Y/cdyOKtUAf15WQCdFc3y
8lmhvlh/NiLwOExeGlDH75k=
=14WD
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bruce Momjian wrote:

Josh, is any of this happening for 8.3?

Should I run with this or let it lay?

---------------------------------------------------------------------------

Josh Berkus wrote:

All,

I'm working on cleaning up postgresql.conf and pg_settings for the
release. Attached is a sample WIP. It's not in patch form because I'm
not done yet; I've just been editing postgresql.conf and need to fix the
docs and pg_settings to match.

Issues encountered and changes made:

PostgreSQL.conf
----------------

suggestions: added section with the 7 most important obvious settings at
the top and suggestions on how to calculate them. If people like this,
I'll add it to the Tutorial in the docs as well.

seq_scan_cost: this is independant of all of the other _costs. I can't
think of any way in which that doesn't make the whole set of costs
unmanageable. For example, if you want to change seq_scan_cost in order
to make query cost more-or-less match up with ms execution time, you
have to modify all 6 settings. If we do implement per-tablespace
costs, then we'll need per-tablespace random_page_cost as well. Or am I
missing something?

(change requires restart): this phrase appears over 20 times in the
notes. This is enough times to be really repetitive and take up a lot
of scrolling space, while not actually covering all startup-time
parameters. We should either (a) remove all such notes and rely on
docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them.
Votes?

Vacuum: all vacuum & autovacuum parameters put under their own section.

Client Cost Defaults: this section became a "catch-all" for all userset
parameters which people weren't sure what to do with. I've divided it
into logical subsections, and moved some parameters to other sections
where they logically belong (for example, explain_pretty_print belongs
in Query Tuning).

pg_settings issues
--------------------

transaction_isolation and transaction_read_only appear more than once in
the pg_settings pseudo_table. The setting column is supposed to be unique.

Given the amount of cleanup/improvement which I'm seeing as necessary
for the GUCs, I'm wondering if I put this off too long for 8.3.

--Josh

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

- --

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997 http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG6XLDATb/zqfZUUQRAt4eAJ93xvOvRRIWnqOgZzj1LmnZF1TvGwCfbMd9
Sm/parspTeRDOqZ7KQ3mHXM=
=Uv7U
-----END PGP SIGNATURE-----

Bruce,

Josh, is any of this happening for 8.3?

Hmmm, just the format cleanup. I haven't heard any objections, but I haven't
heard any comments on the underlying broken functionality either (like
seq_scan_cost), which are beyond me to fix.

Patch next week.

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

This has been saved for the 8.4 release:

http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---------------------------------------------------------------------------

Josh Berkus wrote:

All,

I'm working on cleaning up postgresql.conf and pg_settings for the
release. Attached is a sample WIP. It's not in patch form because I'm
not done yet; I've just been editing postgresql.conf and need to fix the
docs and pg_settings to match.

Issues encountered and changes made:

PostgreSQL.conf
----------------

suggestions: added section with the 7 most important obvious settings at
the top and suggestions on how to calculate them. If people like this,
I'll add it to the Tutorial in the docs as well.

seq_scan_cost: this is independant of all of the other _costs. I can't
think of any way in which that doesn't make the whole set of costs
unmanageable. For example, if you want to change seq_scan_cost in order
to make query cost more-or-less match up with ms execution time, you
have to modify all 6 settings. If we do implement per-tablespace
costs, then we'll need per-tablespace random_page_cost as well. Or am I
missing something?

(change requires restart): this phrase appears over 20 times in the
notes. This is enough times to be really repetitive and take up a lot
of scrolling space, while not actually covering all startup-time
parameters. We should either (a) remove all such notes and rely on
docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them.
Votes?

Vacuum: all vacuum & autovacuum parameters put under their own section.

Client Cost Defaults: this section became a "catch-all" for all userset
parameters which people weren't sure what to do with. I've divided it
into logical subsections, and moved some parameters to other sections
where they logically belong (for example, explain_pretty_print belongs
in Query Tuning).

pg_settings issues
--------------------

transaction_isolation and transaction_read_only appear more than once in
the pg_settings pseudo_table. The setting column is supposed to be unique.

Given the amount of cleanup/improvement which I'm seeing as necessary
for the GUCs, I'm wondering if I put this off too long for 8.3.

--Josh

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +