Units in postgresql.conf

On 7/20/06, Peter Eisentraut <peter_e@gmx.net> wrote:

I think it would be useful to allow units to be added to these settings, for
example

shared_buffers = 1000kB
checkpoint_warning = 30s

This would also allow

shared_buffers = 512MB

which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would people find
this useful?

I agree, a lot of newbies have issues with the configuration file. I
have a tiny bit of code (about 20 lines I think) that will handle K,
M, and G suffixes for memory. It would be equally easy to add S for
seconds, ....

In my code, if no suffix existed, I'd just revert to the default
behavior. This is probably what we'd want to do in PostgreSQL as
well.

The only issue in PostgreSQL is knowing what the unit conversion and
scaling factor is for each parameter (8K, 1K, milliseconds, etc);
though, this wouldn't be hard to add at all.

--
Jonah H. Harris, Software Architect | phone: 732.331.1300
EnterpriseDB Corporation | fax: 732.331.1301
33 Wood Ave S, 2nd Floor | jharris@enterprisedb.com
Iselin, New Jersey 08830 | http://www.enterprisedb.com/

On Thu, 20 Jul 2006, Peter Eisentraut wrote:

One frequent source of confusion are the different units that the parameters
in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB;
bgwriter_delay is in milliseconds, checkpoint_warning is in seconds.
Obviously, we can't change that without inconveniencing a lot of users.

I think it would be useful to allow units to be added to these settings, for
example

shared_buffers = 1000kB
checkpoint_warning = 30s

This would also allow

shared_buffers = 512MB

which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would people find
this useful?

Please! Yes!

Gavin

On 7/20/06, Peter Eisentraut <peter_e@gmx.net> wrote:

One frequent source of confusion are the different units that the parameters
in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB;
bgwriter_delay is in milliseconds, checkpoint_warning is in seconds.
Obviously, we can't change that without inconveniencing a lot of users.

I think it would be useful to allow units to be added to these settings, for
example

shared_buffers = 1000kB
checkpoint_warning = 30s

This would also allow

shared_buffers = 512MB

which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would people find
this useful?

+1. In addition, that would make conffile self-documenting.

--
marko

On Thu, Jul 20, 2006 at 01:49:36PM +0200, Peter Eisentraut wrote:

One frequent source of confusion are the different units that the parameters
in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB;
bgwriter_delay is in milliseconds, checkpoint_warning is in seconds.
Obviously, we can't change that without inconveniencing a lot of users.

I think it would be useful to allow units to be added to these settings, for
example

shared_buffers = 1000kB
checkpoint_warning = 30s

This would also allow

shared_buffers = 512MB

which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would
people find this useful?

Absolutely! :)

Cheers,
D
--
David Fetter <david@fetter.org> http://fetter.org/
phone: +1 415 235 3778 AIM: dfetter666
Skype: davidfetter

Remember to vote!

Peter Eisentraut wrote:

One frequent source of confusion are the different units that the parameters
in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB;
bgwriter_delay is in milliseconds, checkpoint_warning is in seconds.
Obviously, we can't change that without inconveniencing a lot of users.

I think it would be useful to allow units to be added to these settings, for
example

shared_buffers = 1000kB
checkpoint_warning = 30s

This would also allow

shared_buffers = 512MB

which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would people find
this useful?

It is good idea. I going to implement this. There is some proposal:

Time units is easy:
1h = 60min = 3600s = 3600000ms

Memory units:
What kind of unit prefix will we use for memory? 1kB=1000B and
1kiBi=1024B or 1kB=1024kB. See
http://en.wikipedia.org/wiki/Binary_prefix for detail. I suggest use IEC
standard convention. By my opinion it is much better.

And there are some other questions:
1) will be unit required? What will be default unit for value without unit?

I suggest mandatory unit avoid the problem with unclear default value.

2) Each internal representation of setting use different unit. Shell I
convert this representation to milliseconds and bytes? I think it is not
good idea. It should generate some overflow. I suggest to recompute
value and round it to integer.

Zdenek

On Thursday 20 July 2006 05:04, Jonah H. Harris wrote:

On 7/20/06, Peter Eisentraut <peter_e@gmx.net> wrote:

I think it would be useful to allow units to be added to these settings,
for example

shared_buffers = 1000kB
checkpoint_warning = 30s

This would also allow

shared_buffers = 512MB

which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would people
find this useful?

I agree, a lot of newbies have issues with the configuration file. I
have a tiny bit of code (about 20 lines I think) that will handle K,
M, and G suffixes for memory. It would be equally easy to add S for
seconds, ....

In my code, if no suffix existed, I'd just revert to the default
behavior. This is probably what we'd want to do in PostgreSQL as
well.

The only issue in PostgreSQL is knowing what the unit conversion and
scaling factor is for each parameter (8K, 1K, milliseconds, etc);
though, this wouldn't be hard to add at all.

Yummy, Yummy, I'd say this would be a big boost in ability to tune for a lot
of people.

--
Darcy Buskermolen
Wavefire Technologies Corp.

http://www.wavefire.com
ph: 250.717.0200
fx: 250.763.1759

Peter,

One frequent source of confusion are the different units that the parameters
in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB;
bgwriter_delay is in milliseconds, checkpoint_warning is in seconds.
Obviously, we can't change that without inconveniencing a lot of users.

I think it would be useful to allow units to be added to these settings, for
example

shared_buffers = 1000kB
checkpoint_warning = 30s

This would also allow

shared_buffers = 512MB

which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would people find
this useful?

Well, it's on my TODO list for 8.2 to write a simple postgresql.conf
conversion utility in Perl. If you wanted to make a change like that,
it would make finishing that mandatory.

Just as well, right now half the vacuum settings are in a different
section than another half.

--Josh

Peter Eisentraut wrote:

One frequent source of confusion are the different units that the parameters
in postgresql.conf use. shared_buffers is in 8 kB, work_mem is in 1 kB;
bgwriter_delay is in milliseconds, checkpoint_warning is in seconds.
Obviously, we can't change that without inconveniencing a lot of users.

I think it would be useful to allow units to be added to these settings, for
example

<snip>

I haven't thought yet how to parse or implement this, but would people find
this useful?

+1

I'd find this useful myself, and I think it would eliminate many
mistakes by newer admins.

Joe

Josh Berkus wrote:

Well, it's on my TODO list for 8.2 to write a simple postgresql.conf
conversion utility in Perl. If you wanted to make a change like
that, it would make finishing that mandatory.

I don't understand how that is related. Or what a conversion utility
would be for that matter.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Zdenek Kotala wrote:

Time units is easy:
1h = 60min = 3600s = 3600000ms

We don't need anything larger than seconds at the moment.

What kind of unit prefix will we use for memory?

PostgreSQL has always used 1 kB = 1024 B.

1) will be unit required?

No.

What will be default unit for value without
unit?

What we have now.

I suggest mandatory unit avoid the problem with unclear default
value.

Not going to happen.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter,

I don't understand how that is related. Or what a conversion utility
would be for that matter.

Well, the main issue with changing the units of the PostgreSQL.conf file
from a user perspective is that the numbers from you 8.0/8.1 conf file
would no longer work. A little conversion utilitily to turn your 8.0
file into an 8.2 file would help solve that.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Josh Berkus wrote:

Well, the main issue with changing the units of the PostgreSQL.conf
file from a user perspective is that the numbers from you 8.0/8.1
conf file would no longer work.

No one is intending to do any such change.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:

Zdenek Kotala wrote:

Time units is easy:
1h = 60min = 3600s = 3600000ms

We don't need anything larger than seconds at the moment.

What kind of unit prefix will we use for memory?

PostgreSQL has always used 1 kB = 1024 B.

1) will be unit required?

No.

What will be default unit for value without
unit?

What we have now.

I suggest mandatory unit avoid the problem with unclear default
value.

Not going to happen.

Ok. Conclusion is for time s=second, ms=millisecond and for memory B,
kB, MB, GB. Unit is not mandatory and if it will missing the behavior
stays same - backward compatibility (no extra conversion utility).

Last question is if "page" unit should be useful too. For example:

#shared_buffers = 1000 # min 16 or max_connections*2,
8KB each

It means 8000kB. But if somebody compiles postgres with different page
size, than the size will be different. However, somebody should use for
example 8MB and number of buffers will be 8MB/page_size.

Zdenek

PS: I have some GUC patches in the patches queue. Could anybody
test/commit them? I would like continue on the latest version of guc
subsystem. Thanks

Peter Eisentraut wrote:

I think it would be useful to allow units to be added to these settings, for
example...
shared_buffers = 512MB
which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would people find
this useful?

Would this extend to things like "random_page_cost" and similar?

If the random_page_cost were specifiable in seconds or ms it might be easier
to someday write a program to measure such values on particular hardware
platforms. (though I guess for that to work, the config file would also
need to add the reference cost (is it a non-random page access) as well...)

On Thu, 20 Jul 2006, Josh Berkus wrote:

Peter,

I don't understand how that is related. Or what a conversion utility
would be for that matter.

Well, the main issue with changing the units of the PostgreSQL.conf file
from a user perspective is that the numbers from you 8.0/8.1 conf file
would no longer work. A little conversion utilitily to turn your 8.0
file into an 8.2 file would help solve that.

Josh,

I would imagine that Peter intends to handle backward compatibility by
processing values without explicit units in the units assumed pre <8.2.

Thanks,

Gavin

Time units is easy:
1h = 60min = 3600s = 3600000ms

We don't need anything larger than seconds at the moment.

Except for log_rotation_age perhaps?

-- Korry

Gavin, Peter,

I would imagine that Peter intends to handle backward compatibility by
processing values without explicit units in the units assumed pre <8.2.

Aha, I misunderstood.

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

On Thursday 20 July 2006 18:16, Ron Mayer wrote:

Peter Eisentraut wrote:

I think it would be useful to allow units to be added to these settings,
for example...
shared_buffers = 512MB
which is a bit cumbersome to calculate right now (you'd need = 65536).

I haven't thought yet how to parse or implement this, but would people
find this useful?

Would this extend to things like "random_page_cost" and similar?

If the random_page_cost were specifiable in seconds or ms it might be
easier to someday write a program to measure such values on particular
hardware platforms. (though I guess for that to work, the config file
would also need to add the reference cost (is it a non-random page access)
as well...)

I'd think no, since random page cost doesn't actually map to any real world
value. Unless of course we wanted to add MV for "magic value", but then
people would want to use that for everything ;-D

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL