pgsql: Add temp_file_limit GUC parameter to constrain temporary file sp

Started by Tom Laneover 14 years ago8 messages
#1Tom Lane
tgl@sss.pgh.pa.us

Add temp_file_limit GUC parameter to constrain temporary file space usage.

The limit is enforced against the total amount of temp file space used by
each session.

Mark Kirkwood, reviewed by Cédric Villemain and Tatsuo Ishii

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/23e5b16c71f2706978c5f54aabd28ed23a54b6a5

Modified Files
--------------
doc/src/sgml/config.sgml | 33 +++++++++++-
src/backend/storage/file/fd.c | 69 ++++++++++++++++++++++---
src/backend/utils/errcodes.txt | 11 ++--
src/backend/utils/misc/guc.c | 15 +++++
src/backend/utils/misc/postgresql.conf.sample | 5 ++
src/include/utils/guc.h | 2 +
src/include/utils/guc_tables.h | 1 +
7 files changed, 122 insertions(+), 14 deletions(-)

#2Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#1)
Re: [COMMITTERS] pgsql: Add temp_file_limit GUC parameter to constrain temporary file sp

Tom Lane wrote:

Add temp_file_limit GUC parameter to constrain temporary file space usage.

The limit is enforced against the total amount of temp file space used by
each session.

Mark Kirkwood, reviewed by C?dric Villemain and Tatsuo Ishii

Should we document that sessions that exceed this limit generate an
error? I don't see any mention of this in the docs.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#2)
Re: [COMMITTERS] pgsql: Add temp_file_limit GUC parameter to constrain temporary file sp

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

Add temp_file_limit GUC parameter to constrain temporary file space usage.

The limit is enforced against the total amount of temp file space used by
each session.

Mark Kirkwood, reviewed by C?dric Villemain and Tatsuo Ishii

Should we document that sessions that exceed this limit generate an
error? I don't see any mention of this in the docs.

Huh? Seems like a waste of text to me. What else would happen?

regards, tom lane

#4Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#3)
Re: [COMMITTERS] pgsql: Add temp_file_limit GUC parameter to constrain temporary file sp

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

Add temp_file_limit GUC parameter to constrain temporary file space usage.

The limit is enforced against the total amount of temp file space used by
each session.

Mark Kirkwood, reviewed by C?dric Villemain and Tatsuo Ishii

Should we document that sessions that exceed this limit generate an
error? I don't see any mention of this in the docs.

Huh? Seems like a waste of text to me. What else would happen?

Well, if we exceed work_mem, we spill to temp disk. If we exceed temp
disk, we error out. Those different behaviors don't seem obvious to me.
The work_mem docs do mention is spills to disk though, so maybe it is
OK.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

#5Peter Eisentraut
peter_e@gmx.net
In reply to: Bruce Momjian (#4)
Re: Re: [COMMITTERS] pgsql: Add temp_file_limit GUC parameter to constrain temporary file sp

On sön, 2011-07-17 at 18:39 -0400, Bruce Momjian wrote:

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

Add temp_file_limit GUC parameter to constrain temporary file space usage.

The limit is enforced against the total amount of temp file space used by
each session.

Mark Kirkwood, reviewed by C?dric Villemain and Tatsuo Ishii

Should we document that sessions that exceed this limit generate an
error? I don't see any mention of this in the docs.

Huh? Seems like a waste of text to me. What else would happen?

Well, if we exceed work_mem, we spill to temp disk. If we exceed temp
disk, we error out. Those different behaviors don't seem obvious to me.
The work_mem docs do mention is spills to disk though, so maybe it is
OK.

Sounds like it would be good to document the behavior if the limit is
exceeded in each case.

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#4)
Re: Re: [COMMITTERS] pgsql: Add temp_file_limit GUC parameter to constrain temporary file sp

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

Huh? Seems like a waste of text to me. What else would happen?

Well, if we exceed work_mem, we spill to temp disk. If we exceed temp
disk, we error out. Those different behaviors don't seem obvious to me.

[ shrug... ] Feel free to change it.

regards, tom lane

#7Mark Kirkwood
mark.kirkwood@catalyst.net.nz
In reply to: Tom Lane (#6)
Re: Re: [COMMITTERS] pgsql: Add temp_file_limit GUC parameter to constrain temporary file sp

On 19/07/11 02:36, Tom Lane wrote:

Bruce Momjian<bruce@momjian.us> writes:

Tom Lane wrote:

Huh? Seems like a waste of text to me. What else would happen?

Well, if we exceed work_mem, we spill to temp disk. If we exceed temp
disk, we error out. Those different behaviors don't seem obvious to me.

[ shrug... ] Feel free to change it.

No objections from me - can't see any harm in making it very clear what
happens when the limit is exceeded :-)

#8Bruce Momjian
bruce@momjian.us
In reply to: Mark Kirkwood (#7)
1 attachment(s)
Re: Re: [COMMITTERS] pgsql: Add temp_file_limit GUC parameter to constrain temporary file sp

On Tue, Jul 19, 2011 at 10:29:45AM +1200, Mark Kirkwood wrote:

On 19/07/11 02:36, Tom Lane wrote:

Bruce Momjian<bruce@momjian.us> writes:

Tom Lane wrote:

Huh? Seems like a waste of text to me. What else would happen?

Well, if we exceed work_mem, we spill to temp disk. If we exceed temp
disk, we error out. Those different behaviors don't seem obvious to me.

[ shrug... ] Feel free to change it.

No objections from me - can't see any harm in making it very clear
what happens when the limit is exceeded :-)

Documentation patch attached and applied, and backpatched to 9.2.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

Attachments:

temp_file_limit.difftext/x-diff; charset=us-asciiDownload
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
new file mode 100644
index 074afee..216c095
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
*************** SET ENABLE_SEQSCAN TO OFF;
*** 1138,1144 ****
         <para>
          Specifies the maximum amount of disk space that a session can use
          for temporary files, such as sort and hash temporary files, or the
!         storage file for a held cursor.
          The value is specified in kilobytes, and <literal>-1</> (the
          default) means no limit.
          Only superusers can change this setting.
--- 1138,1145 ----
         <para>
          Specifies the maximum amount of disk space that a session can use
          for temporary files, such as sort and hash temporary files, or the
!         storage file for a held cursor.  A transaction attempting to exceed
!         this limit will be cancelled.
          The value is specified in kilobytes, and <literal>-1</> (the
          default) means no limit.
          Only superusers can change this setting.