[doc fix] Correct calculation of vm.nr_hugepages

Started by Tsunakawa, Takayukialmost 8 years ago10 messages

tsunakawa.takay@jp.fujitsu.com

almost 8 years ago

1 attachment(s)

Hello,

The current PostgreSQL documentation overestimates the number of huge pages (vm.nr_hugepages) because the calculation uses the maximum virtual address space. In practice, huge pages are only used for the anonymous shared memory segment. The attached patch fixes the documentation.

FYI, Oracle presents a shell script to calculate the number of huge pages for its shared memory segments:

https://docs.oracle.com/cd/E11882_01/server.112/e10839/appi_vlm.htm#UNXAR385

Regards
Takayuki Tsunakawa

Attachments:

hugepage_size_doc.patchapplication/octet-stream; name=hugepage_size_doc.patchDownload

diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index d162acb..6e65fa7 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -1472,14 +1472,14 @@ export PG_OOM_ADJUST_VALUE=0
     the kernel setting <varname>vm.nr_hugepages</varname>. To estimate the
     number of huge pages needed, start <productname>PostgreSQL</productname>
     without huge pages enabled and check the
-    postmaster's <varname>VmPeak</varname> value, as well as the system's
+    postmaster's anonymous shared memory segment size, as well as the system's
     huge page size, using the <filename>/proc</filename> file system.  This might
     look like:
 <programlisting>
 $ <userinput>head -1 $PGDATA/postmaster.pid</userinput>
 4170
-$ <userinput>grep ^VmPeak /proc/4170/status</userinput>
-VmPeak:  6490428 kB
+$ <userinput>pmap 4170 | grep rw-s | grep zero | awk '{print $2}'</userinput>
+6490428K
 $ <userinput>grep ^Hugepagesize /proc/meminfo</userinput>
 Hugepagesize:       2048 kB
 </programlisting>

Justin Pryzby

pryzby@telsasoft.com

almost 8 years ago

In reply to: Tsunakawa, Takayuki (#1)

Re: [doc fix] Correct calculation of vm.nr_hugepages

On Mon, Feb 19, 2018 at 07:05:47AM +0000, Tsunakawa, Takayuki wrote:

The current PostgreSQL documentation overestimates the number of huge pages
(vm.nr_hugepages) because the calculation uses the maximum virtual address
space. In practice, huge pages are only used for the anonymous shared memory
segment. The attached patch fixes the documentation.

+ <userinput>pmap 4170 | grep rw-s | grep zero | awk '{print $2}'</userinput>

One can do with fewer greps:
pryzbyj@pryzbyj:~$ sudo pmap `pgrep -P 1 -u postgres` |awk '/rw-s/&&/zero/{print $2}' # check PPID=1
144848K

or:
pryzbyj@pryzbyj:~$ sudo pmap `pgrep -u postgres |sed q` |awk '/rw-s/&&/zero/{print $2}' # check any backend, not just postmaster parent
144848K

Or (this is even less clean but gives an alternative which continues to use
/proc directly):
pryzbyj@pryzbyj:~$ sudo cat /proc/`pgrep -P 1 -u postgres`/maps |awk --non-decimal-data 'BEGIN{FS="[ -]"} /zero/{a="0x"$1;b="0x"$2;print (b-a)/1024"kB"}'
144848kB

BTW when huge_pages=try, I've been verifying hugepages are in use by running:
pryzbyj@pryzbyj:~$ sudo sh -c 'cat /proc/`pgrep -u postgres -P1`/smaps |grep -c "KernelPageSize: *2048 kB"' || echo NOT FOUND
0
NOT FOUND

Justin

Tsunakawa, Takayuki

tsunakawa.takay@jp.fujitsu.com

almost 8 years ago

In reply to: Justin Pryzby (#2)

1 attachment(s)

RE: [doc fix] Correct calculation of vm.nr_hugepages

From: Justin Pryzby [mailto:pryzby@telsasoft.com]

One can do with fewer greps:
pryzbyj@pryzbyj:~$ sudo pmap `pgrep -P 1 -u postgres` |awk
'/rw-s/&&/zero/{print $2}' # check PPID=1 144848K

Thanks, I'd like to take this.

Regards
Takayuki Tsunakawa

Attachments:

hugepage_size_doc_v2.patchapplication/octet-stream; name=hugepage_size_doc_v2.patchDownload

diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index d162acb..1aed070 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -1472,14 +1472,14 @@ export PG_OOM_ADJUST_VALUE=0
     the kernel setting <varname>vm.nr_hugepages</varname>. To estimate the
     number of huge pages needed, start <productname>PostgreSQL</productname>
     without huge pages enabled and check the
-    postmaster's <varname>VmPeak</varname> value, as well as the system's
+    postmaster's anonymous shared memory segment size, as well as the system's
     huge page size, using the <filename>/proc</filename> file system.  This might
     look like:
 <programlisting>
 $ <userinput>head -1 $PGDATA/postmaster.pid</userinput>
 4170
-$ <userinput>grep ^VmPeak /proc/4170/status</userinput>
-VmPeak:  6490428 kB
+$ <userinput>pmap 4170 | awk '/rw-s/ && /zero/ {print $2}'</userinput>
+6490428K
 $ <userinput>grep ^Hugepagesize /proc/meminfo</userinput>
 Hugepagesize:       2048 kB
 </programlisting>

Robert Haas

robertmhaas@gmail.com

almost 8 years ago

In reply to: Tsunakawa, Takayuki (#3)

Re: [doc fix] Correct calculation of vm.nr_hugepages

On Mon, Feb 19, 2018 at 9:43 PM, Tsunakawa, Takayuki
<tsunakawa.takay@jp.fujitsu.com> wrote:

Thanks, I'd like to take this.

Why are these values so large? The example in the documentation shows
6490428 kB, and in my test I got 8733888 kB. But 8733888 kB = 8.3 TB!
8.3 GB would make sense, but 8.3 TB does not.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Justin Pryzby

pryzby@telsasoft.com

almost 8 years ago

In reply to: Robert Haas (#4)

Re: [doc fix] Correct calculation of vm.nr_hugepages

On Wed, Feb 21, 2018 at 03:14:57PM -0500, Robert Haas wrote:

On Mon, Feb 19, 2018 at 9:43 PM, Tsunakawa, Takayuki
<tsunakawa.takay@jp.fujitsu.com> wrote:

Thanks, I'd like to take this.

Why are these values so large? The example in the documentation shows
6490428 kB, and in my test I got 8733888 kB. But 8733888 kB = 8.3 TB!
8.3 GB would make sense, but 8.3 TB does not.

pryzbyj@pryzbyj:~$ units -t -v 8733888kB GiB
8733888kB = 8.1340671 GiB

Robert Haas

robertmhaas@gmail.com

almost 8 years ago

In reply to: Justin Pryzby (#5)

Re: [doc fix] Correct calculation of vm.nr_hugepages

On Wed, Feb 21, 2018 at 3:28 PM, Justin Pryzby <pryzby@telsasoft.com> wrote:

On Wed, Feb 21, 2018 at 03:14:57PM -0500, Robert Haas wrote:

On Mon, Feb 19, 2018 at 9:43 PM, Tsunakawa, Takayuki
<tsunakawa.takay@jp.fujitsu.com> wrote:

Thanks, I'd like to take this.

Why are these values so large? The example in the documentation shows
6490428 kB, and in my test I got 8733888 kB. But 8733888 kB = 8.3 TB!
8.3 GB would make sense, but 8.3 TB does not.

pryzbyj@pryzbyj:~$ units -t -v 8733888kB GiB
8733888kB = 8.1340671 GiB

Sigh. It would be nice if I were less stupid.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Vasundhar Boddapati

bvasundhar@gmail.com

almost 8 years ago

In reply to: Robert Haas (#6)

Re: [doc fix] Correct calculation of vm.nr_hugepages

The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: not tested
Spec compliant: not tested
Documentation: tested, passed

It works. Can be Merged.

Andres Freund

andres@anarazel.de

almost 8 years ago

In reply to: Tsunakawa, Takayuki (#3)

Re: [doc fix] Correct calculation of vm.nr_hugepages

On 2018-02-20 02:43:32 +0000, Tsunakawa, Takayuki wrote:

diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml
index d162acb..1aed070 100644
--- a/doc/src/sgml/runtime.sgml
+++ b/doc/src/sgml/runtime.sgml
@@ -1472,14 +1472,14 @@ export PG_OOM_ADJUST_VALUE=0
the kernel setting <varname>vm.nr_hugepages</varname>. To estimate the
number of huge pages needed, start <productname>PostgreSQL</productname>
without huge pages enabled and check the
-    postmaster's <varname>VmPeak</varname> value, as well as the system's
+    postmaster's anonymous shared memory segment size, as well as the system's
huge page size, using the <filename>/proc</filename> file system.  This might
look like:
<programlisting>
$ <userinput>head -1 $PGDATA/postmaster.pid</userinput>
4170
-$ <userinput>grep ^VmPeak /proc/4170/status</userinput>
-VmPeak:  6490428 kB
+$ <userinput>pmap 4170 | awk '/rw-s/ && /zero/ {print $2}'</userinput>
+6490428K
$ <userinput>grep ^Hugepagesize /proc/meminfo</userinput>
Hugepagesize:       2048 kB
</programlisting>

One disadvantage of this is that it relies on the presence of pmap,
which IIRC is not installed by default in a number of distributions. Are
we concerned about that?

Greetings,

Andres Freund

Peter Eisentraut

peter.eisentraut@2ndquadrant.com

almost 8 years ago

In reply to: Vasundhar Boddapati (#7)

Re: [doc fix] Correct calculation of vm.nr_hugepages

On 2/26/18 08:25, Vasundhar Boddapati wrote:

The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: not tested
Spec compliant: not tested
Documentation: tested, passed

It works. Can be Merged.

committed

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

#10

Peter Eisentraut

peter.eisentraut@2ndquadrant.com

almost 8 years ago

In reply to: Andres Freund (#8)

Re: [doc fix] Correct calculation of vm.nr_hugepages

On 3/1/18 04:54, Andres Freund wrote:

One disadvantage of this is that it relies on the presence of pmap,
which IIRC is not installed by default in a number of distributions. Are
we concerned about that?

It's in the same package as "top", so I think that's fine.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services