Memory Utilization Issue

Started by Sachin Srivastavaalmost 11 years ago4 messagesgeneral

ssr.teleatlas@gmail.com

almost 11 years ago

Dear Concern,

Always my server memory utilization is remain >99%. I have 4 DB server and
RAM of the server is (32 GB, 64 GB, 64 GB and 132 GB*). In every server
always we are getting the memory utilization > 99%. *Kindly suggest why
this problem is and which parameter will resolve this problem.

I am using PostgreSQL 9.1.2 and PostGIS 1.5 version on every Red hat Linux
server. Kindly find the "TOP" result, "ulimit -a" result,
("kernel.shmax", "kernel.shmall", "kernel.sem" value) and pg_setting result
of one server as below. If you require any other information then inform to
me.

[root@CPPMOMA_DB01 ~]# top

top - 12:08:08 up 15 days, 15:45, 5 users, load average: 1.05, 1.19, 1.35

Tasks: 605 total, 1 running, 604 sleeping, 0 stopped, 0 zombie

Cpu(s): 6.9%us, 2.1%sy, 0.0%ni, 78.3%id, 12.7%wa, 0.0%hi, 0.1%si,
0.0%st
Mem: 32832364k total, 32621168k used, 211196k free, 77572k buffers

[root@CPPMOMA_DB01 ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256323
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 256323
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[root@CPPMOMA_DB01 ~]#

kernel.shmmax = 32212254720

kernel.shmall = 1073741824

kernel.sem = 250 32000 100 384

postgres=# SELECT name, source, setting FROM pg_settings WHERE source !=
'default' AND source != 'override' ORDER by 2, 1;

name | source | setting

----------------------------+----------------------+----------------------------------

application_name | client | psql.bin

client_encoding | client | UTF8

archive_command | configuration file | cp %p
/dbkup/momacpp_213_live/%f

archive_mode | configuration file | on

autovacuum | configuration file | on

autovacuum_max_workers | configuration file | 3

checkpoint_segments | configuration file | 200

checkpoint_timeout | configuration file | 300

checkpoint_warning | configuration file | 30

DateStyle | configuration file | ISO, MDY

default_text_search_config | configuration file | pg_catalog.english

effective_cache_size | configuration file | 524288

lc_messages | configuration file | en_US.UTF-8

lc_monetary | configuration file | en_US.UTF-8

lc_numeric | configuration file | en_US.UTF-8

lc_time | configuration file | en_US.UTF-8

listen_addresses | configuration file | *

log_destination | configuration file | stderr

log_directory | configuration file | pg_log

logging_collector | configuration file | on

log_line_prefix | configuration file | %t

log_rotation_age | configuration file | 1440

maintenance_work_mem | configuration file | 1638400

max_connections | configuration file | 2000

max_files_per_process | configuration file | 2000

max_wal_senders | configuration file | 5

port | configuration file | 5432

shared_buffers | configuration file | 1572864

temp_buffers | configuration file | 4096

wal_level | configuration file | archive

work_mem | configuration file | 32768

log_timezone | environment variable | Asia/Kolkata

max_stack_depth | environment variable | 2048

TimeZone | environment variable | Asia/Kolkata

(34 rows)

postgres=#

Regards,

*Sachin Srivastava*
Assistant Technical Lead(Oracle/PostgreSQL) | TSG
*Cyient* | www.cyient.com

Laurenz Albe

laurenz.albe@cybertec.at

almost 11 years ago

In reply to: Sachin Srivastava (#1)

Re: Memory Utilization Issue

Sachin Srivastava wrote:

Always my server memory utilization is remain >99%. I have 4 DB server and RAM of the server is (32
GB, 64 GB, 64 GB and 132 GB). In every server always we are getting the memory utilization > 99%.
Kindly suggest why this problem is and which parameter will resolve this problem.

I am using PostgreSQL 9.1.2 and PostGIS 1.5 version on every Red hat Linux server. Kindly find the
"TOP" result, "ulimit -a" result, ("kernel.shmax", "kernel.shmall", "kernel.sem" value) and
pg_setting result of one server as below. If you require any other information then inform to me.

That sounds just fine.
Linux uses memory for the file system cache. That memory is shown as "in use", but
it is available for processes if they need it.

It doesn't look like your machine is swapping.

Do you experience problems?

Yours,
Laurenz Albe

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Mathew Moon

mathew.moon@vipaar.com

almost 11 years ago

In reply to: Sachin Srivastava (#1)

Re: Memory Utilization Issue

What is the output of 'free -m' ? Look at the third column second row (+/- cache). This is the most relevant number. Even if you are swapping that can happen with plenty of RAM available if 'swappiness' is set too low.

Sent from my iPhone

Show quoted text

On May 20, 2015, at 2:25 AM, Sachin Srivastava <ssr.teleatlas@gmail.com> wrote:

Dear Concern,

Always my server memory utilization is remain >99%. I have 4 DB server and RAM of the server is (32 GB, 64 GB, 64 GB and 132 GB). In every server always we are getting the memory utilization > 99%. Kindly suggest why this problem is and which parameter will resolve this problem.

I am using PostgreSQL 9.1.2 and PostGIS 1.5 version on every Red hat Linux server. Kindly find the "TOP" result, "ulimit -a" result, ("kernel.shmax", "kernel.shmall", "kernel.sem" value) and pg_setting result of one server as below. If you require any other information then inform to me.

[root@CPPMOMA_DB01 ~]# top
top - 12:08:08 up 15 days, 15:45, 5 users, load average: 1.05, 1.19, 1.35
Tasks: 605 total, 1 running, 604 sleeping, 0 stopped, 0 zombie
Cpu(s): 6.9%us, 2.1%sy, 0.0%ni, 78.3%id, 12.7%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 32832364k total, 32621168k used, 211196k free, 77572k buffers

[root@CPPMOMA_DB01 ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256323
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 256323
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[root@CPPMOMA_DB01 ~]#

kernel.shmmax = 32212254720

kernel.shmall = 1073741824

kernel.sem = 250 32000 100 384

postgres=# SELECT name, source, setting FROM pg_settings WHERE source != 'default' AND source != 'override' ORDER by 2, 1;

name | source | setting
----------------------------+----------------------+----------------------------------
application_name | client | psql.bin
client_encoding | client | UTF8
archive_command | configuration file | cp %p /dbkup/momacpp_213_live/%f
archive_mode | configuration file | on
autovacuum | configuration file | on
autovacuum_max_workers | configuration file | 3
checkpoint_segments | configuration file | 200
checkpoint_timeout | configuration file | 300
checkpoint_warning | configuration file | 30
DateStyle | configuration file | ISO, MDY
default_text_search_config | configuration file | pg_catalog.english
effective_cache_size | configuration file | 524288
lc_messages | configuration file | en_US.UTF-8
lc_monetary | configuration file | en_US.UTF-8
lc_numeric | configuration file | en_US.UTF-8
lc_time | configuration file | en_US.UTF-8
listen_addresses | configuration file | *
log_destination | configuration file | stderr
log_directory | configuration file | pg_log
logging_collector | configuration file | on
log_line_prefix | configuration file | %t
log_rotation_age | configuration file | 1440
maintenance_work_mem | configuration file | 1638400
max_connections | configuration file | 2000
max_files_per_process | configuration file | 2000
max_wal_senders | configuration file | 5
port | configuration file | 5432
shared_buffers | configuration file | 1572864
temp_buffers | configuration file | 4096
wal_level | configuration file | archive
work_mem | configuration file | 32768
log_timezone | environment variable | Asia/Kolkata
max_stack_depth | environment variable | 2048
TimeZone | environment variable | Asia/Kolkata
(34 rows)

postgres=#

Regards,
Sachin Srivastava
Assistant Technical Lead(Oracle/PostgreSQL) | TSG
Cyient | www.cyient.com

Naveed Shaikh

naveed.shaikh@enterprisedb.com

almost 11 years ago

In reply to: Sachin Srivastava (#1)

Re: Memory Utilization Issue

Could you also please check the Transparent huge page(THP) are enabled on
the server or not, they can also result in intermittent poor performance
along with high system cpu time counted against the database processes.

This can be confirmed by below command:

cat /sys/kernel/mm/redhat_transparent_hugepage/enabled

Here "always" is selected, showing THP are enabled, this needs to be
disable with following command:

echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

Thanks & Regards,
Naveed Shaikh

On Wed, May 20, 2015 at 12:55 PM, Sachin Srivastava <ssr.teleatlas@gmail.com

Show quoted text

wrote:

Dear Concern,

Always my server memory utilization is remain >99%. I have 4 DB server and
RAM of the server is (32 GB, 64 GB, 64 GB and 132 GB*). In every server
always we are getting the memory utilization > 99%. *Kindly suggest why
this problem is and which parameter will resolve this problem.

I am using PostgreSQL 9.1.2 and PostGIS 1.5 version on every Red hat Linux
server. Kindly find the "TOP" result, "ulimit -a" result,
("kernel.shmax", "kernel.shmall", "kernel.sem" value) and pg_setting result
of one server as below. If you require any other information then inform to
me.

[root@CPPMOMA_DB01 ~]# top

top - 12:08:08 up 15 days, 15:45, 5 users, load average: 1.05, 1.19,
1.35

Tasks: 605 total, 1 running, 604 sleeping, 0 stopped, 0 zombie

Cpu(s): 6.9%us, 2.1%sy, 0.0%ni, 78.3%id, 12.7%wa, 0.0%hi, 0.1%si,
0.0%st
Mem: 32832364k total, 32621168k used, 211196k free, 77572k buffers

[root@CPPMOMA_DB01 ~]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256323
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 256323
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[root@CPPMOMA_DB01 ~]#

kernel.shmmax = 32212254720

kernel.shmall = 1073741824

kernel.sem = 250 32000 100 384

postgres=# SELECT name, source, setting FROM pg_settings WHERE source !=
'default' AND source != 'override' ORDER by 2, 1;

name | source | setting

----------------------------+----------------------+----------------------------------

application_name | client | psql.bin

client_encoding | client | UTF8

archive_command | configuration file | cp %p
/dbkup/momacpp_213_live/%f

archive_mode | configuration file | on

autovacuum | configuration file | on

autovacuum_max_workers | configuration file | 3

checkpoint_segments | configuration file | 200

checkpoint_timeout | configuration file | 300

checkpoint_warning | configuration file | 30

DateStyle | configuration file | ISO, MDY

default_text_search_config | configuration file | pg_catalog.english

effective_cache_size | configuration file | 524288

lc_messages | configuration file | en_US.UTF-8

lc_monetary | configuration file | en_US.UTF-8

lc_numeric | configuration file | en_US.UTF-8

lc_time | configuration file | en_US.UTF-8

listen_addresses | configuration file | *

log_destination | configuration file | stderr

log_directory | configuration file | pg_log

logging_collector | configuration file | on

log_line_prefix | configuration file | %t

log_rotation_age | configuration file | 1440

maintenance_work_mem | configuration file | 1638400

max_connections | configuration file | 2000

max_files_per_process | configuration file | 2000

max_wal_senders | configuration file | 5

port | configuration file | 5432

shared_buffers | configuration file | 1572864

temp_buffers | configuration file | 4096

wal_level | configuration file | archive

work_mem | configuration file | 32768

log_timezone | environment variable | Asia/Kolkata

max_stack_depth | environment variable | 2048

TimeZone | environment variable | Asia/Kolkata

(34 rows)

postgres=#

Regards,

*Sachin Srivastava*
Assistant Technical Lead(Oracle/PostgreSQL) | TSG
*Cyient* | www.cyient.com