memory problem with refresh materialized view

Started by Enrico Pirozziabout 10 years ago12 messagesgeneral
Jump to latest
#1Enrico Pirozzi
sscotty71@gmail.com

Hi
I have a new postgresql 9.5.0 installation on a new virtual server debian
8.3 x64 with 4gb RAM, I have compiled postgresql from source.

When I import a dump with materialized views I see that postgres process
takes about all 4 Gb and then I have this error

fork: Cannot allocate memory

Can anyone help me?

Thanks in advantage :)

Enrico

--
PostgreSQL openday Rimini 27-02-2016
<http://www.postgresql.org/about/event/1861/&gt;

Enrico Pirozzi
Tel. +39 0861 1855771 - Mob.+39 328 4164437 - Fax +39 0861 1850310
http://www.pgtraining.com <http://www.pgtraining.com-&gt; -
info@pgtraining.com
www.enricopirozzi.info - info@enricopirozzi.info
Skype sscotty71 - Gtalk sscotty71@gmail.com

#2Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Enrico Pirozzi (#1)
Re: memory problem with refresh materialized view

Enrico Pirozzi wrote:

I have a new postgresql 9.5.0 installation on a new virtual server debian 8.3 x64 with 4gb RAM, I have
compiled postgresql from source.

When I import a dump with materialized views I see that postgres process takes about all 4 Gb and then
I have this error

fork: Cannot allocate memory

What are the memory related settings of the database into which you restore?

Yours,
Laurenz Albe

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#3Peter van Eck
peter@vaneckzone.net
In reply to: Laurenz Albe (#2)
PostgreSQL 9.5 and process REST calls enquiry

Hi, We are looking into setting up a PostgreSQL environment for an
application that inputs JSON through via http rest calls.

Now that we havent been using Postgres for these kind of setups I was
wondering what the course of action is in setting this up.

The development team is coding this in srpingboot with an embedded
mongodb version. Do we have to configure pgrest for instance to enable
postgres for processing JSON rest calls via HTTP or is that not necessary ?
Again just puzzling in how to approach this...

thanks in advance.

Peter

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#4David G. Johnston
david.g.johnston@gmail.com
In reply to: Peter van Eck (#3)
Re: PostgreSQL 9.5 and process REST calls enquiry

On Fri, Feb 12, 2016 at 4:00 AM, Peter van Eck <peter@vaneckzone.net> wrote:

Hi, We are looking into setting up a PostgreSQL environment for an
application that inputs JSON through via http rest calls.

Now that we havent been using Postgres for these kind of setups I was
wondering what the course of action is in setting this up.

The development team is coding this in srpingboot with an embedded mongodb
version. Do we have to configure pgrest for instance to enable postgres for
processing JSON rest calls via HTTP or is that not necessary ?
Again just puzzling in how to approach this...

​PostgreSQL doesn't speak HTTP so you will need some piece of
software/middleware that is capable of speaking both HTTP and PostgreSQL.​
You already have eas access to a HTTP understanding container in
Spring/Java and a JDBC driver but there may be some extensions and/or
applications out there - like this pgrest you speak of though its GitHub
page shows little recent activity - that can provide an abstraction layer
for you. I'll leave it to your favorite search engine and others to
provide recommendations in this area.

David J.

#5Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Peter van Eck (#3)
Re: PostgreSQL 9.5 and process REST calls enquiry

On 02/12/2016 03:00 AM, Peter van Eck wrote:

Hi, We are looking into setting up a PostgreSQL environment for an
application that inputs JSON through via http rest calls.

Now that we havent been using Postgres for these kind of setups I was
wondering what the course of action is in setting this up.

The development team is coding this in srpingboot with an embedded
mongodb version. Do we have to configure pgrest for instance to enable
postgres for processing JSON rest calls via HTTP or is that not necessary ?
Again just puzzling in how to approach this...

The below looks like a good start:

http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html

thanks in advance.

Peter

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#6James Keener
jim@jimkeener.com
In reply to: Adrian Klaver (#5)
Re: PostgreSQL 9.5 and process REST calls enquiry

https://github.com/begriffs/postgrest also looks interesting!

On Fri, Feb 12, 2016 at 10:24 AM, Adrian Klaver <adrian.klaver@aklaver.com>
wrote:

Show quoted text

On 02/12/2016 03:00 AM, Peter van Eck wrote:

Hi, We are looking into setting up a PostgreSQL environment for an
application that inputs JSON through via http rest calls.

Now that we havent been using Postgres for these kind of setups I was
wondering what the course of action is in setting this up.

The development team is coding this in srpingboot with an embedded
mongodb version. Do we have to configure pgrest for instance to enable
postgres for processing JSON rest calls via HTTP or is that not necessary
?
Again just puzzling in how to approach this...

The below looks like a good start:

http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html

thanks in advance.

Peter

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#7Adrian Klaver
adrian.klaver@aklaver.com
In reply to: James Keener (#6)
Re: PostgreSQL 9.5 and process REST calls enquiry

On 02/12/2016 07:43 AM, James Keener wrote:

https://github.com/begriffs/postgrest also looks interesting!

I thought the purpose of Spring/Spring Boot was to provide the REST
services in front of your choice of data store. Not sure how putting
another server in the stack is going to help things.

On Fri, Feb 12, 2016 at 10:24 AM, Adrian Klaver
<adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>> wrote:

On 02/12/2016 03:00 AM, Peter van Eck wrote:

Hi, We are looking into setting up a PostgreSQL environment for an
application that inputs JSON through via http rest calls.

Now that we havent been using Postgres for these kind of setups
I was
wondering what the course of action is in setting this up.

The development team is coding this in srpingboot with an embedded
mongodb version. Do we have to configure pgrest for instance to
enable
postgres for processing JSON rest calls via HTTP or is that not
necessary ?
Again just puzzling in how to approach this...

The below looks like a good start:

http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-sql.html

thanks in advance.

Peter

--
Adrian Klaver
adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org
<mailto:pgsql-general@postgresql.org>)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#8James Keener
jim@jimkeener.com
In reply to: Adrian Klaver (#7)
Re: PostgreSQL 9.5 and process REST calls enquiry

https://github.com/begriffs/postgrest also looks interesting!

I thought the purpose of Spring/Spring Boot was to provide the REST
services in front of your choice of data store. Not sure how putting
another server in the stack is going to help things.

I was simply responding to the original question about accessing the DB
over HTTP.

#9Adrian Klaver
adrian.klaver@aklaver.com
In reply to: James Keener (#8)
Re: PostgreSQL 9.5 and process REST calls enquiry

On 02/12/2016 08:06 AM, James Keener wrote:

https://github.com/begriffs/postgrest also looks interesting!

I thought the purpose of Spring/Spring Boot was to provide the REST
services in front of your choice of data store. Not sure how putting
another server in the stack is going to help things.

I was simply responding to the original question about accessing the DB
over HTTP.

Then I am not sure what the question is.

Do you want to know how to use Spring Boot with Postgres or do you want
suggestions for programs/frameworks/applications that provide REST
services over Postgres?

Or is something else that I am entirely missing:)?

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#10Peter van Eck
peter@vaneckzone.net
In reply to: Adrian Klaver (#9)
Re: PostgreSQL 9.5 and process REST calls enquiry

On 12/02/16 17:34, Adrian Klaver wrote:

On 02/12/2016 08:06 AM, James Keener wrote:

https://github.com/begriffs/postgrest also looks interesting!

I thought the purpose of Spring/Spring Boot was to provide the REST
services in front of your choice of data store. Not sure how putting
another server in the stack is going to help things.

I was simply responding to the original question about accessing the DB
over HTTP.

Then I am not sure what the question is.

Do you want to know how to use Spring Boot with Postgres or do you
want suggestions for programs/frameworks/applications that provide
REST services over Postgres?

Or is something else that I am entirely missing:)?

Hi Adrian, James,

Thanks for your replies sofar.

It is exactly my point that for us the request from the developers
wasn't clear to us as we don't have the knowledge to apparently fully
understand the question.
I think it is as you describe Adrian "Do you want to know how to use
Spring Boot with Postgres"
So we'll go with that and see when the first release is pushed from the
developers.

thnaks again.

Peter

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#11Stefan Keller
sfkeller@gmail.com
In reply to: Peter van Eck (#10)
Re: PostgreSQL 9.5 and process REST calls enquiry

I'd like to extend (and complete) this thread by collecting
programs/frameworks/applications that provide REST services over
Postgres.

I see following categories and open source web frameworks and/or libs:
Those which
* attach Postgres directly, like PostgREST (Haskell) or jsgrest
(JavaScript serverside).
* connect to (m)any db inc. Postgres, like Spring Boot (Java EE) or
sandman2 (Python)

PostgREST looks promising.
But since Postgres 9.5 ff. has good JSON support (and given I like
Python) I'd prefer an even more lightweight solution.
Any suggestions?

:Stefan

2016-02-12 17:46 GMT+01:00 Peter van Eck <peter@vaneckzone.net>:

On 12/02/16 17:34, Adrian Klaver wrote:

On 02/12/2016 08:06 AM, James Keener wrote:

https://github.com/begriffs/postgrest also looks interesting!

I thought the purpose of Spring/Spring Boot was to provide the REST
services in front of your choice of data store. Not sure how putting
another server in the stack is going to help things.

I was simply responding to the original question about accessing the DB
over HTTP.

Then I am not sure what the question is.

Do you want to know how to use Spring Boot with Postgres or do you want
suggestions for programs/frameworks/applications that provide REST services
over Postgres?

Or is something else that I am entirely missing:)?

Hi Adrian, James,

Thanks for your replies sofar.

It is exactly my point that for us the request from the developers wasn't
clear to us as we don't have the knowledge to apparently fully understand
the question.
I think it is as you describe Adrian "Do you want to know how to use Spring
Boot with Postgres"
So we'll go with that and see when the first release is pushed from the
developers.

thnaks again.

Peter

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

#12Adrian Klaver
adrian.klaver@aklaver.com
In reply to: Stefan Keller (#11)
Re: PostgreSQL 9.5 and process REST calls enquiry

On 03/05/2016 05:34 PM, Stefan Keller wrote:

I'd like to extend (and complete) this thread by collecting
programs/frameworks/applications that provide REST services over
Postgres.

I see following categories and open source web frameworks and/or libs:
Those which
* attach Postgres directly, like PostgREST (Haskell) or jsgrest
(JavaScript serverside).
* connect to (m)any db inc. Postgres, like Spring Boot (Java EE) or
sandman2 (Python)

PostgREST looks promising.
But since Postgres 9.5 ff. has good JSON support (and given I like
Python) I'd prefer an even more lightweight solution.
Any suggestions?

Recently ran across the below, have not had time to actually try it out yet:

https://github.com/timothycrosley/hug

:Stefan

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general