ERROR: invalid memory alloc request size 1073741824

Started by Suhal Vemuabout 8 years ago4 messageshackers
Jump to latest
#1Suhal Vemu
suhal@cropextechnology.com

Hi,

I have a tif image of 483 mb, i am uploading the raster into postgis using

"*raster2pgsql -I -C
/home/XXXXXX/Downloads/20161207T055222_20161207T055323_T42QYL/20161207T055222_20161207T055323_T42QYL.ndvi.tif
postgis.gistesttable6 |psql -h localhost -U XXXX -d gisdb -p 5432*"

i have checked with tiles by setting -t auto but when aggregating results
of titles using st_union my system is getting hung.

so, i need to import the tif file of memory greater than 500mb at least .

the error that i am getting is

*ERROR:*
*BEGIN*
*CREATE TABLE*
*ERROR: invalid memory alloc request size 1073741824*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ROLLBACK*

please help me with the pointers and suggestions to fix this

--
Thanks,
Suhal Vemu

--
Please note that this email, including any attachments, is intended solely
for the individual (s) or entity (ies) to whom they are addressed and may
contain information that is private, confidential and privileged. In case
you are not the intended recipient, request you to notify the sender by
reply mail and delete this email, including any copies or attachments from
your system. Any unauthorized dissemination, disclosure and/or use of the
contents of this communication to anyone is strictly prohibited and
punishable by law.

#2David Rowley
dgrowleyml@gmail.com
In reply to: Suhal Vemu (#1)
Re: ERROR: invalid memory alloc request size 1073741824

On 6 April 2018 at 22:06, Suhal Vemu <suhal@cropextechnology.com> wrote:

ERROR:
BEGIN
CREATE TABLE
ERROR: invalid memory alloc request size 1073741824

Can we see the full CREATE TABLE command?

Are you able to get the CREATE TABLE to succeed if you try removing
some combination of columns? Start with the PostGIS ones. Are you able
to determine if it's a single column or type which is causing the
issue?

Which PostgreSQL version? the output of SELECT version(); would be good.

Please note that this email, including any attachments, is intended solely
for the individual (s) or entity (ies) to whom they are addressed and may
contain information that is private, confidential and privileged. In case
you are not the intended recipient, request you to notify the sender by
reply mail and delete this email, including any copies or attachments from
your system. Any unauthorized dissemination, disclosure and/or use of the
contents of this communication to anyone is strictly prohibited and
punishable by law.

This mailing list is public.

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Suhal Vemu (#1)
Re: ERROR: invalid memory alloc request size 1073741824

Suhal Vemu <suhal@cropextechnology.com> writes:

so, i need to import the tif file of memory greater than 500mb at least .

If you're trying to cram that into a single bytea field, it's unsurprising
that it fails. PG is not designed to work with table rows (let alone
individual fields) that exceed some not-very-large fraction of 1GB.
You could stream the value into a "large object", perhaps.

regards, tom lane

#4Suhal Vemu
suhal@cropextechnology.com
In reply to: Suhal Vemu (#1)
Re: ERROR: invalid memory alloc request size 1073741824

Hi all,

This is gisdb and my POSTGIS VERSION
------------------------------------------------------------
---------------------------------
*gisdb=# SELECT PostGIS_version();*
* postgis_version *
*---------------------------------------*
* 2.4 USE_GEOS=1 USE_PROJ=1 USE_STATS=1*
*(1 row)*
*---------------------------------------------------------------------------------------------*
*It didn't create the table it rolled back .*

*Actually the table creating command is: *

*raster2pgsql -I -C
/home/XXXXX/Downloads/20161207T055222_20161207T055323_T42QYL/20161207T055222_20161207T055323_T42QYL.ndvi.tif
postgis.gistesttable6 |psql -h localhost -U XXXX -d gisdb -p 5432*

*output :*
*______________________________________________________________________________________________________________________________*
*:~$ raster2pgsql -I -C
/home/XXXXX/Downloads/20161207T055222_20161207T055323_T42QYL/20161207T055222_20161207T055323_T42QYL.ndvi.tif
postgis.gistesttable6 |psql -h localhost -U XXX -d gisdb -p 5432*
*Processing 1/1:
/home/suhalvemu/Downloads/20161207T055222_20161207T055323_T42QYL/20161207T055222_20161207T055323_T42QYL.ndvi.tif*
*Password for user cropin_suhal: *
*BEGIN*
*CREATE TABLE*
*ERROR: invalid memory alloc request size 1073741824*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ROLLBACK*

On Wed, Apr 11, 2018 at 3:16 PM, Suhal Vemu <suhal@cropextechnology.com>
wrote:

Hi David,

This is gisdb and my POSTGIS VERSION
------------------------------------------------------------
---------------------------------
*gisdb=# SELECT PostGIS_version();*
* postgis_version *
*---------------------------------------*
* 2.4 USE_GEOS=1 USE_PROJ=1 USE_STATS=1*
*(1 row)*

*---------------------------------------------------------------------------------------------*
*It didn't create the table it rolled back .*

*Actually the table creating command is: *

*raster2pgsql -I -C
/home/XXXXX/Downloads/20161207T055222_20161207T055323_T42QYL/20161207T055222_20161207T055323_T42QYL.ndvi.tif
postgis.gistesttable6 |psql -h localhost -U XXXX -d gisdb -p 5432*

*output :*

*______________________________________________________________________________________________________________________________*
*:~$ raster2pgsql -I -C
/home/XXXXX/Downloads/20161207T055222_20161207T055323_T42QYL/20161207T055222_20161207T055323_T42QYL.ndvi.tif
postgis.gistesttable6 |psql -h localhost -U XXX -d gisdb -p 5432*
*Processing 1/1:
/home/suhalvemu/Downloads/20161207T055222_20161207T055323_T42QYL/20161207T055222_20161207T055323_T42QYL.ndvi.tif*
*Password for user cropin_suhal: *
*BEGIN*
*CREATE TABLE*
*ERROR: invalid memory alloc request size 1073741824*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ERROR: current transaction is aborted, commands ignored until end of
transaction block*
*ROLLBACK*

On Wed, Apr 11, 2018 at 2:34 PM, David Rowley <
david.rowley@2ndquadrant.com> wrote:

On 7 April 2018 at 02:14, Suhal Vemu <suhal@cropextechnology.com> wrote:

Hi David,

This is gisdb and my POSTGIS VERSION
------------------------------------------------------------

---------------------------------

gisdb=# SELECT PostGIS_version();
postgis_version

Hi Suhal,

Please reply including the list. You've not really shown the
information I asked for. Best to reply to the list with that
information.

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Thanks,
Suhal Vemu

--
Thanks,
Suhal Vemu

--
Please note that this email, including any attachments, is intended solely
for the individual (s) or entity (ies) to whom they are addressed and may
contain information that is private, confidential and privileged. In case
you are not the intended recipient, request you to notify the sender by
reply mail and delete this email, including any copies or attachments from
your system. Any unauthorized dissemination, disclosure and/or use of the
contents of this communication to anyone is strictly prohibited and
punishable by law.