BUG #17932: Cannot select big bytea values(>500MB)

Started by PG Bug reporting formalmost 3 years ago3 messagesbugs
Jump to latest
#1PG Bug reporting form
noreply@postgresql.org

The following bug has been logged on the website:

Bug reference: 17932
Logged by: Hans Lee
Email address: haduchiep@gmail.com
PostgreSQL version: 13.10
Operating system: linux
Description:

Dear Support Team,

I created a big bytea value and try to select it from a table, I get
an error, something like: "ERROR: invalid memory alloc request size
...".
it means i can insert data into table but then i can't even work
with it. is it a bug? could you please help me to solve this Problem.
Thank you very much for your support

Best Regards
Hiep

#2John Naylor
john.naylor@enterprisedb.com
In reply to: PG Bug reporting form (#1)
Re: BUG #17932: Cannot select big bytea values(>500MB)

On Mon, May 15, 2023 at 5:21 PM PG Bug reporting form <
noreply@postgresql.org> wrote:

I created a big bytea value and try to select it from a table, I get
an error, something like: "ERROR: invalid memory alloc request size
...".
it means i can insert data into table but then i can't even work
with it. is it a bug? could you please help me to solve this Problem.
Thank you very much for your support

This is a known (but not documented as far as I know) issue, but it also
depends on the data contents and how the value is retrieved. Possible
workarounds:

1. If the data is largely printable ASCII, then try
setting "bytea_output" to "escape":

https://www.postgresql.org/docs/devel/runtime-config-client.html#GUC-BYTEA-OUTPUT

2. Request binary transmission in your application (if possible), for
example "show_binary_results" in

https://www.postgresql.org/docs/devel/libpq-example.html

3. COPY out using binary format

--
John Naylor
EDB: http://www.enterprisedb.com

#3duc hiep ha
haduchiep@googlemail.com
In reply to: John Naylor (#2)
Re: BUG #17932: Cannot select big bytea values(>500MB)

Hello John Naylor,

Thank you very much for your support. After setting the "bytea_output" to
"escape" and increasing the Java heap size of DBeaver (to 8GB), I can now
see the big column (bytea > 500MB). I am still in the testing phase and
haven't finished yet, but I wonder if clients with small RAM will be able
to select these big columns or if bytea_output=escape will be visible in
the application. Could you please explain this to me in more detail?

Thanks

Vào Th 3, 16 thg 5, 2023 vào lúc 08:58 John Naylor <
john.naylor@enterprisedb.com> đã viết:

Show quoted text

On Mon, May 15, 2023 at 5:21 PM PG Bug reporting form <
noreply@postgresql.org> wrote:

I created a big bytea value and try to select it from a table, I get
an error, something like: "ERROR: invalid memory alloc request size
...".
it means i can insert data into table but then i can't even work
with it. is it a bug? could you please help me to solve this Problem.
Thank you very much for your support

This is a known (but not documented as far as I know) issue, but it also
depends on the data contents and how the value is retrieved. Possible
workarounds:

1. If the data is largely printable ASCII, then try
setting "bytea_output" to "escape":

https://www.postgresql.org/docs/devel/runtime-config-client.html#GUC-BYTEA-OUTPUT

2. Request binary transmission in your application (if possible), for
example "show_binary_results" in

https://www.postgresql.org/docs/devel/libpq-example.html

3. COPY out using binary format

--
John Naylor
EDB: http://www.enterprisedb.com