weird - invalid string enlargement request size

Walter Cruz escribi�:

A simple query:

select prec.bl_considerada_prioritaria
AS "A proposta � considerada priorit�ria por Conselho Municipal ou Estadual"
from consultaprevia2008.tab_urb_asse_prec prec

The result: ERROR: invalid string enlargement request size 1073741823

Weird. Probably encoding mismatch. What are your locale and encoding
settings, and what encoding is the actual client? And what client is
it? Easiest way to show server settings is:

select name, setting
from pg_settings
where name ~~ any ( array['lc_%', '%_encoding']);

--
Alvaro Herrera http://www.advogato.org/person/alvherre
"La Primavera ha venido. Nadie sabe como ha sido" (A. Machado)

Yes, encoding mismatch [:)]

"client_encoding";"UNICODE"
"lc_collate";"pt_BR.UTF-8"
"lc_ctype";"pt_BR.UTF-8"
"lc_messages";"pt_BR.UTF_8"
"lc_monetary";"pt_BR.UTF_8"
"lc_numeric";"pt_BR.UTF_8"
"lc_time";"pt_BR.UTF_8"
"server_encoding";"LATIN1"

I'm with some troubles to set the encoding In my PHP program, but this
is not hackers issue :)

I posted on hackers cause I think that was a bug, os something.

[]'s
- Walter

"Walter Cruz" <walter.php@gmail.com> writes:

I posted on hackers cause I think that was a bug, os something.

Yeah, I think so too. Can you extract a reproducible test case?

regards, tom lane

On Dec 4, 2007 7:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

"Walter Cruz" <walter.php@gmail.com> writes:

I posted on hackers cause I think that was a bug, os something.

Yeah, I think so too. Can you extract a reproducible test case?

I'm trying to extract that case!

[]'s
- Walter

Yes, I did.

What have I done.

My Linux version: Debian Etch.

My Postgres version:

"PostgreSQL 8.2.5 on x86_64-unknown-linux-gnu, compiled by GCC gcc
(GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)"

It was compiled by myself.

My initdb was:
initdb -E LATIN1 --locale=pt_BR

By that initdb, the $LANG of the system was pt_BR.UTF-8 .

The pt_BR was not even enabled.

A simple query that shows the problem:

select true AS "áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

(it's needed this exact number of chars, with one of them accented.)

The output of the SQL of Alvaro in this server is:

"client_encoding";"UNICODE"
"lc_collate";"pt_BR.UTF-8"
"lc_ctype";"pt_BR.UTF-8"
"lc_messages";"pt_BR"
"lc_monetary";"pt_BR"
"lc_numeric";"pt_BR"
"lc_time";"pt_BR"
"server_encoding";"LATIN1"

(I have enabled the pt_BR locale before the tests, but lc_collate and
lc_type are still pt_BR.UTF-8, due to initdb).

Just after that, I changed my $LANG to pt_BR, did a initdb.

The query

select true AS "áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";

returns:

template1-# ;
NOTICE: identifier
"áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
will be truncated to
"áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

On this second initdb:

template1=# select name, setting
template1-# from pg_settings
template1-# where name ~~ any ( array['lc_%', '%_encoding']);
name | setting
-----------------+---------
client_encoding | LATIN1
lc_collate | pt_BR
lc_ctype | pt_BR
lc_messages | pt_BR
lc_monetary | pt_BR
lc_numeric | pt_BR
lc_time | pt_BR
server_encoding | LATIN1

Is that information enough? I can give you more details, if needed.

[]'s
- Walter

"Walter Cruz" <walter.php@gmail.com> writes:

My initdb was:
initdb -E LATIN1 --locale=pt_BR
By that initdb, the $LANG of the system was pt_BR.UTF-8 .
A simple query that shows the problem:
select true AS "�aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

OK, I was able to reproduce this here:

postgres=# select true AS "�aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
;
ERROR: invalid string enlargement request size 1073741823

What is happening is that vsnprintf() is failing because it doesn't like
its input data. elog.c's EVALUATE_MESSAGE macro thinks that the only
possible reason for failure is that the output buffer isn't large
enough, so it enlarges and tries again, leading soon to the palloc error
displayed above.

Possibly we should put some effort into producing a more useful error
message here, but I'm reluctant to fool with it, because our historical
experience is that vsnprintf's behavior just isn't very consistent
across platforms.

In any case, the bottom-line reason why you're having a problem is that
the database encoding (LATIN1) is inconsistent with the encoding
specified by the server's locale (UTF-8), thus allowing vsnprintf to see
what it thinks is bad data.

PG 8.3 contains defenses to prevent that sort of inconsistency, and
I think that's probably all we need to do about this.

regards, tom lane