BUG #19771: to_char(): NUL byte in result for formats with a trailing decimal point (regression from ec352b37d)

Started by PG Bug reporting formabout 1 month ago5 messagesbugs
Jump to latest
#1PG Bug reporting form
noreply@postgresql.org

The following bug has been logged on the website:

Bug reference: 19771
Logged by: Gowtham Reddy
Email address: gowthamdoma18@gmail.com
PostgreSQL version: 19rc1
Operating system: Linux x86_64
Description:

Hello,

On master (tested at 45fc51fcbf45), to_char() for numbers returns a text
value containing a NUL byte when the format has a decimal point with no
digit positions after it:

SELECT encode(textsend(to_char(7, '9.')), 'hex');
-- master: 203700 REL_19: 2037
SELECT encode(textsend(to_char(7, '9.PL')), 'hex');
-- master: 2037002b REL_19: 2037
SELECT length(to_char(7, '9.')), octet_length(to_char(7, '9.'));
-- master: 2, 3
SELECT to_char(7, '9.') = ' 7';
-- master: false REL_19: true
SELECT encode(textsend(to_char('100'::numeric, 'FM999.')), 'hex');
-- master: 31303000

The NUL byte is also written to tables: CREATE TABLE t AS SELECT
to_char(7, '9.PL') stores 2037002b. The existing regression query in
numeric.sql (SELECT to_char('100'::numeric, 'FM999.')) returns '100\0' on
master but still passes, because psql stops printing at the NUL.

Bisected to:

ec352b37d Replace formatting.c's fixed-size output buffers with
StringInfos.

Its parent 28c5d0724 returns 2037 for the second query.

The underlying problem is older. In NUM_numpart_to_char(), the "Write
Digits" branch copies *Np->input_p even when the number string is already
exhausted, so it copies the terminating '\0'. With the old fixed-size
buffer that NUL ended the C string, so in released branches everything
after the trailing decimal point is silently dropped, including the sign:

SELECT to_char(-7, '9.MI'); -- REL_19: '7' (sign lost)
SELECT to_char(-7, '9.PR'); -- REL_19: '<7' (closing bracket lost)
SELECT to_char(7, '9.PL'); -- REL_19: ' 7' (PL lost)
SELECT to_char(7, '9."x"'); -- REL_19: ' 7' (literal lost)

With the StringInfo output buffer the NUL is kept, and the suffix appears
after it.

Proposed patch (attached, v1-0001): do not emit the string terminator as a
digit. It also adds a regression test to numeric.sql, which fails without
the fix. The core change is:

--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -5659,7 +5659,7 @@ NUM_numpart_to_char(NUMProc *Np, int id)
                                                NUM_EMITC('0');
                                        }
                                }
-                               else
+                               else if (*Np->input_p != '\0')
                                {
                                        NUM_EMITC(*Np->input_p);        /*
Write DIGIT */
                                        Np->num_in = true;

With this, master returns ' 7', ' 7+', '7-', '<7>', ' 7x' for the queries
above, ' 1,234' for to_char(1234, '9,999.'), and '100' for the numeric.sql
case. Formats without a trailing decimal point are unchanged ('9.9',
'FM9.9', '99.99PL', '99PR'). make check passes (239/239) on master
45fc51fcbf45 with the patch.

This changes the result of e.g. to_char(-7, '9.MI') from '7' to '7-'
compared with released branches. I think that is the correct result, but
you may prefer to handle the back-branches differently.

A regression test that would have caught it:

SELECT octet_length(to_char(7, '9.')), to_char(-7, '9.MI'),
to_char(-7, '9.PR');

We found this with a cross-version differential fuzzer (random to_char,
to_number and to_date calls on REL_19_STABLE vs master), written for a
university course project (CS-5187).

Regards,
Gowtham

#2Gowtham Reddy
gowthamdoma18@gmail.com
In reply to: PG Bug reporting form (#1)
Re: BUG #19771: to_char(): NUL byte in result for formats with a trailing decimal point (regression from ec352b37d)

Hello,

Please find attached a proposed patch for BUG #NNNNN.

It stops NUM_numpart_to_char() from emitting the number string's
terminating '\0' as a digit, and adds a regression test to numeric.sql
(the test fails without the fix). make check passes (239/239) on
master 45fc51fcbf45.

Tom, CCing you since this became visible with ec352b37d.

Regards,
Gowtham

On Mon, 19 Oct 2026 at 02:11, PG Bug reporting form
<noreply@postgresql.org> wrote:

Show quoted text

The following bug has been logged on the website:

Bug reference: 19771
Logged by: Gowtham Reddy
Email address: gowthamdoma18@gmail.com
PostgreSQL version: 19rc1
Operating system: Linux x86_64
Description:

Hello,

On master (tested at 45fc51fcbf45), to_char() for numbers returns a text
value containing a NUL byte when the format has a decimal point with no
digit positions after it:

SELECT encode(textsend(to_char(7, '9.')), 'hex');
-- master: 203700 REL_19: 2037
SELECT encode(textsend(to_char(7, '9.PL')), 'hex');
-- master: 2037002b REL_19: 2037
SELECT length(to_char(7, '9.')), octet_length(to_char(7, '9.'));
-- master: 2, 3
SELECT to_char(7, '9.') = ' 7';
-- master: false REL_19: true
SELECT encode(textsend(to_char('100'::numeric, 'FM999.')), 'hex');
-- master: 31303000

The NUL byte is also written to tables: CREATE TABLE t AS SELECT
to_char(7, '9.PL') stores 2037002b. The existing regression query in
numeric.sql (SELECT to_char('100'::numeric, 'FM999.')) returns '100\0' on
master but still passes, because psql stops printing at the NUL.

Bisected to:

ec352b37d Replace formatting.c's fixed-size output buffers with
StringInfos.

Its parent 28c5d0724 returns 2037 for the second query.

The underlying problem is older. In NUM_numpart_to_char(), the "Write
Digits" branch copies *Np->input_p even when the number string is already
exhausted, so it copies the terminating '\0'. With the old fixed-size
buffer that NUL ended the C string, so in released branches everything
after the trailing decimal point is silently dropped, including the sign:

SELECT to_char(-7, '9.MI'); -- REL_19: '7' (sign lost)
SELECT to_char(-7, '9.PR'); -- REL_19: '<7' (closing bracket lost)
SELECT to_char(7, '9.PL'); -- REL_19: ' 7' (PL lost)
SELECT to_char(7, '9."x"'); -- REL_19: ' 7' (literal lost)

With the StringInfo output buffer the NUL is kept, and the suffix appears
after it.

Proposed patch (attached, v1-0001): do not emit the string terminator as a
digit. It also adds a regression test to numeric.sql, which fails without
the fix. The core change is:

--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -5659,7 +5659,7 @@ NUM_numpart_to_char(NUMProc *Np, int id)
NUM_EMITC('0');
}
}
-                               else
+                               else if (*Np->input_p != '\0')
{
NUM_EMITC(*Np->input_p);        /*
Write DIGIT */
Np->num_in = true;

With this, master returns ' 7', ' 7+', '7-', '<7>', ' 7x' for the queries
above, ' 1,234' for to_char(1234, '9,999.'), and '100' for the numeric.sql
case. Formats without a trailing decimal point are unchanged ('9.9',
'FM9.9', '99.99PL', '99PR'). make check passes (239/239) on master
45fc51fcbf45 with the patch.

This changes the result of e.g. to_char(-7, '9.MI') from '7' to '7-'
compared with released branches. I think that is the correct result, but
you may prefer to handle the back-branches differently.

A regression test that would have caught it:

SELECT octet_length(to_char(7, '9.')), to_char(-7, '9.MI'),
to_char(-7, '9.PR');

We found this with a cross-version differential fuzzer (random to_char,
to_number and to_date calls on REL_19_STABLE vs master), written for a
university course project (CS-5187).

Regards,
Gowtham

Attachments:

t254278_2
v1-0001-Fix-to_char-for-number-formats-ending-in-a-decima.patchapplication/octet-stream; name=v1-0001-Fix-to_char-for-number-formats-ending-in-a-decima.patchDownload+18-2
#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Gowtham Reddy (#2)
Re: BUG #19771: to_char(): NUL byte in result for formats with a trailing decimal point (regression from ec352b37d)

Gowtham Reddy <gowthamdoma18@gmail.com> writes:

Please find attached a proposed patch for BUG #NNNNN.
It stops NUM_numpart_to_char() from emitting the number string's
terminating '\0' as a digit, and adds a regression test to numeric.sql
(the test fails without the fix). make check passes (239/239) on
master 45fc51fcbf45.
Tom, CCing you since this became visible with ec352b37d.

That's definitely a bug, but on what basis do you say that it
wasn't visible before ec352b37d? AFAICT it's ancient.
I get this as far back as 9.2, for instance:

regression=# select to_char(7, 'FM99.SG') AS d;
d
---
7
(1 row)

Normally I'd think we should back-patch, but it's possible that
people didn't realize this behavior was unintended --- else we'd
have heard bug reports sooner. (It's not instantly obvious that
we should drop the decimal point but not the sign, I think.)
So I'm inclined to leave the stable branches alone. Maybe there's
still room to squeeze the fix into v19, though.

regards, tom lane

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: PG Bug reporting form (#1)
Re: BUG #19771: to_char(): NUL byte in result for formats with a trailing decimal point (regression from ec352b37d)

I wrote:

Normally I'd think we should back-patch, but it's possible that
people didn't realize this behavior was unintended --- else we'd
have heard bug reports sooner. (It's not instantly obvious that
we should drop the decimal point but not the sign, I think.)
So I'm inclined to leave the stable branches alone. Maybe there's
still room to squeeze the fix into v19, though.

Hearing no objections, pushed to 19 and master only.

A fair question about this is why dropping the decimal point is a good
idea at all. Maybe the actual bug is that there's not a decimal point
available in the input string. But that would be a bigger behavioral
change, and again we've heard no complaints in this area. Even if we
did change how the initial conversion works, this patch would remain a
good safety feature, so I'm content for now to do this and stop here.

regards, tom lane

#5Gowtham Reddy
gowthamdoma18@gmail.com
In reply to: Tom Lane (#4)
Re: BUG #19771: to_char(): NUL byte in result for formats with a trailing decimal point (regression from ec352b37d)

Hi Tom,

Thank you for committing this, and for the explanation about the
back-branches. I understand the reasoning for leaving the
decimal-point question as it is for now.

Regards, Gowtham

Show quoted text

On Tue, 20 Oct 2026 at 22:33, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I wrote:

Normally I'd think we should back-patch, but it's possible that
people didn't realize this behavior was unintended --- else we'd
have heard bug reports sooner. (It's not instantly obvious that
we should drop the decimal point but not the sign, I think.)
So I'm inclined to leave the stable branches alone. Maybe there's
still room to squeeze the fix into v19, though.

Hearing no objections, pushed to 19 and master only.

A fair question about this is why dropping the decimal point is a good
idea at all. Maybe the actual bug is that there's not a decimal point
available in the input string. But that would be a bigger behavioral
change, and again we've heard no complaints in this area. Even if we
did change how the initial conversion works, this patch would remain a
good safety feature, so I'm content for now to do this and stop here.

regards, tom lane