Fix DEFAULT marker handling in COPY FROM ... HEADER MATCH

On Nov 4, 2026, at 08:21, Ayoub Kazar <ayoub.kazar@data-bene.io> wrote:

Hello,

I noticed an issue when combining the HEADER MATCH option (added in v15) with the DEFAULT option in COPY FROM (added in v16).

If any column name in the header line matches the configured DEFAULT marker,
two problematic behaviors occur depending on whether the column has a default:

-When the column has a default value, the input value of the first data row
is discarded and replaced with the column's default expression. All
subsequent data rows are loaded normally:

CREATE TABLE sales (id int, "default" int DEFAULT 0, note text);

COPY sales FROM STDIN WITH (FORMAT csv, HEADER MATCH, DEFAULT 'default');
id,default,note
1,5,first
2,7,second
\.

SELECT * FROM sales ORDER BY id;
id | default | note
----+---------+--------
1 | 0 | first -- Expected 5 here ?
2 | 7 | second
(2 rows)

-When the column does not have a default value, COPY aborts during header
parsing, but not for the correct reason i guess:

CREATE TABLE t (d int, b text);
COPY t FROM STDIN WITH (FORMAT csv, HEADER MATCH, DEFAULT 'd');
d,b
5,x
\.
ERROR: unexpected default marker in COPY data
DETAIL: Column "d" has no default value.
CONTEXT: COPY t, line 1: "d,b"

This also happens with foreign tables via file_fdw when used with
OPTIONS (header 'match', default '...').

Because header parsing and the first row of data are parsed within the same
first call to NextCopyFrom(), and cstate->defaults is initialized to false
only at the top of NextCopyFrom(), the flag set while reading the header
line survived into the first data row. CopyFromTextLikeOneRow() then sees
cstate->defaults[m] set to true and evaluate the default expression instead of
the supplied field.

The attached fix temporarily disables default marker recognition while parsing
the header line when HEADER MATCH. I also added two tests for this.

This affects REL_16_STABLE through master.

Thoughts?

Regards,
Ayoub Kazar<0001-Fix-DEFAULT-marker-handling-in-COPY-FROM-.-HEADER-MATCH.patch>

Nice catch. I reproduced the problem and confirmed that the v1 fix works.

The root cause is that CopyReadAttributesCSV() and CopyReadAttributesText() use the same logic to parse header lines and data lines, so a column name may be incorrectly interpreted as a DEFAULT marker. From this perspective, I think adding an is_header parameter to these two functions to distinguish header parsing from data parsing would be a more general solution.

With your finding, I realized that NULL markers have the same problem:
```
evantest=# create table t (id int, a int);
CREATE TABLE
evantest=# copy t from stdin with (format csv, header match, null 'a');
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself, or an EOF signal.

id,a
1,2
\.

ERROR: column name mismatch in header line field 2: got null value ("a"), expected "a"
CONTEXT: COPY t, line 1: "id,a"
```

Here, a is a valid column name in the header line and should not be rejected simply because it matches the NULL marker.

Adding an is_header parameter allows us to disable both NULL and DEFAULT marker recognition when parsing header lines, addressing both issues with the same approach. It also avoids temporarily modifying the COPY options.

So, I’m proposing v2 with this approach. I kept the regression tests you added in v1 and added tests for NULL markers and the text format. Would you please review v2?

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

Hi,
Thanks for the review!

On 04/11/2026 07:54, Chao Li wrote:

ERROR: column name mismatch in header line field 2: got null value ("a"), expected "a"
CONTEXT: COPY t, line 1: "id,a"
```

Here, a is a valid column name in the header line and should not be rejected simply because it matches the NULL marker.

Adding an is_header parameter allows us to disable both NULL and DEFAULT marker recognition when parsing header lines, addressing both issues with the same approach. It also avoids temporarily modifying the COPY options.

So, I’m proposing v2 with this approach. I kept the regression tests you added in v1 and added tests for NULL markers and the text format. Would you please review v2?

I have mixed feelings about this:

CREATE TABLE t (id int, "N" text);

If user inputs this in COPY:
id   \N
1    \N

same is in diffed existing regression test for it:

|-ERROR: column name mismatch in header line field 3: got null value
("\N"), expected "c" +ERROR: column name mismatch in header line field
3: got "N", expected "c"|

IMO, i prefer having this error out for NULL marker reason in a column
name, rather than reading '\N' as 'N' in header, and this might be the
intention of who wrote it... maybe.
I agree this should be data rows only, but the documentation doesn't
specify that either, at least not as expected as the case of DEFAULT
marker. So i'm not sure...

CSV case:
In CSV, CopyReadAttributesCSV() already checks `!saw_quote`. If a user has a
column named "a" and sets `null 'a'`, quoting the column name in the
header (`id,"a"`) already prevents it from being treated as NULL.

I have an extra worry about performance regressions, previously from
[1]: /messages/by-id/CAOzEurSW8cNr6TPKsjrstnPfhf4QyQqB4tnPXGGe8N4e_v7Jig@mail.gmail.com
branching (without going into much details), therefore having 1-3%
performance regressions or improvements that appear just like that (this
applies everywhere in Postgres really...), CopyReadAttributesCSV() is
very sensitive, it runs millions of times very easily and its a high
registers-pressured code.

A not too serious benchmark on v2 already shows 5% regressions.

Regards,
Ayoub Kazar

[1]: /messages/by-id/CAOzEurSW8cNr6TPKsjrstnPfhf4QyQqB4tnPXGGe8N4e_v7Jig@mail.gmail.com
/messages/by-id/CAOzEurSW8cNr6TPKsjrstnPfhf4QyQqB4tnPXGGe8N4e_v7Jig@mail.gmail.com