PostgreSQL 19 Beta 2: COPY HEADER N exceeds file length - test observation
Hi PostgreSQL community,
I have been testing the new integer form of the COPY FROM HEADER option in
PostgreSQL 19 Beta 2 and wanted to share an edge-case observation for
confirmation.
I tested a CSV file containing only 5 lines:
-
3 metadata lines
-
1 CSV column-header line
-
1 data row
Then executed:
TRUNCATE employees;
COPY employees (emp_id, emp_name, dept)
FROM '/tmp/employees_header_overflow.csv'
WITH (FORMAT csv, HEADER 10);
Result:
COPY 0
The table remained empty:
emp_id | emp_name | dept
--------+----------+------
(0 rows)
So, when HEADER N is greater than the total number of lines available in
the file, PostgreSQL 19 Beta 2 does not raise an error. It consumes the
available input as header lines and returns COPY 0.
From an ETL perspective, this caught my attention because if the HEADER
value is externally configured and is incorrect, data rows could
potentially be skipped without an explicit error.
Could someone please confirm whether this is the expected/intended behavior
for HEADER ?
I am sharing this as a Beta 2 test observation so that it can be reviewed
before the PostgreSQL 19 final release, if any change or documentation
clarification is appropriate.
I have also documented the complete hands-on PoC, including the test cases
and results:
https://github.com/softclement/PostgreSQL-19-COPY-HEADER-PoC
Test environment:
-
PostgreSQL 19 Beta 2
-
Podman
-
WSL2 / Ubuntu
-
COPY FROM CSV
Regards,
Mariyan Clement
Database Architectect | PostgreSQL | Oracle
On Thu, 13 Aug 2026 at 21:12, Mariyan Clement S <softclement@gmail.com> wrote:
Hi PostgreSQL community,
I have been testing the new integer form of the COPY FROM HEADER option in PostgreSQL 19 Beta 2 and wanted to share an edge-case observation for confirmation.
I tested a CSV file containing only 5 lines:
3 metadata lines
So, it's not really a CSV file, or at least not compliant with the
relevant RFC 4180?
Could someone please confirm whether this is the expected/intended behavior for HEADER ?
The docs are clear: "HEADER [...] On input, [i]f set to a non-negative
integer, that number of lines are discarded."
Unless you suggest we raise an error when the input file doesn't
contain at least as many lines as indicated by the user's integer
input to HEADER, the behaviour of ignoring lines indicated by the user
to be headers is fully expected.
Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)
Hi Matthias,
Thank you for the clarification.
Yes, I agree with your interpretation. My intention was not to suggest that
the current behavior is necessarily a bug.
The test file was intentionally constructed with metadata lines before the
CSV header to simulate a real-world batch/export file. I understand that
such a file is not a conventional RFC 4180-compliant CSV file.
My main observation was about the case where the configured HEADER value is
greater than the number of available input lines.
As you pointed out, the documentation clearly states that when HEADER is
specified with a non-negative integer, that number of lines are discarded.
Therefore, consuming the available input and returning COPY 0 is consistent
with the documented behavior.
I wanted to validate this edge case during PostgreSQL 19 Beta testing,
particularly from an ETL perspective where the HEADER value could
potentially be supplied through external configuration and could be
incorrect.
Thanks again for taking the time to clarify this.
Regards,
Mariyan Clement
Database Architect | PostgreSQL | Oracle
LinkedIn: https://www.linkedin.com/in/mariyanclement/
On Fri, Aug 14, 2026 at 4:16 PM Matthias van de Meent <
boekewurm+postgres@gmail.com> wrote:
Show quoted text
On Thu, 13 Aug 2026 at 21:12, Mariyan Clement S <softclement@gmail.com>
wrote:Hi PostgreSQL community,
I have been testing the new integer form of the COPY FROM HEADER option
in PostgreSQL 19 Beta 2 and wanted to share an edge-case observation for
confirmation.I tested a CSV file containing only 5 lines:
3 metadata lines
So, it's not really a CSV file, or at least not compliant with the
relevant RFC 4180?Could someone please confirm whether this is the expected/intended
behavior for HEADER ?
The docs are clear: "HEADER [...] On input, [i]f set to a non-negative
integer, that number of lines are discarded."
Unless you suggest we raise an error when the input file doesn't
contain at least as many lines as indicated by the user's integer
input to HEADER, the behaviour of ignoring lines indicated by the user
to be headers is fully expected.Kind regards,
Matthias van de Meent
Databricks (https://www.databricks.com)