file_fdw: Fix ANALYZE crashes with DEFAULT markers
Hi,
Running ANALYZE on a file_fdw table with a DEFAULT marker in the input
caused an assertion failure. Here is how to reproduce the issue:
---------------------------------------------------------------
printf 'DEFAULT\n' > /tmp/file_fdw_default.csv
Then run:
CREATE EXTENSION file_fdw;
CREATE SERVER fs FOREIGN DATA WRAPPER file_fdw;
CREATE FOREIGN TABLE ft (a integer DEFAULT 42)
SERVER fs
OPTIONS (filename '/tmp/file_fdw_default.csv',
format 'csv', "default" 'DEFAULT');
ANALYZE ft;
---------------------------------------------------------------
This causes the following assertion failure:
TRAP: failed Assert("econtext != NULL"), File: "copyfromparse.c",
Line: 1062, PID: 94428
Builds without assertions can also crash.
The issue occurs because the sampling code in file_fdw passes
a NULL ExprContext to NextCopyFrom(), which needs one to evaluate
column defaults.
Attached patch fixes this by providing an expression context for sampling.
This should be backpatched to v16, where DEFAULT marker support was
introduced.
Regards,
--
Fujii Masao
Hi Fujii
On 04/11/2026 11:13, Fujii Masao wrote:
Attached patch fixes this by providing an expression context for sampling.
This should be backpatched to v16, where DEFAULT marker support was
introduced.
I was able to reproduce the bug and your patch fixes it. Just for my
understanding: why use ReScanExprContext instead of ResetExprContext here?
- /* Fetch next row */
- MemoryContextReset(tupcontext);
- MemoryContextSwitchTo(tupcontext);
+ /* Clean up the previous row's expression evaluation state */
+ ReScanExprContext(econtext);
Thanks for the patch!
Best, Jim
On Wed, Nov 4, 2026 at 7:46 PM Jim Jones <jim.jones@uni-muenster.de> wrote:
Just for my
understanding: why use ReScanExprContext instead of ResetExprContext here?
Yeah, since AFAICT there seems no need to invoke registered callbacks,
ResetExprContext() might be sufficient. Also, ResetPerTupleExprContext()
would be better for consistency with CopyFrom(), which uses it?
Regards,
--
Fujii Masao
On 04/11/2026 14:37, Fujii Masao wrote:
Yeah, since AFAICT there seems no need to invoke registered callbacks,
ResetExprContext() might be sufficient. Also, ResetPerTupleExprContext()
would be better for consistency with CopyFrom(), which uses it?
IIUC es_per_tuple_exprcontext will always be non-NULL because of
GetPerTupleExprContext(), so in theory the extra check in
ResetPerTupleExprContext() is a bit redundant, but I agree it's a good
idea to keep the code consistent.
Thanks!
Best, Jim
Hello Fujii and Jim,
I can confirm reproduction aswell (planned to report it tomorrow but
Fujii beat me to it).
There's also another bug very close to this code: ANALYZE does not
enforce REJECT_LIMIT when on_error 'ignore' is enabled.
That's for a different unrelated commit, i'll report this soon, if it
interests you to take a look.
On 04/11/2026 16:26, Jim Jones wrote:
On 04/11/2026 14:37, Fujii Masao wrote:
Yeah, since AFAICT there seems no need to invoke registered callbacks,
ResetExprContext() might be sufficient. Also, ResetPerTupleExprContext()
would be better for consistency with CopyFrom(), which uses it?IIUC es_per_tuple_exprcontext will always be non-NULL because of
GetPerTupleExprContext(), so in theory the extra check in
ResetPerTupleExprContext() is a bit redundant, but I agree it's a good
idea to keep the code consistent.
This also applies to:
- MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
+ MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
For the same exact reasons, it stays consistent with
fileIterateForeignScan(), if we want of course...
Regards,
Ayoub Kazar
On Nov 4, 2026, at 23:26, Jim Jones <jim.jones@uni-muenster.de> wrote:
On 04/11/2026 14:37, Fujii Masao wrote:
Yeah, since AFAICT there seems no need to invoke registered callbacks,
ResetExprContext() might be sufficient. Also, ResetPerTupleExprContext()
would be better for consistency with CopyFrom(), which uses it?IIUC es_per_tuple_exprcontext will always be non-NULL because of
GetPerTupleExprContext(), so in theory the extra check in
ResetPerTupleExprContext() is a bit redundant, but I agree it's a good
idea to keep the code consistent.Thanks!
Best, Jim
CopyFrom() does this combination:
```
ResetPerTupleExprContext(estate);
MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
```
So, if we use ResetPerTupleExprContext(estate), then MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); should be changed accordingly. I think Ayoub commented the same thing.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Thu, Nov 5, 2026 at 4:50 PM Chao Li <li.evan.chao@gmail.com> wrote:
So, if we use ResetPerTupleExprContext(estate), then MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); should be changed accordingly. I think Ayoub commented the same thing.
Agreed. I've updated the patch to use ResetPerTupleExprContext(estate) and
MemoryContextSwitchTo(GetPerTupleMemoryContext(estate)). Attached.
Regards,
--
Fujii Masao
On 05/11/2026 09:54, Fujii Masao wrote:
On Thu, Nov 5, 2026 at 4:50 PM Chao Li <li.evan.chao@gmail.com> wrote:
So, if we use ResetPerTupleExprContext(estate), then
MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); should be
changed accordingly. I think Ayoub commented the same thing.Agreed. I've updated the patch to use ResetPerTupleExprContext(estate)
and MemoryContextSwitchTo(GetPerTupleMemoryContext(estate)). Attached.
Looks good.
Regards,
Ayoub Kazar
On Nov 5, 2026, at 18:55, Ayoub Kazar <ayoub.kazar@data-bene.io> wrote:
On 05/11/2026 09:54, Fujii Masao wrote:
On Thu, Nov 5, 2026 at 4:50 PM Chao Li <li.evan.chao@gmail.com> wrote:
So, if we use ResetPerTupleExprContext(estate), then MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); should be changed accordingly. I think Ayoub commented the same thing.
Agreed. I've updated the patch to use ResetPerTupleExprContext(estate) and MemoryContextSwitchTo(GetPerTupleMemoryContext(estate)). Attached.
Looks good.
+1
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/