file_fdw: Fix ANALYZE crashes with DEFAULT markers

Started by Fujii Masao12 days ago10 messageshackers
Jump to latest
#1Fujii Masao
masao.fujii@gmail.com

Hi,

Running ANALYZE on a file_fdw table with a DEFAULT marker in the input
caused an assertion failure. Here is how to reproduce the issue:

---------------------------------------------------------------
printf 'DEFAULT\n' > /tmp/file_fdw_default.csv

Then run:

CREATE EXTENSION file_fdw;
CREATE SERVER fs FOREIGN DATA WRAPPER file_fdw;
CREATE FOREIGN TABLE ft (a integer DEFAULT 42)
SERVER fs
OPTIONS (filename '/tmp/file_fdw_default.csv',
format 'csv', "default" 'DEFAULT');

ANALYZE ft;
---------------------------------------------------------------

This causes the following assertion failure:

TRAP: failed Assert("econtext != NULL"), File: "copyfromparse.c",
Line: 1062, PID: 94428

Builds without assertions can also crash.

The issue occurs because the sampling code in file_fdw passes
a NULL ExprContext to NextCopyFrom(), which needs one to evaluate
column defaults.

Attached patch fixes this by providing an expression context for sampling.
This should be backpatched to v16, where DEFAULT marker support was
introduced.

Regards,

--
Fujii Masao

Attachments:

t254557_1
v1-0001-file_fdw-Fix-ANALYZE-crashes-with-DEFAULT-markers.patchapplication/octet-stream; name=v1-0001-file_fdw-Fix-ANALYZE-crashes-with-DEFAULT-markers.patchDownload+28-12
#2Jim Jones
jim.jones@uni-muenster.de
In reply to: Fujii Masao (#1)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

Hi Fujii

On 04/11/2026 11:13, Fujii Masao wrote:

Attached patch fixes this by providing an expression context for sampling.
This should be backpatched to v16, where DEFAULT marker support was
introduced.

I was able to reproduce the bug and your patch fixes it. Just for my
understanding: why use ReScanExprContext instead of ResetExprContext here?

-		/* Fetch next row */
-		MemoryContextReset(tupcontext);
-		MemoryContextSwitchTo(tupcontext);
+		/* Clean up the previous row's expression evaluation state */
+		ReScanExprContext(econtext);

Thanks for the patch!

Best, Jim

#3Fujii Masao
masao.fujii@gmail.com
In reply to: Jim Jones (#2)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

On Wed, Nov 4, 2026 at 7:46 PM Jim Jones <jim.jones@uni-muenster.de> wrote:

Just for my
understanding: why use ReScanExprContext instead of ResetExprContext here?

Yeah, since AFAICT there seems no need to invoke registered callbacks,
ResetExprContext() might be sufficient. Also, ResetPerTupleExprContext()
would be better for consistency with CopyFrom(), which uses it?

Regards,

--
Fujii Masao

#4Jim Jones
jim.jones@uni-muenster.de
In reply to: Fujii Masao (#3)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

On 04/11/2026 14:37, Fujii Masao wrote:

Yeah, since AFAICT there seems no need to invoke registered callbacks,
ResetExprContext() might be sufficient. Also, ResetPerTupleExprContext()
would be better for consistency with CopyFrom(), which uses it?

IIUC es_per_tuple_exprcontext will always be non-NULL because of
GetPerTupleExprContext(), so in theory the extra check in
ResetPerTupleExprContext() is a bit redundant, but I agree it's a good
idea to keep the code consistent.

Thanks!

Best, Jim

#5Ayoub Kazar
kazarayoub2004@gmail.com
In reply to: Jim Jones (#4)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

Hello Fujii and Jim,

I can confirm reproduction aswell (planned to report it tomorrow but
Fujii beat me to it).
There's also another bug very close to this code: ANALYZE does not
enforce REJECT_LIMIT when on_error 'ignore' is enabled.
That's for a different unrelated commit, i'll report this soon, if it
interests you to take a look.

On 04/11/2026 16:26, Jim Jones wrote:

On 04/11/2026 14:37, Fujii Masao wrote:

Yeah, since AFAICT there seems no need to invoke registered callbacks,
ResetExprContext() might be sufficient. Also, ResetPerTupleExprContext()
would be better for consistency with CopyFrom(), which uses it?

IIUC es_per_tuple_exprcontext will always be non-NULL because of
GetPerTupleExprContext(), so in theory the extra check in
ResetPerTupleExprContext() is a bit redundant, but I agree it's a good
idea to keep the code consistent.

This also applies to:
- MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
+ MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));

For the same exact reasons, it stays consistent with
fileIterateForeignScan(), if we want of course...

Regards,
Ayoub Kazar

#6Chao Li
li.evan.chao@gmail.com
In reply to: Jim Jones (#4)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

On Nov 4, 2026, at 23:26, Jim Jones <jim.jones@uni-muenster.de> wrote:

On 04/11/2026 14:37, Fujii Masao wrote:

Yeah, since AFAICT there seems no need to invoke registered callbacks,
ResetExprContext() might be sufficient. Also, ResetPerTupleExprContext()
would be better for consistency with CopyFrom(), which uses it?

IIUC es_per_tuple_exprcontext will always be non-NULL because of
GetPerTupleExprContext(), so in theory the extra check in
ResetPerTupleExprContext() is a bit redundant, but I agree it's a good
idea to keep the code consistent.

Thanks!

Best, Jim

CopyFrom() does this combination:
```
ResetPerTupleExprContext(estate);

MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
```

So, if we use ResetPerTupleExprContext(estate), then MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); should be changed accordingly. I think Ayoub commented the same thing.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

#7Fujii Masao
masao.fujii@gmail.com
In reply to: Chao Li (#6)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

On Thu, Nov 5, 2026 at 4:50 PM Chao Li <li.evan.chao@gmail.com> wrote:

So, if we use ResetPerTupleExprContext(estate), then MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); should be changed accordingly. I think Ayoub commented the same thing.

Agreed. I've updated the patch to use ResetPerTupleExprContext(estate) and
MemoryContextSwitchTo(GetPerTupleMemoryContext(estate)). Attached.

Regards,

--
Fujii Masao

Attachments:

t254557_7
v2-0001-file_fdw-Fix-ANALYZE-crashes-with-DEFAULT-markers.patchapplication/octet-stream; name=v2-0001-file_fdw-Fix-ANALYZE-crashes-with-DEFAULT-markers.patchDownload+28-12
#8Ayoub Kazar
kazarayoub2004@gmail.com
In reply to: Fujii Masao (#7)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

On 05/11/2026 09:54, Fujii Masao wrote:

On Thu, Nov 5, 2026 at 4:50 PM Chao Li <li.evan.chao@gmail.com> wrote:

So, if we use ResetPerTupleExprContext(estate), then
MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); should be
changed accordingly. I think Ayoub commented the same thing.

Agreed. I've updated the patch to use ResetPerTupleExprContext(estate)
and MemoryContextSwitchTo(GetPerTupleMemoryContext(estate)). Attached.

Looks good.

Regards,
Ayoub Kazar

#9Chao Li
li.evan.chao@gmail.com
In reply to: Ayoub Kazar (#8)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

On Nov 5, 2026, at 18:55, Ayoub Kazar <ayoub.kazar@data-bene.io> wrote:

On 05/11/2026 09:54, Fujii Masao wrote:

On Thu, Nov 5, 2026 at 4:50 PM Chao Li <li.evan.chao@gmail.com> wrote:

So, if we use ResetPerTupleExprContext(estate), then MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory); should be changed accordingly. I think Ayoub commented the same thing.

Agreed. I've updated the patch to use ResetPerTupleExprContext(estate) and MemoryContextSwitchTo(GetPerTupleMemoryContext(estate)). Attached.

Looks good.

+1

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

#10Fujii Masao
masao.fujii@gmail.com
In reply to: Chao Li (#9)
Re: file_fdw: Fix ANALYZE crashes with DEFAULT markers

On Thu, Nov 5, 2026 at 8:09 PM Chao Li <li.evan.chao@gmail.com> wrote:

Looks good.

+1

I've pushed the patch. Thanks!

Regards,

--
Fujii Masao