could not open file "base/XX/XX": Interrupted system call

Started by Torsten Krahover 2 years ago7 messagesgeneral
Jump to latest
#1Torsten Krah
krah.tm@gmail.com

Hi,

I am running the postgres docker image with that version:

2023-09-20 10:36:32.478 CEST [1] LOG: starting PostgreSQL 13.12 (Debian 13.12-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2023-09-20 10:36:32.564 CEST [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-09-20 10:36:32.564 CEST [1] LOG: listening on IPv6 address "::", port 5432

And this is happening for some queries:

2023-09-20 10:38:25.076 CEST [47] ERROR: could not open file "base/16386/17328": Interrupted system call
2023-09-20 10:38:41.897 CEST [49] ERROR: could not open file "base/16386/68359": Interrupted system call

I can enter the container and view the files above with e.g. strings
$file | less and I can create a md5sum from that file without an error.

What can I do about that EINTR on open, how to fix it?

The host (6.1.53-060153-generic #202309130436 SMP PREEMPT_DYNAMIC) and
the docker daemon (Version: 24.0.6) do have nothing suspicious in there
dmesg / logs.

Anyone an idea how to debug / progress with that one?

Torsten

#2Laurenz Albe
laurenz.albe@cybertec.at
In reply to: Torsten Krah (#1)
Re: could not open file "base/XX/XX": Interrupted system call

On Wed, 2023-09-20 at 10:49 +0200, Torsten Krah wrote:

I am running the postgres docker image with that version:

2023-09-20 10:36:32.478 CEST [1] LOG:  starting PostgreSQL 13.12 (Debian 13.12-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2023-09-20 10:36:32.564 CEST [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2023-09-20 10:36:32.564 CEST [1] LOG:  listening on IPv6 address "::", port 5432

And this is happening for some queries:

2023-09-20 10:38:25.076 CEST [47] ERROR:  could not open file "base/16386/17328": Interrupted system call
2023-09-20 10:38:41.897 CEST [49] ERROR:  could not open file "base/16386/68359": Interrupted system call

Is that an NFS mount? What are the mount options?

Yours,
Laurenz Albe

#3Torsten Krah
krah.tm@gmail.com
In reply to: Laurenz Albe (#2)
Re: could not open file "base/XX/XX": Interrupted system call

Am Mittwoch, dem 20.09.2023 um 13:11 +0200 schrieb Laurenz Albe:

Is that an NFS mount?  What are the mount options?

Yours,
Laurenz Albe

No mount at all.

Just the docker container filesystem using the overlay2 storage driver,
there is nothing externally mounted in that container, the data is
already in the image (via docker COPY) from which the container gets
created (and that data directory was created in the image build process
via pg_restore from the dump file).

The whole container / image just involves overlay2 fs which itself is
sitting on ext4.

Torsten

#4Slava Shpitalny
slava.shpitalny@tactilemobility.com
In reply to: Torsten Krah (#3)
Re: could not open file "base/XX/XX": Interrupted system call

Hi,
We are facing a similar problem using a docker from
https://hub.docker.com/_/postgres on AWS EC2 machine.
The version we are using is 17 (docker image "postgres:17")
The issue happens on our CI server, and it happens consistently.
The table is a big one (related to
/messages/by-id/16832-943d33fd58eb26e4@postgresql.org
)

Do you happen to have any updates on this issue?

*Slava Shpitalny*

Software Development Team Lead

Tactile Mobility

E-mail: slava.shpitalny@tactilemobility.com

On Thu, Dec 26, 2024 at 1:43 PM Torsten Krah <krah.tm@gmail.com> wrote:

Show quoted text

Am Mittwoch, dem 20.09.2023 um 13:11 +0200 schrieb Laurenz Albe:

Is that an NFS mount? What are the mount options?

Yours,
Laurenz Albe

No mount at all.

Just the docker container filesystem using the overlay2 storage driver,
there is nothing externally mounted in that container, the data is
already in the image (via docker COPY) from which the container gets
created (and that data directory was created in the image build process
via pg_restore from the dump file).

The whole container / image just involves overlay2 fs which itself is
sitting on ext4.

Torsten

#5Torsten Krah
krah.tm@gmail.com
In reply to: Slava Shpitalny (#4)
Re: could not open file "base/XX/XX": Interrupted system call

Am Donnerstag, dem 26.12.2024 um 13:48 +0200 schrieb Slava Shpitalny:

Do you happen to have any updates on this issue?

Hi,

unfortunately not, the only thing I found is this in "man 2 open":

...

EINTR While blocked waiting to complete an open of a slow device
(e.g., a FIFO; see fifo(7)), the call was interrupted by a signal
handler; see signal(7).
...

To me this reads that postgres should handle that error code (e.g. try
again n times before failing) - but maybe it does that already, I did
not verify all open calls, if or how postgres does handle EINTR.

The only thing I did change is, that I use a volume (an anonymous one
in my usecase) again, because with that, I never encountered that error
- if I omit that volume and write directly to the containers fs (which
would be sufficient for me), I still had that error.

kind regards

Torsten

#6Andreas Froede
psql@andreas.spider-net.de
In reply to: Slava Shpitalny (#4)
Re: could not open file "base/XX/XX": Interrupted system call

Am 26.12.24 um 12:48 schrieb Slava Shpitalny:

We are facing a similar problem using a docker from https://
hub.docker.com/_/postgres <https://hub.docker.com/_/postgres&gt; on AWS EC2
machine.

In this case it's a docker issue.
By default docker limit the size of shared memory (i think 64M).
You can change this in your dockerfile.

CIAO
andreas

#7Andreas Froede
psql@andreas.spider-net.de
In reply to: Andreas Froede (#6)
Re: could not open file "base/XX/XX": Interrupted system call

Am 20.01.25 um 15:24 schrieb Andreas Froede:

Am 26.12.24 um 12:48 schrieb Slava Shpitalny:

We are facing a similar problem using a docker from https://
hub.docker.com/_/postgres <https://hub.docker.com/_/postgres&gt; on AWS
EC2 machine.

In this case it's a docker issue.
By default docker limit the size of shared memory (i think 64M).
You can change this in your dockerfile.

Sorry. Wrong discussion.
Please forgot it.

CIAO
andreas