Does PostgreSQL use atomic file creation of FS?

On Wed, Dec 12, 2018 at 02:48:12PM +0300, Dmitry Lazurkin wrote:

Does PostgreSQL use atomic file creation on FS? How does PostgreSQL
catch situation when system crashes between open call and write call? I
am interesting in this because I would like use PostgreSQL on network
file system.

I doubt we can get more certainty than this:

https://www.postgresql.org/docs/devel/creating-cluster.html#CREATING-CLUSTER-NFS

Best,
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

Thank you. But I have read this. I said about network file system only
for example. I would like to known how PostgreSQL handles this specific
case (of course if someone knowns a answer):

fd = open(file, "w");
write(fd, data);
// crash and now I have empty file which isn't correct
fsync(fd);

PS. I think PostgreSQL doesn't have this problem.

Dmitry Lazurkin wrote:

Does PostgreSQL use atomic file creation on FS? How does PostgreSQL
catch situation when system crashes between open call and write call? I
am interesting in this because I would like use PostgreSQL on network
file system.

If there is a crash, the file would be left behind.
This is slightly annoying, but shouldn't be a major problem.
Persisting such information in a crash-safe way would require
fsyncs that hurt I/O performance.

Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com

On Wed, Dec 12, 2018 at 11:52 PM Dmitry Lazurkin <dilaz03@gmail.com> wrote:

Thank you. But I have read this. I said about network file system only
for example. I would like to known how PostgreSQL handles this specific
case (of course if someone knowns a answer):

fd = open(file, "w");
write(fd, data);
// crash and now I have empty file which isn't correct
fsync(fd);

PS. I think PostgreSQL doesn't have this problem.

It depends on the context, but in general PostgreSQL knows about that
sort of thing. When the cluster shuts down, it records that it shut
down cleanly, meaning that everything that should be on disk is on
disk. When you start the cluster up, if it sees that it didn't shut
down cleanly, it enters recovery. During recovery it tolerates files
being too short while it's replaying the WAL to get back to a
consistent state. See the comment in mdread() for example:

https://github.com/postgres/postgres/blob/master/src/backend/storage/smgr/md.c#L755

It's called "write-ahead log" because we log our intention before we
write to data files (and make sure it's on disk first), so we'll
always replay the same effects again if we're interrupted. The WAL is
a magic source of reliability (we can do it again if things go wrong)
and also performance (IO becomes serial, optimised for the storage
hardware).

https://www.postgresql.org/docs/current/wal-intro.html

--
Thomas Munro
http://www.enterprisedb.com