True ACID under linux (no fsync)?

I must have missed that one. How do you turn off fsync()??
I have lots of batch loading to do during which ACID is of no use.

++ kevin

"Poul L. Christiansen" wrote:

Strange.

I did a test on Redhat Linux 7.0 with fsync() turned off and my
batchfile of inserts ran 10 times as fast. But that was on an slow PC
with a slow IDE harddrive.

What Linux distribution and kernel version are you running?

Poul L. Christiansen

"Gary Howland (During daytime)" wrote:

Hi,

Just a quickie - I heard that linux does not have a working fsync() call
(it has no concept of raw devices). In other words, fsync cannot be used
to ensure that data is on disk (try it with a floppy - it returns IMMEDIATELY!!! - long before the write finishes)

This is for ALL linuxes.

Does this mean that postgres is unrealiable (or non ACID-conforming) under linux?

Gary

--
Friends help you move.
Real friends help you move bodies.

_______________________________________________________________________

Free Unlimited Internet Access! Try it now!
http://www.zdnet.com/downloads/altavista/index.html

_______________________________________________________________________

--
Kevin O'Gorman (805) 650-6274 mailto:kogorman@pacbell.net
Permanent e-mail forwarder: mailto:Kevin.O'Gorman.64@Alum.Dartmouth.org
At school: mailto:kogorman@cs.ucsb.edu
Web: http://www.cs.ucsb.edu/~kogorman/index.html
Web: http://trixie.kosman.via.ayuda.com/~kevin/index.html

"There is a freedom lying beyond circumstance,
derived from the direct intuition that life can
be grounded upon its absorption in what is
changeless amid change"
-- Alfred North Whitehead

Strange.

I did a test on Redhat Linux 7.0 with fsync() turned off and my
batchfile of inserts ran 10 times as fast. But that was on an slow PC
with a slow IDE harddrive.

What Linux distribution and kernel version are you running?

Poul L. Christiansen

"Gary Howland (During daytime)" wrote:

Use the "-F" option.

I start PostgreSQL with this line:
su -l postgres -c "/usr/bin/pg_ctl -D $PGDATA -p /usr/bin/postmaster -o
'-i -B 4096 -o -F' start >/dev/null 2>&1" < /dev/null

Interesting... trying to start postgres 7.0.2 with the "-F" simply returns
a list of options to me, as if it can't understand it. It sure does sound
appealing to me, though. With a few hundred megs of disk cache (between the
kernel and the RAID controller), I've always wished that PostgreSQL would
take advantage of it when writing. : )

steve

On Tue, 31 Oct 2000, Steve Wolfe wrote:

Interesting... trying to start postgres 7.0.2 with the "-F" simply returns
a list of options to me, as if it can't understand it. It sure does sound
appealing to me, though. With a few hundred megs of disk cache (between the
kernel and the RAID controller), I've always wished that PostgreSQL would
take advantage of it when writing. : )

postmaster -o -F -D /your/datadirectory

The -o tells postmaster to pass -F as backend option.

man postmaster and man postgres for more information.

--
Dominic J. Eidson
"Baruk Khazad! Khazad ai-menu!" - Gimli
-------------------------------------------------------------------------------
http://www.the-infinite.org/ http://www.the-infinite.org/~dominic/

Use the "-F" option.

I start PostgreSQL with this line:
su -l postgres -c "/usr/bin/pg_ctl -D $PGDATA -p /usr/bin/postmaster -o
'-i -B 4096 -o -F' start >/dev/null 2>&1" < /dev/null

Poul L. Christiansen

Kevin O'Gorman wrote:

"Gary Howland (During daytime)" wrote:

Hi,

Just a quickie - I heard that linux does not have a working fsync() call
(it has no concept of raw devices). In other words, fsync cannot be used
to ensure that data is on disk (try it with a floppy - it returns IMMEDIATELY!!! - long before the write finishes)

Are you sure? Where is this information?

I may be horriblly out of my element here but, I'm going to jump in anyway
:)

If you mount a floppy and copy a large file to it, you will get a prompt
back fairly quickly.

If you type: sync right after

The sync command writes everything to the floppy as expected.

Shawn

On Tue, 31 Oct 2000, markw wrote:

"Gary Howland (During daytime)" wrote:

Just a quickie - I heard that linux does not have a working fsync()
call (it has no concept of raw devices). In other words, fsync cannot
be used to ensure that data is on disk (try it with a floppy - it
returns IMMEDIATELY!!! - long before the write finishes)

That's because the mounted drive is buffered in memory, so when you copy a
file, you are copying it to memory (which is very fast), and the kernel
implicitly syncs with the media when the buffer fills or when an explicit
sync is done or when it is unmounted.

If there is no working fsync (and fdatasync), why do the Linux man pages
claim it to be POSIX.1b compliant?

Can you back up your claim with actual documentation? Saying 'I heard'
puts it in the realm of urban legendry...

Brett W. McCoy
http://www.chapelperilous.net
---------------------------------------------------------------------------
This is the ____LAST time I take travel suggestions from Ray Bradbury!

shawn everett wrote:

I may be horriblly out of my element here but, I'm going to jump in anyway
:)

If you mount a floppy and copy a large file to it, you will get a prompt
back fairly quickly.

If you type: sync right after

The sync command writes everything to the floppy as expected.

Yes, but does cp call fsync()? That is the question. I would suspect that it does not. A non-functional fsync call is a
serious error, and one should think more people would mention it. This is the first I have heard.

* bmccoy@chapelperilous.net <bmccoy@chapelperilous.net> [001031 12:28] wrote:

"Gary Howland (During daytime)" wrote:

Just a quickie - I heard that linux does not have a working fsync()
call (it has no concept of raw devices). In other words, fsync cannot
be used to ensure that data is on disk (try it with a floppy - it
returns IMMEDIATELY!!! - long before the write finishes)

That's because the mounted drive is buffered in memory, so when you copy a
file, you are copying it to memory (which is very fast), and the kernel
implicitly syncs with the media when the buffer fills or when an explicit
sync is done or when it is unmounted.

If there is no working fsync (and fdatasync), why do the Linux man pages
claim it to be POSIX.1b compliant?

Can you back up your claim with actual documentation? Saying 'I heard'
puts it in the realm of urban legendry...

You guys are missing the difference between fsync(2) and sync(1),
on Linux fsync(2) (called on a file descriptor) WILL sync out all
the data halfway to specification, HOWEVER afaik it will not fsync
out all directory entries that may contain this file.

If this is still true, that means that on Linux you must walk '..'
calling fsync on each directory above the file before considering
the file really there.

Now sync(2) and sync(1) (the userland interface to the sync(2)
syscall) work beyond the spec, on Linux sync(2) shouldn't return
until all data has been sync'd to disk, however the standard only
says that sync(2) should schedule the data to be sync'd out, not
that it needs to wait until it's all done.

<blatant plug>
Of course FreeBSD fsync(2) does properly walk '..' for you and
sync(2) behaves as specified by spec, mearly scheduling the IO,
not doing it syncronously.
</blatant plug>

--
-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
"I have the heart of a child; I keep it in a jar on my desk."

On Tue, 31 Oct 2000, Alfred Perlstein wrote:

You guys are missing the difference between fsync(2) and sync(1),
on Linux fsync(2) (called on a file descriptor) WILL sync out all
the data halfway to specification, HOWEVER afaik it will not fsync
out all directory entries that may contain this file.

That all depends on a filesystem. To my belief, all post-ext2 filesystems
(such as tux2, reiserfs, xfs) will properly flush metadata when
necessary..

-alex

On 31 Oct 2000, Gary Howland wrote:

Just a quickie - I heard that linux does not have a working fsync() call

At least the manpage for fsync says that it does.

The implementation:

/* .. finally sync the buffers to disk */
dev = inode->i_dev;
return sync_buffers(dev, 1);

It really looks like it IS implemented. But probably on Linux not just the
file data/metadata is synced, also all that device's data, which makes it
very inefficient, but presumably `safe'.

NB: don't forget that fsync() merely ensures that data was sent to the
disk controller. Maybe this one has a cache (e.g. a fast SCSI harddrive),
and if power fails, well. If fsync() was calling the SCSI FLUSH command,
maybe that could be done, but that would not just sync the file.

On Wed, Nov 01, 2000 at 11:16:24AM +0100, Marc SCHAEFER wrote:

Just a quickie - I heard that linux does not have a working fsync() call

At least the manpage for fsync says that it does.

The Linux fsync call (at least on ext2, I don't know if it's a
filesystem semantics issue or a VFS semantics issue) works slightly
different than the BSD call: On Linux, fsync flushes all the file data
and *file* metadata, but doesn't handle the directory metadata.
Directories are treated as seperate files which can be fsync'd
seperately, and need to be if you're adding or removing files
synchronously.
--
Bruce Guenter <bruceg@em.ca> http://em.ca/~bruceg/