BLOBs and a virtual file system

Peter Martini <PeterM@processflo.com> writes:

Lately I've been planning work on a patch to postgres and linux on my
system to allow access to BLOBs as a virtual filesystem, so I can see
any file I put in there even through network shares while avoiding
duplication / broken link issues. Does this sound like something worth
doing / is there a better way to safely reference files from both inside
and outside postgres?

How would you do that without breaking transactional integrity for
blobs? There'd be no way to deal with multiple row versions in such
a representation.

regards, tom lane

Peter Martini writes:

Lately I've been planning work on a patch to postgres and linux on my
system to allow access to BLOBs as a virtual filesystem, so I can see
any file I put in there even through network shares while avoiding
duplication / broken link issues. Does this sound like something worth
doing / is there a better way to safely reference files from both inside
and outside postgres?

I have been looking into using the Tcl VFS features for this kind of thing.
I work with AOLserver and PostgreSQL so I have Tcl and PostgreSQL available
to me.

http://nnsa.dl.ac.uk/MIDAS/manual/ActiveTcl8.4.4.0-html/tclvfs/doc/vfs.html

I haven't thought about accessing the VFS from inside PostgreSQL though.

Dave Bauer
dave@thedesignexperience.org

I'm not sure what you mean by breaking blobs transactional integrity.
Do you mean by allowing filesystem type access, the blobs won't be
properly locked and updated during a transaction? If so, that's exactly
what I'm trying to achieve - a compromise between forcing the files to
be stored solely in the database (maintaining integrity but sacrificing
functionality) or referencing their names to gain functionality, but at
a greater cost.
More to the point, wouldn't file locking mechanisms, and the possibility
of limiting file visibility through the virtual filesystem, allow
transactional integrity to be maintained? If not, could you explain
where the problem is so I can look further into it?

Thanks,
Peter

On Wed, 23 Jun 2004 15:07:42 -0400
Peter Martini <peterm@processflo.com> wrote:

I'm not sure what you mean by breaking blobs transactional integrity.
Do you mean by allowing filesystem type access, the blobs won't be
properly locked and updated during a transaction? If so, that's
exactly what I'm trying to achieve - a compromise between forcing the
files to be stored solely in the database (maintaining integrity but
sacrificing functionality) or referencing their names to gain
functionality, but at a greater cost.
More to the point, wouldn't file locking mechanisms, and the
possibility of limiting file visibility through the virtual
filesystem, allow transactional integrity to be maintained? If not,
could you explain where the problem is so I can look further into it?

I believe the problem is that the blob could be in several different
states inside of several different transactions. How do you
determine which you show in the filesystem?

Even if the file system is read only you still have this problem of
"which" of the many possible BLOBs to reveal.

---------------------------------
Frank Wiles <frank@wiles.org>
http://frank.wiles.org
---------------------------------