Storing large files in multiple schemas: BLOB or BYTEA

Started by Nonameover 13 years ago1 messagesgeneral
Jump to latest
#1Noname
tigran2-postgres@riatest.com

Yeah, a pg_dump mode that dumped everything but large objects would be

nice.

There is option -b for pg_dump which controls whether large objects are
dumped or no. The problem is that with option -b it dumps all large objects
regardless of what schema you requested it to dump using option -n.
Otherwise it works fine.

I'm now wondering about the idea of implementing a pg_dump option that
dumped large objects into a directory tree like
lobs/[loid]/[lob_md5]
and wrote out a restore script that loaded them using `lo_import`.

During dumping temporary copies could be written to something like
lobs/[loid]/.tmp.nnnn with the md5 being calculated on the fly as the
byte stream is read. If the dumped file had the same md5 as the existing
one it'd just delete the tempfile; otherwise the tempfile would be
renamed to the calculated md5.

That way incremental backup systems could manage the dumped LOB tree
without quite the same horrible degree of duplication as is currently
faced when using lo in the database with pg_dump.

A last_modified timestamp on `pg_largeobject_metadata` would be even
better, allowing the cost of reading and discarding rarely-changed large
objects to be avoided.

Definitely interesting idea with incremental backups.