Proposal: new large object API
I would like to propose new large object client side API for 8.4.
Currently we have:
Oid lo_import(PGconn *conn, const char *filename);
But we do not have an API which imports a large object specifying the
object id. This is inconvenient and inconsistent since we already have
lo_create() and lo_open() which allow to specify the large object id.
So I propose to add new API:
int lo_import_with_oid(PGconn *conn, Oid lobjId, const char *filename);
Another idea is changing the signature of lo_import:
Oid lo_import(PGconn *conn, Oid lobjId, const char *filename);
which will be cleaner but break the backward compatibility.
Comments are welcome.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Here is the proposed patches. If there's no objection, I will
commit(or Shall I wait for next "commit festa"?).
Note that I decide to use lo_import_with_oid, but the signature is
changed(the second and the third arguments are swapped because it
seems more natural).
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Show quoted text
I would like to propose new large object client side API for 8.4.
Currently we have:
Oid lo_import(PGconn *conn, const char *filename);
But we do not have an API which imports a large object specifying the
object id. This is inconvenient and inconsistent since we already have
lo_create() and lo_open() which allow to specify the large object id.So I propose to add new API:
int lo_import_with_oid(PGconn *conn, Oid lobjId, const char *filename);
Another idea is changing the signature of lo_import:
Oid lo_import(PGconn *conn, Oid lobjId, const char *filename);
which will be cleaner but break the backward compatibility.
Comments are welcome.
--
Tatsuo Ishii
SRA OSS, Inc. Japan--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
lob.patchtext/plain; charset=us-asciiDownload
? src/interfaces/libpq/exports.list
? src/interfaces/libpq/libpq.so.5.2
Index: doc/src/sgml/lobj.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/lobj.sgml,v
retrieving revision 1.46
diff -c -r1.46 lobj.sgml
*** doc/src/sgml/lobj.sgml 14 Mar 2007 00:15:26 -0000 1.46
--- doc/src/sgml/lobj.sgml 18 Mar 2008 01:02:10 -0000
***************
*** 161,166 ****
--- 161,188 ----
the server; so it must exist in the client file system and be readable
by the client application.
</para>
+
+ <para>
+ The function
+ <synopsis>
+ Oid lo_import_with_oid(PGconn *conn, const char *filename, Oid lobjId);
+ </synopsis>
+ <indexterm><primary>lo_import_with_oid</></>
+ also imports a new large object. The OID to be assigned can be
+ specified by <replaceable class="parameter">lobjId</replaceable>;
+ if so, failure occurs if that OID is already in use for some large
+ object. If <replaceable class="parameter">lobjId</replaceable>
+ is <symbol>InvalidOid</symbol> (zero) then <function>lo_import_with_oid</> assigns an unused
+ OID (this is the same behavior as <function>lo_import</>).
+ The return value is the OID that was assigned to the new large object,
+ or <symbol>InvalidOid</symbol> (zero) on failure.
+ </para>
+
+ <para>
+ <function>lo_import_with_oid</> is new as of <productname>PostgreSQL</productname>
+ 8.4 and uses <function>lo_create</function> internally which is new in 8.1; if this function is run against 8.0 or before, it will
+ fail and return <symbol>InvalidOid</symbol>.
+ </para>
</sect2>
<sect2>
Index: src/interfaces/libpq/exports.txt
===================================================================
RCS file: /cvsroot/pgsql/src/interfaces/libpq/exports.txt,v
retrieving revision 1.18
diff -c -r1.18 exports.txt
*** src/interfaces/libpq/exports.txt 9 Dec 2007 19:01:40 -0000 1.18
--- src/interfaces/libpq/exports.txt 18 Mar 2008 01:02:10 -0000
***************
*** 140,142 ****
--- 140,143 ----
PQconnectionUsedPassword 138
pg_valid_server_encoding_id 139
PQconnectionNeedsPassword 140
+ lo_import_with_oid 141
Index: src/interfaces/libpq/fe-lobj.c
===================================================================
RCS file: /cvsroot/pgsql/src/interfaces/libpq/fe-lobj.c,v
retrieving revision 1.64
diff -c -r1.64 fe-lobj.c
*** src/interfaces/libpq/fe-lobj.c 1 Jan 2008 19:46:00 -0000 1.64
--- src/interfaces/libpq/fe-lobj.c 18 Mar 2008 01:02:11 -0000
***************
*** 41,46 ****
--- 41,48 ----
static int lo_initialize(PGconn *conn);
+ static Oid
+ lo_import_internal(PGconn *conn, const char *filename, const Oid oid);
/*
* lo_open
***************
*** 484,489 ****
--- 486,512 ----
Oid
lo_import(PGconn *conn, const char *filename)
{
+ return lo_import_internal(conn, filename, InvalidOid);
+ }
+
+ /*
+ * lo_import_with_oid -
+ * imports a file as an (inversion) large object.
+ * large object id can be specified.
+ *
+ * returns the oid of that object upon success,
+ * returns InvalidOid upon failure
+ */
+
+ Oid
+ lo_import_with_oid(PGconn *conn, const char *filename, Oid lobjId)
+ {
+ return lo_import_internal(conn, filename, lobjId);
+ }
+
+ static Oid
+ lo_import_internal(PGconn *conn, const char *filename, Oid oid)
+ {
int fd;
int nbytes,
tmp;
***************
*** 507,516 ****
/*
* create an inversion object
*/
! lobjOid = lo_creat(conn, INV_READ | INV_WRITE);
if (lobjOid == InvalidOid)
{
! /* we assume lo_creat() already set a suitable error message */
(void) close(fd);
return InvalidOid;
}
--- 530,543 ----
/*
* create an inversion object
*/
! if (oid == InvalidOid)
! lobjOid = lo_creat(conn, INV_READ | INV_WRITE);
! else
! lobjOid = lo_create(conn, oid);
!
if (lobjOid == InvalidOid)
{
! /* we assume lo_create() already set a suitable error message */
(void) close(fd);
return InvalidOid;
}
Index: src/interfaces/libpq/libpq-fe.h
===================================================================
RCS file: /cvsroot/pgsql/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.141
diff -c -r1.141 libpq-fe.h
*** src/interfaces/libpq/libpq-fe.h 1 Jan 2008 19:46:00 -0000 1.141
--- src/interfaces/libpq/libpq-fe.h 18 Mar 2008 01:02:12 -0000
***************
*** 495,500 ****
--- 495,501 ----
extern int lo_truncate(PGconn *conn, int fd, size_t len);
extern int lo_unlink(PGconn *conn, Oid lobjId);
extern Oid lo_import(PGconn *conn, const char *filename);
+ extern Oid lo_import_with_oid(PGconn *conn, const char *filename, Oid lobjId);
extern int lo_export(PGconn *conn, Oid lobjId, const char *filename);
/* === in fe-misc.c === */
I have posted proposed patches to pgsql-patches.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Show quoted text
I would like to propose new large object client side API for 8.4.
Currently we have:
Oid lo_import(PGconn *conn, const char *filename);
But we do not have an API which imports a large object specifying the
object id. This is inconvenient and inconsistent since we already have
lo_create() and lo_open() which allow to specify the large object id.So I propose to add new API:
int lo_import_with_oid(PGconn *conn, Oid lobjId, const char *filename);
Another idea is changing the signature of lo_import:
Oid lo_import(PGconn *conn, Oid lobjId, const char *filename);
which will be cleaner but break the backward compatibility.
Comments are welcome.
--
Tatsuo Ishii
SRA OSS, Inc. Japan--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Tatsuo Ishii <ishii@postgresql.org> writes:
Here is the proposed patches. If there's no objection, I will
commit(or Shall I wait for next "commit festa"?).
No need to wait IMHO.
regards, tom lane
lo_import_with_oid added.
Note that actually committed function signature is:
Oid lo_import_with_oid(PGconn *conn, const char *filename, Oid lobjId);
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Show quoted text
I have posted proposed patches to pgsql-patches.
--
Tatsuo Ishii
SRA OSS, Inc. JapanI would like to propose new large object client side API for 8.4.
Currently we have:
Oid lo_import(PGconn *conn, const char *filename);
But we do not have an API which imports a large object specifying the
object id. This is inconvenient and inconsistent since we already have
lo_create() and lo_open() which allow to specify the large object id.So I propose to add new API:
int lo_import_with_oid(PGconn *conn, Oid lobjId, const char *filename);
Another idea is changing the signature of lo_import:
Oid lo_import(PGconn *conn, Oid lobjId, const char *filename);
which will be cleaner but break the backward compatibility.
Comments are welcome.
--
Tatsuo Ishii
SRA OSS, Inc. Japan--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
lo_import_with_oid added.
Note that actually committed function signature is:
Oid lo_import_with_oid(PGconn *conn, const char *filename, Oid lobjId);
--
Tatsuo Ishii
SRA OSS, Inc. Japan
It seems I forgot about the serer side lo_import. Included are the
patches to add new form of lo_import which accepts the large object id
as the second argument.
Comments, objection?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Attachments:
lobj.difftext/plain; charset=us-asciiDownload
Index: src/include/catalog/pg_proc.h
===================================================================
RCS file: /cvsroot/pgsql/src/include/catalog/pg_proc.h,v
retrieving revision 1.482
diff -c -r1.482 pg_proc.h
*** src/include/catalog/pg_proc.h 1 Jan 2008 19:45:57 -0000 1.482
--- src/include/catalog/pg_proc.h 20 Mar 2008 05:58:38 -0000
***************
*** 1027,1032 ****
--- 1027,1034 ----
DATA(insert OID = 764 ( lo_import PGNSP PGUID 12 1 0 f f t f v 1 26 "25" _null_ _null_ _null_ lo_import - _null_ _null_ ));
DESCR("large object import");
+ DATA(insert OID = 767 ( lo_import PGNSP PGUID 12 1 0 f f t f v 2 26 "25 26" _null_ _null_ _null_ lo_import - _null_ _null_ ));
+ DESCR("large object import");
DATA(insert OID = 765 ( lo_export PGNSP PGUID 12 1 0 f f t f v 2 23 "26 25" _null_ _null_ _null_ lo_export - _null_ _null_ ));
DESCR("large object export");
Index: src/backend/libpq/be-fsstubs.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/libpq/be-fsstubs.c,v
retrieving revision 1.87
diff -c -r1.87 be-fsstubs.c
*** src/backend/libpq/be-fsstubs.c 1 Jan 2008 19:45:49 -0000 1.87
--- src/backend/libpq/be-fsstubs.c 20 Mar 2008 05:58:38 -0000
***************
*** 327,333 ****
char fnamebuf[MAXPGPATH];
LargeObjectDesc *lobj;
Oid lobjOid;
!
#ifndef ALLOW_DANGEROUS_LO_FUNCTIONS
if (!superuser())
ereport(ERROR,
--- 327,333 ----
char fnamebuf[MAXPGPATH];
LargeObjectDesc *lobj;
Oid lobjOid;
!
#ifndef ALLOW_DANGEROUS_LO_FUNCTIONS
if (!superuser())
ereport(ERROR,
***************
*** 336,341 ****
--- 336,346 ----
errhint("Anyone can use the client-side lo_import() provided by libpq.")));
#endif
+ if (PG_NARGS() > 1)
+ lobjOid = PG_GETARG_OID(1);
+ else
+ lobjOid = InvalidOid;
+
CreateFSContext();
/*
***************
*** 356,362 ****
/*
* create an inversion object
*/
! lobjOid = inv_create(InvalidOid);
/*
* read in from the filesystem and write to the inversion object
--- 361,367 ----
/*
* create an inversion object
*/
! lobjOid = inv_create(lobjOid);
/*
* read in from the filesystem and write to the inversion object
Index: doc/src/sgml/lobj.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/lobj.sgml,v
retrieving revision 1.47
diff -c -r1.47 lobj.sgml
*** doc/src/sgml/lobj.sgml 19 Mar 2008 00:39:33 -0000 1.47
--- doc/src/sgml/lobj.sgml 20 Mar 2008 05:58:38 -0000
***************
*** 438,443 ****
--- 438,450 ----
The client-side functions can be used by any
<productname>PostgreSQL</productname> user.
</para>
+
+ <para>
+ As of 8.4, a different form of the server-side
+ <function>lo_import</function> added, which accepts the large
+ object id as the second argument. The usage of this form is same as the client
+ side function <function>lo_import_with_oid</function>.
+ </para>
</sect1>
<sect1 id="lo-examplesect">
Index: src/test/regress/expected/opr_sanity.out
===================================================================
RCS file: /cvsroot/pgsql/src/test/regress/expected/opr_sanity.out,v
retrieving revision 1.79
diff -c -r1.79 opr_sanity.out
*** src/test/regress/expected/opr_sanity.out 27 Nov 2007 12:21:05 -0000 1.79
--- src/test/regress/expected/opr_sanity.out 20 Mar 2008 05:58:39 -0000
***************
*** 86,94 ****
p1.proretset != p2.proretset OR
p1.provolatile != p2.provolatile OR
p1.pronargs != p2.pronargs);
! oid | proname | oid | proname
! -----+---------+-----+---------
! (0 rows)
-- Look for uses of different type OIDs in the argument/result type fields
-- for different aliases of the same built-in function.
--- 86,95 ----
p1.proretset != p2.proretset OR
p1.provolatile != p2.provolatile OR
p1.pronargs != p2.pronargs);
! oid | proname | oid | proname
! -----+-----------+-----+-----------
! 764 | lo_import | 767 | lo_import
! (1 row)
-- Look for uses of different type OIDs in the argument/result type fields
-- for different aliases of the same built-in function.
Tatsuo Ishii <ishii@postgresql.org> writes:
It seems I forgot about the serer side lo_import. Included are the
patches to add new form of lo_import which accepts the large object id
as the second argument.
Comments, objection?
Breaking the type_sanity test is not acceptable. Put in a second C
function.
regards, tom lane
Tatsuo Ishii <ishii@postgresql.org> writes:
It seems I forgot about the serer side lo_import. Included are the
patches to add new form of lo_import which accepts the large object id
as the second argument.Comments, objection?
Breaking the type_sanity test is not acceptable. Put in a second C
function.
Are you talking opr_sanity?
From what I read from opr_sanity.sql:
-- Look for uses of different type OIDs in the argument/result type fields
-- for different aliases of the same built-in function.
-- This indicates that the types are being presumed to be binary-equivalent,
-- or that the built-in function is prepared to deal with different types.
-- That's not wrong, necessarily, but we make lists of all the types being
-- so treated. Note that the expected output of this part of the test will
-- need to be modified whenever new pairs of types are made binary-equivalent,
-- or when new polymorphic built-in functions are added!
-- Note: ignore aggregate functions here, since they all point to the same
-- dummy built-in function.
What is evil with a polymorphic function?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Tatsuo Ishii <ishii@postgresql.org> writes:
Breaking the type_sanity test is not acceptable. Put in a second C
function.
Are you talking opr_sanity?
Sorry, yes, too little caffeine ...
What is evil with a polymorphic function?
(1) It's creating a false match --- your proposed entry in the opr_sanity
results has nothing at all to do with what the test is looking for.
(2) Refactoring to have two separate C functions will make the code
clearer, and not noticeably longer.
regards, tom lane
What is evil with a polymorphic function?
(1) It's creating a false match --- your proposed entry in the opr_sanity
results has nothing at all to do with what the test is looking for.(2) Refactoring to have two separate C functions will make the code
clearer, and not noticeably longer.
Ok, here is the revised patch.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Attachments:
lobj.difftext/plain; charset=us-asciiDownload
Index: src/backend/libpq/be-fsstubs.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/libpq/be-fsstubs.c,v
retrieving revision 1.87
diff -c -r1.87 be-fsstubs.c
*** src/backend/libpq/be-fsstubs.c 1 Jan 2008 19:45:49 -0000 1.87
--- src/backend/libpq/be-fsstubs.c 21 Mar 2008 14:35:02 -0000
***************
*** 79,84 ****
--- 79,85 ----
static int newLOfd(LargeObjectDesc *lobjCookie);
static void deleteLOfd(int fd);
+ static Oid lo_import_internal(text *filename, Oid lobjOid);
/*****************************************************************************
***************
*** 320,333 ****
lo_import(PG_FUNCTION_ARGS)
{
text *filename = PG_GETARG_TEXT_P(0);
File fd;
int nbytes,
tmp;
char buf[BUFSIZE];
char fnamebuf[MAXPGPATH];
LargeObjectDesc *lobj;
! Oid lobjOid;
!
#ifndef ALLOW_DANGEROUS_LO_FUNCTIONS
if (!superuser())
ereport(ERROR,
--- 321,354 ----
lo_import(PG_FUNCTION_ARGS)
{
text *filename = PG_GETARG_TEXT_P(0);
+
+ PG_RETURN_OID(lo_import_internal(filename, InvalidOid));
+ }
+
+ /*
+ * lo_import_with_oid -
+ * imports a file as an (inversion) large object specifying oid.
+ */
+ Datum
+ lo_import_with_oid(PG_FUNCTION_ARGS)
+ {
+ text *filename = PG_GETARG_TEXT_P(0);
+ Oid oid = PG_GETARG_OID(1);
+
+ PG_RETURN_OID(lo_import_internal(filename, oid));
+ }
+
+ static Oid
+ lo_import_internal(text *filename, Oid lobjOid)
+ {
File fd;
int nbytes,
tmp;
char buf[BUFSIZE];
char fnamebuf[MAXPGPATH];
LargeObjectDesc *lobj;
! Oid oid;
!
#ifndef ALLOW_DANGEROUS_LO_FUNCTIONS
if (!superuser())
ereport(ERROR,
***************
*** 356,367 ****
/*
* create an inversion object
*/
! lobjOid = inv_create(InvalidOid);
/*
* read in from the filesystem and write to the inversion object
*/
! lobj = inv_open(lobjOid, INV_WRITE, fscxt);
while ((nbytes = FileRead(fd, buf, BUFSIZE)) > 0)
{
--- 377,388 ----
/*
* create an inversion object
*/
! oid = inv_create(lobjOid);
/*
* read in from the filesystem and write to the inversion object
*/
! lobj = inv_open(oid, INV_WRITE, fscxt);
while ((nbytes = FileRead(fd, buf, BUFSIZE)) > 0)
{
***************
*** 378,384 ****
inv_close(lobj);
FileClose(fd);
! PG_RETURN_OID(lobjOid);
}
/*
--- 399,405 ----
inv_close(lobj);
FileClose(fd);
! return oid;
}
/*
Index: src/include/catalog/pg_proc.h
===================================================================
RCS file: /cvsroot/pgsql/src/include/catalog/pg_proc.h,v
retrieving revision 1.482
diff -c -r1.482 pg_proc.h
*** src/include/catalog/pg_proc.h 1 Jan 2008 19:45:57 -0000 1.482
--- src/include/catalog/pg_proc.h 21 Mar 2008 14:35:07 -0000
***************
*** 1027,1032 ****
--- 1027,1034 ----
DATA(insert OID = 764 ( lo_import PGNSP PGUID 12 1 0 f f t f v 1 26 "25" _null_ _null_ _null_ lo_import - _null_ _null_ ));
DESCR("large object import");
+ DATA(insert OID = 767 ( lo_import PGNSP PGUID 12 1 0 f f t f v 2 26 "25 26" _null_ _null_ _null_ lo_import_with_oid - _null_ _null_ ));
+ DESCR("large object import");
DATA(insert OID = 765 ( lo_export PGNSP PGUID 12 1 0 f f t f v 2 23 "26 25" _null_ _null_ _null_ lo_export - _null_ _null_ ));
DESCR("large object export");
Index: src/include/catalog/catversion.h
===================================================================
RCS file: /cvsroot/pgsql/src/include/catalog/catversion.h,v
retrieving revision 1.442
diff -c -r1.442 catversion.h
*** src/include/catalog/catversion.h 10 Mar 2008 13:53:35 -0000 1.442
--- src/include/catalog/catversion.h 21 Mar 2008 14:35:07 -0000
***************
*** 53,58 ****
*/
/* yyyymmddN */
! #define CATALOG_VERSION_NO 200803101
#endif
--- 53,58 ----
*/
/* yyyymmddN */
! #define CATALOG_VERSION_NO 200803211
#endif
Index: src/include/libpq/be-fsstubs.h
===================================================================
RCS file: /cvsroot/pgsql/src/include/libpq/be-fsstubs.h,v
retrieving revision 1.30
diff -c -r1.30 be-fsstubs.h
*** src/include/libpq/be-fsstubs.h 1 Jan 2008 19:45:58 -0000 1.30
--- src/include/libpq/be-fsstubs.h 21 Mar 2008 14:35:07 -0000
***************
*** 20,25 ****
--- 20,26 ----
* LO functions available via pg_proc entries
*/
extern Datum lo_import(PG_FUNCTION_ARGS);
+ extern Datum lo_import_with_oid(PG_FUNCTION_ARGS);
extern Datum lo_export(PG_FUNCTION_ARGS);
extern Datum lo_creat(PG_FUNCTION_ARGS);
Index: doc/src/sgml/lobj.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/lobj.sgml,v
retrieving revision 1.47
diff -c -r1.47 lobj.sgml
*** doc/src/sgml/lobj.sgml 19 Mar 2008 00:39:33 -0000 1.47
--- doc/src/sgml/lobj.sgml 21 Mar 2008 14:35:07 -0000
***************
*** 438,443 ****
--- 438,450 ----
The client-side functions can be used by any
<productname>PostgreSQL</productname> user.
</para>
+
+ <para>
+ As of 8.4, a different form of the server-side
+ <function>lo_import</function> added, which accepts the large
+ object id as the second argument. The usage of this form is same as the client
+ side function <function>lo_import_with_oid</function>.
+ </para>
</sect1>
<sect1 id="lo-examplesect">
Tatsuo Ishii <ishii@postgresql.org> writes:
Ok, here is the revised patch.
This looks sane to me, but I'd suggest leaving out the mention of 8.4
in the docs. Actually, I'm not sure you need a paragraph at all ---
just adding an example would be enough, I think.
SELECT lo_unlink(173454); -- deletes large object with OID 173454
INSERT INTO image (name, raster)
VALUES ('beautiful image', lo_import('/etc/motd'));
+
+ INSERT INTO image (name, raster) -- same as above, but specify OID to use
+ VALUES ('beautiful image', lo_import('/etc/motd', 68583));
SELECT lo_export(image.raster, '/tmp/motd') FROM image
WHERE name = 'beautiful image';
</programlisting>
regards, tom lane
Tatsuo Ishii <ishii@postgresql.org> writes:
Ok, here is the revised patch.
This looks sane to me, but I'd suggest leaving out the mention of 8.4
in the docs. Actually, I'm not sure you need a paragraph at all ---
just adding an example would be enough, I think.SELECT lo_unlink(173454); -- deletes large object with OID 173454
INSERT INTO image (name, raster) VALUES ('beautiful image', lo_import('/etc/motd')); + + INSERT INTO image (name, raster) -- same as above, but specify OID to use + VALUES ('beautiful image', lo_import('/etc/motd', 68583));SELECT lo_export(image.raster, '/tmp/motd') FROM image
WHERE name = 'beautiful image';
</programlisting>
Thanks for the comment. I have committed with your suggested doc
changing.
--
Tatsuo Ishii
SRA OSS, Inc. Japan