problems transfering databases

Started by Miroslav Koncarover 24 years ago20 messagesgeneral
Jump to latest
#1Miroslav Koncar
miroslav.koncar@etk.ericsson.se

Hello,

I'm a newbie in the postgreSQL world, so if I'm asking what's rather
straightforward, I appologize.

This is what I'm trying to do: my collegue and I have set up three
postgreSQL databases; he has set it up on Linux RedHat 6.2, and I've
been working on RedHat 6.2 as well on Solaris 7. We are trying to
transfer all the data from his database to mine (my primary database is
on Solaris). So, we consulted the docs, and he has used the command:

pg_dumpall > backups/2001-06-04db.out

and sent me that file. But, when I try to load it to my database, I get
the error message:

obonjan[postgres-> ~/tmp]%psql -f 2001-08-21db.out
You are now connected to database template1.
SELECT
DELETE 0
DROP
'sql:2001-08-21db.out:5: ERROR: copy: line 1, Bad abstime external
representation '\N
psql:2001-08-21db.out:5: PQendcopy: resetting connection

I thougth, it might be something to do with Solaris, but on Linux, like
expected, the same error.

What amd I doing wrong? Please, contact me directly, since I'm not on
the mailing list.

Thanks,
Miroslav

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

'sql:2001-08-21db.out:5: ERROR: copy: line 1, Bad abstime external
representation '\N
psql:2001-08-21db.out:5: PQendcopy: resetting connection

Given the weird line-wrapping of the message, I'm going to bet that the
problem is that newlines in the dump file have gotten converted to DOS
format (ie, \n became \r\n), and COPY is unhappy because it sees the \r
as part of the data. When it echoes the data it didn't like, the
presence of the \r messes up the format of the error message.

Not sure *how* that happened in a Linux-to-Solaris transfer, though;
what tool did you use to transfer the dump file?

regards, tom lane

#3Miroslav Koncar
miroslav.koncar@etk.ericsson.se
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

Hello Tom,

thanks for the answer. This is the procedure, how we tried to transfer the
data from one machine to another.
My collegue has used the pg_dumpall command, in the following way:

pg_dumpall > backups/2001-06-04db.out

Since it is a simple test database, the 2001-06-04db.out file was around
40kB, so he has sent it to me via email.
I tried to load the data from the file in two different ways:

psql -f 2001-08-21db.out
cat 2001-06-04db.out | psql

but got the error like stated before. I've tried to do this on Solaris and
Linux (I've set postgreSQL up on my Linux box for testing purposes), but
the same error pops up.

Could it have something to do with the versions of the databases, and how
it was installed? The version I use is
psql (PostgreSQL) 7.1.2 (the output of psql -V command), but I'm not sure
what version did my collegue use (I'll find that out, but I think it is
7.x.x).

Regards,
Miroslav

Tom Lane wrote:

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

'sql:2001-08-21db.out:5: ERROR: copy: line 1, Bad abstime external
representation '\N
psql:2001-08-21db.out:5: PQendcopy: resetting connection

Given the weird line-wrapping of the message, I'm going to bet that the
problem is that newlines in the dump file have gotten converted to DOS
format (ie, \n became \r\n), and COPY is unhappy because it sees the \r
as part of the data. When it echoes the data it didn't like, the
presence of the \r messes up the format of the error message.

Not sure *how* that happened in a Linux-to-Solaris transfer, though;
what tool did you use to transfer the dump file?

regards, tom lane

--
Miroslav Koncar
Software Engineer
Ericsson Nikola Tesla
ETK/D/R
Tel: +385 1 365 3479
Fax: +385 1 365 3548
mailto:miroslav.koncar@etk.ericsson.se

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Miroslav Koncar (#3)
Re: problems transfering databases

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

Since it is a simple test database, the 2001-06-04db.out file was around
40kB, so he has sent it to me via email.

Email huh? I bet some part of that process was Windows? Almost
certainly that's where the carriage return characters came from.

regards, tom lane

#5Jeff Eckermann
jeckermann@verio.net
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

Looks like you already have the answer. The error message is a dead giveaway. Some field, in this case having a null value (represented as '\N') contains a carriage return character, like: '\NCR'. PostgreSQL cannot make sense out of this combination, so it returns an error. The CR contained in the error message causes the output to return to the start of the line, outputting the rest of the message from the beginning: see how the final " ' " overwrites the "p" in "psql".
Edit out the CR characters, and the problem will go away.
This is what Tom already said; I am expanding on it.
----- Original Message -----
From: Miroslav Koncar
To: Tom Lane
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 3:18 AM
Subject: Re: [GENERAL] problems transfering databases

Hello Tom,
thanks for the answer. This is the procedure, how we tried to transfer the data from one machine to another.
My collegue has used the pg_dumpall command, in the following way:

pg_dumpall > backups/2001-06-04db.out

Since it is a simple test database, the 2001-06-04db.out file was around 40kB, so he has sent it to me via email.
I tried to load the data from the file in two different ways:

psql -f 2001-08-21db.out
cat 2001-06-04db.out | psql

but got the error like stated before. I've tried to do this on Solaris and Linux (I've set postgreSQL up on my Linux box for testing purposes), but the same error pops up.

Could it have something to do with the versions of the databases, and how it was installed? The version I use is
psql (PostgreSQL) 7.1.2 (the output of psql -V command), but I'm not sure what version did my collegue use (I'll find that out, but I think it is 7.x.x).

Regards,
Miroslav

Tom Lane wrote:

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

'sql:2001-08-21db.out:5: ERROR: copy: line 1, Bad abstime external
representation '\N
psql:2001-08-21db.out:5: PQendcopy: resetting connection

Given the weird line-wrapping of the message, I'm going to bet that the
problem is that newlines in the dump file have gotten converted to DOS
format (ie, \n became \r\n), and COPY is unhappy because it sees the \r
as part of the data. When it echoes the data it didn't like, the
presence of the \r messes up the format of the error message.

Not sure *how* that happened in a Linux-to-Solaris transfer, though;
what tool did you use to transfer the dump file?

regards, tom lane

--
Miroslav Koncar
Software Engineer
Ericsson Nikola Tesla
ETK/D/R
Tel: +385 1 365 3479
Fax: +385 1 365 3548
mailto:miroslav.koncar@etk.ericsson.se

#6Gilles Darold
gilles@darold.net
In reply to: Miroslav Koncar (#1)
Problem with libpsqlodbc

Hi,

I have send some patches to the OpenLDAP team to have it work with
PostgreSQL and they should be applied asap. This works fine but with
a not usefull workaround.

I we want to have PG fully compatible with OpenLDAP we need to resolve
a problem regarding the libpsqlodbc library.

In OpenLDAP they call the function SQLBindParameter as follow:

SQLBindParameter(sth,1,SQL_PARAM_OUTPUT,SQL_C_ULONG,SQL_INTEGER,0,0,&new_keyval,0,0);

And then the statement is executed with SQLExecDirect as follow:

rc=SQLExecDirect(sth,oc->create_proc,SQL_NTS);

What they expected is that the 'new_keyval' pointer may be modified at
execution time,
with the new unique identifier returned by the query, typically a
sequence number.

It works fine with other database (mySQL, Oracle, etc.) but not with
PostgreSQL.

What's the problem ? Is the SQLBindParameter call is wrong ?
My knowledge is not enougth to find it.

If someone could explain me what's wrong !

Regards,

Gilles DAROLD

#7rob
rob@cabrion.com
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

I've had this problem before. The data is stored correctly in PG, it just doesn't dump right (the CR thing I guess. I didn't know the reason at the time). I think this was on 7.0.x. I worked around it by doing a pg_dump that dumped the table as "proper inserts". That make the load MUCH slower, but it works around the issue. Now that I know what the cause might have been, I guess I better go look at my data . . .

--rob
----- Original Message -----
From: Jeff Eckermann
To: Miroslav Koncar
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 10:59 AM
Subject: Re: problems transfering databases

Looks like you already have the answer. The error message is a dead giveaway. Some field, in this case having a null value (represented as '\N') contains a carriage return character, like: '\NCR'. PostgreSQL cannot make sense out of this combination, so it returns an error. The CR contained in the error message causes the output to return to the start of the line, outputting the rest of the message from the beginning: see how the final " ' " overwrites the "p" in "psql".
Edit out the CR characters, and the problem will go away.
This is what Tom already said; I am expanding on it.
----- Original Message -----
From: Miroslav Koncar
To: Tom Lane
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 3:18 AM
Subject: Re: [GENERAL] problems transfering databases

Hello Tom,
thanks for the answer. This is the procedure, how we tried to transfer the data from one machine to another.
My collegue has used the pg_dumpall command, in the following way:

pg_dumpall > backups/2001-06-04db.out

Since it is a simple test database, the 2001-06-04db.out file was around 40kB, so he has sent it to me via email.
I tried to load the data from the file in two different ways:

psql -f 2001-08-21db.out
cat 2001-06-04db.out | psql

but got the error like stated before. I've tried to do this on Solaris and Linux (I've set postgreSQL up on my Linux box for testing purposes), but the same error pops up.

Could it have something to do with the versions of the databases, and how it was installed? The version I use is
psql (PostgreSQL) 7.1.2 (the output of psql -V command), but I'm not sure what version did my collegue use (I'll find that out, but I think it is 7.x.x).

Regards,
Miroslav

Tom Lane wrote:

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

'sql:2001-08-21db.out:5: ERROR: copy: line 1, Bad abstime external
representation '\N
psql:2001-08-21db.out:5: PQendcopy: resetting connection

Given the weird line-wrapping of the message, I'm going to bet that the
problem is that newlines in the dump file have gotten converted to DOS
format (ie, \n became \r\n), and COPY is unhappy because it sees the \r
as part of the data. When it echoes the data it didn't like, the
presence of the \r messes up the format of the error message.

Not sure *how* that happened in a Linux-to-Solaris transfer, though;
what tool did you use to transfer the dump file?

regards, tom lane

--
Miroslav Koncar
Software Engineer
Ericsson Nikola Tesla
ETK/D/R
Tel: +385 1 365 3479
Fax: +385 1 365 3548
mailto:miroslav.koncar@etk.ericsson.se

#8Miroslav Koncar
miroslav.koncar@etk.ericsson.se
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

Hello, Tom,

nope, no windows here. The file was sent direclty from the Linux to my
mailbox, which I read on Solaris.

The only thing I can think of is the PostgreSQL versions. My collegue is
using 7.0.3, and I use 7.1.2. Comments?

Miroslav

Tom Lane wrote:

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

Since it is a simple test database, the 2001-06-04db.out file was around
40kB, so he has sent it to me via email.

Email huh? I bet some part of that process was Windows? Almost
certainly that's where the carriage return characters came from.

regards, tom lane

--
Miroslav Koncar
Software Engineer
Ericsson Nikola Tesla
ETK/D/R
Tel: +385 1 365 3479
Fax: +385 1 365 3548
mailto:miroslav.koncar@etk.ericsson.se

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Miroslav Koncar (#8)
Re: problems transfering databases

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

The only thing I can think of is the PostgreSQL versions. My collegue is
using 7.0.3, and I use 7.1.2. Comments?

That wouldn't produce a carriage-return problem.

It's possible that we're barking up the wrong tree, and that the issue
is not carriage returns but something else --- I just jumped to that
conclusion based on the way you presented the error message. But maybe
the error message was mangled because you were sloppy about cutting-and-
pasting it into your email, or some such. Have you looked at the data
to try to narrow down exactly which line is being rejected?

regards, tom lane

#10Miroslav Koncar
miroslav.koncar@etk.ericsson.se
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

Hello Tom,

problem solved! The error was caused by interpreting \N as a special
character, and not what postgreSQL wants it to be. I've solved it by
replaceing all \N with \\N in a texteditor, which made the shell not to
interpret the character.

Thanks for your suggestions and help.

Miroslav

Tom Lane wrote:

Show quoted text

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

The only thing I can think of is the PostgreSQL versions. My collegue is
using 7.0.3, and I use 7.1.2. Comments?

That wouldn't produce a carriage-return problem.

It's possible that we're barking up the wrong tree, and that the issue
is not carriage returns but something else --- I just jumped to that
conclusion based on the way you presented the error message. But maybe
the error message was mangled because you were sloppy about cutting-and-
pasting it into your email, or some such. Have you looked at the data
to try to narrow down exactly which line is being rejected?

regards, tom lane

#11Jeff Eckermann
jeckermann@verio.net
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

I very much doubt that the data wouldn't "dump right". The dump will include any data that is in the table, including CRs if they are in there.
In the current case, the CRs were not already in there, because PostgreSQL would not have recognized that field as null ("\N") if they were.
Any transfer of a file via a Windows machine is apt to cause line endings to be silently changed, which would account for this case. The real danger is where the rightmost field is a text type, because the CRs would be silently imported into the new installation, where they could cause real problems with any app that tries to use those fields, and a headache to debug.
----- Original Message -----
From: Rob Arnold
To: Jeff Eckermann ; Miroslav Koncar
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 6:32 PM
Subject: Re: problems transfering databases

I've had this problem before. The data is stored correctly in PG, it just doesn't dump right (the CR thing I guess. I didn't know the reason at the time). I think this was on 7.0.x. I worked around it by doing a pg_dump that dumped the table as "proper inserts". That make the load MUCH slower, but it works around the issue. Now that I know what the cause might have been, I guess I better go look at my data . . .

--rob
----- Original Message -----
From: Jeff Eckermann
To: Miroslav Koncar
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 10:59 AM
Subject: Re: problems transfering databases

Looks like you already have the answer. The error message is a dead giveaway. Some field, in this case having a null value (represented as '\N') contains a carriage return character, like: '\NCR'. PostgreSQL cannot make sense out of this combination, so it returns an error. The CR contained in the error message causes the output to return to the start of the line, outputting the rest of the message from the beginning: see how the final " ' " overwrites the "p" in "psql".
Edit out the CR characters, and the problem will go away.
This is what Tom already said; I am expanding on it.
----- Original Message -----
From: Miroslav Koncar
To: Tom Lane
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 3:18 AM
Subject: Re: [GENERAL] problems transfering databases

Hello Tom,
thanks for the answer. This is the procedure, how we tried to transfer the data from one machine to another.
My collegue has used the pg_dumpall command, in the following way:

pg_dumpall > backups/2001-06-04db.out

Since it is a simple test database, the 2001-06-04db.out file was around 40kB, so he has sent it to me via email.
I tried to load the data from the file in two different ways:

psql -f 2001-08-21db.out
cat 2001-06-04db.out | psql

but got the error like stated before. I've tried to do this on Solaris and Linux (I've set postgreSQL up on my Linux box for testing purposes), but the same error pops up.

Could it have something to do with the versions of the databases, and how it was installed? The version I use is
psql (PostgreSQL) 7.1.2 (the output of psql -V command), but I'm not sure what version did my collegue use (I'll find that out, but I think it is 7.x.x).

Regards,
Miroslav

Tom Lane wrote:

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

'sql:2001-08-21db.out:5: ERROR: copy: line 1, Bad abstime external
representation '\N
psql:2001-08-21db.out:5: PQendcopy: resetting connection

Given the weird line-wrapping of the message, I'm going to bet that the
problem is that newlines in the dump file have gotten converted to DOS
format (ie, \n became \r\n), and COPY is unhappy because it sees the \r
as part of the data. When it echoes the data it didn't like, the
presence of the \r messes up the format of the error message.

Not sure *how* that happened in a Linux-to-Solaris transfer, though;
what tool did you use to transfer the dump file?

regards, tom lane

--
Miroslav Koncar
Software Engineer
Ericsson Nikola Tesla
ETK/D/R
Tel: +385 1 365 3479
Fax: +385 1 365 3548
mailto:miroslav.koncar@etk.ericsson.se

#12Jeff Eckermann
jeckermann@verio.net
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

I think we may be using too many words in this discussion.
Perhaps there is a message in this thread that I have not seen.
Miroslav, have you tested for carriage returns in your data, and if so, what
did you find?
If that is the problem, the carriage returns are easily edited out, and
speculation about the cause may use more time than it is worth...
If that is not the problem, we definitely have something weird.
The original error message shows the problem occurring at the first line of
your data. Perhaps you could pipe the first few lines through some utility
that shows nonprinting characters (like "vis -lw"), and see what that shows?

----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Miroslav Koncar" <miroslav.koncar@etk.ericsson.se>
Cc: <pgsql-general@postgresql.org>
Sent: Friday, August 24, 2001 8:13 AM
Subject: Re: [GENERAL] problems transfering databases

Show quoted text

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

The only thing I can think of is the PostgreSQL versions. My collegue is
using 7.0.3, and I use 7.1.2. Comments?

That wouldn't produce a carriage-return problem.

It's possible that we're barking up the wrong tree, and that the issue
is not carriage returns but something else --- I just jumped to that
conclusion based on the way you presented the error message. But maybe
the error message was mangled because you were sloppy about cutting-and-
pasting it into your email, or some such. Have you looked at the data
to try to narrow down exactly which line is being rejected?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

#13rob
rob@cabrion.com
In reply to: Jeff Eckermann (#12)
Re: problems transfering databases

I never transfered the file anywhere, it was all done under linux. The
bottom line is pg_dump follwed by a psql import failed exactly like what was
described in this thread. When I did a dump with "proper inserts" it worked
fine. As I said, I don't know what was causing the problem, just that the CR
thing sounded plausible. I only do dumps with "proper inserts" now since I
know that this problem can occur and that "proper inserts" is a workaround.
I'm just confirming the behavior that someone else reported.

--rob

Jeff Eckermann <jeckermann@verio.net> said:

I very much doubt that the data wouldn't "dump right". The dump will

include any data that is in the table, including CRs if they are in there.

In the current case, the CRs were not already in there, because PostgreSQL

would not have recognized that field as null ("\N") if they were.

Any transfer of a file via a Windows machine is apt to cause line endings

to be silently changed, which would account for this case. The real danger
is where the rightmost field is a text type, because the CRs would be
silently imported into the new installation, where they could cause real
problems with any app that tries to use those fields, and a headache to debug.

----- Original Message -----
From: Rob Arnold
To: Jeff Eckermann ; Miroslav Koncar
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 6:32 PM
Subject: Re: problems transfering databases

I've had this problem before. The data is stored correctly in PG, it

just doesn't dump right (the CR thing I guess. I didn't know the reason at
the time). I think this was on 7.0.x. I worked around it by doing a pg_dump
that dumped the table as "proper inserts". That make the load MUCH slower,
but it works around the issue. Now that I know what the cause might have
been, I guess I better go look at my data . . .

--rob
----- Original Message -----
From: Jeff Eckermann
To: Miroslav Koncar
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 10:59 AM
Subject: Re: problems transfering databases

Looks like you already have the answer. The error message is a dead

giveaway. Some field, in this case having a null value (represented as '\N')
contains a carriage return character, like: '\NCR'. PostgreSQL cannot make
sense out of this combination, so it returns an error. The CR contained in
the error message causes the output to return to the start of the line,
outputting the rest of the message from the beginning: see how the final " '
" overwrites the "p" in "psql".

Edit out the CR characters, and the problem will go away.
This is what Tom already said; I am expanding on it.
----- Original Message -----
From: Miroslav Koncar
To: Tom Lane
Cc: pgsql-general@postgresql.org
Sent: Thursday, August 23, 2001 3:18 AM
Subject: Re: [GENERAL] problems transfering databases

Hello Tom,
thanks for the answer. This is the procedure, how we tried to

transfer the data from one machine to another.

My collegue has used the pg_dumpall command, in the following way:

pg_dumpall > backups/2001-06-04db.out

Since it is a simple test database, the 2001-06-04db.out file was

around 40kB, so he has sent it to me via email.

I tried to load the data from the file in two different ways:

psql -f 2001-08-21db.out
cat 2001-06-04db.out | psql

but got the error like stated before. I've tried to do this on

Solaris and Linux (I've set postgreSQL up on my Linux box for testing
purposes), but the same error pops up.

Could it have something to do with the versions of the databases, and

how it was installed? The version I use is

psql (PostgreSQL) 7.1.2 (the output of psql -V command), but I'm not

sure what version did my collegue use (I'll find that out, but I think it is
7.x.x).

Regards,
Miroslav

Tom Lane wrote:

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

'sql:2001-08-21db.out:5: ERROR: copy: line 1, Bad abstime

external

representation '\N
psql:2001-08-21db.out:5: PQendcopy: resetting connection

Given the weird line-wrapping of the message, I'm going to bet that

the

problem is that newlines in the dump file have gotten converted to

DOS

format (ie, \n became \r\n), and COPY is unhappy because it sees

the \r

as part of the data. When it echoes the data it didn't like, the
presence of the \r messes up the format of the error message.

Not sure *how* that happened in a Linux-to-Solaris transfer,

though;

Show quoted text

what tool did you use to transfer the dump file?

regards, tom lane

--
Miroslav Koncar
Software Engineer
Ericsson Nikola Tesla
ETK/D/R
Tel: +385 1 365 3479
Fax: +385 1 365 3548
mailto:miroslav.koncar@etk.ericsson.se

#14rob
rob@cabrion.com
In reply to: Miroslav Koncar (#1)
Re: problems transfering databases

Have him dump the table again (if he can) using pg_dump's "proper inserts" switch turned on. Tom, I'm not contradicting that the problem may be directly linked to CR's in the data. I have encountered this before and I worked around it with proper inserts. I was working in Linux and never transferred the file anywhere.

--rob

----- Original Message -----
From: Miroslav Koncar
To: Tom Lane
Cc: pgsql-general@postgresql.org
Sent: Friday, August 24, 2001 2:23 AM
Subject: Re: problems transfering databases

Hello, Tom,
nope, no windows here. The file was sent direclty from the Linux to my mailbox, which I read on Solaris.

The only thing I can think of is the PostgreSQL versions. My collegue is using 7.0.3, and I use 7.1.2. Comments?

Miroslav

Tom Lane wrote:

Miroslav Koncar <miroslav.koncar@etk.ericsson.se> writes:

Since it is a simple test database, the 2001-06-04db.out file was around
40kB, so he has sent it to me via email.

Email huh? I bet some part of that process was Windows? Almost
certainly that's where the carriage return characters came from.

regards, tom lane

--
Miroslav Koncar
Software Engineer
Ericsson Nikola Tesla
ETK/D/R
Tel: +385 1 365 3479
Fax: +385 1 365 3548
mailto:miroslav.koncar@etk.ericsson.se

#15Gilles Darold
gilles@darold.net
In reply to: Miroslav Koncar (#1)
Problem with libpsqlodbc

Hi all,

I have sent some patches to the OpenLDAP dev team to have it work with
PostgreSQL and they should be applied asap, I also write a HOWTO.
This works but with a not usefull or silly workaround.

If we want to have PG fully compatible with OpenLDAP we need to resolve
a problem regarding the libpsqlodbc library.

In OpenLDAP they call the function SQLBindParameter as follow:

SQLBindParameter(sth,1,SQL_PARAM_OUTPUT,SQL_C_ULONG,SQL_INTEGER,0,0,&new_keyval,0,0);

And then the statement is executed with SQLExecDirect as follow:

rc=SQLExecDirect(sth,oc->create_proc,SQL_NTS);

What they expected is that the 'new_keyval' pointer may be modified at
execution time, with the new unique identifier returned by the query,
typically a sequence number.

It works fine with other database (mySQL, Oracle, etc.) but not with
PostgreSQL, so I suppose that the problem comes from the odbc library.

What's the problem ? Is the SQLBindParameter call is wrong ?
My knowledge is not enougth to find it, please help...

If someone could explain me what's wrong I should probably correct the
problem !

Regards,

Gilles DAROLD

#16Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Miroslav Koncar (#1)
Re: Problem with libpsqlodbc

Gilles DAROLD wrote:

Hi all,

I have sent some patches to the OpenLDAP dev team to have it work with
PostgreSQL and they should be applied asap, I also write a HOWTO.
This works but with a not usefull or silly workaround.

If we want to have PG fully compatible with OpenLDAP we need to resolve
a problem regarding the libpsqlodbc library.

Did you build libpsqlidbc library on some unix platform ?

In OpenLDAP they call the function SQLBindParameter as follow:

SQLBindParameter(sth,1,SQL_PARAM_OUTPUT,SQL_C_ULONG,SQL_INTEGER,0,0,&new_keyval,0,0);

And then the statement is executed with SQLExecDirect as follow:

rc=SQLExecDirect(sth,oc->create_proc,SQL_NTS);

Are you executing a procedure using an escape sequence
{[?=]call procedure-name[([parameter][,[parameter]]...)]}
?

regards,
Hiroshi Inoue

#17Gilles Darold
gilles@darold.net
In reply to: Miroslav Koncar (#1)
Re: Problem with libpsqlodbc

Hi,

Yes it is build on a Linux RH6.2 and I use procedure as follow:

create_proc => 'SELECT create_person(?)'
add_proc => 'SELECT set_person_name(?,?)'

and the function are:

CREATE FUNCTION create_person(int4)
RETURNS int4 AS '
INSERT INTO persons (id, name) VALUES (nextval(''persons_id_seq''), '' '');
SELECT currval(''persons_id_seq'') AS keyval;
'
LANGUAGE 'sql';

CREATE FUNCTION set_person_name(int4, varchar)
RETURNS int4 AS '
UPDATE persons SET name=text($2) WHERE $1=id;
SELECT currval(''persons_id_seq'') AS keyval;
'
LANGUAGE 'sql';

Regards,

Gilles DAROLD

Hiroshi Inoue wrote:

Show quoted text

Gilles DAROLD wrote:

Hi all,

I have sent some patches to the OpenLDAP dev team to have it work with
PostgreSQL and they should be applied asap, I also write a HOWTO.
This works but with a not usefull or silly workaround.

If we want to have PG fully compatible with OpenLDAP we need to resolve
a problem regarding the libpsqlodbc library.

Did you build libpsqlidbc library on some unix platform ?

In OpenLDAP they call the function SQLBindParameter as follow:

SQLBindParameter(sth,1,SQL_PARAM_OUTPUT,SQL_C_ULONG,SQL_INTEGER,0,0,&new_keyval,0,0);

And then the statement is executed with SQLExecDirect as follow:

rc=SQLExecDirect(sth,oc->create_proc,SQL_NTS);

Are you executing a procedure using an escape sequence
{[?=]call procedure-name[([parameter][,[parameter]]...)]}
?

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

#18Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Gilles Darold (#17)
RE: Problem with libpsqlodbc

-----Original Message-----
From: darold [mailto:darold]On Behalf Of Gilles DAROLD

Hi,

Yes it is build on a Linux RH6.2 and I use procedure as follow:

create_proc => 'SELECT create_person(?)'
add_proc => 'SELECT set_person_name(?,?)'

Do they really work with other databases using
SQLBindParameter ?
SQLBindParameter(stmt, 1, ..) for above queries
means a binding to the first parameter "?".
Your functions seem to return the currval of a
sequence but doesn't change/set the first para-
meter(it's impossible in PostgreSQL). SELECT
doesn't return a parameter but returns a result
set in ODBC.

regards,
Hiroshi Inoue

#19Gilles Darold
gilles@darold.net
In reply to: Hiroshi Inoue (#18)
Re: Problem with libpsqlodbc

Yes it work on other database but probably i'm missing something...

For oracle they do

create_proc => '{call create_person(?)}'
add_proc => '{call set_person_name(?,?)}'

Is that the problem ? Should I use call ? I use select instead of call
because when I've tried it under pgsql call is not a keyword. So do I
definitively missed something ?

Sorry for asking that but could you give me a little sample modifying the
first parameter in a function as I could be less stupid and understand
a little ODBC ?

I request your help because I really don't know about ODBC and after
finding the solution for openldap I will probably never use ODBC :-((

Regards,
Gilles Darold

Hiroshi Inoue wrote:

Show quoted text

-----Original Message-----
From: darold [mailto:darold]On Behalf Of Gilles DAROLD

Hi,

Yes it is build on a Linux RH6.2 and I use procedure as follow:

create_proc => 'SELECT create_person(?)'
add_proc => 'SELECT set_person_name(?,?)'

Do they really work with other databases using
SQLBindParameter ?
SQLBindParameter(stmt, 1, ..) for above queries
means a binding to the first parameter "?".
Your functions seem to return the currval of a
sequence but doesn't change/set the first para-

meter(it's impossible in PostgreSQL). SELECT
doesn't return a parameter but returns a result
set in ODBC.

regards,
Hiroshi Inoue

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

#20Hiroshi Inoue
Inoue@tpf.co.jp
In reply to: Hiroshi Inoue (#18)
Re: Problem with libpsqlodbc

Gilles DAROLD wrote:

Yes it work on other database but probably i'm missing something...

It's a limited spec of PostgreSQL that SELECT can call
function(procedure)s but SELECT returns a result set
not a parameter.

For oracle they do

create_proc => '{call create_person(?)}'
add_proc => '{call set_person_name(?,?)}'

It's an ODBC's spec to call a procedure.

Is that the problem ? Should I use call ?

There seems to be 2 ways.

1) You could use the following.
create_proc => '{?=call create_person(?)}'
add_proc => '{?=call set_person_name(?, ?)}'
(optional parameter marker ?= at the start of the syntax
is needed if you need to accept the return value)

But psqlodbc driver couldn't handle the syntax
other than that of the latest snapshot.
Could you try it ?

2) SELECT returns a result set.
You could fetch and get the result using
SQLFetch(), SQLGetData() etc.

regards,
Hiroshi Inoue