debugging C functions
Hi there
I wrote a C function to call from PostgreSQL8.2.4 under Linux. The functions returns unexpected results. I did an extensive analysis to the function and it seems correct. I want to know if there is a way to debug C functions that are passed to PostgreSQL.
Thanks
Islam
Islam Hegazy wrote:
I wrote a C function to call from PostgreSQL8.2.4 under Linux. The
functions returns unexpected results. I did an extensive analysis to the
function and it seems correct. I want to know if there is a way to debug
C functions that are passed to PostgreSQL.
Yes. Something along these lines (where plr.so is an example shared
object library with a function called throw_notice installed in a
database called contrib_regression):
1. Build and install your function. Ensure both postgres and your
library are built with debug symbols (--enable-debug)
2. start a psql session in the database where your function has
been created
#psql contrib_regression
3. Load the shared object library in psql
contrib_regression=# load '$libdir/plr';
LOAD
4. Start another console and determine the PID for the backend
session (this will wrap poorly -- I'll do my best to make it
readable)
ps -ef | grep postgres
postgres 24496 1 0 18:23 ? 00:00:00
/usr/local/pgsql-dev/bin/postgres -D /opt/data/pgsql/data -p
65432 -i -F
postgres 24498 24496 0 18:23 ? 00:00:00
postgres: writer process
postgres 24499 24496 0 18:23 ? 00:00:00
postgres: stats collector process
postgres 24500 24496 0 18:23 ? 00:00:00
postgres: autovacuum launcher process
postgres 31233 24496 1 20:37 ? 00:00:00
postgres: postgres contrib_regression [local] idle
You want the PID associated with the idle session -- 31233
5. Run gdb and attach to the backend in question
gdb /usr/local/pgsql-dev/bin/postgres 31233
6. Set breakpoints, etc, and then continue the gdb session
[...]
Reading symbols from
/usr/lib64/R/library/stats/libs/stats.so...done.
Loaded symbols for /usr/lib64/R/library/stats/libs/stats.so
0x0000003fef4cdf45 in recv () from /lib64/libc.so.6
(gdb) break throw_notice
Breakpoint 1 at 0x636cb7: file plr.c, line 2908.
(gdb) continue
Continuing.
7. Return to the psql session, run your function
contrib_regression=# select throw_notice('hello');
8. Return to gdb session, debug away...
HTH,
Joe
Joe Conway <mail@joeconway.com> writes:
[ much good advice snipped, but I have to weigh in on one point ]
4. Start another console and determine the PID for the backend
session (this will wrap poorly -- I'll do my best to make it
readable)
"select pg_backend_pid()" is another alternative for finding the PID.
Personally I've gotten to the point where manually determining the
backend PID at all is tedious, and so I tend to use this script:
#!/bin/sh
# tee /dev/tty is for user to see the set of procs considered
PROCS=`ps auxww | \
grep postgres: | \
grep -v -e 'grep postgres:' -e 'postgres: stats' -e 'postgres: writer' -e 'postgres: archiver' -e 'postgres: logger' | \
tee /dev/tty | \
awk '{print $2}'`
if [ `echo "$PROCS" | wc -w` -eq 1 ]
then
exec gdb $PGINSTROOT/bin/postgres -silent "$PROCS"
else
exec gdb $PGINSTROOT/bin/postgres -silent
fi
This fails (but gives you a list of processes to consider attaching to)
if there's more than one candidate.
regards, tom lane
Thanks for your replies, they were very helpful to me. Unfortuantely, I
can't trace the C function. PostgreSQL returns the results directly and the
debugger doesn't stop at the breakpoints in the C function.
I think the problem is in the pointers. I use pointers in my function and I
defined them as static to be preserved between calls, my function returns a
set of records. When I comment the pointers portion, the function works
well. But with the pointers, it hangs.
Any idea on how to deal with pointers issue?
Regards
Islam Hegazy
----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Joe Conway" <mail@joeconway.com>
Cc: "Islam Hegazy" <islheg@gawab.com>; <pgsql-general@postgresql.org>
Sent: Friday, June 01, 2007 11:38 PM
Subject: Re: [GENERAL] debugging C functions
Show quoted text
Joe Conway <mail@joeconway.com> writes:
[ much good advice snipped, but I have to weigh in on one point ]
4. Start another console and determine the PID for the backend
session (this will wrap poorly -- I'll do my best to make it
readable)"select pg_backend_pid()" is another alternative for finding the PID.
Personally I've gotten to the point where manually determining the
backend PID at all is tedious, and so I tend to use this script:#!/bin/sh
# tee /dev/tty is for user to see the set of procs considered
PROCS=`ps auxww | \
grep postgres: | \
grep -v -e 'grep postgres:' -e 'postgres: stats' -e 'postgres:
writer' -e 'postgres: archiver' -e 'postgres: logger' | \
tee /dev/tty | \
awk '{print $2}'`if [ `echo "$PROCS" | wc -w` -eq 1 ]
then
exec gdb $PGINSTROOT/bin/postgres -silent "$PROCS"
else
exec gdb $PGINSTROOT/bin/postgres -silent
fiThis fails (but gives you a list of processes to consider attaching to)
if there's more than one candidate.regards, tom lane
"Islam Hegazy" <islheg@gawab.com> writes:
Thanks for your replies, they were very helpful to me. Unfortuantely, I
can't trace the C function. PostgreSQL returns the results directly and the
debugger doesn't stop at the breakpoints in the C function.
Well, you need to deal with that last, because you will never get very
far if you can't debug your code.
My experience is that gdb needs help to recognize a shared library's
symbols. It works for me to LOAD the shared library (from SQL) before
attaching to the backend with gdb. If you can't do that, gdb's
"sharedlibrary" command might do it.
regards, tom lane
I do the same but I use the ddd debugger
1) Load the shared library from the SQL
2) Open the .c file of my function
3) Place the break points
4) Execute the sql statement 'Select * from Myfn(...);'
The result is displayed and the debugger doesn't stop at the breakpoints.
Are there any steps missing?
Regards
Islam Hegazy
----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Islam Hegazy" <islheg@gawab.com>
Cc: "Joe Conway" <mail@joeconway.com>; <pgsql-general@postgresql.org>
Sent: Sunday, June 03, 2007 10:00 PM
Subject: Re: [GENERAL] debugging C functions
Show quoted text
"Islam Hegazy" <islheg@gawab.com> writes:
Thanks for your replies, they were very helpful to me. Unfortuantely, I
can't trace the C function. PostgreSQL returns the results directly and
the
debugger doesn't stop at the breakpoints in the C function.Well, you need to deal with that last, because you will never get very
far if you can't debug your code.My experience is that gdb needs help to recognize a shared library's
symbols. It works for me to LOAD the shared library (from SQL) before
attaching to the backend with gdb. If you can't do that, gdb's
"sharedlibrary" command might do it.regards, tom lane
"Islam Hegazy" <islheg@gawab.com> writes:
I do the same but I use the ddd debugger
1) Load the shared library from the SQL
2) Open the .c file of my function
3) Place the break points
4) Execute the sql statement 'Select * from Myfn(...);'The result is displayed and the debugger doesn't stop at the breakpoints.
Are you sure you're attaching to the right process?
One way to do it is to run select pg_backend_pid() from psql and attach to
that pid.
--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Yes, I am sure. If I placed a breakpoint in any file, e.g. execMain.c, the
debugger would enter this file.
Islam Hegazy
----- Original Message -----
From: "Gregory Stark" <stark@enterprisedb.com>
To: "Islam Hegazy" <islheg@gawab.com>
Cc: "Tom Lane" <tgl@sss.pgh.pa.us>; "Joe Conway" <mail@joeconway.com>;
<pgsql-general@postgresql.org>
Sent: Monday, June 04, 2007 5:50 AM
Subject: Re: [GENERAL] debugging C functions
Show quoted text
"Islam Hegazy" <islheg@gawab.com> writes:
I do the same but I use the ddd debugger
1) Load the shared library from the SQL
2) Open the .c file of my function
3) Place the break points
4) Execute the sql statement 'Select * from Myfn(...);'The result is displayed and the debugger doesn't stop at the breakpoints.
Are you sure you're attaching to the right process?
One way to do it is to run select pg_backend_pid() from psql and attach to
that pid.--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
It worked today with me and I discovered what is the problem. The problem is
that I have 2 structures that contain pointers. I inistiate variables from
these structures as static. I can't declare the pointers inside the struct
as static. Now the problem is that when the function is called a second
time, it returns set of records, it finds the static pointers in the
function but the pointers inside the struct are deleted. Here is a portion
of the code:
typedef struct
{
int value;
struct TuplesList *next;
}TuplesList;
typedef struct
{
TuplesList *tuplesHead;
TuplesList *tuplesTail;
struct AggrQuery *nextQuery;
}AggrQuery;
Datum AggrFn(PG_FUNCTION_ARGS)
{
static AggrQuery *queriesHead;
static AggrQuery *queriesTail;
}
First call to AggrFn is correct, in the second call queriesHead and
queriesTail are preserved but queriesHead->tuplesHead, for example, is
rubbish from memory. I read in the PostgreSQL documentation 'However, if you
want to allocate any data structures to live across calls, you need to put
them somewhere else. The memory context referenced by multi_call_memory_ctx
is a suitable location for any data that needs to survive until the SRF is
finished running'. I don't understand this statement so I created the
variables as static but it is not working with struct....
Regards
Islam Hegazy
----- Original Message -----
From: "Gregory Stark" <stark@enterprisedb.com>
To: "Islam Hegazy" <islheg@gawab.com>
Cc: "Tom Lane" <tgl@sss.pgh.pa.us>; "Joe Conway" <mail@joeconway.com>;
<pgsql-general@postgresql.org>
Sent: Monday, June 04, 2007 5:50 AM
Subject: Re: [GENERAL] debugging C functions
Show quoted text
"Islam Hegazy" <islheg@gawab.com> writes:
I do the same but I use the ddd debugger
1) Load the shared library from the SQL
2) Open the .c file of my function
3) Place the break points
4) Execute the sql statement 'Select * from Myfn(...);'The result is displayed and the debugger doesn't stop at the breakpoints.
Are you sure you're attaching to the right process?
One way to do it is to run select pg_backend_pid() from psql and attach to
that pid.--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
This post got me thinking, is there a similar procedure for PL/pgSQL functions?
---
David Gardner, IT
The Yucaipa Companies
(310) 228-2855
-----Original Message-----
From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Joe Conway
Sent: Friday, June 01, 2007 9:00 PM
To: Islam Hegazy
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] debugging C functions
Islam Hegazy wrote:
I wrote a C function to call from PostgreSQL8.2.4 under Linux. The
functions returns unexpected results. I did an extensive analysis to the
function and it seems correct. I want to know if there is a way to debug
C functions that are passed to PostgreSQL.
Yes. Something along these lines (where plr.so is an example shared
object library with a function called throw_notice installed in a
database called contrib_regression):
1. Build and install your function. Ensure both postgres and your
library are built with debug symbols (--enable-debug)
2. start a psql session in the database where your function has
been created
#psql contrib_regression
3. Load the shared object library in psql
contrib_regression=# load '$libdir/plr';
LOAD
4. Start another console and determine the PID for the backend
session (this will wrap poorly -- I'll do my best to make it
readable)
ps -ef | grep postgres
postgres 24496 1 0 18:23 ? 00:00:00
/usr/local/pgsql-dev/bin/postgres -D /opt/data/pgsql/data -p
65432 -i -F
postgres 24498 24496 0 18:23 ? 00:00:00
postgres: writer process
postgres 24499 24496 0 18:23 ? 00:00:00
postgres: stats collector process
postgres 24500 24496 0 18:23 ? 00:00:00
postgres: autovacuum launcher process
postgres 31233 24496 1 20:37 ? 00:00:00
postgres: postgres contrib_regression [local] idle
You want the PID associated with the idle session -- 31233
5. Run gdb and attach to the backend in question
gdb /usr/local/pgsql-dev/bin/postgres 31233
6. Set breakpoints, etc, and then continue the gdb session
[...]
Reading symbols from
/usr/lib64/R/library/stats/libs/stats.so...done.
Loaded symbols for /usr/lib64/R/library/stats/libs/stats.so
0x0000003fef4cdf45 in recv () from /lib64/libc.so.6
(gdb) break throw_notice
Breakpoint 1 at 0x636cb7: file plr.c, line 2908.
(gdb) continue
Continuing.
7. Return to the psql session, run your function
contrib_regression=# select throw_notice('hello');
8. Return to gdb session, debug away...
HTH,
Joe
---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?
2007/6/5, David Gardner <David.Gardner@yucaipaco.com>:
This post got me thinking, is there a similar procedure for PL/pgSQL functions?
No. You can debug PL/pgSQL function via debug plugin API. Currently
exists only one debugger, which can do it - Enterprisedb debugger.
Regards
Pavel Stehule
------- Original Message -------
From: "Pavel Stehule" <pavel.stehule@gmail.com>
To: "David Gardner" <David.Gardner@yucaipaco.com>
Sent: 05/06/07, 21:01:49
Subject: Re: pl/pgsql debuging, was Re: [GENERAL] debugging C functions2007/6/5, David Gardner <David.Gardner@yucaipaco.com>:
This post got me thinking, is there a similar procedure for PL/pgSQL functions?
No. You can debug PL/pgSQL function via debug plugin API. Currently
exists only one debugger, which can do it - Enterprisedb debugger.
Or dev builds of pgAdmin - but you still need the plugin.
Regards, Dave
Import Notes
Resolved by subject fallback
I grabbed the May 10th dev snapshot of pgAdmin3, first a little bit of praise to the pgAdmin3 team for allowing me to run both pgAdmin3 1.7 and 1.6.2 side by side.
However what is the debug plugin API? I looked around in postgresql/contrib, and PostgreSQL.org. I'm assuming this plugin is something that needs to run server side?
---
David Gardner, IT
The Yucaipa Companies
(310) 228-2855
-----Original Message-----
From: Dave Page [mailto:dpage@postgresql.org]
Sent: Tuesday, June 05, 2007 1:38 PM
To: Pavel Stehule
Cc: David Gardner; pgsql-general@postgresql.org
Subject: Re: pl/pgsql debuging, was Re: [GENERAL] debugging C functions
------- Original Message -------
From: "Pavel Stehule" <pavel.stehule@gmail.com>
To: "David Gardner" <David.Gardner@yucaipaco.com>
Sent: 05/06/07, 21:01:49
Subject: Re: pl/pgsql debuging, was Re: [GENERAL] debugging C functions2007/6/5, David Gardner <David.Gardner@yucaipaco.com>:
This post got me thinking, is there a similar procedure for PL/pgSQL functions?
No. You can debug PL/pgSQL function via debug plugin API. Currently
exists only one debugger, which can do it - Enterprisedb debugger.
Or dev builds of pgAdmin - but you still need the plugin.
Regards, Dave
------- Original Message -------
From: David Gardner <David.Gardner@yucaipaco.com>
To: "'Dave Page'" <dpage@postgresql.org>
Sent: 06/06/07, 00:14:52
Subject: RE: pl/pgsql debuging, was Re: [GENERAL] debugging C functionsI grabbed the May 10th dev snapshot of pgAdmin3, first a little bit of praise to the pgAdmin3 team for allowing me to run both pgAdmin3 1.7 and 1.6.2 side by side.
However what is the debug plugin API? I looked around in postgresql/contrib, and PostgreSQL.org. I'm assuming this plugin is something that needs to run server side?
Yes, but at present it's only available as part of EnterpriseDB. A PostgreSQL release should be available Real Soon Now.
Regards, Dave
Import Notes
Resolved by subject fallback
As someone who would greatly benefit from this feature, is there
something I can do to help out in development/testing of this feature? I
have a test server I could install a cvs release of pgsql and know my
way around a makefile.
Dave Page wrote:
Show quoted text
------- Original Message -------
From: David Gardner <David.Gardner@yucaipaco.com>
To: "'Dave Page'" <dpage@postgresql.org>
Sent: 06/06/07, 00:14:52
Subject: RE: pl/pgsql debuging, was Re: [GENERAL] debugging C functionsI grabbed the May 10th dev snapshot of pgAdmin3, first a little bit of praise to the pgAdmin3 team for allowing me to run both pgAdmin3 1.7 and 1.6.2 side by side.
However what is the debug plugin API? I looked around in postgresql/contrib, and PostgreSQL.org. I'm assuming this plugin is something that needs to run server side?
Yes, but at present it's only available as part of EnterpriseDB. A PostgreSQL release should be available Real Soon Now.
Regards, Dave
David Gardner wrote:
As someone who would greatly benefit from this feature, is there
something I can do to help out in development/testing of this feature? I
have a test server I could install a cvs release of pgsql and know my
way around a makefile.
Hi David,
At the moment it's just a case of us finding some time to pull the
appropriate code from EnterpriseDB and rebundle it as a PostgreSQL add on.
If you wish to test the debugger client though, you can always download
a copy of EnterpriseDB to test against (get the latest 8.2 build). There
will be some more changes to the client in the next few days though, so
look out for an update to pgAdmin.
Regards, Dave.