HUGE Stack space is gettiing consumed

On Fri, Apr 14, 2006 at 09:13:26AM +0530, Mavinakuli, Prasanna (STSD) wrote:

Hello All,

We are getting very strange problem ..

********************************************************
We Have one data field in some table..type of this field
Is *TEXT*.
We are storing around 1,000 (One Thousand) lines of text
In that field.

If it's run though separate thread,
When we say select <columnname> from <tablename>
We are getting
*could not receive data from server: No such file or directory *
messages without Result

If it was in main thread,
It Works fine.
********************************************************

So you have a multithreaded program. Have you setup something so that
you don't have two threads trying to do things with libpq at the same
time? Because that won't work.

Secondly, it can't have anything to do with stack-space because when
you run out of stack space you get a segmentation fault, not a nice
error.

Either post a complete example of the failure (with source code) or
perhaps the strace output would be enough.
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Hello Martijin,
Thx for u'r Suggetions..

Here is Complete source code..

Desc about following code::::

Code has a function named as queryDB, which makes a connection to our DB
and then tries to query
Particular table which has huge string in it's field.

If this method called from main thread then it *DOES* work properly.

But if we make that as routine function for a thread it doesn't work
instead we will get that following status :::
( PGRES_FATAL_ERROR )
***could not receive data from server: Invalid argument***

******************************CODE*****************************

#include <iostream>
#include "libpq-fe.h"
#include "libpq/libpq-fs.h"
#include <string>
#include <stdlib.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/timeb.h>
#include <time.h>
#include <unistd.h>
#include <pwd.h>
#include <sys/timeb.h>
#include <errno.h>
#include <sys/socket.h>
#include <netdb.h>
#include <netinet/in.h>
#include "common/gs/utils/UtilsConsts.h"
#include <utmp.h>
#include <utmpx.h>
#include <sys/stat.h>
#include <cstdlib>
#include <cstdio>

void* queryDB(void* p)
{

cout<<"I Am in Routine Function *** "<<endl;
PGconn *_conn;
PGresult *_res;
char *dbn,*usn;
char *error;
cout<<"i am connecting:"<<endl;
_conn=PQconnectdb("dbname = LOGDB user = sfmdb port=10864 ");
if(PQstatus(_conn)==CONNECTION_OK)
cout<<"connection is ready"<<endl;
dbn=PQdb(_conn);
usn=PQuser(_conn);
if(PQstatus(_conn)==CONNECTION_BAD)
cout<<"the connection is not hapened"<<endl;
string string1,string2,string3;
_res=PQexec(_conn,"select * from throttlingconfig ;");
if (!( PQresultStatus(_res) == PGRES_COMMAND_OK
||PQresultStatus(_res) == PGRES_TUPLES_OK ) )
{
/*If this Function is *NOT* called from main thread then
we will end up by entering here. */
string msg = PQresultErrorMessage(_res);
PQclear(_res);
cout<<"Query Execution is Not OK **********"<<msg<<endl;
}
else
{
/*If this Function is called from main thread then we will
end up by entering here. */
int no_of_tuples,no_of_columns;
char *str1;
string stg;
no_of_columns=PQnfields(_res);
no_of_tuples=PQntuples(_res);

cout<<"no of tuples "<<no_of_tuples<<endl;
cout<<"Number Of Columns "<<no_of_columns<<endl;

for(int u=0;u<no_of_tuples;u++)
{
for(int y=0;y<no_of_columns;y++)
{
str1=PQgetvalue(_res,u,y);
cout<<"*************Printing *** "<<u<<"th Row
"<<y<<"th Column****************"<<endl;
cout<<str1<<endl;
}
cout<<endl;
}
PQclear(_res);
}
PQfinish(_conn);
return NULL;
}
int main()
{
//size_t _stackSize(512 * 1024 * 2);

//calling queryDB from other thread ..DOESN'T Work here
size_t _stackSize(512 * 1024 * 2 * 20) ;
pthread_attr_t attr;
pthread_attr_init ( &attr ) ;
pthread_t thread_t;
int ret = pthread_attr_setstacksize( &attr, _stackSize) ;
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
int retVal = pthread_create(&thread_t, &attr,queryDB,NULL );
int rc,status;
rc = pthread_join(thread_t, (void **)&status);

//calling queryDB from main thread WORKS here..

queryDB(NULL);
return 1;
}

Thx a lot,
And looking for u'r reply,

Prasanna.

-----Original Message-----
From: Martijn van Oosterhout [mailto:kleptog@svana.org]
Sent: Saturday, April 15, 2006 7:22 PM
To: Mavinakuli, Prasanna (STSD)
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] HUGE Stack space is gettiing consumed

On Fri, Apr 14, 2006 at 09:13:26AM +0530, Mavinakuli, Prasanna (STSD)
wrote:

Hello All,

We are getting very strange problem ..

********************************************************
We Have one data field in some table..type of this field Is *TEXT*.
We are storing around 1,000 (One Thousand) lines of text In that
field.

If it's run though separate thread,
When we say select <columnname> from <tablename> We are getting *could

not receive data from server: No such file or directory * messages
without Result

If it was in main thread,
It Works fine.
********************************************************

So you have a multithreaded program. Have you setup something so that
you don't have two threads trying to do things with libpq at the same
time? Because that won't work.

Secondly, it can't have anything to do with stack-space because when you
run out of stack space you get a segmentation fault, not a nice error.

Either post a complete example of the failure (with source code) or
perhaps the strace output would be enough.
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

On Tue, Apr 18, 2006 at 12:31:57PM +0530, Mavinakuli, Prasanna (STSD) wrote:

Hello Martijin,
Thx for u'r Suggetions..

Here is Complete source code..

Ok, I can't get it to fail. I cleaned up the code because it wouldn't
compile as is (g++ 3.3.5 on Debian). I've put the cleaned up version
here:

http://svana.org/kleptog/temp/test.cpp

You might need to change the QUERY_STRING and the CONNECT_STRING back.
Could you try running this to see if you can get it to fail? If it's
does fail, could you post the output of:

strace -f <program>

somewhere (*don't* post it to the list). Somewhere like:
http://phpfi.com/
should work well.

Desc about following code::::

Code has a function named as queryDB, which makes a connection to our DB
and then tries to query
Particular table which has huge string in it's field.

If this method called from main thread then it *DOES* work properly.

But if we make that as routine function for a thread it doesn't work
instead we will get that following status :::
( PGRES_FATAL_ERROR )
***could not receive data from server: Invalid argument***

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Hello Martijin,

I am not able to make it ..Code what u sent is *not* giving desired
result..
That is -when we try to fetch huge data in a thread routine it says
"could not receive data from server: Error 0"
If it was in main thread then it goes thorugh..I.e It's able to get the
huge data.

Environment..

Postgres Version:7.4.3
Compiler:aCC 5.0
OS:HP-UX PARISC 11.23

Code has put in the following link..
http://phpfi.com/113850

Will send the tusc output (stack trace)

Thx,
Prasanna.

-----Original Message-----
From: Martijn van Oosterhout [mailto:kleptog@svana.org]
Sent: Tuesday, April 18, 2006 2:46 PM
To: Mavinakuli, Prasanna (STSD)
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] HUGE Stack space is gettiing consumed

On Tue, Apr 18, 2006 at 12:31:57PM +0530, Mavinakuli, Prasanna (STSD)
wrote:

Hello Martijin,
Thx for u'r Suggetions..

Here is Complete source code..

Ok, I can't get it to fail. I cleaned up the code because it wouldn't
compile as is (g++ 3.3.5 on Debian). I've put the cleaned up version
here:

http://svana.org/kleptog/temp/test.cpp

You might need to change the QUERY_STRING and the CONNECT_STRING back.
Could you try running this to see if you can get it to fail? If it's
does fail, could you post the output of:

strace -f <program>

somewhere (*don't* post it to the list). Somewhere like:
http://phpfi.com/
should work well.

Desc about following code::::

Code has a function named as queryDB, which makes a connection to our
DB and then tries to query Particular table which has huge string in
it's field.

If this method called from main thread then it *DOES* work properly.

But if we make that as routine function for a thread it doesn't work
instead we will get that following status :::
( PGRES_FATAL_ERROR )
***could not receive data from server: Invalid argument***

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

On Thu, Apr 20, 2006 at 08:20:30PM +0530, Mavinakuli, Prasanna (STSD) wrote:

Hello Martijin,

I am not able to make it ..Code what u sent is *not* giving desired
result..
That is -when we try to fetch huge data in a thread routine it says
"could not receive data from server: Error 0"
If it was in main thread then it goes thorugh..I.e It's able to get the
huge data.

Well, I don't understand. There's no interesting difference between the
code I sent and the code you sent...

Will send the tusc output (stack trace)

Not stack trace, strace which is a system call trace. Don't forget -f
to trace the threads also.
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

Hello Martijin,

I have one more doudt ..while building libpq.so we are not specifying

--enable-thread-safety

Will it make any difference..?

Thx,
Prasanna.
-----Original Message-----
From: Mavinakuli, Prasanna (STSD)
Sent: Friday, April 21, 2006 5:46 PM
To: 'Martijn van Oosterhout'
Subject: RE: [GENERAL] HUGE Stack space is gettiing consumed

Hello Martijin,

Here is System call trace ..
Details http://phpfi.com/114019
Working V/s Non-Working http://phpfi.com/114021 (deleted obv art from
above one)

Please go through it..

Thx,
Prasanna.
-----Original Message-----
From: Martijn van Oosterhout [mailto:kleptog@svana.org]
Sent: Thursday, April 20, 2006 9:00 PM
To: Mavinakuli, Prasanna (STSD)
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] HUGE Stack space is gettiing consumed

On Thu, Apr 20, 2006 at 08:20:30PM +0530, Mavinakuli, Prasanna (STSD)
wrote:

Hello Martijin,

I am not able to make it ..Code what u sent is *not* giving desired
result..
That is -when we try to fetch huge data in a thread routine it says
"could not receive data from server: Error 0"
If it was in main thread then it goes thorugh..I.e It's able to get
the huge data.

Well, I don't understand. There's no interesting difference between the
code I sent and the code you sent...

Will send the tusc output (stack trace)

Not stack trace, strace which is a system call trace. Don't forget -f to
trace the threads also.
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

From each according to his ability. To each according to his ability

to litigate.

On Sat, Apr 22, 2006 at 08:45:49AM +0530, Mavinakuli, Prasanna (STSD) wrote:

Hello Martijin,

I have one more doudt ..while building libpq.so we are not specifying

--enable-thread-safety

Will it make any difference..?

Well, if you don't, the libpq won't know anything about threads and may
indeed have problems. I can't see anything in your other output that
would suggest that the system is doing anything wrong, so if you havn't
got --enable-thread-safety on, perhaps the main and the subthreads are
using a different C library or something obscure.

I'd try that, see if it helps,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/