Why does app fail?

Started by Paul Simpsonabout 22 years ago11 messagesgeneral
Jump to latest
#1Paul Simpson
Paul.Simpson@spgs.org

Hi,

Like many here, I am fairly new to PostgreSQL having been forced to cut my teeth on M$ SQLServer. My apologies if this is the wrong forum for this question, please advise if so!

We use a third party application to manage our student population. Part of the reason we purchased it is that it allegedly will run against "Any ODBC-Compliant back-end database". As I mentioned above, for various reasons we used the M$ product, but now want to move away and PostgreSQL on Linux is the obvious choice.

My problem is that the app won't initialise against PostgreSQL. I have contacted the vendor who simply says they don't support PostgreSQL. They only support M£ SQLServer, M$Access, Oracle and Ingres. No help from them!

I suspect the issue is with data types. I have already created a domain to map DATETIME to TIMESTAMP but I haven't got much further. Unfortunately, with nearly 700 tables, scanning them all isn't an option!

I have two possible solutions which I'd like advice on (which is better and / or how to do each).

1) If there is a log somewhere showing failed requests, I might be able to see what the application is trying to do and then figure out a way around it.

2) Is there a way I can configure PostgreSQL (probably temporarily!) to just look like M$ or one of the other supported systems?

I should mention that I am in the process of developing a new Web-Based front end to the database with the long-term aim of ditching the current application.

Any help would be most gratefully received.

Many thanks,

Paul Simpson

ABOUT THIS E-MAIL
The information contained within this e-mail is privileged, confidential and protected from disclosure.

Any opinions expressed in this electronic communication do not necessarily reflect official policy of St Paul's Girls' School.

If you are not the intended recipient of this document or any part then please:

1) : inform the sender at the above address
2) : delete this e-mail immediately
3) : do not take copies nor disclose the contents of this document to anyone

PUPILS' COMMUNICATIONS
Only Senior School pupils of St Paul's Girls' School are permitted to send electronic mail and then only via our controlled system.
It is contrary to school rules for any other pupil to do so and they are regularly reminded of this fact.

VIRUS PROTECTION
All electronic mail traffic sent from this establishment is monitored and scanned for viruses prior to sending.

#2Shridhar Daithankar
shridhar@frodo.hserus.net
In reply to: Paul Simpson (#1)
Re: Why does app fail?

On Tuesday 24 February 2004 16:51, Paul Simpson wrote:

1) If there is a log somewhere showing failed requests, I might be able to
see what the application is trying to do and then figure out a way around
it.

You can log in every query server is executing. You need to enable the
corresponding log statement in pg_hba.conf. I think it is called
log_statement.

And of course, don't throw away server logs. If they are redirected
to /dev/null, they are useless anyways..

I am sure you will find lot of information in there..

HTH

Shridhar

#3Paul Simpson
Paul.Simpson@spgs.org
In reply to: Shridhar Daithankar (#2)
Re: Why does app fail?

Thanks for that! I'll have a look...

Paul

Shridhar Daithankar <shridhar@frodo.hserus.net> 24/02/2004 12:02:30 >>>

On Tuesday 24 February 2004 16:51, Paul Simpson wrote:

1) If there is a log somewhere showing failed requests, I might be able to
see what the application is trying to do and then figure out a way around
it.

You can log in every query server is executing. You need to enable the
corresponding log statement in pg_hba.conf. I think it is called
log_statement.

And of course, don't throw away server logs. If they are redirected
to /dev/null, they are useless anyways..

I am sure you will find lot of information in there..

HTH

Shridhar

ABOUT THIS E-MAIL
The information contained within this e-mail is privileged, confidential and protected from disclosure.

Any opinions expressed in this electronic communication do not necessarily reflect official policy of St Paul's Girls' School.

If you are not the intended recipient of this document or any part then please:

1) : inform the sender at the above address
2) : delete this e-mail immediately
3) : do not take copies nor disclose the contents of this document to anyone

PUPILS' COMMUNICATIONS
Only Senior School pupils of St Paul's Girls' School are permitted to send electronic mail and then only via our controlled system.
It is contrary to school rules for any other pupil to do so and they are regularly reminded of this fact.

VIRUS PROTECTION
All electronic mail traffic sent from this establishment is monitored and scanned for viruses prior to sending.

#4Paul Simpson
Paul.Simpson@spgs.org
In reply to: Paul Simpson (#3)
Re: Why does app fail?

Ok, I've done that and it would appear to be a table naming issue...

When I look at the logfile at the point where a table is creates, I get the following lines (for each table, as far as I can tell, although the third line only appears for some):-

select relname, nspname, relkind from pg_catalog.pg_class, pg_catalog.pg_namespace where relkind in ('r', 'v')and nspname like 'public' and relname like 'TTATTASSOC' and relname !~ '^pg_|^dd_' and pg_namespace.oid = relnamespace order by nspname, relname

Create Table TTATTASSOC (AssocId varchar(10), SetId varchar(10), SlotId int4)

select u.nspname, c.relname, a.attname, a.atttypid, t.typname, a.attnum, a.attlen, a.atttypmod, a.attnotnull,c.relhasrules, c.relkind from pg_catalog.pg_namespace u, pg_catalog.pg_class c, pg_catalog.pg_attribute a, pg_catalog.pg_type t where u.oid = c.relnamespace and (not a.attisdropped) and c.oid= a.attrelid and a.atttypid = t.oid and (a.attnum > 0) and c.relname = 'TTATTASSOC' and u.nspname = 'public' order by u.nspname, c.relname, attnum

select relname, nspname, relkind from pg_catalog.pg_class, pg_catalog.pg_namespace where relkind in ('r', 'v') and nspname like 'public' and relname like 'TTATTASSOC' and relname !~ '^pg_|^dd_' and pg_namespace.oid = relnamespace order by nspname, relname

If I am correct (comments please) then I assume it is looking for the existence of the table, when it doesn't find it, it is creating it. I'm not entirely sure what the third line does, but the fourth seems to be checking again for the table's existence.

I have run the statements manually and they all seem to work, however the first (and fourth) always return zero rows. If I replace the table name with a lower case version, then I get one row returned.

I have, therefore, come to the conclusion that the app requires table names in Upper case. How can I force this to work?

Sorry for the long question...!

Paul

Shridhar Daithankar <shridhar@frodo.hserus.net> 24/02/2004 12:02:30 >>>

On Tuesday 24 February 2004 16:51, Paul Simpson wrote:

1) If there is a log somewhere showing failed requests, I might be able to
see what the application is trying to do and then figure out a way around
it.

You can log in every query server is executing. You need to enable the
corresponding log statement in pg_hba.conf. I think it is called
log_statement.

And of course, don't throw away server logs. If they are redirected
to /dev/null, they are useless anyways..

I am sure you will find lot of information in there..

HTH

Shridhar

ABOUT THIS E-MAIL
The information contained within this e-mail is privileged, confidential and protected from disclosure.

Any opinions expressed in this electronic communication do not necessarily reflect official policy of St Paul's Girls' School.

If you are not the intended recipient of this document or any part then please:

1) : inform the sender at the above address
2) : delete this e-mail immediately
3) : do not take copies nor disclose the contents of this document to anyone

PUPILS' COMMUNICATIONS
Only Senior School pupils of St Paul's Girls' School are permitted to send electronic mail and then only via our controlled system.
It is contrary to school rules for any other pupil to do so and they are regularly reminded of this fact.

VIRUS PROTECTION
All electronic mail traffic sent from this establishment is monitored and scanned for viruses prior to sending.

#5Shridhar Daithankar
shridhar@frodo.hserus.net
In reply to: Paul Simpson (#4)
Re: Why does app fail?

On Tuesday 24 February 2004 22:10, Paul Simpson wrote:

Create Table TTATTASSOC (AssocId varchar(10), SetId varchar(10), SlotId
int4)
I have, therefore, come to the conclusion that the app requires table names
in Upper case. How can I force this to work?

I believe postgresql is noisy about the case in tablename. In the create table
statement above if you want to preserve the case in tablename, you have to
quote it.

Try it and let us know. (This is out of memory so might need further tuning)

Now how to force you app. to do that remains altogether different question..

Shridhar

#6Paul Simpson
Paul.Simpson@spgs.org
In reply to: Shridhar Daithankar (#5)
Re: Why does app fail?

Thank you for the advice, unfortunately, that isn't an option, you see I didn't write the application and so cannot control the calls it makes. What I need is for PG to do as it's told!

I can only think of three solutions which I'd appreciate comments / advice on....

1) Hack PG Source and re-compile to make upper-case the default table name.
2) Hack the ODBC driver source to get it to always add quotes to table names.
3) Given that the statement seems to be looking for the table name in one of the system tables, put a trigger on that table to automatically convert any new additions to upper case.

Of the three, I suspect #3 would be the easiest, but would I screw up anything else? If anyone suggests #1 or #2 as the "best" alternative, then could they offer advice on how to proceed? (Actually advice on #3 would be welcome too....!)

As an aside, it would seem that from searching the web, this is a common problem. The "quote the names" solution just doesn't work for a lot of people. I would suggest that if PG is to be seen as a realistic migration path from other DB's, this really needs to be fixed by the developers.

Anyway, thanks again for the help so far - hopefully not much further to go!

Paul

Shridhar Daithankar <shridhar@frodo.hserus.net> 25/02/2004 08:58:31 >>>

On Tuesday 24 February 2004 22:10, Paul Simpson wrote:

Create Table TTATTASSOC (AssocId varchar(10), SetId varchar(10), SlotId
int4)
I have, therefore, come to the conclusion that the app requires table names
in Upper case. How can I force this to work?

I believe postgresql is noisy about the case in tablename. In the create table
statement above if you want to preserve the case in tablename, you have to
quote it.

Try it and let us know. (This is out of memory so might need further tuning)

Now how to force you app. to do that remains altogether different question..

Shridhar

ABOUT THIS E-MAIL
The information contained within this e-mail is privileged, confidential and protected from disclosure.

Any opinions expressed in this electronic communication do not necessarily reflect official policy of St Paul's Girls' School.

If you are not the intended recipient of this document or any part then please:

1) : inform the sender at the above address
2) : delete this e-mail immediately
3) : do not take copies nor disclose the contents of this document to anyone

PUPILS' COMMUNICATIONS
Only Senior School pupils of St Paul's Girls' School are permitted to send electronic mail and then only via our controlled system.
It is contrary to school rules for any other pupil to do so and they are regularly reminded of this fact.

VIRUS PROTECTION
All electronic mail traffic sent from this establishment is monitored and scanned for viruses prior to sending.

#7Paul Simpson
Paul.Simpson@spgs.org
In reply to: Paul Simpson (#6)
Re: Why does app fail?

I don't know about that. I'm assuming that it is something the ODBC driver is supplying. The supplier simply states they don't support PostgreSQL, so there isn't much I can do in that arena.

I accept the back-end converts them. What I'm saying is could I not re-compile the back-end to make it convert to upper case instead? There must be a call somewhere in all that C code that does the deed and change it from a call to "tolower()" (or equivalent) to a call to "toupper()" (or equivalent).

I'm not bothered about having to remember the table name case. The application uses all upper-case anyway (hence the problem!) and other methods of access don't use ODBC.

Shame about the triggers on the system tables.....

Paul

Martijn van Oosterhout <kleptog@svana.org> 25/02/2004 10:25:50 >>>

On Wed, Feb 25, 2004 at 10:08:02AM +0000, Paul Simpson wrote:

Thank you for the advice, unfortunately, that isn't an option, you
see I didn't write the application and so cannot control the calls it
makes. What I need is for PG to do as it's told!

Now hang a moment. This application has code is specifically written to
handle postgresql (no other database has pg_catalog) and yet they can't
tweak it to make the queries case insensetive.

I can only think of three solutions which I'd appreciate comments / advice on....

1) Hack PG Source and re-compile to make upper-case the default table name.

The backend converts them to lower case anyway.

2) Hack the ODBC driver source to get it to always add quotes to table names.

Bad idea, then you have to remember the case. Best to always leave the
quotes off.

3) Given that the statement seems to be looking for the table name in
one of the system tables, put a trigger on that table to
automatically convert any new additions to upper case.

Triggers on system tables don't work.

As an aside, it would seem that from searching the web, this is a
common problem. The "quote the names" solution just doesn't work for
a lot of people. I would suggest that if PG is to be seen as a
realistic migration path from other DB's, this really needs to be
fixed by the developers.

If you write code specific for postgres to check for tables, add a
lower() to the query so it matches. Like so:

select relname, nspname, relkind from pg_catalog.pg_class,
pg_catalog.pg_namespace where relkind in ('r', 'v')and nspname like
'public' and relname like lower('TTATTASSOC') and relname !~ '^pg_|^dd_' and
pg_namespace.oid = relnamespace order by nspname, relname

--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

If the Catholic church can survive the printing press, science fiction
will certainly weather the advent of bookwarez.
http://craphound.com/ebooksneitherenorbooks.txt - Cory Doctorow

ABOUT THIS E-MAIL
The information contained within this e-mail is privileged, confidential and protected from disclosure.

Any opinions expressed in this electronic communication do not necessarily reflect official policy of St Paul's Girls' School.

If you are not the intended recipient of this document or any part then please:

1) : inform the sender at the above address
2) : delete this e-mail immediately
3) : do not take copies nor disclose the contents of this document to anyone

PUPILS' COMMUNICATIONS
Only Senior School pupils of St Paul's Girls' School are permitted to send electronic mail and then only via our controlled system.
It is contrary to school rules for any other pupil to do so and they are regularly reminded of this fact.

VIRUS PROTECTION
All electronic mail traffic sent from this establishment is monitored and scanned for viruses prior to sending.

#8Tom Lane
tgl@sss.pgh.pa.us
In reply to: Paul Simpson (#7)
Re: Why does app fail?

"Paul Simpson" <Paul.Simpson@spgs.org> writes:

I accept the back-end converts them. What I'm saying is could I not re-comp
ile the back-end to make it convert to upper case instead?

If that were a simple change, somebody would have done it before you.
(Hint: the problem is not in changing a few lines in the lexer, it's in
fixing everything else that will break.)

What might be easier is to hack the ODBC driver so that those
system-catalog queries it issues are case-insensitive --- for instance,
use "ilike" in place of "like". I am not certain that would fix your
problem completely, but it'd be worth trying.

regards, tom lane

#9Paul Simpson
Paul.Simpson@spgs.org
In reply to: Tom Lane (#8)
Re: Why does app fail?

I have downloaded it, thanks.

I'm not sure about this "foldtoupper" idea (particularly as they are also talking about leaving the tablenames alone in the system tables.

What I don't understand is why the system can't be told simply to be case sensitive. That way, if a table is created as MyTaBlE then it is called MyTaBlE without requiring quotes, or am I missing something here?

Paul

Jeff Eckermann <jeff_eckermann@yahoo.com> 25/02/2004 14:40:41 >>>

Just a couple of notes:

Make sure that you are using the latest available
version of the ODBC driver. You can get the latest
snapshot of the .dll from Hiroshi's page:

http://www.geocities.jp/inocchichichi/psqlodbc/

There was discussion of, and agreement in principle
to, a user setting "foldtoupper", which would take
care of your problem, and all similar ones. But
apparently that has not been picked up by anyone. See
the following:

http://groups.google.com/groups?hl=en&amp;lr=&amp;ie=UTF-8&amp;threadm=15001.1027109813%40sss.pgh.pa.us&amp;rnum=1&amp;prev=/groups%3Fhl%3Den%26lr%3D%26ie%3DISO-8859-1%26q%3Dfoldtoupper%26btnG%3DGoogle%2BSearch%26meta%3Dgroup%253Dcomp.databases.postgresql.*

I am thinking that providing this option would be
important if we want to encourage more people to port
their apps from other RDBMS's. But I am not
comptetent to contribute to that, so this is all I
will say about it.

--- Paul Simpson <Paul.Simpson@spgs.org> wrote:

Shridhar Daithankar

<shridhar_daithankar@persistent.co.in> 25/02/2004
10:31:16 >>>

On Wednesday 25 February 2004 15:38, Paul Simpson

wrote:

Thank you for the advice, unfortunately, that

isn't an option, you see I

didn't write the application and so cannot

control the calls it makes. What

I need is for PG to do as it's told!

I can only think of three solutions which I'd

appreciate comments / advice

on....

1) Hack PG Source and re-compile to make

upper-case the default table name.

2) Hack the ODBC driver source to get it to

always add quotes to table

names. 3) Given that the statement seems to be

looking for the table name

in one of the system tables, put a trigger on

that table to automatically

convert any new additions to upper case.

I vote for option 2 It might take care of any other

complaints such as

mismatch in locale etc. as well. I don't know

really.

Do we need to submit a TODO to hackers or on ODBC

List? I am not too sure..

Shridhar

Having spent a lot of time searching on the 'net, I
suspect this would be a very popular option for a
lot of people. What are the chances of getting this
done fairly quickly, do you think?

Paul

ABOUT THIS E-MAIL
The information contained within this e-mail is
privileged, confidential and protected from
disclosure.

Any opinions expressed in this electronic
communication do not necessarily reflect official
policy of St Paul's Girls' School.

If you are not the intended recipient of this
document or any part then please:

1) : inform the sender at the above address
2) : delete this e-mail immediately
3) : do not take copies nor disclose the contents of
this document to anyone

PUPILS' COMMUNICATIONS
Only Senior School pupils of St Paul's Girls' School
are permitted to send electronic mail and then only
via our controlled system.
It is contrary to school rules for any other pupil
to do so and they are regularly reminded of this
fact.

VIRUS PROTECTION
All electronic mail traffic sent from this
establishment is monitored and scanned for viruses
prior to sending.

__________________________________
Do you Yahoo!?
Yahoo! Mail SpamGuard - Read only the mail you want.
http://antispam.yahoo.com/tools

ABOUT THIS E-MAIL
The information contained within this e-mail is privileged, confidential and protected from disclosure.

Any opinions expressed in this electronic communication do not necessarily reflect official policy of St Paul's Girls' School.

If you are not the intended recipient of this document or any part then please:

1) : inform the sender at the above address
2) : delete this e-mail immediately
3) : do not take copies nor disclose the contents of this document to anyone

PUPILS' COMMUNICATIONS
Only Senior School pupils of St Paul's Girls' School are permitted to send electronic mail and then only via our controlled system.
It is contrary to school rules for any other pupil to do so and they are regularly reminded of this fact.

VIRUS PROTECTION
All electronic mail traffic sent from this establishment is monitored and scanned for viruses prior to sending.

#10Stephan Szabo
sszabo@megazone23.bigpanda.com
In reply to: Paul Simpson (#9)
Re: Why does app fail?

On Wed, 25 Feb 2004, Paul Simpson wrote:

I have downloaded it, thanks.
I'm not sure about this "foldtoupper" idea (particularly as they are
also talking about leaving the tablenames alone in the system tables.

What I don't understand is why the system can't be told simply to be
case sensitive. That way, if a table is created as MyTaBlE then it is
called MyTaBlE without requiring quotes, or am I missing something here?

It could, but I think that'd be even further from spec complience than
what we do now (fold to lowercase). Foldtoupper should provide a setting
that is not backward compatible with postgresql versions but meets the SQL
spec for handling unquoted names.

#11Scott Ribe
scott_ribe@killerbytes.com
In reply to: Paul Simpson (#4)
Re: Why does app fail?

I have, therefore, come to the conclusion that the app requires table names in
Upper case. How can I force this to work?

Check your ODBC driver; some ODBC drivers have an option to quote
identifiers. If yours doesn't, then maybe you should look around and see if
you can find one that does. (In general, when I've been forced to buy
commercial ODBC drivers, I've had good luck with OpenLink's products.)

--
Scott Ribe
scott_ribe@killerbytes.com
http://www.killerbytes.com/
(303) 665-7007 voice