python patch

Hi Greg,

If you're looking at the Python code, do you feel like trying to submit a
patch to make it respec the new 'attisdropped' attribute of the
'pg_attribute' catalog. This is a flag that indicates that a column is
dropped and I notice that Python accesses the pg_attribute relation, and
probably needs to skip over attisdropped columns.

Oh yeah, you'd have to be working with CVS postgres to do this...

Just a thought...no pressure :)

Chris

I don't have a problem looking into it but I can't promise I can get it
right. My python skills are fairly good...my postgres internal skills
are still sub-par IMO.

From a cursory review, if attisdropped is true then the attribute/column
should be ignored/skipped?! Seems pretty dang straight forward.

I'll have a look at it and see what I can come up with.

FYI, I'm currently working off of anonymous CVS. The patch I submitted
was against CVS, current within the last couple of hours.

Greg

I don't have a problem looking into it but I can't promise I can get it
right. My python skills are fairly good...my postgres internal skills
are still sub-par IMO.

From a cursory review, if attisdropped is true then the attribute/column
should be ignored/skipped?! Seems pretty dang straight forward.

Basically, yep. Just grep the source code for pg_attribute most likely...

I'm interested in knowing what it uses pg_attribute for as well...?

Chris

Well, that certainly appeared to be very straight forward. pg.py and
syscat.py scripts were both modified. pg.py uses it to cache a list of
pks (which is seemingly does for every db connection) and various
attributes. syscat uses it to walk the list of system tables and
queries the various attributes from these tables.

In both cases, it seemingly makes sense to apply what you've requested.

Please find attached the quested patch below.

Greg

Your patch has been added to the PostgreSQL unapplied patches list at:

http://candle.pha.pa.us/cgi-bin/pgpatches

I will try to apply it within the next 48 hours.

---------------------------------------------------------------------------

Greg Copeland wrote:

Checking application/pgp-signature: FAILURE
-- Start of PGP signed section.

Okay, I read
http://archives.postgresql.org/pgsql-bugs/2002-06/msg00086.php and never
saw a fix offered up. Since I'm gearing up to use Postgres and Python
soon, I figured I'd have a hand at trying to get this sucker addressed.
Apologies if this has already been plugged. I looked in the archives
and never saw a response.

At any rate, I must admit I don't think I fully understand the
implications of some of the changes I made even though they appear to be
straight forward. We all know the devil is in the details. Anyone more
knowledgeable is requested to review my changes. :(

I also updated the advanced.py script in a somewhat nonsensical fashion
to make use of an int8 field in an effort to test this change. It seems
to run okay, however, this is by no means an all exhaustive test. So,
it's possible that a bumpy road may lay ahead for some. On the other
hand...overflows (hopefully) previously lurked (long -> int conversion).

This is my first submission. Please be kind if I submitted to the wrong
list. ;)

Thank you,
Greg Copeland

[ text/x-diff is unsupported, treating like TEXT/PLAIN ]

? lib_pgmodule.so.0.0
? postgres-python.patch
? tutorial/advanced.pyc
Index: pgmodule.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/pgmodule.c,v
retrieving revision 1.38
diff -u -r1.38 pgmodule.c
--- pgmodule.c	2002/03/29 07:45:39	1.38
+++ pgmodule.c	2002/08/08 02:46:12
@@ -289,23 +289,26 @@
{
case INT2OID:
case INT4OID:
-			case INT8OID:
case OIDOID:
typ[j] = 1;
break;

+			case INT8OID:
+				typ[j] = 2;
+				break;
+
case FLOAT4OID:
case FLOAT8OID:
case NUMERICOID:
-				typ[j] = 2;
+				typ[j] = 3;
break;

case CASHOID:
-				typ[j] = 3;
+				typ[j] = 4;
break;

default:
- typ[j] = 4;
+ typ[j] = 5;
break;
}
}
@@ -1797,23 +1800,26 @@
{
case INT2OID:
case INT4OID:
- case INT8OID:
case OIDOID:
typ[j] = 1;
break;

+			case INT8OID:
+				typ[j] = 2;
+				break;
+
case FLOAT4OID:
case FLOAT8OID:
case NUMERICOID:
-				typ[j] = 2;
+				typ[j] = 3;
break;

case CASHOID:
-				typ[j] = 3;
+				typ[j] = 4;
break;

default:
- typ[j] = 4;
+ typ[j] = 5;
break;
}
}
@@ -1846,10 +1852,14 @@
break;

case 2:
-						val = PyFloat_FromDouble(strtod(s, NULL));
+						val = PyLong_FromLong(strtol(s, NULL, 10));
break;

case 3:
+						val = PyFloat_FromDouble(strtod(s, NULL));
+						break;
+
+					case 4:
{
int			mult = 1;

@@ -1946,11 +1956,14 @@
{
case INT2OID:
case INT4OID:
- case INT8OID:
case OIDOID:
typ[j] = 1;
break;

+			case INT8OID:
+				typ[j] = 2;
+				break;
+
case FLOAT4OID:
case FLOAT8OID:
case NUMERICOID:
@@ -1995,10 +2008,14 @@
break;

case 2:
-						val = PyFloat_FromDouble(strtod(s, NULL));
+						val = PyLong_FromLong(strtol(s, NULL, 10));
break;

case 3:
+						val = PyFloat_FromDouble(strtod(s, NULL));
+						break;
+
+					case 4:
{
int			mult = 1;

Index: tutorial/advanced.py
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/tutorial/advanced.py,v
retrieving revision 1.5
diff -u -r1.5 advanced.py
--- tutorial/advanced.py	2000/10/02 03:46:24	1.5
+++ tutorial/advanced.py	2002/08/08 02:46:12
@@ -109,11 +109,13 @@
print "CREATE TABLE sal_emp ("
print "    name            text,"
print "    pay_by_quarter  int4[],"
+	print "    pay_by_extra_quarter  int8[],"
print "    schedule        text[][]"
print ")"
pgcnx.query("""CREATE TABLE sal_emp (
name              text,
pay_by_quarter    int4[],
+        pay_by_extra_quarter    int8[],
schedule          text[][])""")
wait_key()
print
@@ -123,18 +125,22 @@
print "INSERT INTO sal_emp VALUES ("
print "    'Bill',"
print "    '{10000,10000,10000,10000}',"
+	print "    '{9223372036854775800,9223372036854775800,9223372036854775800}',"
print "    '{{\"meeting\", \"lunch\"}, {}}')"
print
print "INSERT INTO sal_emp VALUES ("
print "    'Carol',"
print "    '{20000,25000,25000,25000}',"
+	print "    '{9223372036854775807,9223372036854775807,9223372036854775807}',"
print "    '{{\"talk\", \"consult\"}, {\"meeting\"}}')"
print
pgcnx.query("""INSERT INTO sal_emp VALUES (
'Bill', '{10000,10000,10000,10000}',
+	'{9223372036854775800,9223372036854775800,9223372036854775800}',
'{{\"meeting\", \"lunch\"}, {}}')""")
pgcnx.query("""INSERT INTO sal_emp VALUES (
'Carol', '{20000,25000,25000,25000}',
+	'{9223372036854775807,9223372036854775807,9223372036854775807}',
'{{\"talk\", \"consult\"}, {\"meeting\"}}')""")
wait_key()
print
@@ -148,11 +154,25 @@
print pgcnx.query("""SELECT name FROM sal_emp WHERE
sal_emp.pay_by_quarter[1] <> sal_emp.pay_by_quarter[2]""")
print
+	print pgcnx.query("""SELECT name FROM sal_emp WHERE
+        sal_emp.pay_by_extra_quarter[1] <> sal_emp.pay_by_extra_quarter[2]""")
+	print
print "-- retrieve third quarter pay of all employees"
print 
print "SELECT sal_emp.pay_by_quarter[3] FROM sal_emp"
print
print pgcnx.query("SELECT sal_emp.pay_by_quarter[3] FROM sal_emp")
+	print
+	print "-- retrieve third quarter extra pay of all employees"
+	print 
+	print "SELECT sal_emp.pay_by_extra_quarter[3] FROM sal_emp"
+	print pgcnx.query("SELECT sal_emp.pay_by_extra_quarter[3] FROM sal_emp")
+	print 
+	print "-- retrieve first two quarters of extra quarter pay of all employees"
+	print 
+	print "SELECT sal_emp.pay_by_extra_quarter[1:2] FROM sal_emp"
+	print
+	print pgcnx.query("SELECT sal_emp.pay_by_extra_quarter[1:2] FROM sal_emp")
print
print "-- select subarrays"
print 

-- End of PGP section, PGP failed!

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Your patch has been added to the PostgreSQL unapplied patches list at:

http://candle.pha.pa.us/cgi-bin/pgpatches

I will try to apply it within the next 48 hours.

---------------------------------------------------------------------------

Greg Copeland wrote:

Checking application/pgp-signature: FAILURE
-- Start of PGP signed section.

Well, that certainly appeared to be very straight forward. pg.py and
syscat.py scripts were both modified. pg.py uses it to cache a list of
pks (which is seemingly does for every db connection) and various
attributes. syscat uses it to walk the list of system tables and
queries the various attributes from these tables.

In both cases, it seemingly makes sense to apply what you've requested.

Please find attached the quested patch below.

Greg

On Wed, 2002-08-07 at 22:16, Christopher Kings-Lynne wrote:

I don't have a problem looking into it but I can't promise I can get it
right. My python skills are fairly good...my postgres internal skills
are still sub-par IMO.

From a cursory review, if attisdropped is true then the attribute/column
should be ignored/skipped?! Seems pretty dang straight forward.

Basically, yep. Just grep the source code for pg_attribute most likely...

I'm interested in knowing what it uses pg_attribute for as well...?

Chris

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

[ text/x-patch is unsupported, treating like TEXT/PLAIN ]

Index: pg.py
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/pg.py,v
retrieving revision 1.9
diff -u -r1.9 pg.py
--- pg.py	2002/03/19 13:20:52	1.9
+++ pg.py	2002/08/08 03:29:48
@@ -69,7 +69,8 @@
WHERE pg_class.oid = pg_attribute.attrelid AND
pg_class.oid = pg_index.indrelid AND
pg_index.indkey[0] = pg_attribute.attnum AND 
-							pg_index.indisprimary = 't'""").getresult():
+							pg_index.indisprimary = 't' AND
+							pg_attribute.attisdropped = 'f'""").getresult():
self.__pkeys__[rel] = att

# wrap query for debugging
@@ -111,7 +112,8 @@
WHERE pg_class.relname = '%s' AND
pg_attribute.attnum > 0 AND
pg_attribute.attrelid = pg_class.oid AND
-						pg_attribute.atttypid = pg_type.oid"""
+						pg_attribute.atttypid = pg_type.oid AND
+						pg_attribute.attisdropped = 'f'"""

l = {}
for attname, typname in self.db.query(query % cl).getresult():
Index: tutorial/syscat.py
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/tutorial/syscat.py,v
retrieving revision 1.7
diff -u -r1.7 syscat.py
--- tutorial/syscat.py	2002/05/03 14:21:38	1.7
+++ tutorial/syscat.py	2002/08/08 03:29:48
@@ -37,7 +37,7 @@
FROM pg_class bc, pg_class ic, pg_index i, pg_attribute a
WHERE i.indrelid = bc.oid AND i.indexrelid = bc.oid
AND i.indkey[0] = a.attnum AND a.attrelid = bc.oid
-				AND i.indproc = '0'::oid
+				AND i.indproc = '0'::oid AND a.attisdropped = 'f'
ORDER BY class_name, index_name, attname""")
return result

@@ -48,6 +48,7 @@
WHERE c.relkind = 'r' and c.relname !~ '^pg_'
AND c.relname !~ '^Inv' and a.attnum > 0
AND a.attrelid = c.oid and a.atttypid = t.oid
+                        AND a.attisdropped = 'f'
ORDER BY relname, attname""")
return result

-- End of PGP section, PGP failed!

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

I wouldn't apply this _just_ yet Bruce as I'm not certain all the changes
are necessary... I intend to look into it but I haven't had the time yet
(sorry Greg!)

Chris

On Sun, 11 Aug 2002, Bruce Momjian wrote:

Your patch has been added to the PostgreSQL unapplied patches list at:

http://candle.pha.pa.us/cgi-bin/pgpatches

I will try to apply it within the next 48 hours.

---------------------------------------------------------------------------

Greg Copeland wrote:

Checking application/pgp-signature: FAILURE

Not a problem. I would rather them be correct.

Worth noting that the first patch is what attempts to fix the long ->
int overflow issue. The second patch attempts to resolve "attisdropped"
column use issues with the python scripts. The third patch addresses
issues generated by the implicate to explicate use of "cascade".

I assume your reservations are only with the second patch and not the
first and third patches?

Greg

OK, great to have people reviewing them. I will hold on all the python
patches until I hear back from Christopher:

http://candle.pha.pa.us/cgi-bin/pgpatches

---------------------------------------------------------------------------

Greg Copeland wrote:

Checking application/pgp-signature: FAILURE
-- Start of PGP signed section.

Not a problem. I would rather them be correct.

Worth noting that the first patch is what attempts to fix the long ->
int overflow issue. The second patch attempts to resolve "attisdropped"
column use issues with the python scripts. The third patch addresses
issues generated by the implicate to explicate use of "cascade".

I assume your reservations are only with the second patch and not the
first and third patches?

Greg

On Sun, 2002-08-11 at 04:43, Christopher Kings-Lynne wrote:

I wouldn't apply this _just_ yet Bruce as I'm not certain all the changes
are necessary... I intend to look into it but I haven't had the time yet
(sorry Greg!)

Chris

On Sun, 11 Aug 2002, Bruce Momjian wrote:

Your patch has been added to the PostgreSQL unapplied patches list at:

http://candle.pha.pa.us/cgi-bin/pgpatches

I will try to apply it within the next 48 hours.

---------------------------------------------------------------------------

Greg Copeland wrote:

Checking application/pgp-signature: FAILURE

-- Start of PGP signed section.

Well, that certainly appeared to be very straight forward. pg.py and
syscat.py scripts were both modified. pg.py uses it to cache a list of
pks (which is seemingly does for every db connection) and various
attributes. syscat uses it to walk the list of system tables and
queries the various attributes from these tables.

In both cases, it seemingly makes sense to apply what you've requested.

Please find attached the quested patch below.

Greg

On Wed, 2002-08-07 at 22:16, Christopher Kings-Lynne wrote:

I don't have a problem looking into it but I can't promise I can get it
right. My python skills are fairly good...my postgres internal skills
are still sub-par IMO.

From a cursory review, if attisdropped is true then the attribute/column
should be ignored/skipped?! Seems pretty dang straight forward.

Basically, yep. Just grep the source code for pg_attribute most likely...

I'm interested in knowing what it uses pg_attribute for as well...?

Chris

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

[ text/x-patch is unsupported, treating like TEXT/PLAIN ]

Index: pg.py
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/pg.py,v
retrieving revision 1.9
diff -u -r1.9 pg.py
--- pg.py	2002/03/19 13:20:52	1.9
+++ pg.py	2002/08/08 03:29:48
@@ -69,7 +69,8 @@
WHERE pg_class.oid = pg_attribute.attrelid AND
pg_class.oid = pg_index.indrelid AND
pg_index.indkey[0] = pg_attribute.attnum AND
-							pg_index.indisprimary = 't'""").getresult():
+							pg_index.indisprimary = 't' AND
+							pg_attribute.attisdropped = 'f'""").getresult():
self.__pkeys__[rel] = att

# wrap query for debugging
@@ -111,7 +112,8 @@
WHERE pg_class.relname = '%s' AND
pg_attribute.attnum > 0 AND
pg_attribute.attrelid = pg_class.oid AND
-						pg_attribute.atttypid = pg_type.oid"""
+						pg_attribute.atttypid = pg_type.oid AND
+						pg_attribute.attisdropped = 'f'"""

l = {}
for attname, typname in self.db.query(query % cl).getresult():
Index: tutorial/syscat.py
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/tutorial/syscat.py,v
retrieving revision 1.7
diff -u -r1.7 syscat.py
--- tutorial/syscat.py	2002/05/03 14:21:38	1.7
+++ tutorial/syscat.py	2002/08/08 03:29:48
@@ -37,7 +37,7 @@
FROM pg_class bc, pg_class ic, pg_index i, pg_attribute a
WHERE i.indrelid = bc.oid AND i.indexrelid = bc.oid
AND i.indkey[0] = a.attnum AND a.attrelid = bc.oid
-				AND i.indproc = '0'::oid
+				AND i.indproc = '0'::oid AND a.attisdropped = 'f'
ORDER BY class_name, index_name, attname""")
return result

@@ -48,6 +48,7 @@
WHERE c.relkind = 'r' and c.relname !~ '^pg_'
AND c.relname !~ '^Inv' and a.attnum > 0
AND a.attrelid = c.oid and a.atttypid = t.oid
+                        AND a.attisdropped = 'f'
ORDER BY relname, attname""")
return result

-- End of PGP section, PGP failed!

--
Bruce Momjian                        |  http://candle.pha.pa.us
pgman@candle.pha.pa.us               |  (610) 359-1001
+  If your life is a hard drive,     |  13 Roberts Road
+  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

-- End of PGP section, PGP failed!

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Not a problem. I would rather them be correct.

Worth noting that the first patch is what attempts to fix the long ->
int overflow issue. The second patch attempts to resolve "attisdropped"
column use issues with the python scripts. The third patch addresses
issues generated by the implicate to explicate use of "cascade".

I assume your reservations are only with the second patch and not the
first and third patches?

Correct. I'm pretty sure you don't need to exclude attisdropped from the
primary key list because all it's doing is finding the column that a primary
key is over and that should never be over a dropped column. I can't
remember what you said the second query did?

Chris

On Sun, 2002-08-11 at 21:15, Christopher Kings-Lynne wrote:

Not a problem. I would rather them be correct.

Worth noting that the first patch is what attempts to fix the long ->
int overflow issue. The second patch attempts to resolve "attisdropped"
column use issues with the python scripts. The third patch addresses
issues generated by the implicate to explicate use of "cascade".

I assume your reservations are only with the second patch and not the
first and third patches?

Correct. I'm pretty sure you don't need to exclude attisdropped from the
primary key list because all it's doing is finding the column that a primary
key is over and that should never be over a dropped column. I can't
remember what you said the second query did?

Hmmm. Sounds okay but I'm just not sure that holds true (as I
previously stated, I'm ignorant on the topic). Obviously I'll defer to
you on this.

Here's the queries and what they do:

From pg.py:
Used to locate primary keys -- or so the comment says. It does create a
dictionary of keys and attribute values for each returned row so I
assume it really is attempting to do something of the like.

SELECT pg_class.relname, pg_attribute.attname
FROM pg_class, pg_attribute, pg_index
WHERE pg_class.oid = pg_attribute.attrelid AND
pg_class.oid = pg_index.indrelid AND
pg_index.indkey[0] = pg_attribute.attnum AND
pg_index.indisprimary = 't' AND
pg_attribute.attisdropped = 'f' ;

So, everyone is in agreement that any attribute which is indexed as a
primary key will never be able to have attisdtopped = 't'?

According to the code:
SELECT pg_attribute.attname, pg_type.typname
FROM pg_class, pg_attribute, pg_type
WHERE pg_class.relname = '%s' AND
pg_attribute.attnum > 0 AND
pg_attribute.attrelid = pg_class.oid AND
pg_attribute.atttypid = pg_type.oid AND
pg_attribute.attisdropped = 'f' ;

is used to obtain all attributes (column names) and their types for a
given table ('%s'). It then attempts to build a column/type cache. I'm
assuming that this really does need to be there. Please correct
accordingly.

From syscat.py:
SELECT bc.relname AS class_name,
ic.relname AS index_name, a.attname
FROM pg_class bc, pg_class ic, pg_index i, pg_attribute a
WHERE i.indrelid = bc.oid AND i.indexrelid = bc.oid
AND i.indkey[0] = a.attnum AND a.attrelid = bc.oid
AND i.indproc = '0'::oid AND a.attisdropped = 'f'
ORDER BY class_name, index_name, attname ;

According to the nearby documentation, it's supposed to be fetching a
list of "all simple indicies". If that's the case, is it safe to assume
that any indexed column will never have attisdropped = 't'? If so, we
can remove that check from the file as well. Worth pointing out, this
is from syscat.py, which is sample source and not used as actual
interface. So, worse case, it would appear to be redundant in nature
with no harm done.

This should conclude the patched items offered in the second patch.

What ya think?

Thanks,
Greg

All of that said, the cost of the check is so small it may save someones
ass some day when they have a corrupted catalog and the below
assumptions are no longer true.

Well, I tend to agree with that. Overall, I can't say that I see bad
things coming out of accepting the patch as is. It's not exactly
causing an extra join or other wise a significant waste of resources.
At worst, it appears to be ambiguous. Since Christopher has not offered
any additional follow up, can we assume that he agrees? In not, please
let me know and I'll resubmit patch #2.

In the mean time, patches #1 and #3 should be good to go. Bruce, feel
free to apply those whenever time allows.

Thanks,
Greg Copeland

Yep - alright, just commit it I guess.

Chris

Patch applied. Thanks.

---------------------------------------------------------------------------

Greg Copeland wrote:

Checking application/pgp-signature: FAILURE
-- Start of PGP signed section.

Okay, I read
http://archives.postgresql.org/pgsql-bugs/2002-06/msg00086.php and never
saw a fix offered up. Since I'm gearing up to use Postgres and Python
soon, I figured I'd have a hand at trying to get this sucker addressed.
Apologies if this has already been plugged. I looked in the archives
and never saw a response.

At any rate, I must admit I don't think I fully understand the
implications of some of the changes I made even though they appear to be
straight forward. We all know the devil is in the details. Anyone more
knowledgeable is requested to review my changes. :(

I also updated the advanced.py script in a somewhat nonsensical fashion
to make use of an int8 field in an effort to test this change. It seems
to run okay, however, this is by no means an all exhaustive test. So,
it's possible that a bumpy road may lay ahead for some. On the other
hand...overflows (hopefully) previously lurked (long -> int conversion).

This is my first submission. Please be kind if I submitted to the wrong
list. ;)

Thank you,
Greg Copeland

[ text/x-diff is unsupported, treating like TEXT/PLAIN ]

? lib_pgmodule.so.0.0
? postgres-python.patch
? tutorial/advanced.pyc
Index: pgmodule.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/pgmodule.c,v
retrieving revision 1.38
diff -u -r1.38 pgmodule.c
--- pgmodule.c	2002/03/29 07:45:39	1.38
+++ pgmodule.c	2002/08/08 02:46:12
@@ -289,23 +289,26 @@
{
case INT2OID:
case INT4OID:
-			case INT8OID:
case OIDOID:
typ[j] = 1;
break;

+			case INT8OID:
+				typ[j] = 2;
+				break;
+
case FLOAT4OID:
case FLOAT8OID:
case NUMERICOID:
-				typ[j] = 2;
+				typ[j] = 3;
break;

case CASHOID:
-				typ[j] = 3;
+				typ[j] = 4;
break;

default:
- typ[j] = 4;
+ typ[j] = 5;
break;
}
}
@@ -1797,23 +1800,26 @@
{
case INT2OID:
case INT4OID:
- case INT8OID:
case OIDOID:
typ[j] = 1;
break;

+			case INT8OID:
+				typ[j] = 2;
+				break;
+
case FLOAT4OID:
case FLOAT8OID:
case NUMERICOID:
-				typ[j] = 2;
+				typ[j] = 3;
break;

case CASHOID:
-				typ[j] = 3;
+				typ[j] = 4;
break;

default:
- typ[j] = 4;
+ typ[j] = 5;
break;
}
}
@@ -1846,10 +1852,14 @@
break;

case 2:
-						val = PyFloat_FromDouble(strtod(s, NULL));
+						val = PyLong_FromLong(strtol(s, NULL, 10));
break;

case 3:
+						val = PyFloat_FromDouble(strtod(s, NULL));
+						break;
+
+					case 4:
{
int			mult = 1;

@@ -1946,11 +1956,14 @@
{
case INT2OID:
case INT4OID:
- case INT8OID:
case OIDOID:
typ[j] = 1;
break;

+			case INT8OID:
+				typ[j] = 2;
+				break;
+
case FLOAT4OID:
case FLOAT8OID:
case NUMERICOID:
@@ -1995,10 +2008,14 @@
break;

case 2:
-						val = PyFloat_FromDouble(strtod(s, NULL));
+						val = PyLong_FromLong(strtol(s, NULL, 10));
break;

case 3:
+						val = PyFloat_FromDouble(strtod(s, NULL));
+						break;
+
+					case 4:
{
int			mult = 1;

Index: tutorial/advanced.py
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/tutorial/advanced.py,v
retrieving revision 1.5
diff -u -r1.5 advanced.py
--- tutorial/advanced.py	2000/10/02 03:46:24	1.5
+++ tutorial/advanced.py	2002/08/08 02:46:12
@@ -109,11 +109,13 @@
print "CREATE TABLE sal_emp ("
print "    name            text,"
print "    pay_by_quarter  int4[],"
+	print "    pay_by_extra_quarter  int8[],"
print "    schedule        text[][]"
print ")"
pgcnx.query("""CREATE TABLE sal_emp (
name              text,
pay_by_quarter    int4[],
+        pay_by_extra_quarter    int8[],
schedule          text[][])""")
wait_key()
print
@@ -123,18 +125,22 @@
print "INSERT INTO sal_emp VALUES ("
print "    'Bill',"
print "    '{10000,10000,10000,10000}',"
+	print "    '{9223372036854775800,9223372036854775800,9223372036854775800}',"
print "    '{{\"meeting\", \"lunch\"}, {}}')"
print
print "INSERT INTO sal_emp VALUES ("
print "    'Carol',"
print "    '{20000,25000,25000,25000}',"
+	print "    '{9223372036854775807,9223372036854775807,9223372036854775807}',"
print "    '{{\"talk\", \"consult\"}, {\"meeting\"}}')"
print
pgcnx.query("""INSERT INTO sal_emp VALUES (
'Bill', '{10000,10000,10000,10000}',
+	'{9223372036854775800,9223372036854775800,9223372036854775800}',
'{{\"meeting\", \"lunch\"}, {}}')""")
pgcnx.query("""INSERT INTO sal_emp VALUES (
'Carol', '{20000,25000,25000,25000}',
+	'{9223372036854775807,9223372036854775807,9223372036854775807}',
'{{\"talk\", \"consult\"}, {\"meeting\"}}')""")
wait_key()
print
@@ -148,11 +154,25 @@
print pgcnx.query("""SELECT name FROM sal_emp WHERE
sal_emp.pay_by_quarter[1] <> sal_emp.pay_by_quarter[2]""")
print
+	print pgcnx.query("""SELECT name FROM sal_emp WHERE
+        sal_emp.pay_by_extra_quarter[1] <> sal_emp.pay_by_extra_quarter[2]""")
+	print
print "-- retrieve third quarter pay of all employees"
print 
print "SELECT sal_emp.pay_by_quarter[3] FROM sal_emp"
print
print pgcnx.query("SELECT sal_emp.pay_by_quarter[3] FROM sal_emp")
+	print
+	print "-- retrieve third quarter extra pay of all employees"
+	print 
+	print "SELECT sal_emp.pay_by_extra_quarter[3] FROM sal_emp"
+	print pgcnx.query("SELECT sal_emp.pay_by_extra_quarter[3] FROM sal_emp")
+	print 
+	print "-- retrieve first two quarters of extra quarter pay of all employees"
+	print 
+	print "SELECT sal_emp.pay_by_extra_quarter[1:2] FROM sal_emp"
+	print
+	print pgcnx.query("SELECT sal_emp.pay_by_extra_quarter[1:2] FROM sal_emp")
print
print "-- select subarrays"
print 

-- End of PGP section, PGP failed!

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Patch applied. Thanks.

---------------------------------------------------------------------------

Greg Copeland wrote:

Checking application/pgp-signature: FAILURE
-- Start of PGP signed section.

Well, that certainly appeared to be very straight forward. pg.py and
syscat.py scripts were both modified. pg.py uses it to cache a list of
pks (which is seemingly does for every db connection) and various
attributes. syscat uses it to walk the list of system tables and
queries the various attributes from these tables.

In both cases, it seemingly makes sense to apply what you've requested.

Please find attached the quested patch below.

Greg

On Wed, 2002-08-07 at 22:16, Christopher Kings-Lynne wrote:

I don't have a problem looking into it but I can't promise I can get it
right. My python skills are fairly good...my postgres internal skills
are still sub-par IMO.

From a cursory review, if attisdropped is true then the attribute/column
should be ignored/skipped?! Seems pretty dang straight forward.

Basically, yep. Just grep the source code for pg_attribute most likely...

I'm interested in knowing what it uses pg_attribute for as well...?

Chris

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

[ text/x-patch is unsupported, treating like TEXT/PLAIN ]

Index: pg.py
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/pg.py,v
retrieving revision 1.9
diff -u -r1.9 pg.py
--- pg.py	2002/03/19 13:20:52	1.9
+++ pg.py	2002/08/08 03:29:48
@@ -69,7 +69,8 @@
WHERE pg_class.oid = pg_attribute.attrelid AND
pg_class.oid = pg_index.indrelid AND
pg_index.indkey[0] = pg_attribute.attnum AND 
-							pg_index.indisprimary = 't'""").getresult():
+							pg_index.indisprimary = 't' AND
+							pg_attribute.attisdropped = 'f'""").getresult():
self.__pkeys__[rel] = att

# wrap query for debugging
@@ -111,7 +112,8 @@
WHERE pg_class.relname = '%s' AND
pg_attribute.attnum > 0 AND
pg_attribute.attrelid = pg_class.oid AND
-						pg_attribute.atttypid = pg_type.oid"""
+						pg_attribute.atttypid = pg_type.oid AND
+						pg_attribute.attisdropped = 'f'"""

l = {}
for attname, typname in self.db.query(query % cl).getresult():
Index: tutorial/syscat.py
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/python/tutorial/syscat.py,v
retrieving revision 1.7
diff -u -r1.7 syscat.py
--- tutorial/syscat.py	2002/05/03 14:21:38	1.7
+++ tutorial/syscat.py	2002/08/08 03:29:48
@@ -37,7 +37,7 @@
FROM pg_class bc, pg_class ic, pg_index i, pg_attribute a
WHERE i.indrelid = bc.oid AND i.indexrelid = bc.oid
AND i.indkey[0] = a.attnum AND a.attrelid = bc.oid
-				AND i.indproc = '0'::oid
+				AND i.indproc = '0'::oid AND a.attisdropped = 'f'
ORDER BY class_name, index_name, attname""")
return result

@@ -48,6 +48,7 @@
WHERE c.relkind = 'r' and c.relname !~ '^pg_'
AND c.relname !~ '^Inv' and a.attnum > 0
AND a.attrelid = c.oid and a.atttypid = t.oid
+                        AND a.attisdropped = 'f'
ORDER BY relname, attname""")
return result

-- End of PGP section, PGP failed!

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

OK, I have applied all three of Greg's python patches.

---------------------------------------------------------------------------

Christopher Kings-Lynne wrote:

Yep - alright, just commit it I guess.

Chris

-----Original Message-----
From: Greg Copeland [mailto:greg@copelandconsulting.net]
Sent: Thursday, 15 August 2002 11:09 AM
To: Rod Taylor
Cc: Christopher Kings-Lynne; Bruce Momjian; PostgresSQL Hackers Mailing
List
Subject: Re: [HACKERS] python patch

Well, I tend to agree with that. Overall, I can't say that I see bad
things coming out of accepting the patch as is. It's not exactly
causing an extra join or other wise a significant waste of resources.
At worst, it appears to be ambiguous. Since Christopher has not offered
any additional follow up, can we assume that he agrees? In not, please
let me know and I'll resubmit patch #2.

In the mean time, patches #1 and #3 should be good to go. Bruce, feel
free to apply those whenever time allows.

Thanks,
Greg Copeland

On Mon, 2002-08-12 at 18:33, Rod Taylor wrote:

All of that said, the cost of the check is so small it may save someones
ass some day when they have a corrupted catalog and the below
assumptions are no longer true.

On Mon, 2002-08-12 at 18:40, Greg Copeland wrote:

On Sun, 2002-08-11 at 21:15, Christopher Kings-Lynne wrote:

Not a problem. I would rather them be correct.

Worth noting that the first patch is what attempts to fix

the long ->

int overflow issue. The second patch attempts to resolve

"attisdropped"

column use issues with the python scripts. The third

patch addresses

issues generated by the implicate to explicate use of "cascade".

I assume your reservations are only with the second patch

and not the

first and third patches?

Correct. I'm pretty sure you don't need to exclude

attisdropped from the

primary key list because all it's doing is finding the

column that a primary

key is over and that should never be over a dropped column. I can't
remember what you said the second query did?

Hmmm. Sounds okay but I'm just not sure that holds true (as I
previously stated, I'm ignorant on the topic). Obviously

I'll defer to

you on this.

Here's the queries and what they do:

From pg.py:

Used to locate primary keys -- or so the comment says. It

does create a

dictionary of keys and attribute values for each returned row so I
assume it really is attempting to do something of the like.

SELECT pg_class.relname, pg_attribute.attname
FROM pg_class, pg_attribute, pg_index
WHERE pg_class.oid = pg_attribute.attrelid AND
pg_class.oid = pg_index.indrelid AND
pg_index.indkey[0] = pg_attribute.attnum AND
pg_index.indisprimary = 't' AND
pg_attribute.attisdropped = 'f' ;

So, everyone is in agreement that any attribute which is indexed as a
primary key will never be able to have attisdtopped = 't'?

According to the code:
SELECT pg_attribute.attname, pg_type.typname
FROM pg_class, pg_attribute, pg_type
WHERE pg_class.relname = '%s' AND
pg_attribute.attnum > 0 AND
pg_attribute.attrelid = pg_class.oid AND
pg_attribute.atttypid = pg_type.oid AND
pg_attribute.attisdropped = 'f' ;

is used to obtain all attributes (column names) and their types for a
given table ('%s'). It then attempts to build a column/type

cache. I'm

assuming that this really does need to be there. Please correct
accordingly.

From syscat.py:

SELECT bc.relname AS class_name,
ic.relname AS index_name, a.attname
FROM pg_class bc, pg_class ic, pg_index i, pg_attribute a
WHERE i.indrelid = bc.oid AND i.indexrelid = bc.oid
AND i.indkey[0] = a.attnum AND a.attrelid = bc.oid
AND i.indproc = '0'::oid AND a.attisdropped = 'f'
ORDER BY class_name, index_name, attname ;

According to the nearby documentation, it's supposed to be fetching a
list of "all simple indicies". If that's the case, is it

safe to assume

that any indexed column will never have attisdropped = 't'? If so, we
can remove that check from the file as well. Worth pointing out, this
is from syscat.py, which is sample source and not used as actual
interface. So, worse case, it would appear to be redundant in nature
with no harm done.

This should conclude the patched items offered in the second patch.

What ya think?

Thanks,
Greg

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Thanks.

-Greg