CVS regression test failure on OBSD
When I run 'make check' on the CVS version (today and for the last
serveral days), I have been getting interesting failures. Some of the
tests fail, but the interesting part is that not the same tests always
fail. Example:
bpalmer@mizer:~/APPS/pgsql>diff 1 2
1c1
< boolean ... FAILED
---
boolean ... ok
3c3
< name ... ok
---
name ... FAILED
5c5
< text ... FAILED
---
text ... ok
9,10c9,10
< oid ... FAILED
< float4 ... ok
---
oid ... ok
float4 ... FAILED
In the regression.diff file, I keep seeing this:
*** ./expected/name.out Tue Jan 4 11:19:34 2000
--- ./results/name.out Sat Jan 6 16:36:34 2001
***************
*** 1,124 ****
! --
! -- NAME
! -- all inputs are silently truncated at NAMEDATALEN (32) characters
! --
! -- fixed-length by reference
! SELECT name 'name string' = name 'name string' AS "True";
! True
! ------
...
...
...
! (2 rows)
!
! DROP TABLE NAME_TBL;
--- 1 ----
! psql: Backend startup failed======================================================================
Any ideas where to look for this one?
When I run the regression test WITH ANOTHER INSTANCE OF PGSQL running, I
got an error telling me that I needed more SEMMNI or SEMMNS. I recompiled
my kernel with the values of 1024 and 2048 respectivly, and that seems to
have solved the problem, but they may still be related.
Any ideas?
- Brandon
b. palmer, bpalmer@crimelabs.net
pgp: www.crimelabs.net/bpalmer.pgp5
bpalmer <bpalmer@crimelabs.net> writes:
! psql: Backend startup failed
Any ideas where to look for this one?
In the postmaster: the fork() to launch a backend is failing. There
should be a more detailed message in the postmaster's stderr log,
but almost certainly it's a resource-exhaustion issue. Does your
kernel enforce a per-userid limit on the number of processes, for
example?
regards, tom lane
In the postmaster: the fork() to launch a backend is failing. There
should be a more detailed message in the postmaster's stderr log,
but almost certainly it's a resource-exhaustion issue. Does your
kernel enforce a per-userid limit on the number of processes, for
example?
Looks like that was part of the problem. Process ulimit and openfile
limit were both problems. I have rolled up the process ulimit (soft /
hard) to 256 and the Open Files to 256.
There are still, however, 4 tests that fail.
Attached.
- Brandon
uname -a
OpenBSD mizer 2.8 a#0 i386 (2.8 snapshot NOV 6 install)
b. palmer, bpalmer@crimelabs.net
pgp: www.crimelabs.net/bpalmer.pgp5
Attachments:
regression.diffstext/plain; charset=US-ASCII; name=regression.diffsDownload
*** ./expected/int2.out Tue Jan 4 11:19:34 2000
--- ./results/int2.out Sat Jan 6 17:56:29 2001
***************
*** 14,20 ****
INSERT INTO INT2_TBL(f1) VALUES ('-32767');
-- bad input values -- should give warnings
INSERT INTO INT2_TBL(f1) VALUES ('100000');
! ERROR: pg_atoi: error reading "100000": Numerical result out of range
INSERT INTO INT2_TBL(f1) VALUES ('asdf');
ERROR: pg_atoi: error in "asdf": can't parse "asdf"
SELECT '' AS five, INT2_TBL.*;
--- 14,20 ----
INSERT INTO INT2_TBL(f1) VALUES ('-32767');
-- bad input values -- should give warnings
INSERT INTO INT2_TBL(f1) VALUES ('100000');
! ERROR: pg_atoi: error reading "100000": Result too large
INSERT INTO INT2_TBL(f1) VALUES ('asdf');
ERROR: pg_atoi: error in "asdf": can't parse "asdf"
SELECT '' AS five, INT2_TBL.*;
======================================================================
*** ./expected/int4.out Tue Mar 14 18:06:56 2000
--- ./results/int4.out Sat Jan 6 17:56:30 2001
***************
*** 14,20 ****
INSERT INTO INT4_TBL(f1) VALUES ('-2147483647');
-- bad input values -- should give warnings
INSERT INTO INT4_TBL(f1) VALUES ('1000000000000');
! ERROR: pg_atoi: error reading "1000000000000": Numerical result out of range
INSERT INTO INT4_TBL(f1) VALUES ('asdf');
ERROR: pg_atoi: error in "asdf": can't parse "asdf"
SELECT '' AS five, INT4_TBL.*;
--- 14,20 ----
INSERT INTO INT4_TBL(f1) VALUES ('-2147483647');
-- bad input values -- should give warnings
INSERT INTO INT4_TBL(f1) VALUES ('1000000000000');
! ERROR: pg_atoi: error reading "1000000000000": Result too large
INSERT INTO INT4_TBL(f1) VALUES ('asdf');
ERROR: pg_atoi: error in "asdf": can't parse "asdf"
SELECT '' AS five, INT4_TBL.*;
======================================================================
*** ./expected/float8.out Mon Mar 20 00:19:10 2000
--- ./results/float8.out Sat Jan 6 17:56:30 2001
***************
*** 241,249 ****
INSERT INTO FLOAT8_TBL(f1) VALUES ('-10e400');
ERROR: Input '-10e400' is out of range for float8
INSERT INTO FLOAT8_TBL(f1) VALUES ('10e-400');
- ERROR: Input '10e-400' is out of range for float8
INSERT INTO FLOAT8_TBL(f1) VALUES ('-10e-400');
- ERROR: Input '-10e-400' is out of range for float8
-- maintain external table consistency across platforms
-- delete all values and reinsert well-behaved ones
DELETE FROM FLOAT8_TBL;
--- 241,247 ----
======================================================================
*** ./expected/geometry.out Tue Sep 12 17:07:16 2000
--- ./results/geometry.out Sat Jan 6 17:56:52 2001
***************
*** 114,120 ****
| (5.1,34.5) | [(1,2),(3,4)] | (3,4)
| (-5,-12) | [(1,2),(3,4)] | (1,2)
| (10,10) | [(1,2),(3,4)] | (3,4)
! | (0,0) | [(0,0),(6,6)] | (-0,0)
| (-10,0) | [(0,0),(6,6)] | (0,0)
| (-3,4) | [(0,0),(6,6)] | (0.5,0.5)
| (5.1,34.5) | [(0,0),(6,6)] | (6,6)
--- 114,120 ----
| (5.1,34.5) | [(1,2),(3,4)] | (3,4)
| (-5,-12) | [(1,2),(3,4)] | (1,2)
| (10,10) | [(1,2),(3,4)] | (3,4)
! | (0,0) | [(0,0),(6,6)] | (0,0)
| (-10,0) | [(0,0),(6,6)] | (0,0)
| (-3,4) | [(0,0),(6,6)] | (0.5,0.5)
| (5.1,34.5) | [(0,0),(6,6)] | (6,6)
***************
*** 150,160 ****
six | box
-----+----------------------------------------------------------------------------
| (2.12132034355964,2.12132034355964),(-2.12132034355964,-2.12132034355964)
! | (71.7106781186548,72.7106781186548),(-69.7106781186548,-68.7106781186548)
! | (4.53553390593274,6.53553390593274),(-2.53553390593274,-0.535533905932738)
! | (3.12132034355964,4.12132034355964),(-1.12132034355964,-0.121320343559643)
| (107.071067811865,207.071067811865),(92.9289321881345,192.928932188135)
! | (170.710678118655,70.7106781186548),(29.2893218813452,-70.7106781186548)
(6 rows)
-- translation
--- 150,160 ----
six | box
-----+----------------------------------------------------------------------------
| (2.12132034355964,2.12132034355964),(-2.12132034355964,-2.12132034355964)
! | (71.7106781186547,72.7106781186547),(-69.7106781186547,-68.7106781186547)
! | (4.53553390593274,6.53553390593274),(-2.53553390593274,-0.535533905932737)
! | (3.12132034355964,4.12132034355964),(-1.12132034355964,-0.121320343559642)
| (107.071067811865,207.071067811865),(92.9289321881345,192.928932188135)
! | (170.710678118655,70.7106781186547),(29.2893218813453,-70.7106781186547)
(6 rows)
-- translation
***************
*** 224,233 ****
twentyfour | rotation
------------+-----------------------------
| (0,0),(0,0)
! | (-0,0),(-20,-20)
! | (-0,2),(-14,0)
| (0,79.2),(-58.8,0)
! | (14,-0),(0,-34)
| (0,40),(0,0)
| (0,0),(0,0)
| (-10,-10),(-30,-30)
--- 224,233 ----
twentyfour | rotation
------------+-----------------------------
| (0,0),(0,0)
! | (0,0),(-20,-20)
! | (0,2),(-14,0)
| (0,79.2),(-58.8,0)
! | (14,0),(0,-34)
| (0,40),(0,0)
| (0,0),(0,0)
| (-10,-10),(-30,-30)
***************
*** 254,264 ****
WHERE (p.f1 <-> point '(0,0)') >= 1;
twenty | rotation
--------+-----------------------------------------------------------------------------------
! | (0,-0),(-0.2,-0.2)
| (-0.1,-0.1),(-0.3,-0.3)
| (-0.25,-0.25),(-0.25,-0.35)
| (-0.3,-0.3),(-0.3,-0.3)
! | (0.08,-0),(0,-0.56)
| (0.12,-0.28),(0.04,-0.84)
| (0.26,-0.7),(0.1,-0.82)
| (0.12,-0.84),(0.12,-0.84)
--- 254,264 ----
WHERE (p.f1 <-> point '(0,0)') >= 1;
twenty | rotation
--------+-----------------------------------------------------------------------------------
! | (0,0),(-0.2,-0.2)
| (-0.1,-0.1),(-0.3,-0.3)
| (-0.25,-0.25),(-0.25,-0.35)
| (-0.3,-0.3),(-0.3,-0.3)
! | (0.08,0),(0,-0.56)
| (0.12,-0.28),(0.04,-0.84)
| (0.26,-0.7),(0.1,-0.82)
| (0.12,-0.84),(0.12,-0.84)
***************
*** 266,272 ****
| (0.0976764836465887,-0.0241724631246608),(0.0325588278821962,-0.0725173893739825)
| (0.109762715208919,-0.0562379754328844),(0.0813970697054906,-0.0604311578116521)
| (0.0976764836465887,-0.0725173893739825),(0.0976764836465887,-0.0725173893739825)
! | (-0,0.0828402366863905),(-0.201183431952663,0)
| (-0.100591715976331,0.124260355029586),(-0.301775147928994,0.0414201183431953)
| (-0.251479289940828,0.103550295857988),(-0.322485207100592,0.0739644970414201)
| (-0.301775147928994,0.124260355029586),(-0.301775147928994,0.124260355029586)
--- 266,272 ----
| (0.0976764836465887,-0.0241724631246608),(0.0325588278821962,-0.0725173893739825)
| (0.109762715208919,-0.0562379754328844),(0.0813970697054906,-0.0604311578116521)
| (0.0976764836465887,-0.0725173893739825),(0.0976764836465887,-0.0725173893739825)
! | (0,0.0828402366863905),(-0.201183431952663,0)
| (-0.100591715976331,0.124260355029586),(-0.301775147928994,0.0414201183431953)
| (-0.251479289940828,0.103550295857988),(-0.322485207100592,0.0739644970414201)
| (-0.301775147928994,0.124260355029586),(-0.301775147928994,0.124260355029586)
***************
*** 445,452 ****
-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| ((-3,0),(-2.59807621135076,1.50000000000442),(-1.49999999999116,2.59807621135842),(1.53102359078377e-11,3),(1.50000000001768,2.59807621134311),(2.59807621136607,1.4999999999779),(3,-3.06204718156754e-11),(2.59807621133545,-1.50000000003094),(1.49999999996464,-2.59807621137373),(-4.59307077235131e-11,-3),(-1.5000000000442,-2.5980762113278),(-2.59807621138138,-1.49999999995138))
| ((-99,2),(-85.6025403783588,52.0000000001473),(-48.9999999997054,88.602540378614),(1.00000000051034,102),(51.0000000005893,88.6025403781036),(87.6025403788692,51.9999999992634),(101,1.99999999897932),(87.6025403778485,-48.0000000010313),(50.9999999988214,-84.6025403791243),(0.999999998468976,-98),(-49.0000000014732,-84.6025403775933),(-85.6025403793795,-47.9999999983795))
! | ((-4,3),(-3.33012701891794,5.50000000000737),(-1.49999999998527,7.3301270189307),(1.00000000002552,8),(3.50000000002946,7.33012701890518),(5.33012701894346,5.49999999996317),(6,2.99999999994897),(5.33012701889242,0.499999999948437),(3.49999999994107,-1.33012701895622),(0.999999999923449,-2),(-1.50000000007366,-1.33012701887966),(-3.33012701896897,0.500000000081028))
! | ((-2,2),(-1.59807621135076,3.50000000000442),(-0.499999999991161,4.59807621135842),(1.00000000001531,5),(2.50000000001768,4.59807621134311),(3.59807621136607,3.4999999999779),(4,1.99999999996938),(3.59807621133545,0.499999999969062),(2.49999999996464,-0.59807621137373),(0.999999999954069,-1),(-0.500000000044197,-0.598076211327799),(-1.59807621138138,0.500000000048617))
| ((90,200),(91.3397459621641,205.000000000015),(95.0000000000295,208.660254037861),(100.000000000051,210),(105.000000000059,208.66025403781),(108.660254037887,204.999999999926),(110,199.999999999898),(108.660254037785,194.999999999897),(104.999999999882,191.339745962088),(99.9999999998469,190),(94.9999999998527,191.339745962241),(91.3397459620621,195.000000000162))
| ((0,0),(13.3974596216412,50.0000000001473),(50.0000000002946,86.602540378614),(100.00000000051,100),(150.000000000589,86.6025403781036),(186.602540378869,49.9999999992634),(200,-1.02068239385585e-09),(186.602540377848,-50.0000000010313),(149.999999998821,-86.6025403791243),(99.999999998469,-100),(49.9999999985268,-86.6025403775933),(13.3974596206205,-49.9999999983795))
(6 rows)
--- 445,452 ----
-----+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| ((-3,0),(-2.59807621135076,1.50000000000442),(-1.49999999999116,2.59807621135842),(1.53102359078377e-11,3),(1.50000000001768,2.59807621134311),(2.59807621136607,1.4999999999779),(3,-3.06204718156754e-11),(2.59807621133545,-1.50000000003094),(1.49999999996464,-2.59807621137373),(-4.59307077235131e-11,-3),(-1.5000000000442,-2.5980762113278),(-2.59807621138138,-1.49999999995138))
| ((-99,2),(-85.6025403783588,52.0000000001473),(-48.9999999997054,88.602540378614),(1.00000000051034,102),(51.0000000005893,88.6025403781036),(87.6025403788692,51.9999999992634),(101,1.99999999897932),(87.6025403778485,-48.0000000010313),(50.9999999988214,-84.6025403791243),(0.999999998468976,-98),(-49.0000000014732,-84.6025403775933),(-85.6025403793795,-47.9999999983795))
! | ((-4,3),(-3.33012701891794,5.50000000000737),(-1.49999999998527,7.3301270189307),(1.00000000002552,8),(3.50000000002946,7.33012701890518),(5.33012701894346,5.49999999996317),(6,2.99999999994897),(5.33012701889242,0.499999999948437),(3.49999999994107,-1.33012701895622),(0.999999999923449,-2),(-1.50000000007366,-1.33012701887967),(-3.33012701896897,0.500000000081028))
! | ((-2,2),(-1.59807621135076,3.50000000000442),(-0.499999999991161,4.59807621135842),(1.00000000001531,5),(2.50000000001768,4.59807621134311),(3.59807621136607,3.4999999999779),(4,1.99999999996938),(3.59807621133545,0.499999999969062),(2.49999999996464,-0.598076211373729),(0.999999999954069,-1),(-0.500000000044197,-0.598076211327799),(-1.59807621138138,0.500000000048616))
| ((90,200),(91.3397459621641,205.000000000015),(95.0000000000295,208.660254037861),(100.000000000051,210),(105.000000000059,208.66025403781),(108.660254037887,204.999999999926),(110,199.999999999898),(108.660254037785,194.999999999897),(104.999999999882,191.339745962088),(99.9999999998469,190),(94.9999999998527,191.339745962241),(91.3397459620621,195.000000000162))
| ((0,0),(13.3974596216412,50.0000000001473),(50.0000000002946,86.602540378614),(100.00000000051,100),(150.000000000589,86.6025403781036),(186.602540378869,49.9999999992634),(200,-1.02068239385585e-09),(186.602540377848,-50.0000000010313),(149.999999998821,-86.6025403791243),(99.999999998469,-100),(49.9999999985268,-86.6025403775933),(13.3974596206205,-49.9999999983795))
(6 rows)
***************
*** 503,513 ****
WHERE (p1.f1 <-> c1.f1) > 0
ORDER BY distance, circle, point using <<;
twentyfour | circle | point | distance
! ------------+----------------+------------+-------------------
! | <(100,0),100> | (5.1,34.5) | 0.976531926977964
| <(1,2),3> | (-3,4) | 1.47213595499958
| <(0,0),3> | (-3,4) | 2
! | <(100,0),100> | (-3,4) | 3.07764064044151
| <(100,0),100> | (-5,-12) | 5.68348972285122
| <(1,3),5> | (-10,0) | 6.40175425099138
| <(1,3),5> | (10,10) | 6.40175425099138
--- 503,513 ----
WHERE (p1.f1 <-> c1.f1) > 0
ORDER BY distance, circle, point using <<;
twentyfour | circle | point | distance
! ------------+----------------+------------+------------------
! | <(100,0),100> | (5.1,34.5) | 0.97653192697797
| <(1,2),3> | (-3,4) | 1.47213595499958
| <(0,0),3> | (-3,4) | 2
! | <(100,0),100> | (-3,4) | 3.07764064044152
| <(100,0),100> | (-5,-12) | 5.68348972285122
| <(1,3),5> | (-10,0) | 6.40175425099138
| <(1,3),5> | (10,10) | 6.40175425099138
***************
*** 519,525 ****
| <(0,0),3> | (10,10) | 11.142135623731
| <(1,3),5> | (-5,-12) | 11.1554944214035
| <(1,2),3> | (-5,-12) | 12.2315462117278
! | <(1,3),5> | (5.1,34.5) | 26.7657047773224
| <(1,2),3> | (5.1,34.5) | 29.757594539282
| <(0,0),3> | (5.1,34.5) | 31.8749193547455
| <(100,200),10> | (5.1,34.5) | 180.778038568384
--- 519,525 ----
| <(0,0),3> | (10,10) | 11.142135623731
| <(1,3),5> | (-5,-12) | 11.1554944214035
| <(1,2),3> | (-5,-12) | 12.2315462117278
! | <(1,3),5> | (5.1,34.5) | 26.7657047773223
| <(1,2),3> | (5.1,34.5) | 29.757594539282
| <(0,0),3> | (5.1,34.5) | 31.8749193547455
| <(100,200),10> | (5.1,34.5) | 180.778038568384
======================================================================
bpalmer <bpalmer@crimelabs.net> writes:
There are still, however, 4 tests that fail.
Those all look like trivial platform dependencies. Please read
http://www.postgresql.org/devel-corner/docs/postgres/regress.htm
http://www.postgresql.org/devel-corner/docs/postgres/regress-platform.htm
and submit resultmap patches as required for your platform.
regards, tom lane
Those all look like trivial platform dependencies. Please read
http://www.postgresql.org/devel-corner/docs/postgres/regress.htm
http://www.postgresql.org/devel-corner/docs/postgres/regress-platform.htm
and submit resultmap patches as required for your platform.
Sweet. Thanks, looks like the problem is solved. Should me make note of
the changes what were needed for OBSD somewhere?
b. palmer, bpalmer@crimelabs.net
pgp: www.crimelabs.net/bpalmer.pgp5
Attachments:
resultmap.difftext/plain; charset=US-ASCII; name=resultmap.diffDownload
bpalmer@mizer:~/APPS/pgsql/src/test/regress>diff -c resultmap resultmap.orig
*** resultmap Sat Jan 6 19:35:02 2001
--- resultmap.orig Sat Jan 6 19:12:57 2001
***************
*** 7,13 ****
float4/.*-qnx=float4-exp-three-digits
float8/.*-bsdi=float8-small-is-zero
float8/.*-freebsd=float8-small-is-zero
- float8/.*-openbsd=float8-small-is-zero
float8/i.86-.*-netbsd=float8-small-is-zero
float8/.*-qnx=float8-exp-three-digits
float8/alpha.*-dec-osf.*:cc=float8-fp-exception
--- 7,12 ----
***************
*** 16,22 ****
geometry/.*-darwin=geometry-powerpc-darwin
geometry/.*-freebsd=geometry-positive-zeros
geometry/.*-freebsd4=geometry-positive-zeros-bsd
- geometry/.*-openbsd=geometry-positive-zeros-bsd
geometry/.*-irix6=geometry-irix
geometry/.*-netbsd=geometry-positive-zeros
geometry/.*-sysv5uw7.*:cc=geometry-uw7-cc
--- 15,20 ----
***************
*** 45,51 ****
int2/.*-irix6=int2-too-large
int2/.*-netbsd=int2-too-large
int2/.*-qnx=int2-too-large
- int2/.*-openbsd=int2-too-large
int2/alpha.*-dec-osf=int2-too-large
int2/hppa=int2-too-large
int2/i.86-pc-cygwin=int2-math-result-out-of-range
--- 43,48 ----
***************
*** 62,68 ****
int4/.*-irix6=int4-too-large
int4/.*-netbsd=int4-too-large
int4/.*-qnx=int4-too-large
- int4/.*-openbsd=int4-too-large
int4/alpha.*-dec-osf=int4-too-large
int4/hppa=int4-too-large
int4/i.86-pc-cygwin=int4-math-result-out-of-range
--- 59,64 ----
bpalmer <bpalmer@crimelabs.net> writes:
[ resultmap patches for OpenBSD ]
Applied, thanks!
Sweet. Thanks, looks like the problem is solved. Should me make note of
the changes what were needed for OBSD somewhere?
If there's anything that users need to know beyond the standard build
instructions, perhaps we should add a doc/FAQ_OBSD ...
regards, tom lane
Peter Eisentraut <peter_e@gmx.net> writes:
Apparently these resultmap entries were needed at some point.
I think you read the message backwards --- we were adding the entries,
not removing them. (The patch was reversed as given, which is a common
mistake.) As patched, the entries for OpenBSD look pretty much like
those for the other flavors of BSD, which gives me a nice warm fuzzy
feeling about it...
regards, tom lane
Import Notes
Reply to msg id not found: Pine.LNX.4.30.0101070319020.755-100000@peter.localdomainReference msg id not found: Pine.LNX.4.30.0101070319020.755-100000@peter.localdomain | Resolved by subject fallback
bpalmer writes:
Those all look like trivial platform dependencies. Please read
http://www.postgresql.org/devel-corner/docs/postgres/regress.htm
http://www.postgresql.org/devel-corner/docs/postgres/regress-platform.htm
and submit resultmap patches as required for your platform.Sweet. Thanks, looks like the problem is solved. Should me make note of
the changes what were needed for OBSD somewhere?
Apparently these resultmap entries were needed at some point. It wouldn't
surprise me if the error messages changed between releases of the OS --
it's happened to FreeBSD as well. Perhaps they should be qualified by
version.
--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/
A few problems came up with 7.1CVS for i86 OpenBSD. Here is a writeup on
what I learned and how I solved the problems.
- Brandon
b. palmer, bpalmer@crimelabs.net
pgp: www.crimelabs.net/bpalmer.pgp5
Attachments:
bpalmer writes:
A few problems came up with 7.1CVS for i86 OpenBSD. Here is a writeup on
what I learned and how I solved the problems.
Can you check
http://www.postgresql.org/devel-corner/docs/postgres/kernel-resources.htm#SYSVIPC
to see if the description for FreeBSD is also applicable (at least in
similar form) to OpenBSD?
I'm currently writing up a section about the process and file limit issue,
which will end up nearby the section linked to above.
--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/
Can you check
http://www.postgresql.org/devel-corner/docs/postgres/kernel-resources.htm#SYSVIPC
to see if the description for FreeBSD is also applicable (at least in
similar form) to OpenBSD?
Close enough.
I'm currently writing up a section about the process and file limit issue,
which will end up nearby the section linked to above.
I still don't know how to know what the max allowed (by the system) are.
Anyone?
- b
b. palmer, bpalmer@crimelabs.net
pgp: www.crimelabs.net/bpalmer.pgp5