@(#) Mordred Labs advisory 0x0001: Buffer overflow in PostgreSQL (fwd)

Hi Vince,

Glad he made the advisory for something there's a fix for. :)

Regards and best wishes,

Justin Clift

Vince Vielhaber wrote:

Surprised it took this long.

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net
56K Nationwide Dialup from $16.00/mo at Pop4 Networking
http://www.camping-usa.com http://www.cloudninegifts.com
http://www.meanstreamradio.com http://www.unknown-artists.com
==========================================================================

---------- Forwarded message ----------
Date: Mon, 19 Aug 2002 15:40:28 +0000
From: Sir Mordred The Traitor <mordred@s-mail.com>
To: bugtraq@securityfocus.com
Subject: @(#) Mordred Labs advisory 0x0001: Buffer overflow in PostgreSQL

// @(#) Mordred Labs Advisory 0x0001

Release data: 19/08/02
Name: Buffer overflow in PostgreSQL
Versions affected: <= 7.2
Risk: average

--[ Description:
PostgreSQL is an advanced object-relational database management system
that supports an extended subset of the SQL standard, including
transactions,
foreign keys, subqueries, triggers, user-defined types and functions.

There exists a stack based buffer overflow in cash_words() function, that
potentially allows an attacker to execute malicious code.

--[ How to reproduce:
psql> select cash_words('-700000000000000000000000000000');
pgReadData() -- backend closed the channel unexpectedly.
.... ....
The connection to the server was lost...

--[ Solution:
Upgrade to version 7.2.1.

________________________________________________________________________
This letter has been delivered unencrypted. We'd like to remind you that
the full protection of e-mail correspondence is provided by S-mail
encryption mechanisms if only both, Sender and Recipient use S-mail.
Register at S-mail.com: http://www.s-mail.com/inf/en

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

Justin Clift <justin@postgresql.org> writes:

Glad he made the advisory for something there's a fix for. :)

The claim that this bug allows execution of arbitrary code is bogus anyway.
The overflow at INT_MIN will clobber the stack, yes, but in an absolutely
predetermined way; an attacker will have no opportunity to insert code
of his choosing.

regards, tom lane

Vince,

Do you reckon it's worth you responding to "Sir Mordred" and pointing
out that he overstated the vulnerability?

:-)

Regards and best wishes,

Justin Clift

Tom Lane wrote:

Justin Clift <justin@postgresql.org> writes:

Glad he made the advisory for something there's a fix for. :)

The claim that this bug allows execution of arbitrary code is bogus anyway.
The overflow at INT_MIN will clobber the stack, yes, but in an absolutely
predetermined way; an attacker will have no opportunity to insert code
of his choosing.

regards, tom lane

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

On Tue, 20 Aug 2002, Justin Clift wrote:

Vince,

Do you reckon it's worth you responding to "Sir Mordred" and pointing
out that he overstated the vulnerability?

Not me. Tom (pref) or Marc would be the proper respondent.

:-)

Regards and best wishes,

Justin Clift

Tom Lane wrote:

Justin Clift <justin@postgresql.org> writes:

Glad he made the advisory for something there's a fix for. :)

The claim that this bug allows execution of arbitrary code is bogus anyway.
The overflow at INT_MIN will clobber the stack, yes, but in an absolutely
predetermined way; an attacker will have no opportunity to insert code
of his choosing.

regards, tom lane

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net
56K Nationwide Dialup from $16.00/mo at Pop4 Networking
http://www.camping-usa.com http://www.cloudninegifts.com
http://www.meanstreamradio.com http://www.unknown-artists.com
==========================================================================

On Tue, 20 Aug 2002, Justin Clift wrote:

Vince,

Do you reckon it's worth you responding to "Sir Mordred" and pointing
out that he overstated the vulnerability?

Not me. Tom (pref) or Marc would be the proper respondent.

Has it actually been fixed?

Chris

Christopher Kings-Lynne wrote:

On Tue, 20 Aug 2002, Justin Clift wrote:

Vince,

Do you reckon it's worth you responding to "Sir Mordred" and pointing
out that he overstated the vulnerability?

Not me. Tom (pref) or Marc would be the proper respondent.

Has it actually been fixed?

The TODO list only mentions the cash_out(2) problem, whilst the email
archives mention them both.

From the info still around, this looks to mean that the cash_words()
problem was fixed, but the cash_out() problem was harder to fix.

Tom/Bruce, is that correct?

+ Justin

Chris

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

Vince Vielhaber wrote:

Surprised it took this long.

Yes, me too, and it says the solution is to upgrade to 7.2.1. Nope.

---------- Forwarded message ----------
Date: Mon, 19 Aug 2002 15:40:28 +0000
From: Sir Mordred The Traitor <mordred@s-mail.com>
To: bugtraq@securityfocus.com
Subject: @(#) Mordred Labs advisory 0x0001: Buffer overflow in PostgreSQL

// @(#) Mordred Labs Advisory 0x0001

Release data: 19/08/02
Name: Buffer overflow in PostgreSQL
Versions affected: <= 7.2
Risk: average

--[ Description:
PostgreSQL is an advanced object-relational database management system
that supports an extended subset of the SQL standard, including
transactions,
foreign keys, subqueries, triggers, user-defined types and functions.

There exists a stack based buffer overflow in cash_words() function, that
potentially allows an attacker to execute malicious code.

--[ How to reproduce:
psql> select cash_words('-700000000000000000000000000000');
pgReadData() -- backend closed the channel unexpectedly.
.... ....
The connection to the server was lost...

--[ Solution:
Upgrade to version 7.2.1.

________________________________________________________________________
This letter has been delivered unencrypted. We'd like to remind you that
the full protection of e-mail correspondence is provided by S-mail
encryption mechanisms if only both, Sender and Recipient use S-mail.
Register at S-mail.com: http://www.s-mail.com/inf/en

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:

Has it actually been fixed?

I couldn't reproduce a problem with his example. The buffer size in
cash_words is awfully tight though --- it's dimensioned 128 which looks
like a round number rather than a carefully calculated one, and the
required size is at least 115. I was thinking of bumping it up to 256
just to be sure.

regards, tom lane

Justin Clift wrote:

Hi Vince,

Glad he made the advisory for something there's a fix for. :)

Oh, I see he jumpe on cash_words() and didn't mention cash_out.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Justin Clift wrote:

Christopher Kings-Lynne wrote:

On Tue, 20 Aug 2002, Justin Clift wrote:

Vince,

Do you reckon it's worth you responding to "Sir Mordred" and pointing
out that he overstated the vulnerability?

Not me. Tom (pref) or Marc would be the proper respondent.

Has it actually been fixed?

The TODO list only mentions the cash_out(2) problem, whilst the email
archives mention them both.

From the info still around, this looks to mean that the cash_words()

problem was fixed, but the cash_out() problem was harder to fix.

Tom/Bruce, is that correct?

Looks like cash_words is fixed in current CVS, so I guess in 7.2.1:

Welcome to psql 7.3devel, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit

test=> select cash_words('-700000000000000000000000000000');
cash_words

--------------------------------------------------------------------------------------------------------------------

Minus twenty one million four hundred seventy four thousand eight
hundred thirty six dollars and forty eight cents
(1 row)

Looks like cash_out still bombs:

test=> select cash_out(2);
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Justin Clift <justin@postgresql.org> writes:

From the info still around, this looks to mean that the cash_words()
problem was fixed, but the cash_out() problem was harder to fix.

Tom/Bruce, is that correct?

The cash_out problem can't really be fixed until we do something about
subdividing type "opaque" into multiple pseudo-types with more carefully
defined meanings. cash_out is declared cash_out(opaque) which does not
really mean that it accepts any input type ... but one of the several
meanings of "opaque" is "accepts any type", so the parser doesn't reject
cash_out(2).

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

regards, tom lane

Tom Lane wrote:

Justin Clift <justin@postgresql.org> writes:

From the info still around, this looks to mean that the cash_words()
problem was fixed, but the cash_out() problem was harder to fix.

Tom/Bruce, is that correct?

The cash_out problem can't really be fixed until we do something about
subdividing type "opaque" into multiple pseudo-types with more carefully
defined meanings. cash_out is declared cash_out(opaque) which does not
really mean that it accepts any input type ... but one of the several
meanings of "opaque" is "accepts any type", so the parser doesn't reject
cash_out(2).

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Hang on, you seem to be suggesting we release a major new upgrade, with
major new functionality, knowing it contains a way to trivially crash
the backend.

Err.. hang on. What happened to our reputation for quality and
releasing "when it's ready"?

Since when were we Microsoft-ized?

;-)

Regards and best wishes,

Justin Clift

regards, tom lane

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

Looks like cash_words is fixed in current CVS, so I guess in 7.2.1:

Welcome to psql 7.3devel, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit

test=> select cash_words('-700000000000000000000000000000');

cash_words

------------------------------------------------------------------
--------------------------------------------------

Minus twenty one million four hundred seventy four thousand eight
hundred thirty six dollars and forty eight cents
(1 row)

It doesn't crash - but it sure is giving the wrong answer ;)

Chris

Justin Clift <justin@postgresql.org> writes:

Hang on, you seem to be suggesting we release a major new upgrade, with
major new functionality, knowing it contains a way to trivially crash
the backend.

This particular hole has been in *every* release since Postgres 1.01.

I'm really not interested in responding to any argument that we cannot
release 7.3 until we have fixed everything that could be labeled a DOS
threat. 7.3 already contains a bunch of bug fixes; shall we postpone
releasing those because there are other unfixed bugs?

regards, tom lane

Tom Lane wrote:

Justin Clift <justin@postgresql.org> writes:

Hang on, you seem to be suggesting we release a major new upgrade, with
major new functionality, knowing it contains a way to trivially crash
the backend.

This particular hole has been in *every* release since Postgres 1.01.

How many releases have we known about this and still done a major
release?

I'm really not interested in responding to any argument that we cannot
release 7.3 until we have fixed everything that could be labeled a DOS
threat. 7.3 already contains a bunch of bug fixes; shall we postpone
releasing those because there are other unfixed bugs?

How trivial are they to exploit?

For example, thinking about something like the various ISP's around who
host PostgreSQL databases; how much effort would it take to fix the
vulnerabilities that let someone with remote access, but no ability to
run a "trusted" language, take out the backend?

Regards and best wishes,

Justin Clift

regards, tom lane

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> writes:

test=> select cash_words('-700000000000000000000000000000');
Minus twenty one million four hundred seventy four thousand eight
hundred thirty six dollars and forty eight cents

It doesn't crash - but it sure is giving the wrong answer ;)

Well, yeah, it's only a 32-bit storage value. Overflow per se is not
the issue here. (One reason why I'm not excited about fixing this on an
emergency basis is that I can't believe anyone is using the money type
for any mission-critical applications... it's too limited.)

regards, tom lane

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Hang on, you seem to be suggesting we release a major new upgrade, with
major new functionality, knowing it contains a way to trivially crash
the backend.

Err.. hang on. What happened to our reputation for quality and
releasing "when it's ready"?

Since when were we Microsoft-ized?

I personally agree with Justin that it should be fixed for 7.3 (just imagine
all those people selling colo postgres services). There should be a 7.2.2
as well that fixes the date parser problem.

However, if you let people just run anything they want on your server (eg.
select cash_out(2);) then you're already in a world of pain because they can
quite easily DOS you by doing large, expensive queries, creating 1000
billion row tables, etc., etc.

Chris

On Mon, 2002-08-19 at 23:50, Christopher Kings-Lynne wrote:

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Hang on, you seem to be suggesting we release a major new upgrade, with
major new functionality, knowing it contains a way to trivially crash
the backend.

Err.. hang on. What happened to our reputation for quality and
releasing "when it's ready"?

Since when were we Microsoft-ized?

I personally agree with Justin that it should be fixed for 7.3 (just imagine
all those people selling colo postgres services). There should be a 7.2.2
as well that fixes the date parser problem.

Has anyone actually considered the time required to make the appropriate
fix (clean up use of OPAQUE)? I don't think this bug is worthy of
pushing the 7.3 release out a few weeks.

The simple fix is to drop the money type entirely as it serves few
purposes -- but there are some who use it.

Justin Clift <justin@postgresql.org> writes:

Tom Lane wrote:

This particular hole has been in *every* release since Postgres 1.01.

How many releases have we known about this and still done a major
release?

Several; a quick glance in the archives shows this has been on the TODO
list since 7.0.something.

I really have zero patience for arguments that "problem FOO is so bad
that we should drop everything else until we've fixed it". There are
too many possible candidates for problem FOO, and there are still only
24 hours in the day.

regards, tom lane

Rod Taylor dijo:

The simple fix is to drop the money type entirely as it serves few
purposes -- but there are some who use it.

There are a lot of other functions that cause the same problem. Just
dropping or fixing cash_out is not the solution.

--
Alvaro Herrera (<alvherre[a]atentus.com>)
One man's impedance mismatch is another man's layer of abstraction.
(Lincoln Yeoh)

On Tue, 20 Aug 2002 13:40, Justin Clift wrote:
[snip]

For example, thinking about something like the various ISP's around who
host PostgreSQL databases; how much effort would it take to fix the
vulnerabilities that let someone with remote access, but no ability to
run a "trusted" language, take out the backend?

I believe its been said before, in this forum no less, that PostgreSQL should
focus on its primary role as an RDBMS and not be paranoid about security. I
believe it was the thread on SSL connections, and Tom suggested a simple ssh
tunnel or vpn.

Of course, lets not leave the door wide open, but perhaps the developer's time
would be better spent on features such as schemas and replication.

I know that all of my clients have their databases behind several layers of
firewalls, and taking advantage of a vulnerability such as this remotely is
extremely difficult.

Finally, question the due dilligence process that selects an ISP partner who
would leave a database open to the world, even if they run "unbreakable"
Oracle :)

Cheers

Mark

Hi Mark,

Mark Pritchard wrote:

<snip>

I believe its been said before, in this forum no less, that PostgreSQL should
focus on its primary role as an RDBMS and not be paranoid about security. I
believe it was the thread on SSL connections, and Tom suggested a simple ssh
tunnel or vpn.

Of course, lets not leave the door wide open, but perhaps the developer's time
would be better spent on features such as schemas and replication.

What you're saying has a lot of merit, these are important features.
However, part of our reputation is as being a higher-end, more powerful,
more mature, database offering than other solutions around.

As Tom pointed out, we've already had several releases without getting
rid of this OPAQUE DOS bug, and as pointed out he also really doesn't
have the patience for fixing it (yet).

Was hoping we could get serious bugs fixed sooner rather than later,
otherwise I'm afraid of us just adding more bugs over time until we
start to affect our "having come good" reputation.

I know that all of my clients have their databases behind several layers of
firewalls, and taking advantage of a vulnerability such as this remotely is
extremely difficult.

Totally agreed. Lots of higher end or corporate places are, and that's
not a bad thing in any way.

For your clients, this particular bugfix doesn't sounds like its
necessary.

Finally, question the due dilligence process that selects an ISP partner who
would leave a database open to the world, even if they run "unbreakable"
Oracle :)

This is the one point of yours I don't feel you've quite got down pat.
Err... *why* is it safe to leave a HTTP, SSH, IMAP, etc server open to
the world (configured correctly of course), but somehow fundamentally
wrong to leave a database server open to the world. If we've got a
product without these bugs, there wouldnt be a security vulnerability
would there?

The schema side of things could be *really* useful for ISPs' for
example, yet a large number of them probably wouldn't be so comfortable
having PostgreSQL available for their clients whilst knowing that these
kinds of DOS bugs exists. Mentally, it's not a good thing.

Not trying to be a pain here, but instead trying to keep our QA level
up.

:)

Regards and best wishes,

Justin Clift

Cheers

Mark

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

Mark Pritchard <mark@tangent.net.au> writes:

I believe its been said before, in this forum no less, that
PostgreSQL should focus on its primary role as an RDBMS and not be
paranoid about security. I believe it was the thread on SSL
connections, and Tom suggested a simple ssh tunnel or vpn.

I'd say the two issues are pretty different. IMHO, buffer overruns and
similar security problems are just a special class of software bug
(it's interesting to note that most of the buffer overruns have been
found in the less-maintained parts of the system, like the cash type
or contrib/). Therefore, the justification for fixing buffer overruns
(and avoiding them in the first place) is the same as for fixing other
kinds of bugs: it makes the system more reliable.

On the other hand, adding something like SSL tends to make the system
more complex (and therefore *less* reliable). There may or may not be
a pay-off from a user's POV, but it's not the clear win that avoiding
buffer overruns is, IMHO.

Of course, lets not leave the door wide open, but perhaps the
developer's time would be better spent on features such as schemas
and replication.

It's probably worth noting that the "barrier to entry" for fixing
buffer overruns or doing a code audit is much, much lower than for
implementing advanced features like schemas or replication. The main
thing that auditing code requires is time, rather than coding
skill/knowledge.

Cheers,

Neil

--
Neil Conway <neilc@samurai.com> || PGP Key ID: DB3C29FC

As Tom pointed out, we've already had several releases without getting
rid of this OPAQUE DOS bug, and as pointed out he also really doesn't
have the patience for fixing it (yet).

Was hoping we could get serious bugs fixed sooner rather than later,
otherwise I'm afraid of us just adding more bugs over time until we
start to affect our "having come good" reputation.

Getting a buggy rep on BugTraq has got to be a Bad Thing. We already have
enough trouble competing with MySQL mindshare, and they haven't had BugTraq
problems.

Chris

-----Original Message-----
From: Neil Conway [mailto:neilc@samurai.com]
Sent: Monday, August 19, 2002 10:22 PM
To: Mark Pritchard
Cc: Justin Clift; Tom Lane; Christopher Kings-Lynne;
pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] @(#) Mordred Labs advisory 0x0001:
Buffer overflow in

Mark Pritchard <mark@tangent.net.au> writes:

I believe its been said before, in this forum no less, that

PostgreSQL

should focus on its primary role as an RDBMS and not be

paranoid about

security. I believe it was the thread on SSL connections, and Tom
suggested a simple ssh tunnel or vpn.

I'd say the two issues are pretty different. IMHO, buffer
overruns and similar security problems are just a special
class of software bug (it's interesting to note that most of
the buffer overruns have been found in the less-maintained
parts of the system, like the cash type or contrib/).
Therefore, the justification for fixing buffer overruns (and
avoiding them in the first place) is the same as for fixing
other kinds of bugs: it makes the system more reliable.

On the other hand, adding something like SSL tends to make
the system more complex (and therefore *less* reliable).
There may or may not be a pay-off from a user's POV, but it's
not the clear win that avoiding buffer overruns is, IMHO.

Of course, lets not leave the door wide open, but perhaps the
developer's time would be better spent on features such as

schemas and

replication.

It's probably worth noting that the "barrier to entry" for
fixing buffer overruns or doing a code audit is much, much
lower than for implementing advanced features like schemas or
replication. The main thing that auditing code requires is
time, rather than coding skill/knowledge.

Most computer virus problems are caused by buffer overrun. Someone
decided it wasn't very important.

Some computer viruses have caused billions of dollars in damage. Sounds
important to me.

"Please try our database. Someday, we hope to close off all the virus
entry points, but right now, we figure it isn't too important."

Will you trust your multi-million dollar database to someone who says
the above? I think the priorities are upside down. Any *known*
buffer-overrun _must_ be repaired, and as quickly as possible. And
potential overruns should be identified. A grep for memcpy, strcpy,
gets, etc. should hunt down most of them. A known buffer overrun should
fill the designer of a product with abject terror. And I really mean
that, literally. If you *know* of a buffer overrun, and simply decide
not to fix it, that sounds very negligent to me. For a public project
like PostgreSQL, there is probably very little liability for the
programmers, but I am thinking of the damage that can be inflicted upon
potential clients using the database.

"Dann Corbit" <DCorbit@connx.com> writes:

If you *know* of a buffer overrun, and simply decide not to fix it,
that sounds very negligent to me.

*sigh*, no one is doing that, and it is pure negligence on your part
for replying to a thread that you clearly have not read.

Cheers,

Neil

--
Neil Conway <neilc@samurai.com> || PGP Key ID: DB3C29FC

"Dann Corbit" <DCorbit@connx.com> writes:

I read (in some other message) that this buffer overrun problem has been
known for a very, very long time.

No, the problem you're referring to (cash_out() and friends) is *not*
a buffer overrun.

Cheers,

Neil

--
Neil Conway <neilc@samurai.com> || PGP Key ID: DB3C29FC

-----Original Message-----
From: Neil Conway [mailto:neilc@samurai.com]
Sent: Monday, August 19, 2002 10:42 PM
To: Dann Corbit
Cc: Neil Conway; Mark Pritchard; Justin Clift; Tom Lane;
Christopher Kings-Lynne; pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] @(#) Mordred Labs advisory 0x0001:
Buffer overflow in

"Dann Corbit" <DCorbit@connx.com> writes:

If you *know* of a buffer overrun, and simply decide not to fix it,
that sounds very negligent to me.

*sigh*, no one is doing that, and it is pure negligence on
your part for replying to a thread that you clearly have not read.

I read (in some other message) that this buffer overrun problem has been
known for a very, very long time.

To simply decide not to fix it means:
"It's on the todo list"
For generation after generation after generation.

It does not mean that "Someday, we hope to fix this."

What I am saying is that there is nothing that could possibly be more
important than fixing this, except some other known problem that could
also cause billions of dollars worth of damage. Are there any such
problems besides the buffer overrun problems?

-----Original Message-----
From: Neil Conway [mailto:neilc@samurai.com]
Sent: Monday, August 19, 2002 10:48 PM
To: Dann Corbit
Cc: Neil Conway; Mark Pritchard; Justin Clift; Tom Lane;
Christopher Kings-Lynne; pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] @(#) Mordred Labs advisory 0x0001:
Buffer overflow in

"Dann Corbit" <DCorbit@connx.com> writes:

I read (in some other message) that this buffer overrun problem has
been known for a very, very long time.

No, the problem you're referring to (cash_out() and friends)
is *not* a buffer overrun.

I did miss the one message that said it was not a buffer overrun (I just
got back from vacation, sorry).

However, if it *can* crash the server, that sounds pretty important to
me. Another message in this thread seemed to indicate that security was
not a major focus (lagging behind adding new features). I do hope that
is not true.

On Tue, 20 Aug 2002 15:22, Neil Conway wrote:

I'd say the two issues are pretty different. IMHO, buffer overruns and
similar security problems are just a special class of software bug
(it's interesting to note that most of the buffer overruns have been
found in the less-maintained parts of the system, like the cash type
or contrib/). Therefore, the justification for fixing buffer overruns
(and avoiding them in the first place) is the same as for fixing other
kinds of bugs: it makes the system more reliable.

Agreed - different issues, similar argument. They should be fixed, I just
don't think its a sky is falling type problem. Not saying you said I was
(*grin*), just that a competent network administrator has taken steps to
secure the database over and above that expected of the developers.

It's probably worth noting that the "barrier to entry" for fixing
buffer overruns or doing a code audit is much, much lower than for
implementing advanced features like schemas or replication. The main
thing that auditing code requires is time, rather than coding
skill/knowledge.

Definitely, and I wish I had some to spend on Postgres! Time that is :)

As you noted, most of the issues are in contrib - obviously due to the
skills/knowledge of the core team and the strength of the development model.
However, if the quality of programmers in the market is anything to go by, I
don't hold out for the future unless Postgres is rewritten in something that
holds hands as well as Java :)

Cheers

Mark

To simply decide not to fix it means:

<snip>

What I am saying is that there is nothing that could possibly be more
important than fixing this, except some other known problem that could
also cause billions of dollars worth of damage. Are there any such
problems besides the buffer overrun problems?

This is an open source project. If you, or others with similar strong
feelings about what is important to you, would like to submit patches in
those areas I'm sure that they would be looked on favorably.

To simply insist that everyone else have the same priorities on any
topic is a bit unrealistic. However, I'd hope that if there are folks
who look at this particular issue with your PoV they would speak up and
think about helping out. If you didn't state a strong opinion on the
topic then others might never catch on that there is a potential issue,
let alone that they could contribute to a solution...

- Thomas

On Tue, 20 Aug 2002 15:35, Dann Corbit wrote:

Most computer virus problems are caused by buffer overrun. Someone
decided it wasn't very important.

Some computer viruses have caused billions of dollars in damage. Sounds
important to me.

"Please try our database. Someday, we hope to close off all the virus
entry points, but right now, we figure it isn't too important."

This sounds a little hysterical to me...don't happen to have a remotely
accessible database do you? :)

Will you trust your multi-million dollar database to someone who says
the above? I think the priorities are upside down. Any *known*
buffer-overrun _must_ be repaired, and as quickly as possible. And

As always, feedback accepted in diff -c format.

Seriously though, Oracle was unbreakable for what, two days? Software has
bugs. I'm sure there are a stack more in PostgreSQL.

You limit your exposure to bugs/defects/etc through the use of multiple layers
of protection. If you leave your database out in the wild, you deserve to be
hacked.

potential overruns should be identified. A grep for memcpy, strcpy,
gets, etc. should hunt down most of them. A known buffer overrun should
fill the designer of a product with abject terror. And I really mean
that, literally. If you *know* of a buffer overrun, and simply decide

I'd be worried if my IT consultants experienced "abject terror". I much prefer
them to be calm, safe in the knowledge that vulnerabilities such as this will
not cause me any problems, because they had the forethought to plan for
situations like this and limit their exposure.

I worry about two pieces of software - Apache and OpenSSH. I compile from
source, knowing that I can fix the issue (be it the recent issues with either
piece of software) as soon as the fixed source becomes available. I may be in
the minority, but at least I don't experience abject terror too often (well,
unless I let my sister drive my car...but that is another story).

Cheers

Mark

-----Original Message-----
From: Mark Pritchard [mailto:mark@tangent.net.au]
Sent: Monday, August 19, 2002 11:27 PM
To: Dann Corbit; Neil Conway
Cc: Justin Clift; Tom Lane; Christopher Kings-Lynne;
pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] @(#) Mordred Labs advisory 0x0001:
Buffer overflow in

On Tue, 20 Aug 2002 15:35, Dann Corbit wrote:

Most computer virus problems are caused by buffer overrun. Someone
decided it wasn't very important.

Some computer viruses have caused billions of dollars in damage.
Sounds important to me.

"Please try our database. Someday, we hope to close off

all the virus

entry points, but right now, we figure it isn't too important."

This sounds a little hysterical to me...don't happen to have
a remotely
accessible database do you? :)

I tend to be hyperbolic at times.

Will you trust your multi-million dollar database to

someone who says

the above? I think the priorities are upside down. Any *known*
buffer-overrun _must_ be repaired, and as quickly as possible. And

As always, feedback accepted in diff -c format.

Seriously though, Oracle was unbreakable for what, two days?
Software has
bugs. I'm sure there are a stack more in PostgreSQL.

You limit your exposure to bugs/defects/etc through the use
of multiple layers
of protection. If you leave your database out in the wild,
you deserve to be
hacked.

Nobody deserves to be hacked. Security should assume that each link in
the chain is the only way to bar the door. IMO-YMMV.

potential overruns should be identified. A grep for

memcpy, strcpy,

gets, etc. should hunt down most of them. A known buffer overrun
should fill the designer of a product with abject terror. And I
really mean that, literally. If you *know* of a buffer

overrun, and

simply decide

I'd be worried if my IT consultants experienced "abject
terror". I much prefer
them to be calm, safe in the knowledge that vulnerabilities
such as this will
not cause me any problems, because they had the forethought
to plan for
situations like this and limit their exposure.

My comment was meant to emphasize urgency, rather than irrational
behavior. Somewhat hyperbolic, obviously.

On Mon, 19 Aug 2002, Tom Lane wrote:

Justin Clift <justin@postgresql.org> writes:

From the info still around, this looks to mean that the cash_words()
problem was fixed, but the cash_out() problem was harder to fix.

Tom/Bruce, is that correct?

The cash_out problem can't really be fixed until we do something about
subdividing type "opaque" into multiple pseudo-types with more carefully
defined meanings. cash_out is declared cash_out(opaque) which does not
really mean that it accepts any input type ... but one of the several
meanings of "opaque" is "accepts any type", so the parser doesn't reject
cash_out(2).

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Can we trap and just return an error before it goes into the weeds and
put the subdividing opaque fix in later?

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net
56K Nationwide Dialup from $16.00/mo at Pop4 Networking
http://www.camping-usa.com http://www.cloudninegifts.com
http://www.meanstreamradio.com http://www.unknown-artists.com
==========================================================================

On Mon, 19 Aug 2002, Tom Lane wrote:

Justin Clift <justin@postgresql.org> writes:

From the info still around, this looks to mean that the cash_words()
problem was fixed, but the cash_out() problem was harder to fix.

Tom/Bruce, is that correct?

The cash_out problem can't really be fixed until we do something about
subdividing type "opaque" into multiple pseudo-types with more carefully
defined meanings. cash_out is declared cash_out(opaque) which does not
really mean that it accepts any input type ... but one of the several
meanings of "opaque" is "accepts any type", so the parser doesn't reject
cash_out(2).

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Does anyone have an idea about what other functions are affected by this?

As a stop gap measure to remove the *known* DoS issue how about changing the
pg_proc entry to restrict input types, i.e. not cash_out(opaque)? cash_words is
already listed as only taking the money type is cash_out really that different?

On a related topic cash_out() is listed in pg_proc as returning an int4 but
doesn't the code clearly show that is incorrect?

--
Nigel J. Andrews
Director

---
Logictree Systems Limited
Computer Consultants

On August 19, 2002 11:58 pm, Rod Taylor wrote:

On Mon, 2002-08-19 at 23:50, Christopher Kings-Lynne wrote:
The simple fix is to drop the money type entirely as it serves few
purposes -- but there are some who use it.

As the original creator of the type I probably have the most emotional
attachment to it but even I am thinking of dropping its use. I would have
preferred to fix it (more automatic casts and 64 bit storage as well as the
fixes in the current thread) but I seem to be alone so it hardly seems worth
it. I still think that there is some benefit to being able to do integer
arithmetic though. I know that I do a lot of calculations (mostly sums) on
money and going to numeric is going to be a hit. No matter how efficient it
is it can't be as efficient as a cpu register addition.

But maybe I'm wrong and the hit will be negligible.

-- 
D'Arcy J.M. Cain <darcy@{druid|vex}.net>   |  Democracy is three wolves
http://www.druid.net/darcy/                |  and a sheep voting on
+1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.

On Tue, 20 Aug 2002, D'Arcy J.M. Cain wrote:

On August 19, 2002 11:58 pm, Rod Taylor wrote:

On Mon, 2002-08-19 at 23:50, Christopher Kings-Lynne wrote:
The simple fix is to drop the money type entirely as it serves few
purposes -- but there are some who use it.

As the original creator of the type I probably have the most emotional
attachment to it but even I am thinking of dropping its use. I would have
preferred to fix it (more automatic casts and 64 bit storage as well as the
fixes in the current thread) but I seem to be alone so it hardly seems worth
it. I still think that there is some benefit to being able to do integer
arithmetic though. I know that I do a lot of calculations (mostly sums) on
money and going to numeric is going to be a hit. No matter how efficient it
is it can't be as efficient as a cpu register addition.

But maybe I'm wrong and the hit will be negligible.

I used to use the money tag but someone said it was going away in a future
version and to use the numeric type instead. I was under the impression
that it was going to be history long before now - it seems I was told this
back in 6.3.something.

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net
56K Nationwide Dialup from $16.00/mo at Pop4 Networking
http://www.camping-usa.com http://www.cloudninegifts.com
http://www.meanstreamradio.com http://www.unknown-artists.com
==========================================================================

On Mon, Aug 19, 2002 at 22:35:26 -0700,

Most computer virus problems are caused by buffer overrun. Someone
decided it wasn't very important.

I disaggree with this. Most computer viruses that I see are rely on
poorly designed software and poorly trained users to propagate.

Buffer overruns are used by worms and by some tools designed to get unauthorized
access to machines.

Justin Clift wrote:

Hi Mark,

Mark Pritchard wrote:

[...]

Finally, question the due dilligence process that selects an ISP partner who
would leave a database open to the world, even if they run "unbreakable"
Oracle :)

This is the one point of yours I don't feel you've quite got down pat.
Err... *why* is it safe to leave a HTTP, SSH, IMAP, etc server open to
the world (configured correctly of course), but somehow fundamentally
wrong to leave a database server open to the world. If we've got a
product without these bugs, there wouldnt be a security vulnerability
would there?

Because in every system I've seen so far, the application plays a major
role in ensuring the data integrity by implementing at least part of the
business logic. Referential integrity and all the stuff is nice to have
and supports the application developer fighting against concurrency
issues, very hard to solve on the application layer. But the decision if
accountant Victor is permitted to cancel this payment for Hugo is still
up to the application.

If you say your users need direct DB access on the SQL interface level,
I say trash your application because the data it produces isn't worth
the magnetism on the media. It's not that we ugently need to fix such a
bug that can only cause a DOS in case someone finds a way to hack
through the application into the SQL interface. It's that the
application has to be fixed not to allow that, because even if the
server wouldn't be crashable, such a hack would end in a disaster.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

Vince Vielhaber <vev@michvhf.com> writes:

On Mon, 19 Aug 2002, Tom Lane wrote:

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Can we trap and just return an error before it goes into the weeds and
put the subdividing opaque fix in later?

I don't think there's any quick and dirty solution.

One thing we could probably do in a relatively short amount of time,
considering that we already have one pseudo-type in the system, is to
go ahead and invent the "C string" pseudo-type and then change all the
built-in I/O functions to be declared as taking or returning C string
(as appropriate). We couldn't really do strong type checking on this
yet, because we couldn't expect user-defined types' I/O functions to be
declared correctly for awhile yet, but at least it would plug the hole
for built-in types.

What this needs is someone to do the legwork...

regards, tom lane

On Tue, 2002-08-20 at 00:35, Dann Corbit wrote:

Most computer virus problems are caused by buffer overrun. Someone
decided it wasn't very important.

This is true. IMO, it is extremely arrogant to ignore a buffer overrun
and announce it can't be exploited. There is many cases where this
assertion has been proved to be false. The real risk of overrun is not
what you think can be done with it (or lack there of), rather, it's how
someone else might decide to use it in some obscure manner which you can
not currently fathom or foresee.

Will you trust your multi-million dollar database to someone who says
the above? I think the priorities are upside down. Any *known*
buffer-overrun _must_ be repaired, and as quickly as possible. And

I agree with that too. When setting priorities, using a scale of 1-10,
10 being highest priority, anything that effects stability or
reliability should always be a 10. As such, they should always be
repaired even above new wiz-bang features.

IMO, if you don't embrace that rule of thumb, a developer shouldn't be
working on a project where stability and reliability are key components
of the end product.

potential overruns should be identified. A grep for memcpy, strcpy,
gets, etc. should hunt down most of them. A known buffer overrun should
fill the designer of a product with abject terror. And I really mean

Agreed. It is horribly irresponsible to thumb a nose at such things in
this day and age.

that, literally. If you *know* of a buffer overrun, and simply decide
not to fix it, that sounds very negligent to me. For a public project
like PostgreSQL, there is probably very little liability for the
programmers, but I am thinking of the damage that can be inflicted upon
potential clients using the database.

Not a question of it sounding negligent. It is negligent.

If quality and stability is not the core developers #1 goal then
expecting people to trust the resulting product is laughable. Please
tell me why anyone should use a database to maintain important data when
quality and stability is not important to the developers of such a
product. It's an oxymoron.

Time and time again I've read what basically amounts to, "...if someone
can crash it it's because someone is stupid enough to allow someone to
be able to do it in the first place..." Maybe you're right. After all,
if core developers continue to turn a blind eye to such issues, they are
in fact, the facilitators of allowing people to do it to begin with.
That is, they are the ones that allowing people to do it in the first
place. In short, every time I see that response, I immediately think,
"...now that's the pot calling the kettle black."

At some point in time, you have to stand and say, "the buck stops here."

Now then, after that long rant and rave, since these are known issues, I
don't have a problem with the next release going out as planned. Once
it does go out, I would certainly hope that the developers would
readjust their priorities and target a bug fix release to immediately
follow.

You know, I've seen many people trash Oracle's "unbreakable" mantra.
I'm sure no one is confused at the fact that it is nothing but a
marketing ploy, however, perhaps there is a culture behind it. Perhaps
this is their way of saying stability and reliability is very important
to them. Perhaps their mantra is the rule of thumb outlined above.

Sign,

Greg Copeland

Dann Corbit wrote:

[...]

What I am saying is that there is nothing that could possibly be more
important than fixing this, except some other known problem that could
also cause billions of dollars worth of damage. Are there any such
problems besides the buffer overrun problems?

And what others tried to tell you is, that there are different types of
systems and levels of vulnerability. A software that by nature needs to
be exposed to the internet (like an SMTP, HTTP or SSH server) is in high
danger and needs to be fixed immediately. But software that by nature
needs to be well protected from uncontrolled access (like a database, a
backup management system or a logical volume manager) does not.

The matter of the fact is, that if you grant someone access to your
database that gives him the power to execute the statement that triggers
this bug, you're lost anyway. Whatever constraints you have set up, an
empty database is usually very consistent but not neccessarily useful.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

On Tue, 2002-08-20 at 08:05, Jan Wieck wrote:

If you say your users need direct DB access on the SQL interface level,
I say trash your application because the data it produces isn't worth
the magnetism on the media. It's not that we ugently need to fix such a
bug that can only cause a DOS in case someone finds a way to hack
through the application into the SQL interface. It's that the
application has to be fixed not to allow that, because even if the
server wouldn't be crashable, such a hack would end in a disaster.

The problem is, the vast majority of these issues are actually caused by
internal resources (people) rather than external "attacks".

The real fear is internal resources DoSing internal resources. The
reaction on the list is usually to look outward for sources of possible
problems. That in it self is a problem. It's well documented the vast
majority (what, 70+%) of these issues actually originate internally.

It is because of these issues that it is often, no longer an issue of
"application this or that", as an application may of been completely
bypassed.

And yes, you can argue all day long that if this happens, you should be
fearing destruction of your data. While that's true, data can be
restored. It also requires a different personality type. Many people
would be willing to DoS a system, however, that doesn't have to mean
they are willing to destroy data.

Regards,

Greg Copeland

"Nigel J. Andrews" <nandrews@investsystems.co.uk> writes:

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Does anyone have an idea about what other functions are affected by this?

As a first approximation, every output function for a built-in
pass-by-reference datatype will show this same behavior. cash_out is
just getting picked on because it was the one mentioned in the first
complaint. For that matter, every input function for any datatype
has the same problem:
regression=# select cash_in(2);
server closed the connection unexpectedly

Let's see ... I count 264 standard pg_proc entries that are declared
with one or more "opaque" parameters. Many but by no means all are I/O
functions. There are another 13 standard functions declared to return
"opaque". To plug the hole in a credible fashion we'd need to do
something about every one of these; so belay that last suggestion that
just implementing a "C string" pseudo-type would be enough to be
meaningful.

Right offhand it looks like we'd need at least:
"C string" for the I/O functions
"internal type" for index access functions, selectivity, etc
"any array" for array_eq and array_dims
"any type" for count(*) (and not much else!)
"tuple" for the return type of trigger functions
"void" for the return type of things that return void
not sure about encoding conversion functions

The functions handling internal-type stuff are probably things we don't
want the user calling at all. As long as we don't declare any of them
to *return* an internal type, there would be no way to construct a
function call that the parser would accept, so that hole would be
plugged with just one pseudo-type; we don't really need to distinguish
which internal type is meant in any one case. The "any array"
pseudo-type would probably take a special-case kluge in parse_coerce,
but that doesn't seem intolerable.

Anyone see something I missed? Tatsuo, what exactly should the function
signature really be for all those encoding conversion functions you just
added?

regards, tom lane

On Tue, 2002-08-20 at 08:22, D'Arcy J.M. Cain wrote:

On August 19, 2002 11:58 pm, Rod Taylor wrote:

On Mon, 2002-08-19 at 23:50, Christopher Kings-Lynne wrote:
The simple fix is to drop the money type entirely as it serves few
purposes -- but there are some who use it.

As the original creator of the type I probably have the most emotional
attachment to it but even I am thinking of dropping its use. I would have
preferred to fix it (more automatic casts and 64 bit storage as well as the

But maybe I'm wrong and the hit will be negligible.

It wouldn't fix this particular problem anyway :) -- so I'm going to
vote to keep it.

Though for my own work I've created a money domain due to lack of digits
with the type -- not to mention I needed to store hundredths of a cent.

Anyone see something I missed? Tatsuo, what exactly should the function
signature really be for all those encoding conversion functions you just
added?

test=# \df iso8859_1_to_utf8
List of functions
Result data type | Schema | Name | Argument data types
------------------+------------+-------------------+---------------------------------
integer | pg_catalog | iso8859_1_to_utf8 | integer, integer, -, -, integer
(1 row)
--
Tatsuo Ishii

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

Anyone see something I missed? Tatsuo, what exactly should the function
signature really be for all those encoding conversion functions you just
added?

test=# \df iso8859_1_to_utf8
List of functions
Result data type | Schema | Name | Argument data types
------------------+------------+-------------------+---------------------------------
integer | pg_catalog | iso8859_1_to_utf8 | integer, integer, -, -, integer

Right, that's what they are now, but what do the "-" entries really
mean? Also, are the "integer" args and result truthful, or do they
really mean something else?

regards, tom lane

On Tue, 20 Aug 2002, Tom Lane wrote:

"Nigel J. Andrews" <nandrews@investsystems.co.uk> writes:

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Does anyone have an idea about what other functions are affected by this?

As a first approximation, every output function for a built-in
pass-by-reference datatype will show this same behavior. cash_out is
just getting picked on because it was the one mentioned in the first
complaint. For that matter, every input function for any datatype
has the same problem:
regression=# select cash_in(2);
server closed the connection unexpectedly

...

But going back to the idea that it seems that the only problem being publicised
in the 'outside world' is the cash_out(2) version can we not do the restriction
on acceptable input type in order to claim that the fix?

Obviously this is only a marketing ploy but on the basis that a real fix seems
unlikely before beta in 11 days time (I'm still trying to work out what Tom's
suggestion is) perhaps one worth implementing.

--
Nigel J. Andrews
Director

---
Logictree Systems Limited
Computer Consultants

"Nigel J. Andrews" <nandrews@investsystems.co.uk> writes:

But going back to the idea that it seems that the only problem being
publicised in the 'outside world' is the cash_out(2) version can we
not do the restriction on acceptable input type in order to claim that
the fix?

Totally pointless IMHO, when the same problem exists in hundreds of
other functions. Also, there really is no way to patch cash_out per se;
the problem is a system-level problem, namely failure to enforce type
checking. cash_out has no way to know that what it's been passed is the
wrong kind of datum.

Basically, we've used "opaque" as a substitute for accurate type
declarations; that's got to stop.

regards, tom lane

Tom Lane wrote:

"Nigel J. Andrews" <nandrews@investsystems.co.uk> writes:

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Does anyone have an idea about what other functions are affected by this?

As a first approximation, every output function for a built-in
pass-by-reference datatype will show this same behavior. cash_out is
just getting picked on because it was the one mentioned in the first
complaint. For that matter, every input function for any datatype
has the same problem:
regression=# select cash_in(2);
server closed the connection unexpectedly

Let's see ... I count 264 standard pg_proc entries that are declared
with one or more "opaque" parameters. Many but by no means all are I/O
functions. There are another 13 standard functions declared to return
"opaque". To plug the hole in a credible fashion we'd need to do
something about every one of these; so belay that last suggestion that
just implementing a "C string" pseudo-type would be enough to be
meaningful.

Is there ever a reason for a user to call a function with an opaque
parameter directly? If not, can we simply REVOKE EXECUTE for these
functions?

Joe

On Tue, Aug 20, 2002 at 11:28:32AM -0400, Tom Lane wrote:

"Nigel J. Andrews" <nandrews@investsystems.co.uk> writes:

But going back to the idea that it seems that the only problem being
publicised in the 'outside world' is the cash_out(2) version can we
not do the restriction on acceptable input type in order to claim that
the fix?

Totally pointless IMHO, when the same problem exists in hundreds of
other functions. Also, there really is no way to patch cash_out per se;
the problem is a system-level problem, namely failure to enforce type
checking. cash_out has no way to know that what it's been passed is the
wrong kind of datum.

Basically, we've used "opaque" as a substitute for accurate type
declarations; that's got to stop.

Hmm, are there _any_ cases where it's appropriate to call an 'opaque'
function directly from user code? cash_out() and it's kin are type
output functions that are called under controlled conditions, with
backend controlled parameters. Trigger functions also are called with
backend controlled parameters. Is there a 'hack' fix that doesn't allow
opaque returning functions in user-defined locations?

Ross

Joe Conway <mail@joeconway.com> writes:

Is there ever a reason for a user to call a function with an opaque
parameter directly? If not, can we simply REVOKE EXECUTE for these
functions?

Not sure, but that doesn't solve the problem for array_eq and array_dims
in any case.

Good thought though ...

regards, tom lane

On Tuesday 20 August 2002 11:28 am, Tom Lane wrote:

"Nigel J. Andrews" <nandrews@investsystems.co.uk> writes:

But going back to the idea that it seems that the only problem being
publicised in the 'outside world' is the cash_out(2) version can we
not do the restriction on acceptable input type in order to claim that
the fix?

Basically, we've used "opaque" as a substitute for accurate type
declarations; that's got to stop.

Umm, but what about the reply buffer overrun advisory? I've read this whole
thread, and the reply advisory (AFAICT, unless I've just hit delete too
quickly) has NOT been addressed. Let's repeat the salient portion of Florian
Weimer's message:

PostgreSQL 7.2.1 has a buffer overflow bug in the date parser (which
is invoked each time a string is converted to a datetime object). If
a frontend does not perform proper date checking and rejects overlong
date strings, a buffer is overwritten by parser. The string has to
pass some checks of the parser, so it is not immediately obvious that
this can be exploited. Denial of service is possible, though,
especially if the frontend does not automatically reestablish the
database connection. (All connections are affected, not just the one
that is issueing the query.)

In the DATE parser, not money. Not cash_out. Where do we stand on the DATE
overrun, if one actually exists? If it exists, can it be exploited by a bad
date entry on someone's web form or other client application? If it's not
exploitable, then it lessens the impact -- but it is irresponsible to assume
that just because we can't find a way to exploit it that no one else can.

Now it's possible I just hit delete too quickly; but it's also possible that
everyone has just assumed that this was the cash_out problem and has started
hashing on that. Although this reply advisory hasn't been out as long as the
original one, which WAS cash_words.

If there is indeed an exploitable overrun in the DATE parser, then I believe
we should issue a 7.2.2 to fix this problem. I know that MANY people will be
using 7.2.x for quite awhile, as they won't want to make a MAJOR database
upgrade (which is not painless thanks to the need to use 7.3's pg_dump for
things). If the upgrade was painless, I'd agree that 7.3 is the solution --
but a real security fix shouldn't wait for 7.3. But I'm holding judgment on
a proven exploit. A proven exploit will change my mind to say 'we need a
7.2.2 NOW that fixes this overrun.'
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar.owen@wgcr.org> writes:

Umm, but what about the reply buffer overrun advisory? I've read this whole
thread, and the reply advisory (AFAICT, unless I've just hit delete too
quickly) has NOT been addressed.

Yes it has. CVS logs show

2002-08-04 02:44 thomas

* src/backend/utils/adt/: date.c, datetime.c, format_type.c,
nabstime.c, timestamp.c, varlena.c: Add guard code to protect from
buffer overruns on long date/time input strings. [other
comments pruned, but note this commit did a lot of other stuff too]

The original argument was about whether we should push out a 7.2.2
release just because of this fix. AFAIK no one has even troubled to
look at the patch and see whether it applies directly to the 7.2 branch;
Thomas has revised the date/time code quite a bit since 7.2, so I'd
expect that it's not going to apply exactly.

I'd put more stock in the concern level of the people making complaints
if anyone had bothered to do even that much legwork. Without an offered
patch against 7.2 branch, I don't think the folks who push out releases
(which is not me, but Marc, Bruce, you, Trond, etc) should bother to
take notice of the complaints at all.

regards, tom lane

On Tuesday 20 August 2002 12:15 pm, Tom Lane wrote:

Lamar Owen <lamar.owen@wgcr.org> writes:

Umm, but what about the reply buffer overrun advisory? I've read this
whole thread, and the reply advisory (AFAICT, unless I've just hit delete
too quickly) has NOT been addressed.

Yes it has. CVS logs show

I'd put more stock in the concern level of the people making complaints
if anyone had bothered to do even that much legwork. Without an offered
patch against 7.2 branch, I don't think the folks who push out releases
(which is not me, but Marc, Bruce, you, Trond, etc) should bother to
take notice of the complaints at all.

If a patch is proffered to 7.2.1 to fix this, I'll be happy to roll a new
RPMset. I tend to agree with you on this detail, Tom.

I had just apparently missed that portion; thanks for the reminder.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

The cash_out problem can't really be fixed until we do something about
subdividing type "opaque" into multiple pseudo-types with more carefully
defined meanings. cash_out is declared cash_out(opaque) which does not
really mean that it accepts any input type ... but one of the several
meanings of "opaque" is "accepts any type", so the parser
doesn't reject cash_out(2).

Would it be possible to update the system tables, so that cash_out does not take
opaque but really takes type money ?
I mean the first thing cash_out does is PG_GETARG_CASH(0), so it really only copes
with a money type.

I know the problem is that the cat chases its tail here, because of what comes first,
the type or the io functions. But couldn't this be overcome, at least for internal types ?

Andreas

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

Would it be possible to update the system tables, so that cash_out does not take
opaque but really takes type money ?

That is part of the solution, but only part: we have hundreds of
functions that take "opaque" because we don't currently have any way
to declare what they really take. (In particular, all the typinput
functions are like that --- so fixing typoutput functions isn't plugging
even half of the gap.)

See my proposal to make "opaque" obsolete.

regards, tom lane

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

Would it be possible to update the system tables, so that cash_out does not take
opaque but really takes type money ?

That is part of the solution, but only part: we have hundreds of
functions that take "opaque" because we don't currently have any way
to declare what they really take.

So the idea is, that input functions take cstring type, and output functions
take the explicit type they are designed for ?
Is there anything that can be done vs that types only exist after the functions ?
e.g. create it in one tx with constraints deferred ? Or is that no issue ?

(In particular, all the typinput
functions are like that --- so fixing typoutput functions isn't plugging
even half of the gap.)

See my proposal to make "opaque" obsolete.

Yes, it is great that you will look into this !!
About the names, would it be good to use SQL99 reserved words ? (e.g. ROW for tuple)
nice url: http://developer.mimer.se/validator/sql-reserved-words.tml

count(*) --> anynumeric :-) (two flies with one strike)
NULL/NONNULL --> null|nullvalue|anynull ? We only need this internally, no ?

Hard to say what is good for those names imho, don't like "anytype" :-(
(maybe even leave that opaque for now)

I like "cstring", "void" and "internal".
Maybe "anyarray" instead of "anyarraytype".
And I would prefer "row" instead of "tuple".

Andreas

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

Hard to say what is good for those names imho, don't like "anytype" :-(

How about "any"? It's a reserved word per SQL99, I think.

I like "cstring", "void" and "internal".

Okay.

Maybe "anyarray" instead of "anyarraytype".

That would match with "any".

And I would prefer "row" instead of "tuple".

I'm leaning towards agreeing with Stephan: we should use typename
"trigger" to declare triggers. "Tuple" (or "row") is strictly correct
only for BEFORE triggers, not AFTER triggers, so it's a bit of a
misnomer for triggers anyhow.

I'm now also toying with inventing a pseudotype just for procedural
language handlers, which are currently "foo() returns opaque". If we
want the type system to catch misuses of trigger functions, we should
want it for handlers too. Maybe name this type "language_handler"?
(I had thought we could declare handlers to return "internal", but we
can't do that without breaking type safety. We don't want *any* way
for an SQL construct to look like it returns type "internal".)

regards, tom lane

Nigel J. Andrews wrote:

On Tue, 20 Aug 2002, Tom Lane wrote:

"Nigel J. Andrews" <nandrews@investsystems.co.uk> writes:

I'd like to see something done about this fairly soon, but it's not
happening for 7.3 ...

Does anyone have an idea about what other functions are affected by this?

As a first approximation, every output function for a built-in
pass-by-reference datatype will show this same behavior. cash_out is
just getting picked on because it was the one mentioned in the first
complaint. For that matter, every input function for any datatype
has the same problem:
regression=# select cash_in(2);
server closed the connection unexpectedly

...

But going back to the idea that it seems that the only problem being publicized
in the 'outside world' is the cash_out(2) version can we not do the restriction
on acceptable input type in order to claim that the fix?

Obviously this is only a marketing ploy but on the basis that a real fix seems
unlikely before beta in 11 days time (I'm still trying to work out what Tom's
suggestion is) perhaps one worth implementing.

If we wanted to hide the vulnerability, I wouldn't have put it on the
TODO list. One of our styles is not to deceive people; if we have a
problem, we document it so others know about it and so someday someone
will fix it --- today may be that day. ;-)

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

Hard to say what is good for those names imho, don't like

"anytype" :-(

How about "any"? It's a reserved word per SQL99, I think.

I would actually stick to opaque in that case, already used in other db's.

I like "cstring", "void" and "internal".

Okay.

Maybe "anyarray" instead of "anyarraytype".

That would match with "any".

I thought you wanted it separate ?

And I would prefer "row" instead of "tuple".

I'm leaning towards agreeing with Stephan: we should use typename
"trigger" to declare triggers. "Tuple" (or "row") is strictly correct
only for BEFORE triggers, not AFTER triggers, so it's a bit of a
misnomer for triggers anyhow.

Convinced.

I'm now also toying with inventing a pseudotype just for procedural
language handlers, which are currently "foo() returns opaque". If we
want the type system to catch misuses of trigger functions, we should
want it for handlers too. Maybe name this type "language_handler"?

"HANDLER" would again already be a reserved word, sounds good.

Andreas

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

Hard to say what is good for those names imho, don't like
"anytype" :-(

How about "any"? It's a reserved word per SQL99, I think.

I would actually stick to opaque in that case, already used in other db's.

I want to change the name because (a) we are changing the semantics,
(b) we can't throw notices for opaque if we keep it as a valid choice.

Maybe "anyarray" instead of "anyarraytype".

That would match with "any".

I thought you wanted it separate ?

I meant that if the one name is "any", then making the other "anyarray"
(ie, both without "type" on the end) is consistent.

regards, tom lane

Hard to say what is good for those names imho, don't like
"anytype" :-(

How about "any"? It's a reserved word per SQL99, I think.

I would actually stick to opaque in that case, already used in other db's.

I want to change the name because (a) we are changing the semantics,
(b) we can't throw notices for opaque if we keep it as a valid choice.

Hmm, "any" would sound like it is the same as opaque. Would "any" really be
all allowed types ? I think we would want to eliminate that altogether.
If it is not all types I would rather use a more restrictive name (nulltype
/ anynumeric).

Imho opaque is missing a runtime type info, like a descriptor,
and thus only "pass by value" could not be allowed anymore.

I guess I must sleep over this, not convinced about depricating opaque yet :-)

I meant that if the one name is "any", then making the other "anyarray"
(ie, both without "type" on the end) is consistent.

Ah, good.

Andreas

On Tue, 2002-08-20 at 16:46, Zeugswetter Andreas SB SD wrote:

Hard to say what is good for those names imho, don't like
"anytype" :-(

How about "any"? It's a reserved word per SQL99, I think.

I would actually stick to opaque in that case, already used in other db's.

I want to change the name because (a) we are changing the semantics,
(b) we can't throw notices for opaque if we keep it as a valid choice.

Hmm, "any" would sound like it is the same as opaque. Would "any" really be
all allowed types ? I think we would want to eliminate that altogether.

Erm.. count(*) <- * is literally anything.

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

Hmm, "any" would sound like it is the same as opaque. Would "any" really be
all allowed types ? I think we would want to eliminate that altogether.

Do you plan on eliminating the COUNT() aggregate, then?

Imho opaque is missing a runtime type info, like a descriptor,
and thus only "pass by value" could not be allowed anymore.

AFAICS it's only useful for functions that only care whether their
argument is NULL or not, and don't inspect its value. But that just
happens to describe COUNT, as well as nullvalue/nonnullvalue.

I don't really think that using ANY instead of OPAQUE for this purpose
will affect users, because they will never be declaring any functions
that would legitimately take ANY, much less return ANY (the latter
probably makes no sense at all). It seems to me that COUNT, nullvalue,
and nonnullvalue pretty much cover the spectrum of what you can usefully
do with only an isnull bit to look at...

regards, tom lane

On Tue, 2002-08-20 at 17:15, Tom Lane wrote:

Yes it has. CVS logs show

2002-08-04 02:44 thomas

* src/backend/utils/adt/: date.c, datetime.c, format_type.c,
nabstime.c, timestamp.c, varlena.c: Add guard code to protect from
buffer overruns on long date/time input strings. [other
comments pruned, but note this commit did a lot of other stuff too]

The original argument was about whether we should push out a 7.2.2
release just because of this fix. AFAIK no one has even troubled to
look at the patch and see whether it applies directly to the 7.2 branch;
Thomas has revised the date/time code quite a bit since 7.2, so I'd
expect that it's not going to apply exactly.

It doesn't. I tried, since there's a Debian bug requesting those
patches be applied, but as far as I remember every hunk failed.
I didn't have time to try to make it fit.

--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight, UK
http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C
========================================
"But I would not have you to be ignorant, brethren,
concerning them which are asleep, that ye sorrow not,
even as others which have no hope. For if we believe
that Jesus died and rose again, even so them also
which sleep in Jesus will God bring with him."
I Thessalonians 4:13,14

On Tue, 20 Aug 2002 23:48, Greg Copeland wrote:

On Tue, 2002-08-20 at 00:35, Dann Corbit wrote:

Most computer virus problems are caused by buffer overrun. Someone
decided it wasn't very important.

This is true. IMO, it is extremely arrogant to ignore a buffer overrun
and announce it can't be exploited. There is many cases where this
assertion has been proved to be false. The real risk of overrun is not

I would certainly hope you are not misquoting me here. I have never stated we
should ignore the bug. I suggested that with proper network level protection,
the bug should not be exploitable.

If you did indeed misquote me, perhaps extreme arrogance in this case is the
attribution of comments to people without basis?

I agree with that too. When setting priorities, using a scale of 1-10,
10 being highest priority, anything that effects stability or
reliability should always be a 10. As such, they should always be
repaired even above new wiz-bang features.

So, cutting through the self-supporting hyperbole, you believe that stability
or reliability is more important than new features. I don't disagree. What I
do disagree with is the "panic" mentality that seems so evident in this type
of post.

If you have set up your production infrastructure in a sensible manner (of
course sensible is open for interpretation), this bug does not affect you.

This does highlight the root cause of our difference in opinions - I don't
feel this is an issue because it doesn't affect me. You may be concerned by
these issues because your infrastructure allows the general 'net using public
to directly access your database. While I disagree with this
configuration...it doesn't make you fundamentely wrong, simply different in
our respective approaches.

IMO, if you don't embrace that rule of thumb, a developer shouldn't be
working on a project where stability and reliability are key components
of the end product.

I wasn't aware that PostgreSQL as an open source collaborative project had any
specific requirements of upon it at all. While stability and reliability are
obvious goals, if you feel the project does not meet your needs you are more
than welcome to try one of the alternatives. MySQL for example :)

Seriously though, its similar to the people who run Linus' kernels. Oh my god
they say, the stable 2.4.x series has VM issues. We can't trust it anymore!

Why are you using that kernel in the first place? Where has Linus said that
this is suitable for production use? Stable does not mean "production ready".

potential overruns should be identified. A grep for memcpy, strcpy,
gets, etc. should hunt down most of them. A known buffer overrun should
fill the designer of a product with abject terror. And I really mean

Agreed. It is horribly irresponsible to thumb a nose at such things in
this day and age.

I'm not going to restate my previous response to this view, but I will ask,
who is affected by the "horribly irresponsible" approach? If it is you, then
why hasn't your due dilligence process audited the code? Perhaps you would
feel more comfortable with one of the closed source/closed development model
organisations? But what bugs lie within the depths of DBs such as SQL Server?
How will you audit those? Or will you just trust the sales guy?

programmers, but I am thinking of the damage that can be inflicted upon
potential clients using the database.

Not a question of it sounding negligent. It is negligent.

If quality and stability is not the core developers #1 goal then
expecting people to trust the resulting product is laughable. Please

Where is an expectation at all? If you want to use PostgreSQL, then use it. If
it doesn't meet your needs, don't use it, or start fixing the issues
yourself. If you can't do it, buy Oracle, or DB2, or whatever else.

Time and time again I've read what basically amounts to, "...if someone
can crash it it's because someone is stupid enough to allow someone to
be able to do it in the first place..." Maybe you're right. After all,
if core developers continue to turn a blind eye to such issues, they are
in fact, the facilitators of allowing people to do it to begin with.

I'm not sure how you make the jump from knowing that an issue exists and
allowing people to exploit it, to the inference that core developers are
turning a blind eye to it. Forgive me if I misquote you Tom, but I don't
think he has ever said "we should not fix this bug", simply that the effort
is significant, and there are other factors to consider.

You know, I've seen many people trash Oracle's "unbreakable" mantra.
I'm sure no one is confused at the fact that it is nothing but a
marketing ploy, however, perhaps there is a culture behind it. Perhaps
this is their way of saying stability and reliability is very important
to them. Perhaps their mantra is the rule of thumb outlined above.

Perhaps we need a pgsql-hackers-heated_discussions list so we can take these
discussions offline? :)

Cheers

Mark

greg@CopelandConsulting.Net (Greg Copeland) wrote

Time and time again I've read what basically amounts to, "...if someone
can crash it it's because someone is stupid enough to allow someone to
be able to do it in the first place..." Maybe you're right. After all,
if core developers continue to turn a blind eye to such issues, they are
in fact, the facilitators of allowing people to do it to begin with.=20
That is, they are the ones that allowing people to do it in the first
place. In short, every time I see that response, I immediately think,
"...now that's the pot calling the kettle black."

At some point in time, you have to stand and say, "the buck stops here."

I agree here, but at the same time you cannot put 100% of the
responsibility on the developers. If you are the dba/sysadmin/whatever/etc
then it is your responsibility. It is up to you to know about potential
problems and have workarounds or whatever it is you need to do. I think
that is one of the things that seperates a "good" admin from a "not-so-
good" one.

Afterall, when your boss calls you into his office monday morning and asks
for a really good explanation for why the db was cracked, I dont think he
is going to accept the excuse "this guy, you dont know him but his name is
tom lane.. well anyway, he didnt fix this bug i knew about so i didnt do
nothing about it because i shouldnt have to because that would be like the
pot calling the kettle black"

That being said, I do agree the developers should give things like this
more priority. But, its open source... so you either live with it or
write your own patch.

test=# \df iso8859_1_to_utf8
List of functions
Result data type | Schema | Name | Argument data types
------------------+------------+-------------------+---------------------------------
integer | pg_catalog | iso8859_1_to_utf8 | integer, integer, -, -, integer

Right, that's what they are now, but what do the "-" entries really
mean? Also, are the "integer" args and result truthful, or do they
really mean something else?

They are like:

* conv_proc(
* INTEGER, -- source encoding id
* INTEGER, -- destination encoding id
* OPAQUE, -- source string (null terminated C string)
* OPAQUE, -- destination string (null terminated C string)
* INTEGER -- source string length

For the second and third argument they are actually treated as:

unsigned char *src = PG_GETARG_CSTRING(2);
unsigned char *dest = PG_GETARG_CSTRING(3);

The first one is an input parameter(source string), and second one is
an output parameter(destination string). The caller of this function
is responsible for allocationg enough memory for destination string.

The returned integer is actually dummy. The function always returns 1.
--
Tatsuo Ishii

Tatsuo Ishii <t-ishii@sra.co.jp> writes:

They are like:

* conv_proc(
* INTEGER, -- source encoding id
* INTEGER, -- destination encoding id
* OPAQUE, -- source string (null terminated C string)
* OPAQUE, -- destination string (null terminated C string)
* INTEGER -- source string length

Got it (shoulda read the documentation before asking ;-))

The returned integer is actually dummy. The function always returns 1.

Yes. I will change these to be declared as

foo(int, int, cstring, cstring, int) returns void

unless anyone has a problem with that?

regards, tom lane

Hmm, "any" would sound like it is the same as opaque. Would "any" really be
all allowed types ? I think we would want to eliminate that

altogether.

Do you plan on eliminating the COUNT() aggregate, then?

Ah, you want it for aggbasetype in pg_aggregate, I did not
see that.

How could we then disallow it's use in other context ? Seems
if there is no restriction, "any" will be exactly as prone to
"wrong use" as opaque was.

May be a plan could be to leave opaque, but throw a notice
when it is used in a create stmt, like:
NOTICE: the use of type OPAQUE should be avoided where possible

Andreas

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

How could we then disallow it's use in other context ? Seems
if there is no restriction, "any" will be exactly as prone to
"wrong use" as opaque was.

Well, you can always shoot yourself in the foot by creating a C function
that misinterprets its input. I'm not here to prevent that. But it
won't be easy to make a crashable function without superuser privileges,
because all the PL languages will reject function definitions that use
type ANY as an argument or result (ditto the other pseudotypes, except
maybe VOID).

May be a plan could be to leave opaque, but throw a notice
when it is used in a create stmt, like:
NOTICE: the use of type OPAQUE should be avoided where possible

Right, that's exactly the plan.

Actually, I think we can tighten the use of OPAQUE quite a bit, too.
The only supported uses will be (a) as an argument or result of a
datatype's I/O function, (b) as the result of a trigger function.
Since I/O functions have to be coded in C anyway, we can disallow
OPAQUE as an argument type of PL functions without losing any backwards
compatibility. Furthermore, *we do not have to treat OPAQUE as meaning
ANY*. It can become a pseudo-type that's not coercion-compatible to
anything else, thus preventing any direct SQL calls of either I/O
functions or triggers that are declared in the old style. There are
still some holes, for example you could do
select old_input_function(old_trigger_function());
and the type system wouldn't complain. But it's a lot more nearly
watertight than before, even for functions declared with OPAQUE.

regards, tom lane

On Tue, 2002-08-20 at 18:40, Mark Pritchard wrote:

On Tue, 20 Aug 2002 23:48, Greg Copeland wrote:

On Tue, 2002-08-20 at 00:35, Dann Corbit wrote:

Most computer virus problems are caused by buffer overrun. Someone
decided it wasn't very important.

This is true. IMO, it is extremely arrogant to ignore a buffer overrun
and announce it can't be exploited. There is many cases where this
assertion has been proved to be false. The real risk of overrun is not

I would certainly hope you are not misquoting me here. I have never stated we
should ignore the bug. I suggested that with proper network level protection,
the bug should not be exploitable.

Don't think I quoted you at all. I don't see quotation marks and am
including Dan's remark. Perhaps everyone isn't talking about you after
all. Continue with the green pills. ;)

do disagree with is the "panic" mentality that seems so evident in this type
of post.

Hmmm. Interesting. I didn't see panic or anything alarmist in my post.

IMO, if you don't embrace that rule of thumb, a developer shouldn't be
working on a project where stability and reliability are key components
of the end product.

I wasn't aware that PostgreSQL as an open source collaborative project had any
specific requirements of upon it at all. While stability and reliability are
obvious goals, if you feel the project does not meet your needs you are more
than welcome to try one of the alternatives. MySQL for example :)

In other words, if someone if verbal about development goals or
objectives they should go elsewhere.

As for specific requirements, I've said nothing which are not inherently
obvious. Without a stable and reliable product, no one is going to use
it for serious work. As such, it's certainly not a leap of faith to
assume that developers without such ideals in mind should be shunned or
properly guided. Why? Obviously (amazed I have to spell this out),
said developer efforts would be counter to the project's objectives
(stated or otherwise implicitly obvious).

You also seem to imply that "open source collaborative project[s]"
should not aim for high goals. I think it's safe to say, we disagree.

I'm not going to restate my previous response to this view, but I will ask,
who is affected by the "horribly irresponsible" approach? If it is you, then
why hasn't your due dilligence process audited the code? Perhaps you would
feel more comfortable with one of the closed source/closed development model
organisations? But what bugs lie within the depths of DBs such as SQL Server?
How will you audit those? Or will you just trust the sales guy?

Hmm. Keep a blind and pretend it doesn't exist. Not a good response.
Since this thread came out of someone's dilligence (coders or
otherwise), I fail to see the significance of your comments. After all,
mucho-peer review is supposed to be one of the perks of using OSS
software. Considering I'm not asserted I'm performance a code audit,
this seems completely unrelated to the topic at hand.

Where is an expectation at all? If you want to use PostgreSQL, then use it. If
it doesn't meet your needs, don't use it, or start fixing the issues
yourself. If you can't do it, buy Oracle, or DB2, or whatever else.

In other words, if someone if verbal about development goals they should
go elsewhere. Furthermore, you seem to assert that only the core
developers should be using this database. Wow, I'm amazed. Just
because someone does or does not contributes code doesn't suddenly
invalidate observations. That, of course, doesn't mean the observations
have to be valid.

Simply stated, if it meets my needs, shut-up and use it. If it doesn't,
go elsewhere. Bad attitude.

I'm not sure how you make the jump from knowing that an issue exists and
allowing people to exploit it, to the inference that core developers are
turning a blind eye to it. Forgive me if I misquote you Tom, but I don't
think he has ever said "we should not fix this bug", simply that the effort
is significant, and there are other factors to consider.

Because it was stated that these are known issues. Because it was
stated these have been known issues for a very long time. Because it
was stated that, more or less, no one is excited about doing the lots of
effort which is seemingly required to put some of these issues to bed.

The expression, "turning a blind eye", means that something is being
ignored. It means it won't be focused on for now. That does not have
to mean forever. The statement is valid and supported by pretty much
every posting on this list on the topic at hand. I stand by my
statement.

Stop with the misquoting comments already. The quotes I used were
properly attributed. Believe it or not, everyone is not talking about
you or trying to place words in your mouth. Believe it or not, the
quotes are correct. I have no idea what you're talking about.

Perhaps we need a pgsql-hackers-heated_discussions list so we can take these
discussions offline? :)

I'm concerned that you feel comments on corporate culture should be
considered a heated discussion. I in no way, shape or form, asserted
anything other then perhaps there is a cultural emphasis behind the
marketing ploy. You can say no and I can say maybe all day long.
Either way, that's hardly heated.

Somewhere around here, we've wondered from the road.

Regards,

Greg Copeland

On Tue, 2002-08-20 at 19:59, ngpg@grymmjack.com wrote:

greg@CopelandConsulting.Net (Greg Copeland) wrote

At some point in time, you have to stand and say, "the buck stops here."

I agree here, but at the same time you cannot put 100% of the
responsibility on the developers. If you are the dba/sysadmin/whatever/etc
then it is your responsibility. It is up to you to know about potential
problems and have workarounds or whatever it is you need to do. I think
that is one of the things that seperates a "good" admin from a "not-so-
good" one.

I absolutely agree. I was not trying to say that bugs don't happen and
that sometimes those bugs may cause crashes. What I was trying to say
is what amounts to, "dr, when I move my arm like this, it hurts", and
the response is, "don't do that." Humor aside, surely there has to be a
happy medium in between. Perhaps, with a skewing toward fixing rather
than prescribing. ;)

Afterall, when your boss calls you into his office monday morning and asks
for a really good explanation for why the db was cracked, I dont think he
is going to accept the excuse "this guy, you dont know him but his name is

I understand and agree with ya.

That being said, I do agree the developers should give things like this
more priority. But, its open source... so you either live with it or
write your own patch.

Well, the priority portion was what I was shooting for. Perhaps it came
off being over zealous. I'm not really sure. I re-read it and I didn't
think so. But, I'm not you and you're not me...so, it's hard to say how
exactly it was received.

As for the open source comment, that fine and all...but...there are
companies which are paying for postgres' development too. Some of the
developers are being paid to do this. The "write your own patch" has
much more meaning on simple projects. For a project as complex as
postgres, simply asking for a patch is almost meaningless. Along those
lines, I have been reading (code and the list) and learning for sometime
now, as time allows. One day, I will contribute significant patches.
However, until that day comes, I would hope that observational
commentary is simply not invalidated just because they're not one with
the code yet.

Regards,

Greg Copeland

May be a plan could be to leave opaque, but throw a notice
when it is used in a create stmt, like:
NOTICE: the use of type OPAQUE should be avoided where possible

Right, that's exactly the plan.

Actually, I think we can tighten the use of OPAQUE quite a bit, too.
The only supported uses will be (a) as an argument or result of a
datatype's I/O function, (b) as the result of a trigger function.

In my paper I use C functions that take "any tuple".
I do not yet see how I can do that without opaque if we don't
have a "row" but only a "trigger" type.

Andreas

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

In my paper I use C functions that take "any tuple".
I do not yet see how I can do that without opaque if we don't
have a "row" but only a "trigger" type.

For that you would have to use "any", at the moment. This would give
you the same amount of type safety you have with "opaque", ie, none.

We could talk about adding more pseudotypes that are geared to
requirements not existing in the standard set of functions ...
but I'm going to concentrate on those first.

regards, tom lane

In my paper I use C functions that take "any tuple".
I do not yet see how I can do that without opaque if we don't
have a "row" but only a "trigger" type.

For that you would have to use "any", at the moment. This would give
you the same amount of type safety you have with "opaque", ie, none.

I would have to use some pg_proc magic to make "any" appear there,
since the plan was to not make it visible at the sql level, no ?
If you do, you would also have to throw the notice for "any".

Again, I think leaving "any" be "opaque" and throwing the warning NOTICE
would be better. I do not think there is enough time to really see through
the implications of restricting opaque already in 7.3, (at least for me :-)

We could talk about adding more pseudotypes that are geared to
requirements not existing in the standard set of functions ...
but I'm going to concentrate on those first.

Yes, certainly a lot of work.

Andreas

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

For that you would have to use "any", at the moment. This would give
you the same amount of type safety you have with "opaque", ie, none.

I would have to use some pg_proc magic to make "any" appear there,
since the plan was to not make it visible at the sql level, no ?

Huh? It'll be perfectly visible.

There is one little problem with calling it ANY, it turns out: that word
is fully reserved in our parser (and trying to make it less reserved
creates reduce/reduce conflicts). So unless we go back to "anytype"
you'd have to quote the name, eg
create function foo("any") returns ...
I do prefer using "any" because that's what we have historically used
in CREATE AGGREGATE, but maybe the keyword conflict will be too
annoying.

regards, tom lane

For that you would have to use "any", at the moment. This would give
you the same amount of type safety you have with "opaque",

ie, none.

I would have to use some pg_proc magic to make "any" appear there,
since the plan was to not make it visible at the sql level, no ?

Huh? It'll be perfectly visible.

I did not mean visible, I meant useable, like in
create function xx(any) returns text ...;

If that is possible, what is the difference to opaque ?

Andreas

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

I did not mean visible, I meant useable, like in
create function xx(any) returns text ...;
If that is possible, what is the difference to opaque ?

"any" will have the same behavior that "opaque" used to have, yes.

regards, tom lane

I did not mean visible, I meant useable, like in
create function xx(any) returns text ...;
If that is possible, what is the difference to opaque ?

"any" will have the same behavior that "opaque" used to have, yes.

Ok, now I vote, that you don't implement "any" and use "opaque".
I don't think we want two types that do the same thing.
Is it that you like the name "any" more than "opaque" ?
I am confused.

Andreas

"Zeugswetter Andreas SB SD" <ZeugswetterA@spardat.at> writes:

Ok, now I vote, that you don't implement "any" and use "opaque".
I don't think we want two types that do the same thing.
Is it that you like the name "any" more than "opaque" ?

No, it's that I want to deprecate "opaque" so that we can catch old
uses that should not be there anymore. If you look at your code and
you decide that "any" is the correct semantics, then fine: change
"opaque" to "any" and the warnings will go away. But relatively few
existing uses of "opaque" really mean "any", and I don't want the
people who are using "opaque" to mean "cstring", "trigger", etc
to keep using "opaque" for those other purposes. The idea here is
to force a security review.

regards, tom lane

Ok, now I vote, that you don't implement "any" and use "opaque".
I don't think we want two types that do the same thing.
Is it that you like the name "any" more than "opaque" ?

No, it's that I want to deprecate "opaque" so that we can catch old
uses that should not be there anymore. If you look at your code and
you decide that "any" is the correct semantics, then fine: change
"opaque" to "any" and the warnings will go away. But relatively few
existing uses of "opaque" really mean "any", and I don't want the
people who are using "opaque" to mean "cstring", "trigger", etc
to keep using "opaque" for those other purposes. The idea here is
to force a security review.

That is what I have been trying to say, imho "any" should have the same
NOTICE as opaque has, since it is potentially dangerous.
I would suggest a warning NOTICE for opaque and not depricate it.

Imho the NOTICE should *not* go away.

If we want "any" in the future, it should imho always be passed a "safe"
Datum that includes type info. This will allow us to create a type "any"
that does not have the pitfalls of opaque.

Andreas