Re: Bit strings

Bruce Momjian wrote:

Can we get the BIT type working now that 7.1 is branched?

Yep. We never quite finished the discussions before the 7.0 release. The
outstanding issues were

1) should bit operations be allowed between bit strings of different sizes?
They are at the moment, but the longest string is truncated, as you
otherwise get illogical behaviour (imagine padding with zeros and doing an
XOR). Truncation can be somewhat unlogical as well. My current feeling is
that perhaps it would be better to just disallow bit operations on
bitstrings that haven't got the same length completely. Makes the code
easier anyway ;-)

2) A second point brought up by Peter is the conversion of bit-strings to
integers and vice-versa. Postgres does not have unsigned ints, so it can
lead to somewhat surprising behaviour. Implicit conversion is probably out
of the question. So what type of conversion functions would people like to
see, if any?

3) With TOAST bit strings could be used to store large amounts of binary
data. Don mentioned MIME-encoded strings for I/O. Is this useful? I guess
and advantage is that BIT is an SQL type, but I'm not sure that any such
application would be particularly portable. It doesn't look like a lot of
extra work though, (provided somebody has got some MIME en-/decoding code
lying about) so we could add it. Am i right that with TOAST and no
restrictions on the query-buffer lengths in psql one could then upload mime
objects straight into postgres?

So, what do people think?

Adriaan

Adriaan Joubert writes:

My current feeling is that perhaps it would be better to just disallow
bit operations on bitstrings that haven't got the same length
completely.

I feel with you. :)

3) With TOAST bit strings could be used to store large amounts of binary
data.

This is not really the proper application for bit strings. There are BLOBs
for that. Which leads to the question, what are bit strings really useful
for other than genuine bit masks?

--
Peter Eisentraut Sernanders v�g 10:115
peter_e@gmx.net 75262 Uppsala
http://yi.org/peter-e/ Sweden

Can someone comment on this?

Bruce Momjian wrote:

Can we get the BIT type working now that 7.1 is branched?

Yep. We never quite finished the discussions before the 7.0 release. The
outstanding issues were

1) should bit operations be allowed between bit strings of different sizes?
They are at the moment, but the longest string is truncated, as you
otherwise get illogical behaviour (imagine padding with zeros and doing an
XOR). Truncation can be somewhat unlogical as well. My current feeling is
that perhaps it would be better to just disallow bit operations on
bitstrings that haven't got the same length completely. Makes the code
easier anyway ;-)

2) A second point brought up by Peter is the conversion of bit-strings to
integers and vice-versa. Postgres does not have unsigned ints, so it can
lead to somewhat surprising behaviour. Implicit conversion is probably out
of the question. So what type of conversion functions would people like to
see, if any?

3) With TOAST bit strings could be used to store large amounts of binary
data. Don mentioned MIME-encoded strings for I/O. Is this useful? I guess
and advantage is that BIT is an SQL type, but I'm not sure that any such
application would be particularly portable. It doesn't look like a lot of
extra work though, (provided somebody has got some MIME en-/decoding code
lying about) so we could add it. Am i right that with TOAST and no
restrictions on the query-buffer lengths in psql one could then upload mime
objects straight into postgres?

So, what do people think?

Adriaan

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Can we get the BIT type working now that 7.1 is branched?

I did some work on the BIT types a couple months ago. According to
my notes, the following issues are still outstanding before they
can be said to work at all:

Bit and hexstring literals are not handled in a reasonable fashion;
the scanner converts them to integer constants which is bogus.
Probably they need to be converted to some generic 'UNKNOWNBITSTRING'
pseudo-type that can later be coerced to a specific bitstring type.
I didn't touch this because it seems to open up the Pandora's box
of unknown-constant handling, for which we do not have a good
general solution.

SQL92 sez we need a position() function for bitstrings.

Need a regression test for bit types.

scalarltsel() and friends need to cope with bit/varbit types in
order to make good use of indexes on bitstrings.

pg_dump does not handle BIT/VARBIT lengths properly (pjw may have
fixed this by now).

regards, tom lane

Any idea where we are on this?

Can we get the BIT type working now that 7.1 is branched?

I did some work on the BIT types a couple months ago. According to
my notes, the following issues are still outstanding before they
can be said to work at all:

Bit and hexstring literals are not handled in a reasonable fashion;
the scanner converts them to integer constants which is bogus.
Probably they need to be converted to some generic 'UNKNOWNBITSTRING'
pseudo-type that can later be coerced to a specific bitstring type.
I didn't touch this because it seems to open up the Pandora's box
of unknown-constant handling, for which we do not have a good
general solution.

SQL92 sez we need a position() function for bitstrings.

Need a regression test for bit types.

scalarltsel() and friends need to cope with bit/varbit types in
order to make good use of indexes on bitstrings.

pg_dump does not handle BIT/VARBIT lengths properly (pjw may have
fixed this by now).

regards, tom lane

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

OK, I will delete my copy of the email. If there is something to be
added to the TODO list, let me know.

[ People are welcome to modify it themselves too.]

Bruce Momjian writes:

Any idea where we are on this?

Bit and hexstring literals are not handled in a reasonable fashion;

Bit string literals are handled correctly, although it occurred to me that
perhaps

#define IsA_Value(t) \
(IsA(t, Integer) || IsA(t, Float) || IsA(t, String))

should be augmented with BitString. Can someone advise?

Hex literals are still not handled correctly.

SQL92 sez we need a position() function for bitstrings.

We have one now.

Need a regression test for bit types.

We have one now.

scalarltsel() and friends need to cope with bit/varbit types in
order to make good use of indexes on bitstrings.

Not done.

pg_dump does not handle BIT/VARBIT lengths properly (pjw may have
fixed this by now).

Works fine for me.

--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Peter Eisentraut <peter_e@gmx.net> writes:

perhaps
#define IsA_Value(t) \
(IsA(t, Integer) || IsA(t, Float) || IsA(t, String))
should be augmented with BitString. Can someone advise?

Yes, definitely --- see the sole use of that macro, in outfuncs.c.
This is a bug.

regards, tom lane

Bruce Momjian writes:

Any idea where we are on this?

Bit and hexstring literals are not handled in a reasonable fashion;

Bit string literals are handled correctly, although it occurred to me that
perhaps

#define IsA_Value(t) \
(IsA(t, Integer) || IsA(t, Float) || IsA(t, String))

should be augmented with BitString. Can someone advise?

Hex literals are still not handled correctly.

SQL92 sez we need a position() function for bitstrings.

We have one now.

Need a regression test for bit types.

We have one now.

scalarltsel() and friends need to cope with bit/varbit types in
order to make good use of indexes on bitstrings.

Not done.

pg_dump does not handle BIT/VARBIT lengths properly (pjw may have
fixed this by now).

Works fine for me.

--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/

Peter Eisentraut <peter_e@gmx.net> writes:

Yes, definitely --- see the sole use of that macro, in outfuncs.c.

Maybe this macro could be removed and expanded in place. That would make
things more transparent. Or is there any other potential use?

No objection from me. The macro clearly failed to make things better in
that usage, and offhand I think it would be just as likely to be a time
bomb in any other usage.

regards, tom lane

Tom Lane writes:

Peter Eisentraut <peter_e@gmx.net> writes:

perhaps
#define IsA_Value(t) \
(IsA(t, Integer) || IsA(t, Float) || IsA(t, String))
should be augmented with BitString. Can someone advise?

Yes, definitely --- see the sole use of that macro, in outfuncs.c.

Maybe this macro could be removed and expanded in place. That would make
things more transparent. Or is there any other potential use?

--
Peter Eisentraut peter_e@gmx.net http://yi.org/peter-e/