Token length limit

"Ansley, Michael" <Michael.Ansley@intec.co.za> writes:

Now that the query strings are effectively unlimited in length, the point
that I mentioned in a previous mail about the token length being limited to
16kB becomes an issue. One of the reasons for wanting a large query string
length is to allow people to insert long text strings into a text field.
However, if I understand things right, the token length will limit the text
going into the text field to 16kB. Is this right? Should I have a look at
how to increase the token length arbitrarily?

Yes, and yes. It's not a critical issue as long as we have limited
tuple sizes, but we do need to fix this eventually.

(Actually I think the limit is currently 64k not 16k, because of the
hack that parser/Makefile applies to scan.c, but the point is there
shouldn't be any hardwired limit...)

regards, tom lane

(Actually I think the limit is currently 64k not 16k, because of the
hack that parser/Makefile applies to scan.c, but the point is there
shouldn't be any hardwired limit...)

Yes, I noticed that hack.

Yes, and yes. It's not a critical issue as long as we have limited
tuple sizes, but we do need to fix this eventually.

The theory is, of course, that now the query string lengths are unlimited,
there is good reason to remove the tuple block-size limit. Somebody...
somebody.... anybody.....

Anyway, it has been a good learning experience, and thanks for everybody's
help so far. With some luck, the debugging process will be short and sweet.

MikeA