Documenting the Frontend/Backend Protocol update criteria
Hi,
Currently the criteria on updating the F/B protocol is undefined. This makes it hard to update the protocol going forward. It makes it also hard to write library/driver/application implementations that will be more future proof to future server versions.
Ideally the documentation for 9.4 (backport?) would say what kind of things are allowed to change within the v3 protocol, and thus implies what kind of changes need a new v4 protocol. Is there some wishlist page of items to do differently for v4 already?
I did find the following text in the documentation: "binary representations for complex data types might change across server versions". But having more specific rules would help, especially since it seems to be there just to scare: so far changes have been strongly discouraged.
An example to consider: some binary formats have flags (arrays) or version (jsonb) field. We should explicitly say that clients must reject any unknown bits/versions that they do not know about. This guarantees they detect small format updates instead of silently accepting then and possibly returning corrupt data.
My motivation:
Two years ago accidentally I opened a discussion on how to do updates to the binary encoding of data in the protocol [1]http://grokbase.com/t/postgresql/pgsql-hackers/11bwhv1esa/add-minor-version-to-v3-protocol-to-allow-changes-without-breaking-backwards-compatibility. I would like to reopen the discussion now since the jsonb 'binary' encoding is just a version '1' + text json. The result from the last discussion was that having a version or flags as part of the binary format is not enough, since drivers/libraries (fixable) and applications (unfixable) are depending on the current encoding.
And if we add a new bit to the flags or bump the version number we will break backward compatibility.
To summarize the previous discussion:
- there are currently no written rules for modifying the binary encoding formats
- bytea modification was done with a GUC, but GUC was seen as a bad solution in general
- my proposal was to add a minor format version number was not good either since any per session state would be problematic for connection poolers
Jan Urbański made a presentation titled 'Postgres on the wire',
subtitle 'A look at the PostgreSQL wire protocol'. I hope this
covers some of your interest. Presentation slide deck is available
at
http://www.pgcon.org/2014/schedule/attachments/330_postgres-for-the-wire.pdf
Hope it helps;
---
Koichi Suzuki
2014-06-02 7:22 GMT+09:00 Mikko Tiihonen <Mikko.Tiihonen@nitorcreations.com>:
Hi,
Currently the criteria on updating the F/B protocol is undefined. This makes
it hard to update the protocol going forward. It makes it also hard to write
library/driver/application implementations that will be more future proof to
future server versions.Ideally the documentation for 9.4 (backport?) would say what kind of things
are allowed to change within the v3 protocol, and thus implies what kind of
changes need a new v4 protocol. Is there some wishlist page of items to do
differently for v4 already?I did find the following text in the documentation: "binary representations
for complex data types might change across server versions". But having more
specific rules would help, especially since it seems to be there just to
scare: so far changes have been strongly discouraged.An example to consider: some binary formats have flags (arrays) or version
(jsonb) field. We should explicitly say that clients must reject any unknown
bits/versions that they do not know about. This guarantees they detect small
format updates instead of silently accepting then and possibly returning
corrupt data.My motivation:
Two years ago accidentally I opened a discussion on how to do updates to the
binary encoding of data in the protocol [1]. I would like to reopen the
discussion now since the jsonb 'binary' encoding is just a version '1' +
text json. The result from the last discussion was that having a version or
flags as part of the binary format is not enough, since drivers/libraries
(fixable) and applications (unfixable) are depending on the current
encoding.
And if we add a new bit to the flags or bump the version number we will
break backward compatibility.To summarize the previous discussion:
- there are currently no written rules for modifying the binary encoding
formats
- bytea modification was done with a GUC, but GUC was seen as a bad solution
in general
- my proposal was to add a minor format version number was not good either
since any per session state would be problematic for connection poolers
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers