logical decoding documentation?
Where, if anywhere, is the current documentation for writing or using a
logical decoding output plugin consumer thingy?
I'm trying to find my way into it ...
src/backend/replication/logical/logical.c, which textually contains most
of the functions that appear to interact with the test_decoding module,
contains this in the header comment:
"""
The idea is that a consumer provides three callbacks, one to read WAL,
one to prepare a data write, and a final one for actually writing since
their implementation depends on the type of consumer. Check
logicalfunc.c for an example implementations of a fairly simple consumer
and a implementation of a WAL reading callback that's suitable for
simpler consumers.
"""
There is no file logicalfunc.c. And test_decoding actually uses five
callbacks, not three.
Is a consumer the same as a decoder?
test_decoding.c contains this:
/* These must be available to pg_dlsym() */
static void pg_decode_startup(LogicalDecodingContext *ctx,
OutputPluginOptions *opt, bool is_init);
...
which is surely wrong.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
On 2014-03-11 15:57:39 -0400, Peter Eisentraut wrote:
Where, if anywhere, is the current documentation for writing or using a
logical decoding output plugin consumer thingy?
There's a pending patch for it. The corresponding commit is
http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=commit;h=5eeedd55b2d7e53b5fdcdab6a8e74bb666d75bcc
I welcome feedback about it. I've spent a fair bit of time immersed in
this stuff, and I am not really sure anymore what's understandable and
whatnot ;)
It's referencing pg_recvlogical which isn't committed yet (that's the
commit just before), that's why the docs weren't committed with the
feature itself.
src/backend/replication/logical/logical.c, which textually contains most
of the functions that appear to interact with the test_decoding module,
contains this in the header comment:"""
The idea is that a consumer provides three callbacks, one to read WAL,
one to prepare a data write, and a final one for actually writing since
their implementation depends on the type of consumer. Check
logicalfunc.c for an example implementations of a fairly simple consumer
and a implementation of a WAL reading callback that's suitable for
simpler consumers.
"""There is no file logicalfunc.c.
Hrmpf: There's a missing 's', it's logicalfuncs.c.
And test_decoding actually uses five callbacks, not three.
The callbacks logicalfuncs.c header comment is talking about adding a
new method to output data. Currently you can stream out changes via
walsender and via the SQL SRFs. But it might be interesting to
e.g. consume the changes in a bgworker, without going through either SQL
or walsender. To do that you need the three callbacks referenced above.
Is a consumer the same as a decoder?
A consumer is just the recipient of the changestream. I.e the walsender
that streams out the changestream, or the SRF that spills the data into
a tuplestore.
I don't think the term "decoder" is used anywhere, but if it is, it'd
be the output plugin.
test_decoding.c contains this:
/* These must be available to pg_dlsym() */
static void pg_decode_startup(LogicalDecodingContext *ctx,
OutputPluginOptions *opt, bool is_init);
...which is surely wrong.
Hm, these days the comment should be above _PG_init() and
_PG_output_plugin_init(). That changed around several times...
Could you perhaps commit the attached patch fixing the issues you
mentioned?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
minor-logical-decoding-comment-improvements.patchtext/x-patch; charset=us-asciiDownload+7-4
On Tue, Mar 11, 2014 at 4:16 PM, Andres Freund <andres@2ndquadrant.com> wrote:
Could you perhaps commit the attached patch fixing the issues you
mentioned?
I committed this.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers