Refectoring of receivelog.c
I was working on adding the tar streaming functionality we talked about at
the developer meeting to pg_basebackup, and rapidly ran across the issue
that Andres has been complaining about for a while. The code in
receivelog.c just passes an insane number of parameters around. Adding or
changing even a small thing ends up touching a huge number of places.
Here's an attempt to refactor the code to instead pass around a control
structure. I think it's a definite win already now, and we can't just keep
adding new functionality on top of the current one.
I'll proceed to work on the actual functionality I was working on to go on
top of this separately, but would appreciate a review of this part
independently. It's mostly mechanical, but there may definitely be mistakes
- or thinkos in the whole idea...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Attachments:
basebackup.patchtext/x-patch; charset=US-ASCII; name=basebackup.patchDownload+267-247
On 15 February 2016 at 04:48, Magnus Hagander <magnus@hagander.net> wrote:
I was working on adding the tar streaming functionality we talked about at
the developer meeting to pg_basebackup, and rapidly ran across the issue
that Andres has been complaining about for a while. The code in
receivelog.c just passes an insane number of parameters around. Adding or
changing even a small thing ends up touching a huge number of places.
Other than the lack of comments on the fields in StreamCtl to indicate
their functions I think this looks good.
I didn't find any mistakes, but I do admit my eyes started glazing over
after a bit.
I'd rather not have StreamCtl as a typedef of an anonymous struct if it's
exposed in a header though. It should have a name that can be used in
forward declarations etc.
After recently working with the XLogReader I can't emphasise enough how
important *useful* comments on the fields in these sorts of structs are -
the XLogReaderState has ReadRecPtr, readSegNo + readOff + readLen,
currRecPtr AND latestPagePtr. Which actually do have comments, just not
super helpful ones.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Mon, Feb 15, 2016 at 7:15 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 15 February 2016 at 04:48, Magnus Hagander <magnus@hagander.net> wrote:
I was working on adding the tar streaming functionality we talked about
at the developer meeting to pg_basebackup, and rapidly ran across the issue
that Andres has been complaining about for a while. The code in
receivelog.c just passes an insane number of parameters around. Adding or
changing even a small thing ends up touching a huge number of places.Other than the lack of comments on the fields in StreamCtl to indicate
their functions I think this looks good.
I copied that lack of comments from the previous implementation :P
But yeah, I agree, it's probably a good idea to add them.
I didn't find any mistakes, but I do admit my eyes started glazing over
after a bit.I'd rather not have StreamCtl as a typedef of an anonymous struct if it's
exposed in a header though. It should have a name that can be used in
forward declarations etc.
It's not exactly uncommon with anonymous ones either in our code, but I see
no problem adding that.
Thanks!
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 15 Feb 2016, at 14:46, Magnus Hagander <magnus@hagander.net> wrote:
On Mon, Feb 15, 2016 at 7:15 AM, Craig Ringer <craig@2ndquadrant.com <mailto:craig@2ndquadrant.com>> wrote:
On 15 February 2016 at 04:48, Magnus Hagander <magnus@hagander.net <mailto:magnus@hagander.net>> wrote:
I was working on adding the tar streaming functionality we talked about at the developer meeting to pg_basebackup, and rapidly ran across the issue that Andres has been complaining about for a while. The code in receivelog.c just passes an insane number of parameters around. Adding or changing even a small thing ends up touching a huge number of places.Other than the lack of comments on the fields in StreamCtl to indicate their functions I think this looks good.
I copied that lack of comments from the previous implementation :P
But yeah, I agree, it's probably a good idea to add them.
I didn't find any mistakes, but I do admit my eyes started glazing over after a bit.
I'd rather not have StreamCtl as a typedef of an anonymous struct if it's exposed in a header though. It should have a name that can be used in forward declarations etc.
It's not exactly uncommon with anonymous ones either in our code, but I see no problem adding that.
Short review of this patch:
The patch applies, and builds, cleanly on top of master as of today. No new
functionality is introduced and thus no new tests or doc patches etc are
applicable.
The main point of the patch is to improve readability and reduce the number of
parameters passed, goals which are reached. However, I agree with Craig that
comments on the struct fields should be added to improve readability even
further. The comment on ReceiveXlogStream() also now reads a bit strange since
it describes fields inside the StreamCtl without referencing StreamCtl at all.
For first time readers of the code it could perhaps be helpful with a brief
note that the discussed parameters are in StreamCtl to avoid potential
confusion.
Overall I think this patch is an improvement on the current code and consider
it ready for committer.
cheers ./daniel
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Mar 11, 2016 at 9:40 AM, Daniel Gustafsson <daniel@yesql.se> wrote:
On 15 Feb 2016, at 14:46, Magnus Hagander <magnus@hagander.net> wrote:
On Mon, Feb 15, 2016 at 7:15 AM, Craig Ringer <craig@2ndquadrant.com
<mailto:craig@2ndquadrant.com>> wrote:
On 15 February 2016 at 04:48, Magnus Hagander <magnus@hagander.net
<mailto:magnus@hagander.net>> wrote:
I was working on adding the tar streaming functionality we talked about
at the developer meeting to pg_basebackup, and rapidly ran across the issue
that Andres has been complaining about for a while. The code in
receivelog.c just passes an insane number of parameters around. Adding or
changing even a small thing ends up touching a huge number of places.Other than the lack of comments on the fields in StreamCtl to indicate
their functions I think this looks good.
I copied that lack of comments from the previous implementation :P
But yeah, I agree, it's probably a good idea to add them.
I didn't find any mistakes, but I do admit my eyes started glazing over
after a bit.
I'd rather not have StreamCtl as a typedef of an anonymous struct if
it's exposed in a header though. It should have a name that can be used in
forward declarations etc.It's not exactly uncommon with anonymous ones either in our code, but I
see no problem adding that.
Short review of this patch:
The patch applies, and builds, cleanly on top of master as of today. No
new
functionality is introduced and thus no new tests or doc patches etc are
applicable.The main point of the patch is to improve readability and reduce the
number of
parameters passed, goals which are reached. However, I agree with Craig
that
comments on the struct fields should be added to improve readability even
further. The comment on ReceiveXlogStream() also now reads a bit strange
since
it describes fields inside the StreamCtl without referencing StreamCtl at
all.
For first time readers of the code it could perhaps be helpful with a brief
note that the discussed parameters are in StreamCtl to avoid potential
confusion.Overall I think this patch is an improvement on the current code and
consider
it ready for committer.
Pushed with updated comments and a named stsruct.
Thanks!
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 03/11/2016 11:15 AM, Magnus Hagander wrote:
...
Pushed with updated comments and a named stsruct.
Pretty sure this memset call in pg_basebackup.c is incorrect, as it
passes parameters in the wrong order:
MemSet(&stream, sizeof(stream), 0);
It seems benign, because we're setting all the fields explicitly, but
gcc is nagging about it.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Sun, Mar 13, 2016 at 12:15 AM, Tomas Vondra <tomas.vondra@2ndquadrant.com
wrote:
On 03/11/2016 11:15 AM, Magnus Hagander wrote:
...
Pushed with updated comments and a named stsruct.
Pretty sure this memset call in pg_basebackup.c is incorrect, as it passes
parameters in the wrong order:MemSet(&stream, sizeof(stream), 0);
It seems benign, because we're setting all the fields explicitly, but gcc
is nagging about it.
Indeed,that's backwards. Interestingly enough I thought I did a c&p between
that and the one in pg_receivexlog.c, but clearly I did not. And my gcc did
not nag about it.
Fixed, thanks for the pointer!
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/