Too high rate of progress information from pg_basebackup

On Wed, May 25, 2016 at 12:37 PM, hubert depesz lubaczewski <
depesz@depesz.com> wrote:

Hi,
it looks that pg_basebackup, when using --progress information, outputs
so much information that it puts cpu pressure on terminal application
I'm using - whether it's simple xterm/rxvt, "screen" or "tmux" terminal
multiplexers.

My specific usecase is setting multiple slave servers for multiple
clusters in AWS, so the data transmission between servers is pretty
fast, but the jump host, that I run all the pg_basebackup (via ssh)
from, is not really beefy, and with 4 concurrent pg_basebackups, I've
seen tmux use up to 80% of CPU of the jump host.

I filled bug report for tmux too, but since the problem can be repeated
in xterm, rxvt, and screen, I think that perhaps it should be handled in
some way in pg_basebackup too?

For completeness sake:
we're using pg_basebackup from 9.3.11 on ubuntu trusty.

In my quick sanity-check test, it looks that progress information in our
case is outputted with the speed of ~ 320KB/s. Which seems excessive.

Hi!

This was fixed in pg_basebackup in 9.4, see commit
8198a321c9ea2072c5acde438f4d19638e04f6f2.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On Wed, May 25, 2016 at 12:52:55PM +0200, Magnus Hagander wrote:

This was fixed in pg_basebackup in 9.4, see commit
8198a321c9ea2072c5acde438f4d19638e04f6f2.

Hmm .. - we can't upgrade to 9.4. Any chance to have this backported to
9.3? It doesn't look like new functionality, more like bugfix, to me.

Best regards,

depesz

--
The best thing about modern society is how easy it is to avoid contact with it.
http://depesz.com/

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On Wed, May 25, 2016 at 12:54 PM, hubert depesz lubaczewski <
depesz@depesz.com> wrote:

On Wed, May 25, 2016 at 12:52:55PM +0200, Magnus Hagander wrote:

This was fixed in pg_basebackup in 9.4, see commit
8198a321c9ea2072c5acde438f4d19638e04f6f2.

Hmm .. - we can't upgrade to 9.4. Any chance to have this backported to
9.3? It doesn't look like new functionality, more like bugfix, to me.

IIRC that was discussed at the time and we decided not to.

The patch is fairly simple though, it wouldn't surprise me if it applies
cleanly to 9.3 if you want to do a one-off build. Though I guess that means
we could also reconsider it for backpatching now that we haven't seen
side-effects.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On Wed, May 25, 2016 at 12:56:18PM +0200, Magnus Hagander wrote:

IIRC that was discussed at the time and we decided not to.

Fair enough.

The patch is fairly simple though, it wouldn't surprise me if it applies
cleanly to 9.3 if you want to do a one-off build. Though I guess that means
we could also reconsider it for backpatching now that we haven't seen
side-effects.

Yeah, that's a no-go for us. I guess that's one more reason to upgrade.

Thanks Magnus,

Best regards,

depesz

--
The best thing about modern society is how easy it is to avoid contact with it.
http://depesz.com/

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

On Wed, May 25, 2016 at 3:58 AM, hubert depesz lubaczewski
<depesz@depesz.com> wrote:

On Wed, May 25, 2016 at 12:56:18PM +0200, Magnus Hagander wrote:

IIRC that was discussed at the time and we decided not to.

Fair enough.

FWIW, there is one conflict at the top of pg_basebackup.c with a set
of variables declared: last_progress_report should be declared as
non-static for consistency. So that's rather minimal.
--
Michael

--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs