"ago" times on buildfarm status page

On Wed, Aug 21, 2019 at 9:40 AM Peter Eisentraut <
peter.eisentraut@2ndquadrant.com> wrote:

I find the time displays like

01:03 ago

on the buildfarm status page unhelpful.

First, I can never tell whether this is hours-minutes or minutes-seconds
-- there is probably a less ambiguous format available.

But more importantly, the page doesn't say when it was generated, so a
relative time like this is meaningless. The page might have most
recently reloaded last night. That means when I look at the page, I
*always* have to reload it first to make sense of the times.

I notice that the page source actually includes absolute times that are
then converted to relative using some JavaScript. Could we perhaps just
turn that off? Or preferably convert to local time. I can much easier
make sense of an absolute local time: I can compare that to the clock in
the corner of the screen, and I can compare that, say, to a commit
timestamp.

It used to be that the "ago" part was generated on the server, but Andrew
changed that to the fixed timestamp + javascript to improve cachability and
thus performance. Perhaps now that it's that it could be as easy as adding
a checkbox to the page (which could remember your preference in a cookie)
that switches between the two modes?

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/&gt;
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/&gt;

Magnus Hagander <magnus@hagander.net> writes:

On Wed, Aug 21, 2019 at 9:40 AM Peter Eisentraut <
peter.eisentraut@2ndquadrant.com> wrote:

I find the time displays like

01:03 ago

on the buildfarm status page unhelpful.

First, I can never tell whether this is hours-minutes or minutes-seconds
-- there is probably a less ambiguous format available.

But more importantly, the page doesn't say when it was generated, so a
relative time like this is meaningless. The page might have most
recently reloaded last night. That means when I look at the page, I
*always* have to reload it first to make sense of the times.

I notice that the page source actually includes absolute times that are
then converted to relative using some JavaScript. Could we perhaps just
turn that off? Or preferably convert to local time. I can much easier
make sense of an absolute local time: I can compare that to the clock in
the corner of the screen, and I can compare that, say, to a commit
timestamp.

It used to be that the "ago" part was generated on the server, but Andrew
changed that to the fixed timestamp + javascript to improve cachability and
thus performance. Perhaps now that it's that it could be as easy as adding
a checkbox to the page (which could remember your preference in a cookie)
that switches between the two modes?

The Javscript could also be made to update the "ago" part every minute,
and show the absoulte time as a tooltip, which is what pretty much every
other website does.

- ilmari
--
"A disappointingly low fraction of the human race is,
at any given time, on fire." - Stig Sandbeck Mathisen

Magnus Hagander <magnus@hagander.net> writes:

On Wed, Aug 21, 2019 at 9:40 AM Peter Eisentraut <
peter.eisentraut@2ndquadrant.com> wrote:

I find the time displays like
01:03 ago
on the buildfarm status page unhelpful.

I notice that the page source actually includes absolute times that are
then converted to relative using some JavaScript. Could we perhaps just
turn that off?

It used to be that the "ago" part was generated on the server, but Andrew
changed that to the fixed timestamp + javascript to improve cachability and
thus performance. Perhaps now that it's that it could be as easy as adding
a checkbox to the page (which could remember your preference in a cookie)
that switches between the two modes?

FWIW, I'm used to the way it's shown and would not like a change, so
if this can be made user-settable as Magnus suggests, that would be
better IMO.

The real problem with that column though is that it relies on run start
times that are self-reported by the buildfarm clients, and some of them
have system clocks that are many hours off reality. What *I'd* like to
see is for the column to contain time of receipt of the buildfarm report
at the server, less the measured runtime of the test.

regards, tom lane

On 8/21/19 8:32 AM, Dagfinn Ilmari Mannsåker wrote:

Magnus Hagander <magnus@hagander.net> writes:

On Wed, Aug 21, 2019 at 9:40 AM Peter Eisentraut <
peter.eisentraut@2ndquadrant.com> wrote:

I find the time displays like

01:03 ago

on the buildfarm status page unhelpful.

First, I can never tell whether this is hours-minutes or minutes-seconds
-- there is probably a less ambiguous format available.

But more importantly, the page doesn't say when it was generated, so a
relative time like this is meaningless. The page might have most
recently reloaded last night. That means when I look at the page, I
*always* have to reload it first to make sense of the times.

I notice that the page source actually includes absolute times that are
then converted to relative using some JavaScript. Could we perhaps just
turn that off? Or preferably convert to local time. I can much easier
make sense of an absolute local time: I can compare that to the clock in
the corner of the screen, and I can compare that, say, to a commit
timestamp.

It used to be that the "ago" part was generated on the server, but Andrew
changed that to the fixed timestamp + javascript to improve cachability and
thus performance. Perhaps now that it's that it could be as easy as adding
a checkbox to the page (which could remember your preference in a cookie)
that switches between the two modes?

The Javscript could also be made to update the "ago" part every minute,
and show the absoulte time as a tooltip, which is what pretty much every
other website does.

The code for the page is here:
<https://github.com/PGBuildFarm/server-code/blob/master/templates/status.tt&gt;

Patches welcome.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 8/21/19 9:55 AM, Tom Lane wrote:

The real problem with that column though is that it relies on run start
times that are self-reported by the buildfarm clients, and some of them
have system clocks that are many hours off reality. What *I'd* like to
see is for the column to contain time of receipt of the buildfarm report
at the server, less the measured runtime of the test.

That might be possible. I'll put it on my list of things to do. It's not
happening any time soon, though.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 8/21/19 3:40 AM, Peter Eisentraut wrote:

I find the time displays like

01:03 ago

on the buildfarm status page unhelpful.

First, I can never tell whether this is hours-minutes or minutes-seconds
-- there is probably a less ambiguous format available.

This is hours:minutes. days are always explicitly prepended if greater
than 0.

But more importantly, the page doesn't say when it was generated, so a
relative time like this is meaningless. The page might have most
recently reloaded last night. That means when I look at the page, I
*always* have to reload it first to make sense of the times.

I notice that the page source actually includes absolute times that are
then converted to relative using some JavaScript. Could we perhaps just
turn that off? Or preferably convert to local time. I can much easier
make sense of an absolute local time: I can compare that to the clock in
the corner of the screen, and I can compare that, say, to a commit
timestamp.

Maybe.

TBH the whole buildfarm UI is rather dated and clunky. But it needs work
from a web monkey, which I am really not.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 8/21/19 11:07 AM, Andrew Dunstan wrote:

On 8/21/19 9:55 AM, Tom Lane wrote:

The real problem with that column though is that it relies on run start
times that are self-reported by the buildfarm clients, and some of them
have system clocks that are many hours off reality. What *I'd* like to
see is for the column to contain time of receipt of the buildfarm report
at the server, less the measured runtime of the test.

That might be possible. I'll put it on my list of things to do. It's not
happening any time soon, though.

What I have done quickly is to store a measure of the clock skew. We
already calculated it but we didn't store it. Initial indications are
that only a few have significant skew.

Still, if we simply added the skew to the snapshot time that might be
enough to achieve what you want. That would be a one line change, I think.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

What I have done quickly is to store a measure of the clock skew. We
already calculated it but we didn't store it. Initial indications are
that only a few have significant skew.

Oh, I didn't know that the server had the ability to measure that.

(Yes, I agree that there are just a couple with big skews at any
one time.)

Still, if we simply added the skew to the snapshot time that might be
enough to achieve what you want. That would be a one line change, I think.

+1

regards, tom lane

On 8/21/19 4:16 PM, Tom Lane wrote:

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

What I have done quickly is to store a measure of the clock skew. We
already calculated it but we didn't store it. Initial indications are
that only a few have significant skew.

Oh, I didn't know that the server had the ability to measure that.

(Yes, I agree that there are just a couple with big skews at any
one time.)

Still, if we simply added the skew to the snapshot time that might be
enough to achieve what you want. That would be a one line change, I think.

+1

Done. It's only happening prospectively, so we'll need to wait a few
days to see it flow through.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 8/21/19 10:52 AM, Andrew Dunstan wrote:

The Javscript could also be made to update the "ago" part every minute,
and show the absoulte time as a tooltip, which is what pretty much every
other website does.

The code for the page is here:
<https://github.com/PGBuildFarm/server-code/blob/master/templates/status.tt&gt;

Patches welcome.

I have done both of these things.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 8/21/19 4:16 PM, Tom Lane wrote:

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

Still, if we simply added the skew to the snapshot time that might be
enough to achieve what you want. That would be a one line change, I think.

+1

Done. It's only happening prospectively, so we'll need to wait a few
days to see it flow through.

Hm, doesn't seem to have done the trick. The current dashboard page shows
(in the v12 branch)

mule ... 01:17 ago OK [97205d0] Config
loach ... 01:32 ago OK [97205d0] Config
dangomushi ... 02:11 ago OK [97205d0] Config
bowerbird ... 02:23 ago scriptsCheck [97205d0] Details
snapper ... 02:48 ago OK [63fc3b1] Config
caiman ... 03:04 ago OK [97205d0] Config
nightjar ... 03:17 ago recoveryCheck [97205d0] Details
chub ... 03:29 ago OK [97205d0] Config
clam ... 03:34 ago OK [97205d0] Config
demoiselle ... 03:45 ago OK [97205d0] Config

snapper is clearly out of line here: the commit it claims
to have fetched 2:48 ago was obsoleted around seven hours ago.

(Snapper is one of the machines that is typically inconsistent
in this way. I've been assuming that's because its system clock
is a few hours off ... but maybe there's something else going on?)

regards, tom lane

On 8/26/19 2:55 PM, Tom Lane wrote:

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 8/21/19 4:16 PM, Tom Lane wrote:

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

Still, if we simply added the skew to the snapshot time that might be
enough to achieve what you want. That would be a one line change, I think.

+1

Done. It's only happening prospectively, so we'll need to wait a few
days to see it flow through.

Hm, doesn't seem to have done the trick. The current dashboard page shows
(in the v12 branch)

mule ... 01:17 ago OK [97205d0] Config
loach ... 01:32 ago OK [97205d0] Config
dangomushi ... 02:11 ago OK [97205d0] Config
bowerbird ... 02:23 ago scriptsCheck [97205d0] Details
snapper ... 02:48 ago OK [63fc3b1] Config
caiman ... 03:04 ago OK [97205d0] Config
nightjar ... 03:17 ago recoveryCheck [97205d0] Details
chub ... 03:29 ago OK [97205d0] Config
clam ... 03:34 ago OK [97205d0] Config
demoiselle ... 03:45 ago OK [97205d0] Config

snapper is clearly out of line here: the commit it claims
to have fetched 2:48 ago was obsoleted around seven hours ago.

(Snapper is one of the machines that is typically inconsistent
in this way. I've been assuming that's because its system clock
is a few hours off ... but maybe there's something else going on?)

I think this is the problem:

 'scmrepo' => '/home/pgbf/pgmirror.git',

Probably this isn't updated often enough. It probably has little to do with the clock settings.

This is the kind of old-fashioned way of doing things. These days "git_keep_mirror => 1" along with the community repo as the base would avoid these problems.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote:

I think this is the problem:

'scmrepo' => '/home/pgbf/pgmirror.git',

Probably this isn't updated often enough. It probably has little to do with the clock settings.

This is the kind of old-fashioned way of doing things. These days "git_keep_mirror => 1" along with the community repo as the base would avoid these problems.

We've discussed this before (see below).

The configuration is intentionally like that. I specifically configured skate and snapper to build the exact same source, where skate builds with default buildfarm settings, while snapper builds with the settings actually used by Debian source packages.

These animals were set up to avoid cases we had in the past where Debian source packages failed to build on sparc, even though build animals running on Debian sparc were building fine:

/messages/by-id/000001d2f0c2$e2d335a0$a879a0e0$@turelinckx.be

With snapper building the exact same source as skate (as it is now), we have some point of reference if snapper fails but skate succeeds. I could configure snapper to perform an update of the repo before building, but then we give up this comparability in exchange for a bit more clarity regarding timestamps.

Best regards,
Tom Turelinckx

On 8/27/19 4:33 AM, Tom Turelinckx wrote:

On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote:

I think this is the problem:

'scmrepo' => '/home/pgbf/pgmirror.git',

Probably this isn't updated often enough. It probably has little to do with the clock settings.

This is the kind of old-fashioned way of doing things. These days "git_keep_mirror => 1" along with the community repo as the base would avoid these problems.

We've discussed this before (see below).

Hm. So the issue really is that the build timestamp that the buildfarm
client is reporting tells when it pulled from the local repo, not when
that repo was last updated from the community server. Not sure if there's
any simple way to improve that ... Andrew, any thoughts?

Maybe we need an option to use the git commit time. instead of the
snapshot time.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 8/27/19 8:45 AM, Andrew Dunstan wrote:

On 8/27/19 4:33 AM, Tom Turelinckx wrote:

On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote:

I think this is the problem:

'scmrepo' => '/home/pgbf/pgmirror.git',

Probably this isn't updated often enough. It probably has little to do with the clock settings.

This is the kind of old-fashioned way of doing things. These days "git_keep_mirror => 1" along with the community repo as the base would avoid these problems.

We've discussed this before (see below).

Hm. So the issue really is that the build timestamp that the buildfarm
client is reporting tells when it pulled from the local repo, not when
that repo was last updated from the community server. Not sure if there's
any simple way to improve that ... Andrew, any thoughts?

Maybe we need an option to use the git commit time. instead of the
snapshot time.

Scratch that - we use this to calculate the duration of the first stage,
so mangling it would just create another error.

It's tempting to say we should sort the dashboard by git reference time
then snapshot - that should be fairly doable. But what if there isn't a
git reference, as happens when there's a git failure for example. In
those cases Maybe just use the snapshot time?

Storing the git timestanp would involve a table change in our second
largest table, so the team would need to discuss and plan it.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

"Tom Turelinckx" <pgbf@twiska.com> writes:

On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote:

I think this is the problem:
'scmrepo' => '/home/pgbf/pgmirror.git',
Probably this isn't updated often enough. It probably has little to do with the clock settings.

The configuration is intentionally like that. I specifically configured skate and snapper to build the exact same source, where skate builds with default buildfarm settings, while snapper builds with the settings actually used by Debian source packages.

TBH, I don't find that particularly important ... especially not for HEAD
builds, where building a many-hours-old snapshot is pretty much in the
category of "why bother?". On the whole, I think building from the latest
available source is the most useful policy. If there's some platform-
or configuration-specific issue, it usually takes more than one build
cycle for us to notice it anyway, so that ensuring two animals have exactly
comparable builds at any given instant isn't very helpful.

regards, tom lane

On 8/27/19 10:15 AM, Tom Lane wrote:

"Tom Turelinckx" <pgbf@twiska.com> writes:

On Mon, Aug 26, 2019, at 9:08 PM, Andrew Dunstan wrote:

I think this is the problem:
'scmrepo' => '/home/pgbf/pgmirror.git',
Probably this isn't updated often enough. It probably has little to do with the clock settings.

The configuration is intentionally like that. I specifically configured skate and snapper to build the exact same source, where skate builds with default buildfarm settings, while snapper builds with the settings actually used by Debian source packages.

TBH, I don't find that particularly important ... especially not for HEAD
builds, where building a many-hours-old snapshot is pretty much in the
category of "why bother?". On the whole, I think building from the latest
available source is the most useful policy. If there's some platform-
or configuration-specific issue, it usually takes more than one build
cycle for us to notice it anyway, so that ensuring two animals have exactly
comparable builds at any given instant isn't very helpful.

Yeah, point. snapper seems the more important box here.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services