Patch CI statistics

Pipeline reads CI wall-clock time. Corpus reads today's patchsets laid out over PostgreSQL's history.

range24h 7d 30d 90d all

granularityhour day week

Runs

10,658

terminal, in range

Success rate

84.6%

9,017 of 10,658

Median build

2m 52s

successful and failed alike

Median test

5m 18s

successful and failed alike

Median queue wait

0m 00s

queued to started

Infra errors

0.1%

8 runs

GitHub re-attempts

attempt > 1 on one run id

Runs by result

terminal runs only; timeouts fold into their failure band

Show data

When	Result	Runs
10 Aug 00:00	build failed	11
10 Aug 00:00	cancelled	1
10 Aug 00:00	infra error	1
10 Aug 00:00	success	231
10 Aug 00:00	tests failed	27
11 Aug 00:00	build failed	65
11 Aug 00:00	success	1,293
11 Aug 00:00	tests failed	168
12 Aug 00:00	build failed	65
12 Aug 00:00	cancelled	3
12 Aug 00:00	infra error	5
12 Aug 00:00	success	1,298
12 Aug 00:00	tests failed	165
13 Aug 00:00	build failed	65
13 Aug 00:00	cancelled	1
13 Aug 00:00	success	1,293
13 Aug 00:00	tests failed	166
14 Aug 00:00	build failed	65
14 Aug 00:00	cancelled	1
14 Aug 00:00	infra error	1
14 Aug 00:00	success	1,275
14 Aug 00:00	tests failed	165
15 Aug 00:00	build failed	66
15 Aug 00:00	cancelled	1
15 Aug 00:00	infra error	1
15 Aug 00:00	success	1,288
15 Aug 00:00	tests failed	173
16 Aug 00:00	build failed	65
16 Aug 00:00	success	1,284
16 Aug 00:00	tests failed	169
17 Aug 00:00	build failed	55
17 Aug 00:00	success	1,055
17 Aug 00:00	tests failed	136

Success rate

hollow points sit below 5 terminal runs in the bucket

Show data

When	Success rate	Terminal runs
10 Aug 00:00	85.2%	271
11 Aug 00:00	84.7%	1,526
12 Aug 00:00	84.5%	1,536
13 Aug 00:00	84.8%	1,525
14 Aug 00:00	84.6%	1,507
15 Aug 00:00	84.2%	1,529
16 Aug 00:00	84.6%	1,518
17 Aug 00:00	84.7%	1,246

Build and test duration by era

median seconds; successful runs only

Show data

When	Era	Build median	Test median	Build p25-p75
10 Aug 00:00	trixie	2m 51s	5m 21s	2m 47s - 2m 55s
11 Aug 00:00	trixie	2m 52s	5m 22s	2m 46s - 2m 55s
12 Aug 00:00	trixie	2m 52s	5m 21s	2m 46s - 2m 55s
13 Aug 00:00	trixie	2m 52s	5m 21s	2m 47s - 2m 55s
14 Aug 00:00	trixie	2m 52s	5m 22s	2m 47s - 2m 55s
15 Aug 00:00	trixie	2m 53s	5m 22s	2m 47s - 2m 56s
16 Aug 00:00	trixie	2m 53s	5m 19s	2m 47s - 2m 55s
17 Aug 00:00	trixie	2m 53s	5m 19s	2m 47s - 2m 56s

ccache hit rate

median per bucket; the image's warm cache was baked against one reference commit, so a patch on a different base misses most of it

Show data

When	Hit rate	Runs
10 Aug 00:00	2.5%	231
11 Aug 00:00	2.5%	1,293
12 Aug 00:00	2.5%	1,298
13 Aug 00:00	2.5%	1,293
14 Aug 00:00	2.5%	1,275
15 Aug 00:00	1.5%	1,288
16 Aug 00:00	1.5%	1,284
17 Aug 00:00	1.5%	1,055

Build time against ccache hit rate

one point per successful run; table shows the first 50

Show data

Hit rate	Build
1.5%	2m 46s
1.5%	1m 56s
1.5%	2m 53s
1.5%	2m 49s
1.5%	2m 46s
1.5%	2m 48s
1.5%	2m 49s
1.5%	2m 57s
1.5%	2m 55s
1.5%	2m 55s
1.2%	3m 13s
1.5%	2m 54s
1.5%	2m 32s
1.5%	2m 59s
1.5%	2m 47s
1.5%	2m 18s
1.5%	2m 56s
1.5%	2m 55s
1.5%	2m 55s
1.5%	2m 52s
1.5%	2m 14s
1.5%	1m 59s
1.5%	2m 47s
1.5%	2m 54s
1.5%	2m 22s
1.5%	2m 48s
1.5%	2m 43s
1.5%	2m 51s
1.5%	2m 55s
1.5%	2m 36s
1.5%	2m 57s
1.5%	2m 54s
1.5%	2m 56s
1.5%	2m 11s
1.5%	2m 48s
1.5%	2m 52s
1.5%	2m 57s
1.5%	2m 57s
1.5%	2m 59s
1.5%	2m 56s
1.5%	2m 41s
1.5%	2m 56s
1.5%	2m 57s
1.5%	2m 55s
1.5%	2m 14s
1.5%	2m 54s
1.5%	2m 55s
1.5%	2m 58s
1.5%	2m 47s
1.5%	2m 55s

Queue latency

median seconds

Show data

When	Push to queued	Queued to started
10 Aug 00:00	-8657m 07s	0m 00s
11 Aug 00:00	-8650m 05s	0m 00s
12 Aug 00:00	-7208m 02s	0m 00s
13 Aug 00:00	-5767m 45s	0m 00s
14 Aug 00:00	-4325m 22s	0m 00s
15 Aug 00:00	-2884m 38s	0m 00s
16 Aug 00:00	-1442m 03s	0m 00s
17 Aug 00:00	0m 01s	0m 00s

Average concurrency

time-weighted average of runs in flight per bucket, against the 30-slot budget

Show data

When	Avg concurrency
10 Aug 00:00	14.7
11 Aug 00:00	13.6
12 Aug 00:00	13.2
13 Aug 00:00	13.4
14 Aug 00:00	12.6
15 Aug 00:00	12.7
16 Aug 00:00	12.1
17 Aug 00:00	11.3

Push lag

time from the patch being posted to CI queueing it

Show data

When	Lag	Runs
10 Aug 00:00	same day	9
10 Aug 00:00	days	33
10 Aug 00:00	weeks	6
10 Aug 00:00	months	112
10 Aug 00:00	older	111
11 Aug 00:00	same day	20
11 Aug 00:00	days	91
11 Aug 00:00	weeks	150
11 Aug 00:00	months	573
11 Aug 00:00	older	692
12 Aug 00:00	same day	30
12 Aug 00:00	days	82
12 Aug 00:00	weeks	155
12 Aug 00:00	months	576
12 Aug 00:00	older	693
13 Aug 00:00	same day	33
13 Aug 00:00	days	75
13 Aug 00:00	weeks	156
13 Aug 00:00	months	570
13 Aug 00:00	older	691
14 Aug 00:00	same day	14
14 Aug 00:00	days	81
14 Aug 00:00	weeks	152
14 Aug 00:00	months	572
14 Aug 00:00	older	688
15 Aug 00:00	same day	21
15 Aug 00:00	days	77
15 Aug 00:00	weeks	157
15 Aug 00:00	months	581
15 Aug 00:00	older	693
16 Aug 00:00	same day	18
16 Aug 00:00	days	58
16 Aug 00:00	weeks	161
16 Aug 00:00	months	588
16 Aug 00:00	older	693
17 Aug 00:00	same day	15
17 Aug 00:00	days	47
17 Aug 00:00	weeks	130
17 Aug 00:00	months	493
17 Aug 00:00	older	561

Push kind

first push, a re-push on a newer master, or a retry of the same head

Show data

When	Kind	Runs
10 Aug 00:00	first push	9
10 Aug 00:00	rebase re-push	261
10 Aug 00:00	retry	1
11 Aug 00:00	first push	20
11 Aug 00:00	rebase re-push	1,506
12 Aug 00:00	first push	30
12 Aug 00:00	rebase re-push	1,503
12 Aug 00:00	retry	3
13 Aug 00:00	first push	35
13 Aug 00:00	rebase re-push	1,490
14 Aug 00:00	first push	14
14 Aug 00:00	rebase re-push	1,493
15 Aug 00:00	first push	21
15 Aug 00:00	rebase re-push	1,507
15 Aug 00:00	retry	1
16 Aug 00:00	first push	18
16 Aug 00:00	rebase re-push	1,500
17 Aug 00:00	first push	15
17 Aug 00:00	rebase re-push	1,231

Suites failed per failing run

one suite is usually a patch bug; dozens is usually one crash cascading

Show data

Suites failed	Runs
1	388
2	58
3	24
4	344
5	74
6	19
7	45
8	18
9	7
10	5
11	11
12	11
13	4
21	7
31	1
33	7
55	1
56	1
58	5
59	6
60	1
61	1
62	6
76	7
83	7
93	1
102	3
107	3
111	1
122	6
123	1
129	5
131	3
133	2
140	1
155	4
156	1
157	1
158	2
200	4

Top failing suites

total failures, and in brackets the number of distinct patchsets - many failures on few patchsets means one broken patch retried; roughly one per patchset means either a flaky suite or a suite that reruns others and inherits their failures

recovery/027_stream_regress

553 (89 branches)

test_plan_advice/001_replan_regress

514 (79 branches)

pg_upgrade/002_pg_upgrade

505 (79 branches)

regress/collate.icu.utf8

69 (13 branches)

regress/subselect

65 (13 branches)

regress/join

59 (12 branches)

postgres_fdw/postgres_fdw

58 (9 branches)

regress/alter_table

58 (11 branches)

regress/copy2

58 (11 branches)

regress/partition_join

58 (12 branches)

isolation/horizons

57 (9 branches)

regress/partition_split

57 (10 branches)

regress/generated_virtual

55 (10 branches)

regress/stats

52 (11 branches)

isolation/eval-plan-qual

50 (8 branches)

regress/aggregates

50 (10 branches)

regress/plpgsql

50 (9 branches)

regress/predicate

50 (9 branches)

regress/with

50 (9 branches)

regress/opr_sanity

46 (8 branches)

Infra health

Everything here should be zero. Non-zero is worth a look, not a panic.

Condition	Count
infra errors	8
in flight past 48h	0
branches with a failed push	2
payloads we could not parse	0
runs with no payload	7
built but pushed no image	32

Retried heads

same head_sha pushed again, new run id

Verdict changed

100.0%

5 of 5 - meaningless until this denominator grows

discussionsall active only

Tracked patchsets

5,348

current, not superseded, all patchsets

Median base age

943 days

behind master, all patchsets

On recent master

33.9%

base 30 days or newer, all patchsets

Has a verdict

89.5%

terminal CI status, all patchsets

Has a runnable image

86.2%

counted by digest, all patchsets

recent

1,812

base age tier, all patchsets

stale

358

base age tier, all patchsets

ancient

3,121

base age tier, all patchsets

unknown

base age tier, all patchsets

Master re-check recency

when each patchset was last tried against master; overdue is past 26h with master moved since. Unlike the base age tiles above, this follows the discussions control.

current

447

behind

1,158

overdue

296

never

3,447

Base age distribution

all current patchsets against those in active discussions

Show data

Base age	All	Active discussions
<1d	1,273	244
1-7d	309	105
7-30d	230	85
30-90d	46	12
90d-1y	312	8
1-2y	318	2
2-5y	879	0
>5y	1,924	1
unknown	57	8

Outcome by base commit year

current patchsets, by the year of the master commit they apply to

Show data

Base year	Outcome	Patchsets
unknown	never applied	57
2012	build failed	1
2014	never applied	1
2015	build failed	3
2015	never applied	3
2016	success	22
2016	tests failed	2
2016	never applied	2
2017	success	463
2017	tests failed	58
2017	build failed	14
2017	never applied	63
2018	success	281
2018	tests failed	38
2018	build failed	7
2018	never applied	60
2019	success	244
2019	tests failed	23
2019	build failed	10
2019	never applied	74
2020	success	232
2020	tests failed	34
2020	build failed	4
2020	never applied	52
2021	success	241
2021	tests failed	35
2021	build failed	9
2021	never applied	35
2022	success	242
2022	cancelled	1
2022	tests failed	36
2022	build failed	11
2022	never applied	36
2023	success	220
2023	tests failed	26
2023	build failed	9
2023	never applied	32
2024	success	205
2024	tests failed	36
2024	build failed	8
2024	infra error	1
2024	never applied	23
2025	success	260
2025	tests failed	43
2025	build failed	13
2025	infra error	1
2025	never applied	36
2026	success	1,562
2026	cancelled	1
2026	tests failed	205
2026	build failed	75
2026	infra error	2
2026	no verdict	37
2026	never applied	159

Success rate by base commit year

of patchsets carrying a verdict

Show data

Base year	Success rate	With a verdict
2012	0.0%	1
2015	0.0%	3
2016	91.7%	24
2017	86.5%	535
2018	86.2%	326
2019	88.1%	277
2020	85.9%	270
2021	84.6%	285
2022	83.4%	290
2023	86.3%	255
2024	82.0%	250
2025	82.0%	317
2026	85.5%	1,953

Apply failures by base commit year

patchsets that would not apply even to their own detected base - a patch-extraction / base-detection signal, not a staleness measure; the rate is in the table

Show data

Base year	Patchsets	Failed to apply	Failure rate
unknown	57	57	100.0%
2012	1	0	0.0%
2014	1	1	100.0%
2015	6	3	50.0%
2016	26	2	7.7%
2017	598	63	10.5%
2018	386	60	15.5%
2019	351	74	21.1%
2020	322	52	16.1%
2021	320	35	10.9%
2022	326	36	11.0%
2023	287	32	11.1%
2024	273	23	8.4%
2025	353	36	10.2%
2026	2,041	159	7.8%

Build and test time over PostgreSQL history

median seconds of successful runs, by the base commit's year

Show data

Base year	Build median	Test median
2016	1m 41s	3m 18s
2017	1m 46s	4m 37s
2018	1m 50s	2m 38s
2019	1m 55s	2m 35s
2020	2m 00s	2m 52s
2021	2m 11s	3m 09s
2022	2m 14s	3m 58s
2023	2m 51s	3m 58s
2024	2m 43s	3m 39s
2025	2m 45s	4m 26s
2026	2m 52s	5m 21s

Test suites over PostgreSQL history

median executed suites of successful runs, by the base commit's year

Show data

Base year	Suites median
2016	412
2017	451
2018	505
2019	537
2020	566
2021	621
2022	654
2023	777
2024	833
2025	900
2026	1018

Per PostgreSQL major

A major whose era image is still a disabled stub is marked - those patchsets sit in won't-retry until the image exists.

PG	Era	Patchsets	Runs	Success	Build median	Test median	Suites	Images
9	stretch	13	13	61.5%	1m 34s	2m 41s	398	9
10	stretch	404	405	87.9%	1m 45s	5m 20s	443	392
11	stretch	347	348	87.1%	1m 48s	2m 35s	476	339
12	buster	275	275	86.9%	1m 51s	2m 35s	513	267
13	buster	273	273	84.6%	1m 57s	2m 37s	548	267
14	bullseye	290	292	86.3%	2m 08s	3m 06s	590	284
15	bullseye	279	281	86.5%	2m 11s	3m 18s	625	268
16	bookworm	269	268	82.8%	2m 46s	4m 12s	669	256
17	bookworm	250	250	84.4%	2m 54s	3m 27s	793	244
18	trixie	280	281	79.0%	2m 41s	3m 50s	858	267
19	trixie	343	341	85.9%	2m 49s	4m 42s	936	329
20	trixie	1,825	29,209	83.5%	2m 52s	5m 18s	1018	1,689

Files the corpus conflicts on

from apply-stage failures; in brackets the number of distinct patchsets

src/bin/pgbench/pgbench.c

11 (11 branches)

doc/src/sgml/ref/pgbench.sgml

10 (10 branches)

src/bin/pgbench/t/001_pgbench_with_server.pl

6 (6 branches)

src/backend/access/transam/xlog.c

5 (5 branches)

src/backend/parser/gram.y

5 (5 branches)

src/bin/pg_dump/pg_dump.c

5 (5 branches)

doc/src/sgml/config.sgml

4 (4 branches)

src/backend/access/common/reloptions.c

4 (4 branches)

src/backend/commands/tablecmds.c

4 (4 branches)

src/backend/postmaster/autovacuum.c

4 (4 branches)

src/include/utils/float.h

4 (4 branches)

templates/pages/about/policies/coc/ja.html

4 (4 branches)

configure.in

3 (3 branches)

doc/src/sgml/perform.sgml

3 (3 branches)

doc/src/sgml/ref/psql-ref.sgml

3 (3 branches)

doc/src/sgml/ref/select.sgml

3 (3 branches)

indent.c

3 (3 branches)

run_build.pl

3 (3 branches)

src/backend/commands/indexcmds.c

3 (3 branches)

src/backend/commands/vacuum.c

3 (3 branches)

Verdict coverage in active discussions

patchsets in threads with a message in the last 30 days

Show data

Era	With a verdict	No verdict
bullseye	1	0
trixie	425	30
unknown	0	9

Patch size against outcome

files touched per patchset, from the recorded submission paths

Show data

Files touched	Outcome	Patchsets
unknown	success	3,972
unknown	cancelled	2
unknown	tests failed	536
unknown	build failed	164
unknown	infra error	4
unknown	no verdict	37
unknown	never applied	633

Patchset versions per topic

every version, superseded included; split by whether the topic was committed

Show data

Versions	Topic	Topics
1	not committed	4,978
1	committed	77
2	committed	28
2	not committed	154
3	not committed	46
3	committed	7
4	not committed	14
4	committed	3
5	committed	2
5	not committed	9
6	not committed	6
7	not committed	1
8	not committed	1
9	not committed	1
11	not committed	1

Why the orchestrator won't retry

The dashboard shows the top three of this same breakdown.

Reason	Patchsets
base too old	3,011
committed	120
patchset touches .github/	8

Navigation

Patch CI statistics

Runs by result

Success rate

Build and test duration by era

ccache hit rate

Build time against ccache hit rate

Queue latency

Average concurrency

Push lag

Push kind

Suites failed per failing run

Top failing suites

Infra health

Master re-check recency

Base age distribution

Outcome by base commit year

Success rate by base commit year

Apply failures by base commit year

Build and test time over PostgreSQL history

Test suites over PostgreSQL history

Per PostgreSQL major

Files the corpus conflicts on

Verdict coverage in active discussions

Patch size against outcome

Patchset versions per topic

Why the orchestrator won't retry