Patch CI statistics

Pipeline reads CI wall-clock time. Corpus reads today's patchsets laid out over PostgreSQL's history.

Runs
1,739
terminal, in range
Success rate
85.5%
1,487 of 1,739
Median build
2m 50s
successful and failed alike
Median test
5m 15s
successful and failed alike
Median queue wait
0m 00s
queued to started
Infra errors
0.0%
0 runs
GitHub re-attempts
0
attempt > 1 on one run id

Runs by result

terminal runs only; timeouts fold into their failure band

Show data
WhenResultRuns
17 Aug 15:00success3
17 Aug 16:00success8
17 Aug 16:00tests failed1
17 Aug 17:00success1
17 Aug 18:00success1
17 Aug 20:00success1
17 Aug 22:00success1
17 Aug 23:00success3
18 Aug 00:00build failed2
18 Aug 00:00success95
18 Aug 00:00tests failed11
18 Aug 01:00build failed8
18 Aug 01:00success101
18 Aug 01:00tests failed6
18 Aug 02:00build failed6
18 Aug 02:00success91
18 Aug 02:00tests failed9
18 Aug 03:00build failed11
18 Aug 03:00success102
18 Aug 03:00tests failed12
18 Aug 04:00build failed3
18 Aug 04:00success95
18 Aug 04:00tests failed15
18 Aug 05:00build failed3
18 Aug 05:00success106
18 Aug 05:00tests failed14
18 Aug 06:00build failed2
18 Aug 06:00success100
18 Aug 06:00tests failed9
18 Aug 07:00build failed9
18 Aug 07:00success77
18 Aug 07:00tests failed10
18 Aug 08:00build failed4
18 Aug 08:00success94
18 Aug 08:00tests failed12
18 Aug 09:00build failed1
18 Aug 09:00success92
18 Aug 09:00tests failed14
18 Aug 10:00build failed4
18 Aug 10:00success89
18 Aug 10:00tests failed21
18 Aug 11:00build failed8
18 Aug 11:00success87
18 Aug 11:00tests failed19
18 Aug 12:00build failed4
18 Aug 12:00success108
18 Aug 12:00tests failed6
18 Aug 13:00build failed2
18 Aug 13:00success87
18 Aug 13:00tests failed9
18 Aug 14:00build failed5
18 Aug 14:00success111
18 Aug 14:00tests failed8
18 Aug 15:00build failed2
18 Aug 15:00success34
18 Aug 15:00tests failed2

Success rate

hollow points sit below 5 terminal runs in the bucket

Show data
WhenSuccess rateTerminal runs
17 Aug 15:00100.0%3
17 Aug 16:0088.9%9
17 Aug 17:00100.0%1
17 Aug 18:00100.0%1
17 Aug 20:00100.0%1
17 Aug 22:00100.0%1
17 Aug 23:00100.0%3
18 Aug 00:0088.0%108
18 Aug 01:0087.8%115
18 Aug 02:0085.8%106
18 Aug 03:0081.6%125
18 Aug 04:0084.1%113
18 Aug 05:0086.2%123
18 Aug 06:0090.1%111
18 Aug 07:0080.2%96
18 Aug 08:0085.5%110
18 Aug 09:0086.0%107
18 Aug 10:0078.1%114
18 Aug 11:0076.3%114
18 Aug 12:0091.5%118
18 Aug 13:0088.8%98
18 Aug 14:0089.5%124
18 Aug 15:0089.5%38

Build and test duration by era

median seconds; successful runs only

Show data
WhenEraBuild medianTest medianBuild p25-p75
18 Aug 15:00bookworm2m 54s3m 35s2m 50s - 3m 07s
17 Aug 15:00trixie2m 49s5m 31s2m 49s - 2m 53s
17 Aug 16:00trixie2m 50s5m 19s2m 49s - 2m 54s
17 Aug 17:00trixie3m 01s5m 44s3m 01s - 3m 01s
17 Aug 18:00trixie2m 49s5m 06s2m 49s - 2m 49s
17 Aug 20:00trixie2m 12s3m 46s2m 12s - 2m 12s
17 Aug 22:00trixie2m 49s5m 29s2m 49s - 2m 49s
17 Aug 23:00trixie2m 52s5m 24s2m 50s - 2m 55s
18 Aug 00:00trixie2m 47s5m 14s2m 37s - 2m 53s
18 Aug 01:00trixie2m 49s5m 16s2m 32s - 2m 56s
18 Aug 02:00trixie2m 50s5m 19s2m 45s - 2m 55s
18 Aug 03:00trixie2m 52s5m 19s2m 46s - 2m 55s
18 Aug 04:00trixie2m 52s5m 16s2m 34s - 2m 55s
18 Aug 05:00trixie2m 52s5m 19s2m 43s - 2m 55s
18 Aug 06:00trixie2m 53s5m 19s2m 47s - 2m 56s
18 Aug 07:00trixie2m 52s5m 18s2m 46s - 2m 55s
18 Aug 08:00trixie2m 52s5m 17s2m 26s - 2m 55s
18 Aug 09:00trixie2m 51s5m 20s2m 47s - 2m 56s
18 Aug 10:00trixie2m 52s5m 21s2m 47s - 2m 55s
18 Aug 11:00trixie2m 52s5m 18s2m 48s - 2m 55s
18 Aug 12:00trixie2m 52s5m 23s2m 47s - 2m 56s
18 Aug 13:00trixie2m 48s5m 21s2m 39s - 2m 55s
18 Aug 14:00trixie2m 48s4m 44s2m 39s - 2m 53s
18 Aug 15:00trixie2m 48s5m 10s2m 41s - 2m 55s

ccache hit rate

median per bucket; the image's warm cache was baked against one reference commit, so a patch on a different base misses most of it

Show data
WhenHit rateRuns
17 Aug 15:001.5%3
17 Aug 16:001.5%8
17 Aug 17:001.5%1
17 Aug 18:001.5%1
17 Aug 20:001.5%1
17 Aug 22:001.5%1
17 Aug 23:001.5%3
18 Aug 00:001.5%95
18 Aug 01:001.5%101
18 Aug 02:001.5%91
18 Aug 03:001.5%102
18 Aug 04:001.5%95
18 Aug 05:001.5%106
18 Aug 06:001.5%100
18 Aug 07:001.5%77
18 Aug 08:001.5%94
18 Aug 09:001.5%92
18 Aug 10:001.5%89
18 Aug 11:001.5%87
18 Aug 12:001.5%108
18 Aug 13:001.5%87
18 Aug 14:001.5%111
18 Aug 15:001.6%34

Build time against ccache hit rate

one point per successful run; table shows the first 50

Show data
Hit rateBuild
1.5%2m 47s
1.5%2m 48s
4.7%2m 51s
4.7%2m 50s
1.5%2m 55s
4.7%2m 54s
4.7%2m 51s
4.7%3m 04s
1.5%2m 41s
4.7%2m 52s
4.7%2m 52s
1.5%2m 48s
4.7%1m 53s
10.5%2m 48s
1.5%2m 56s
1.5%1m 58s
34.7%0m 43s
1.7%3m 08s
17.2%2m 22s
1.5%2m 55s
1.6%2m 20s
1.6%2m 55s
1.6%3m 11s
1.6%3m 09s
1.6%3m 10s
1.6%3m 07s
1.6%2m 59s
1.5%2m 25s
1.5%2m 53s
1.6%3m 07s
1.6%2m 54s
1.6%3m 09s
1.5%3m 02s
1.2%2m 39s
1.2%2m 40s
1.5%2m 48s
1.2%2m 37s
1.5%2m 21s
1.2%2m 48s
1.5%2m 54s
1.3%2m 49s
1.3%2m 49s
1.3%2m 50s
1.3%1m 45s
1.3%1m 49s
1.5%2m 55s
1.3%2m 37s
1.3%2m 38s
1.5%2m 56s
1.5%2m 46s

Queue latency

median seconds

Show data
WhenPush to queuedQueued to started
17 Aug 15:000m 01s0m 00s
17 Aug 16:000m 02s0m 00s
17 Aug 17:000m 02s0m 00s
17 Aug 18:000m 02s0m 00s
17 Aug 20:000m 02s0m 00s
17 Aug 22:000m 02s0m 00s
17 Aug 23:000m 01s0m 00s
18 Aug 00:000m 01s0m 00s
18 Aug 01:000m 02s0m 00s
18 Aug 02:000m 01s0m 00s
18 Aug 03:000m 01s0m 00s
18 Aug 04:000m 01s0m 00s
18 Aug 05:000m 01s0m 00s
18 Aug 06:000m 01s0m 00s
18 Aug 07:000m 02s0m 00s
18 Aug 08:000m 02s0m 00s
18 Aug 09:000m 01s0m 00s
18 Aug 10:000m 01s0m 00s
18 Aug 11:000m 01s0m 00s
18 Aug 12:000m 02s0m 00s
18 Aug 13:000m 02s0m 00s
18 Aug 14:000m 02s0m 00s
18 Aug 15:000m 02s0m 00s

Average concurrency

time-weighted average of runs in flight per bucket, against the 30-slot budget

Show data
WhenAvg concurrency
17 Aug 15:001.3
17 Aug 16:001.3
17 Aug 17:000.2
17 Aug 18:000.2
17 Aug 19:000.0
17 Aug 20:000.1
17 Aug 21:000.0
17 Aug 22:000.2
17 Aug 23:000.9
18 Aug 00:0024.3
18 Aug 01:0023.5
18 Aug 02:0023.6
18 Aug 03:0025.6
18 Aug 04:0025.2
18 Aug 05:0024.3
18 Aug 06:0021.8
18 Aug 07:0018.8
18 Aug 08:0020.7
18 Aug 09:0022.0
18 Aug 10:0022.5
18 Aug 11:0023.9
18 Aug 12:0022.0
18 Aug 13:0024.3
18 Aug 14:0024.0
18 Aug 15:007.8

Push lag

time from the patch being posted to CI queueing it

Show data
WhenLagRuns
17 Aug 15:00days3
17 Aug 16:00same day1
17 Aug 16:00days7
17 Aug 16:00weeks1
17 Aug 17:00same day1
17 Aug 18:00same day1
17 Aug 20:00same day1
17 Aug 22:00same day1
17 Aug 23:00same day2
17 Aug 23:00days1
18 Aug 00:00same day1
18 Aug 00:00days17
18 Aug 00:00weeks5
18 Aug 00:00months77
18 Aug 00:00older8
18 Aug 01:00same day2
18 Aug 01:00days2
18 Aug 01:00weeks3
18 Aug 01:00months108
18 Aug 02:00days1
18 Aug 02:00weeks3
18 Aug 02:00months82
18 Aug 02:00older20
18 Aug 03:00days3
18 Aug 03:00weeks3
18 Aug 03:00older119
18 Aug 04:00same day1
18 Aug 04:00days3
18 Aug 04:00weeks5
18 Aug 04:00older104
18 Aug 05:00same day2
18 Aug 05:00weeks54
18 Aug 05:00months29
18 Aug 05:00older38
18 Aug 06:00days7
18 Aug 06:00weeks5
18 Aug 06:00months99
18 Aug 07:00same day1
18 Aug 07:00days4
18 Aug 07:00weeks3
18 Aug 07:00months31
18 Aug 07:00older57
18 Aug 08:00same day3
18 Aug 08:00days9
18 Aug 08:00weeks38
18 Aug 08:00months29
18 Aug 08:00older31
18 Aug 09:00same day2
18 Aug 09:00days1
18 Aug 09:00weeks4
18 Aug 09:00months32
18 Aug 09:00older68
18 Aug 10:00same day5
18 Aug 10:00days1
18 Aug 10:00weeks6
18 Aug 10:00older102
18 Aug 11:00same day1
18 Aug 11:00days2
18 Aug 11:00weeks8
18 Aug 11:00older103
18 Aug 12:00same day1
18 Aug 12:00days4
18 Aug 12:00weeks14
18 Aug 12:00months57
18 Aug 12:00older42
18 Aug 13:00same day5
18 Aug 13:00days7
18 Aug 13:00weeks21
18 Aug 13:00months65
18 Aug 14:00same day3
18 Aug 14:00months65
18 Aug 14:00older56
18 Aug 15:00older38

Push kind

first push, a re-push on a newer master, or a retry of the same head

Show data
WhenKindRuns
17 Aug 15:00rebase re-push3
17 Aug 16:00first push1
17 Aug 16:00rebase re-push8
17 Aug 17:00first push1
17 Aug 18:00first push1
17 Aug 20:00first push1
17 Aug 22:00first push1
17 Aug 23:00first push2
17 Aug 23:00rebase re-push1
18 Aug 00:00first push1
18 Aug 00:00rebase re-push107
18 Aug 01:00first push2
18 Aug 01:00rebase re-push113
18 Aug 02:00rebase re-push106
18 Aug 03:00rebase re-push125
18 Aug 04:00first push1
18 Aug 04:00rebase re-push112
18 Aug 05:00first push2
18 Aug 05:00rebase re-push121
18 Aug 06:00rebase re-push111
18 Aug 07:00first push1
18 Aug 07:00rebase re-push95
18 Aug 08:00first push3
18 Aug 08:00rebase re-push107
18 Aug 09:00first push2
18 Aug 09:00rebase re-push105
18 Aug 10:00first push5
18 Aug 10:00rebase re-push109
18 Aug 11:00first push1
18 Aug 11:00rebase re-push113
18 Aug 12:00first push1
18 Aug 12:00rebase re-push117
18 Aug 13:00first push47
18 Aug 13:00rebase re-push51
18 Aug 14:00first push124
18 Aug 15:00first push38

Suites failed per failing run

one suite is usually a patch bug; dozens is usually one crash cascading

Show data
Suites failedRuns
161
29
34
448
513
62
77
82
102
113
122
211
331
581
591
621
681
761
831
1071
1221
1291
1321
1571

Top failing suites

total failures, and in brackets the number of distinct patchsets - many failures on few patchsets means one broken patch retried; roughly one per patchset means either a flaky suite or a suite that reruns others and inherits their failures

recovery/027_stream_regress
80 (80 branches)
pg_upgrade/002_pg_upgrade
75 (75 branches)
test_plan_advice/001_replan_regress
74 (74 branches)
regress/partition_join
10 (10 branches)
regress/subselect
10 (10 branches)
postgres_fdw/postgres_fdw
9 (9 branches)
regress/alter_table
9 (9 branches)
regress/collate.icu.utf8
9 (9 branches)
regress/copy2
9 (9 branches)
isolation/horizons
8 (8 branches)
regress/generated_virtual
8 (8 branches)
regress/partition_split
8 (8 branches)
regress/plpgsql
8 (8 branches)
regress/predicate
8 (8 branches)
regress/stats
8 (8 branches)
regress/with
8 (8 branches)
isolation/eval-plan-qual
7 (7 branches)
regress/domain
7 (7 branches)
regress/explain
7 (7 branches)
regress/foreign_key
7 (7 branches)

Infra health

Everything here should be zero. Non-zero is worth a look, not a panic.

ConditionCount
infra errors0
in flight past 48h0
branches with a failed push2
payloads we could not parse0
runs with no payload0
built but pushed no image18
Retried heads
0
same head_sha pushed again, new run id
Verdict changed
-
0 of 0 - meaningless until this denominator grows
discussionsallactive only
Tracked patchsets
5,628
current, not superseded, all patchsets
Median base age
860 days
behind master, all patchsets
On recent master
34.2%
base 30 days or newer, all patchsets
Has a verdict
89.3%
terminal CI status, all patchsets
Has a runnable image
85.7%
counted by digest, all patchsets
recent
1,926
base age tier, all patchsets
stale
398
base age tier, all patchsets
ancient
3,226
base age tier, all patchsets
unknown
78
base age tier, all patchsets

Master re-check recency

when each patchset was last tried against master; overdue is past 26h with master moved since. Unlike the base age tiles above, this follows the discussions control.

current
5
behind
381
overdue
109
never
16

Base age distribution

all current patchsets against those in active discussions

Show data
Base ageAllActive discussions
<1d1,616318
1-7d6555
7-30d24599
30-90d6216
90d-1y33610
1-2y3632
2-5y9370
>5y1,9261
unknown7810

Outcome by base commit year

current patchsets, by the year of the master commit they apply to

Show data
Base yearOutcomePatchsets
unknownnever applied10
2021success1
2025success2
2025tests failed1
2026success422
2026cancelled1
2026tests failed31
2026build failed6
2026infra error1
2026no verdict6
2026never applied30

Success rate by base commit year

of patchsets carrying a verdict

Show data
Base yearSuccess rateWith a verdict
2021100.0%1
202566.7%3
202691.6%463

Apply failures by base commit year

patchsets that would not apply even to their own detected base - a patch-extraction / base-detection signal, not a staleness measure; the rate is in the table

Show data
Base yearPatchsetsFailed to applyFailure rate
unknown1010100.0%
2021100.0%
2025300.0%
2026497306.0%

Build and test time over PostgreSQL history

median seconds of successful runs, by the base commit's year

Show data
Base yearBuild medianTest median
20212m 17s3m 02s
20252m 32s5m 43s
20262m 53s5m 21s

Test suites over PostgreSQL history

median executed suites of successful runs, by the base commit's year

Show data
Base yearSuites median
2021623
2025905
20261019

Per PostgreSQL major

A major whose era image is still a disabled stub is marked - those patchsets sit in won't-retry until the image exists.

PGEraPatchsetsRunsSuccessBuild medianTest medianSuitesImages
15bullseye11100.0%2m 17s3m 02s6231
19trixie161580.0%2m 47s4m 45s99714
20trixie4834,43091.0%2m 52s5m 21s1019438

Files the corpus conflicts on

from apply-stage failures; in brackets the number of distinct patchsets

src/backend/utils/adt/float.c
3 (3 branches)
src/include/utils/float.h
3 (3 branches)
doc/src/sgml/release-18.sgml
2 (2 branches)
doc/src/sgml/release-19.sgml
2 (2 branches)
src/backend/commands/publicationcmds.c
2 (2 branches)
contrib/pg_buffercache/pg_buffercache_pages.c
1 (1 branches)
doc/src/sgml/perform.sgml
1 (1 branches)
doc/src/sgml/protocol.sgml
1 (1 branches)
doc/src/sgml/ref/explain.sgml
1 (1 branches)
src/backend/access/common/reloptions.c
1 (1 branches)
src/backend/access/common/toast_internals.c
1 (1 branches)
src/backend/access/heap/heapam.c
1 (1 branches)
src/backend/access/table/toast_helper.c
1 (1 branches)
src/backend/optimizer/path/allpaths.c
1 (1 branches)
src/backend/optimizer/path/joinpath.c
1 (1 branches)
src/backend/optimizer/plan/subselect.c
1 (1 branches)
src/backend/optimizer/prep/prepjointree.c
1 (1 branches)
src/backend/optimizer/util/pathnode.c
1 (1 branches)
src/backend/postmaster/checkpointer.c
1 (1 branches)
src/backend/replication/logical/relation.c
1 (1 branches)

Verdict coverage in active discussions

patchsets in threads with a message in the last 30 days

Show data
EraWith a verdictNo verdict
bullseye10
trixie46633
unknown011

Patch size against outcome

files touched per patchset, from the recorded submission paths

Show data
Files touchedOutcomePatchsets
unknownsuccess425
unknowncancelled1
unknowntests failed32
unknownbuild failed6
unknowninfra error1
unknownno verdict6
unknownnever applied40

Patchset versions per topic

every version, superseded included; split by whether the topic was committed

Show data
VersionsTopicTopics
1not committed4,980
1committed335
2not committed157
2committed36
3not committed46
3committed8
4not committed14
4committed5
5committed2
5not committed8
6committed1
6not committed5
7not committed4
8not committed2
10committed1
11not committed1

Why the orchestrator won't retry

The dashboard shows the top three of this same breakdown.

ReasonPatchsets
base too old3,118
committed118
patchset touches .github/11