Re: Cirrus CI (Windows help wanted)
On Sun, Jul 31, 2022 at 01:31:58PM -0700, Andres Freund wrote:
But it failed like:
https://cirrus-ci.com/task/5622728425209856?logs=push#L16
[00:09:53.675] unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in:
https://cloud.google.com/container-registry/docs/advanced-authenticationDoes this require tying my github account to a google account ?
Or paying cirrusci ? Or ??Not sure what the problem here is - it worked for me in the past.
I reported the problem to Fedor at cirrusci who had to fix something.
It works now.
On Wed, Aug 03, 2022 at 09:06:05PM -0700, Andres Freund wrote:
On 2022-07-28 17:23:19 -0500, Justin Pryzby wrote:
I think it could be a lot faster to start, since cirrus caches the generated
docker image locally. Rather than (I gather) pulling the image every time.I'm quite certain that is not true. All the docker images built are just
uploaded to the google container registry and then downloaded onto a
*separate* windows host. The dockerfile: stuff generates a separate task
running on a separate machine...
I think you're right. When I used an image built with with Andrew's original
recipe using "dockerfile-as-a-ci-environment" , it still took ~4+ minutes to
start.
Note that there's now a cirrusci image with visual studio 2022, which allows
building postgres. I tried it here.
https://cirrus-ci.com/task/4939320325832704
Scheduled in 01:19
It makes sense that that's faster since it has none of the goodies or
postgres-specific stuff in your image: debugging tools, perl, python, flex,
bison, ssl, zlib, ICU...
But look: https://cirrus-ci.com/task/4953593575899136
=> I installed most of the goodies, but deferred running the installers until
the image is run, and it starts just as quickly. It has to run the installer
each time, rather than once when building the image. That's crummy, but it's
still 1-2 minutes faster than now. Maybe for some of those, it's not needed to
run an installer at all. (Like if the installer can be extracted into
c:\programfiles).
--
Justin
Import Notes
Reply to msg id not found: 20220804040605.izlmfu6wxmuhl7np@awork3.anarazel.de20220731203158.jzexs6exyyttalep@awork3.anarazel.de
Hi,
On 2022-08-18 19:56:56 -0500, Justin Pryzby wrote:
Note that there's now a cirrusci image with visual studio 2022, which allows
building postgres. I tried it here.
In the past there were repeated issues with the cirrus install of visual
studio missing some things, and modifying the install to install them was very
slow, that's why I had switched to installing VS ourselves.
I suspect that the other issue right now is that they updated the host to
a newer version of windows, and when container and host version don't match,
the windows container stuff gets slower starting up.
Why is the build phase so slow in that image? 12min? That's way slower than
the windows builds normally.
=> I installed most of the goodies, but deferred running the installers until
the image is run, and it starts just as quickly. It has to run the installer
each time, rather than once when building the image. That's crummy, but it's
still 1-2 minutes faster than now. Maybe for some of those, it's not needed to
run an installer at all. (Like if the installer can be extracted into
c:\programfiles).
I am strongly against that. For one, the amount of traffic that causes with
the software providers is substantial. For another, the failure rates of
downloading stuff every time are quite high.
I think pruning the container from unnecessary content, and trying to base it
on the vs 2022 image (which is prel-loaded onto the host) is a better plan.
Greetings,
Andres Freund
On Thu, Aug 18, 2022 at 06:09:39PM -0700, Andres Freund wrote:
Why is the build phase so slow in that image? 12min? That's way slower than
the windows builds normally.
Because I'd just rebased it, and it's using ccache, which I mentioned is
several times slower for cache misses, and only ~50% faster for cache hits :(
20220701191841.GH13040@telsasoft.com
There's a patch to use depend mode, which would make cache misses less
expensive.
=> I installed most of the goodies, but deferred running the installers until
the image is run, and it starts just as quickly. It has to run the installer
each time, rather than once when building the image. That's crummy, but it's
still 1-2 minutes faster than now. Maybe for some of those, it's not needed to
run an installer at all. (Like if the installer can be extracted into
c:\programfiles).I am strongly against that. For one, the amount of traffic that causes with
the software providers is substantial. For another, the failure rates of
downloading stuff every time are quite high.
I think you misunderstood. The installers are *retrieved* when the image is
built, and zipfiles are extracted. But for .exes, the installation is deferred
until the image is run.
--
Justin
Hi,
On 2022-08-18 20:17:09 -0500, Justin Pryzby wrote:
On Thu, Aug 18, 2022 at 06:09:39PM -0700, Andres Freund wrote:
Why is the build phase so slow in that image? 12min? That's way slower than
the windows builds normally.Because I'd just rebased it, and it's using ccache, which I mentioned is
several times slower for cache misses, and only ~50% faster for cache hits :(
20220701191841.GH13040@telsasoft.com
That makes it basically unusable - builds being that slow even just
occasionally is prohibitive for something like cfbot I think.
I'm a bit surprised though - I'm fairly certain that that wasn't the case when
I tried it locally (with ninja at least).
=> I installed most of the goodies, but deferred running the installers until
the image is run, and it starts just as quickly. It has to run the installer
each time, rather than once when building the image. That's crummy, but it's
still 1-2 minutes faster than now. Maybe for some of those, it's not needed to
run an installer at all. (Like if the installer can be extracted into
c:\programfiles).I am strongly against that. For one, the amount of traffic that causes with
the software providers is substantial. For another, the failure rates of
downloading stuff every time are quite high.I think you misunderstood. The installers are *retrieved* when the image is
built, and zipfiles are extracted. But for .exes, the installation is deferred
until the image is run.
Ah, that's better.
I still think it's better to not install things every time and reduce the
install footprint though.
Greetings,
Andres Freund