Postgresql + containerization possible use case

On Fri, Dec 10, 2021 at 3:24 AM Achilleas Mantzios <
achill@matrix.gatewaynet.com> wrote:

Hi
we are running some 140 remote servers (in the 7 seas via satellite
connections), and in each one of them we run:
- jboss
- postgresql
- uucp (not as a daemon)
- gpsd
- samba
- and possibly some other services

Hardware and software upgrades are very hard since there is no physical
access to those servers by trained personnel, and also there is a diversity
of software versions.

The idea for future upgrades is to containerize certain aspects of the
software. The questions are (I am not skilled in docker, only minimal
contact with lxd) :
- is this a valid use case for containerization?
- are there any gotchas around postgersql, the reliability of the system ?
- since we are talking about 4+ basic services (pgsqk, jboss, uucp,
samba), is docker a good fit or should we be looking into lxd as well?
- are there any success stories of other after following a similar path?

My experience with LXD is that upon install you are now on a regular update
plan that is impossible to change.
This means that your very expensive data connection will be preempted for
updates at the whim of the
canonical crew. Suggest not using such (most using such on wireless
connections seem to have found
the resultant issues less than wonderful - - cost (on the data connection)
being #1 and the inability to achieve
solid reliability crowding it for #2).

IMO HTH

On 10/12/21 1:24 μ.μ., o1bigtenor wrote:

On Fri, Dec 10, 2021 at 3:24 AM Achilleas Mantzios <achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>> wrote:

Hi
we are running some 140 remote servers (in the 7 seas via satellite connections), and in each one of them we run:
- jboss
- postgresql
- uucp (not as a daemon)
- gpsd
- samba
- and possibly some other services

Hardware and software upgrades are very hard since there is no physical access to those servers by trained personnel, and also there is a diversity of software versions.

The idea for future upgrades is to containerize certain aspects of the software. The questions are (I am not skilled in docker, only minimal contact with lxd) :
- is this a valid use case for containerization?
- are there any gotchas around postgersql, the reliability of the system ?
- since we are talking about 4+ basic services (pgsqk, jboss, uucp, samba), is docker a good fit or should we be looking into lxd as well?
- are there any success stories of other after following a similar path?

Thanks

My experience with LXD is that upon install you are now on a regular update plan that is impossible to change.

Ehhmmm we are running some old versions there already (jboss, pgsql), LXD would not differ in this regard.
What do you mean? that the updates for LXD are huge? short spaced/very regular?
Can you pls elaborate some more on that?

This means that your very expensive data connection will be preempted for updates at the whim of the
canonical crew. Suggest not using such (most using such on wireless connections seem to have found
the resultant issues less than wonderful - - cost (on the data connection) being #1 and the inability to achieve
solid reliability crowding it for #2).

Crew has their own paid service. Business connection is for business not crew.
What I am interested is, could docker be of any use in the above scenario? Containerization in general?
The guys (admins/mgmt) here seem to be dead set on docker, but I have to guarantee some basic data safety requirements.

IMO     HTH

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

On Fri, Dec 10, 2021 at 6:02 AM Achilleas Mantzios <
achill@matrix.gatewaynet.com> wrote:

On 10/12/21 1:24 μ.μ., o1bigtenor wrote:

On Fri, Dec 10, 2021 at 3:24 AM Achilleas Mantzios <
achill@matrix.gatewaynet.com> wrote:

Hi
we are running some 140 remote servers (in the 7 seas via satellite
connections), and in each one of them we run:
- jboss
- postgresql
- uucp (not as a daemon)
- gpsd
- samba
- and possibly some other services

Hardware and software upgrades are very hard since there is no physical
access to those servers by trained personnel, and also there is a diversity
of software versions.

The idea for future upgrades is to containerize certain aspects of the
software. The questions are (I am not skilled in docker, only minimal
contact with lxd) :
- is this a valid use case for containerization?
- are there any gotchas around postgersql, the reliability of the system ?
- since we are talking about 4+ basic services (pgsqk, jboss, uucp,
samba), is docker a good fit or should we be looking into lxd as well?
- are there any success stories of other after following a similar path?

Thanks

My experience with LXD is that upon install you are now on a regular
update plan that is impossible to change.

Ehhmmm we are running some old versions there already (jboss, pgsql), LXD
would not differ in this regard.
What do you mean? that the updates for LXD are huge? short spaced/very
regular?
Can you pls elaborate some more on that?

Updates seem to happen very very regularly.
That means that the system is often tied up with the updating - - - NOT on
doing the function(s).
If there are any issues with the newest and bestest version - - - - well
you get to deal with not
only a hung system (happened a few times whilst I was trying this out (over
a longer period of
time as well)) but a system that isn't doing what you want it to be doing.
I chose to space the updates out to once a month - - - then followed senior
dev team
suggestions to control that and achieved a system that would not update
anything. To make
things even more interesting it was not possible to even remove snapd and
LXD. I was using
rm -r carefully and there was some error message that I no longer remember.
End result was
that I had to blow the system away and reinstall. I'm not a fan of doing
this nor a need to do
such to remove any program I choose to remove. My experiences told me that
the idea behind
this central management (ubuntu controlled updating and upgrading) was most
likely designed
to facilitate a paid serviced from Canonical which income from would cause
a very nice spike
in value to Canonical benefiting only a very tiny number of hands. The dev
team at LXD was
almost shrill in its defense of the 'we know best' thinking that this
behavior depicted. Somehow
running bleeding edge hasn't ever given me reliability. When it comes to
business - - - well I
want things to work - - - I'm not a programmer geek who is forever trying
to 'improve' something.
My existence is not validated by the umpteen hundred versions of my
software available out
there. My existence is better validated by what I can get done - - - - and
not necessarily what
someone else says I have to do right now (even in the middle of the
night!!!).
Does that help?

This means that your very expensive data connection will be preempted for
updates at the whim of the
canonical crew. Suggest not using such (most using such on wireless
connections seem to have found
the resultant issues less than wonderful - - cost (on the data connection)
being #1 and the inability to achieve
solid reliability crowding it for #2).

Crew has their own paid service. Business connection is for business not
crew.
What I am interested is, could docker be of any use in the above scenario?
Containerization in general?

Know nothing about Docker and as a result of my foray into containerization
- - - - well - - - - I'm not a
fan at present. Much more likely to do something like set up a
RaspberryPi and then use that to do things
and if more is needed well - - - I'm considering micro-controllers linked
into SoCs (not necessarily RaspberryPi
but similar) and then possible one central perhaps full size server - - -
but then that server would be busy.
I also am using test systems for any level of system so I'm experimenting
on testing systems and things
don't move to the 'real work horses' until I'm happy that things are stable
and do what I want them to do.
Doesn't necessarily make for cheap but it has upped reliability and reduced
stress (when a primary use
system gets borked - - - whatever the reason - - - - life isn't fun until
its fixed - - - right?).

The guys (admins/mgmt) here seem to be dead set on docker, but I have to
guarantee some basic data safety requirements.

The 'book' says everything is wonderful - - - - if it were me - - - no
guarantees until 'I' am sure.
If they want it - - - - and want you to guarantee it - - - - I wouldn't
touch it myself!! (That's my opinion and
worth all of what you paid for it. I have pile of software installed on my
main use system - - - I'm looking
for good stuff that works and far too much stuff in the programming world
is hype and not function!!)

Regards

On 12/10/21 01:24, Achilleas Mantzios wrote:

Hi

The idea for future upgrades is to containerize certain aspects of the
software. The questions are (I am not skilled in docker, only minimal
contact with lxd) :
- is this a valid use case for containerization?
- are there any gotchas around postgersql, the reliability of the system ?
- since we are talking about 4+ basic services (pgsqk, jboss, uucp,
samba), is docker a good fit or should we be looking into lxd as well?
- are there any success stories of other after following a similar path?

My question is what is it going to change? The software in the
containers still need to be maintained/upgraded and now you have added
maintenance and management of the container software and communication
between containers.

Thank you!

PS

For those who wonder about UUCP, UUCP was our comms solution prior we
installed TCP/IP on the vessels. However, to this day, it provides a
nice management layer, a sort of protection before data leave the vessel
or reach the vessel, in a user controlled manner. So uucp stayed as it
matched exactly the business as far data transfers and emails are
concerned. It would be hard to uniformly manage data transfers and
emails in/out in a plain TCP/IP setup (rsync, ftp, etc,
sendmail/exim4/postfix or other MTA).

--
Adrian Klaver
adrian.klaver@aklaver.com

On Fri, Dec 10, 2021, 2:24 AM Achilleas Mantzios <
achill@matrix.gatewaynet.com> wrote:

Hi
we are running some 140 remote servers (in the 7 seas via satellite
connections)

How are they used? What is in Postgres? Should that all have the exact same
read only data at all times?

On 12/10/21 12:06, Michael Lewis wrote:

On Fri, Dec 10, 2021, 2:24 AM Achilleas Mantzios
<achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>> wrote:

Hi
we are running some 140 remote servers (in the 7 seas via satellite
connections)

How are they used? What is in Postgres? Should that all have the exact
same read only data at all times?

A previous explanation from Achilleas Mantzios , not sure if this is
still exactly how it works:

/messages/by-id/3124961.tmAbAIay6W@smadev.internal.net

--
Adrian Klaver
adrian.klaver@aklaver.com

Interesting. I'm not sure that moving an image for a VM with Postgres is
compatible with the goal of minimizing time/data but if upgrades are yearly
or something, perhaps that is reasonable. It has been 9+ years since that
post as well. But the full data will need to be included in the image if
the goal is not requiring any (significant) tech support at each location,
right? Unless the data that needs to be stored at each site is small, that
seems a bit unlikely to be a feasible option.

On 10/12/21 4:02 μ.μ., o1bigtenor wrote:

The 'book' says everything is wonderful - - - - if it were me - - - no guarantees until 'I' am sure.
If they want it - - - - and want you to guarantee it - - - - I wouldn't touch it myself!! (That's my opinion and
worth all of what you paid for it. I have pile of software installed on my main use system - - - I'm looking
for good stuff that works and far too much stuff in the programming world is hype and not function!!)

Thanks, this is more than true.

Regards

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

Hi Adrian

On 10/12/21 6:01 μ.μ., Adrian Klaver wrote:

On 12/10/21 01:24, Achilleas Mantzios wrote:

Hi

The idea for future upgrades is to containerize certain aspects of the software. The questions are (I am not skilled in docker, only minimal contact with lxd) :
- is this a valid use case for containerization?
- are there any gotchas around postgersql, the reliability of the system ?
- since we are talking about 4+ basic services (pgsqk, jboss, uucp, samba), is docker a good fit or should we be looking into lxd as well?
- are there any success stories of other after following a similar path?

My question is what is it going to change? The software in the containers still need to be maintained/upgraded and now you have added maintenance and management of the container software and
communication between containers.

I think the rationale is that sysadms may upgrade system and kernel without caring about breaking things, the docker images of the respective services (jboss, pgsql, exim4, etc..) will evolve (more)
independently.

Thank you!

PS

For those who wonder about UUCP, UUCP was our comms solution prior we installed TCP/IP on the vessels. However, to this day, it provides a nice management layer, a sort of protection before data
leave the vessel or reach the vessel, in a user controlled manner. So uucp stayed as it matched exactly the business as far data transfers and emails are concerned. It would be hard to uniformly
manage data transfers and emails in/out in a plain TCP/IP setup (rsync, ftp, etc, sendmail/exim4/postfix or other MTA).

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

Hi Michael, Adrian

On 10/12/21 11:28 μ.μ., Adrian Klaver wrote:

On 12/10/21 12:06, Michael Lewis wrote:

On Fri, Dec 10, 2021, 2:24 AM Achilleas Mantzios <achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>> wrote:

    Hi
    we are running some 140 remote servers (in the 7 seas via satellite
    connections)

How are they used? What is in Postgres? Should that all have the exact same read only data at all times?

A previous explanation from Achilleas Mantzios , not sure if this is still exactly how it works:

/messages/by-id/3124961.tmAbAIay6W@smadev.internal.net

Thank you Adrian, the situation is the similar with the one you posted above (in 2012), albeit sizes/number of remote servers has increased.

In order to respond to Michael,
Remote pgsql servers are masters. A large majority of tables is read-only (by software, not enforced by other means), but those servers they have their very important write functions as well. And the
data are individual for any of those 100+ servers. There are no two identical DBs in the whole topology. Every server has its own data specific for this vessel.

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

On 11/12/21 6:31 μ.μ., Michael Lewis wrote:

Interesting. I'm not sure that moving an image for a VM with Postgres is compatible with the goal of minimizing time/data but if upgrades are yearly or something, perhaps that is reasonable. It has
been 9+ years since that post as well. But the full data will need to be included in the image if the goal is not requiring any (significant) tech support at each location, right? Unless the data
that needs to be stored at each site is small, that seems a bit unlikely to be a feasible option.

The data are independent, read-write, and different for each server. Basically they (sysadm ppl) want to facilitate system/kernel upgrades, which is a must as old hardware is harder to support with
newest kernel versions.
Their idea is for exim4, pgsql, jboss to reside in separate docker images.
However I asked for a detailed description of the problem, in order to focus more formally on the problem.
I know docker and postgresql has been a hot topic for quite some time. First presentations said ... dont do it, but lately we see more and more positive experiences, articles, success stories.

Pgsql major versions and data go hand in hand, I hate to give them the illusion that separating postgresql from data is something to consider.
Serious thought is needed, for sure, I am just thinking that dockerizing exim4, jboss, and other services might be a valid idea, while still keeping postgresql and data on the host ?

I had so much troubles because of unreliable system setup in the past, that I'd hate to repeat the same (or similar) mistakes again.

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

Am 10.12.2021 um 13:01 schrieb Achilleas Mantzios <achill@matrix.gatewaynet.com>:

On 10/12/21 1:24 μ.μ., o1bigtenor wrote:

On Fri, Dec 10, 2021 at 3:24 AM Achilleas Mantzios <achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>> wrote:
Hi
we are running some 140 remote servers (in the 7 seas via satellite connections), and in each one of them we run:
- jboss
- postgresql
- uucp (not as a daemon)
- gpsd
- samba
- and possibly some other services

Hardware and software upgrades are very hard since there is no physical access to those servers by trained personnel, and also there is a diversity of software versions.

The idea for future upgrades is to containerize certain aspects of the software. The questions are (I am not skilled in docker, only minimal contact with lxd) :
- is this a valid use case for containerization?
- are there any gotchas around postgersql, the reliability of the system ?
- since we are talking about 4+ basic services (pgsqk, jboss, uucp, samba), is docker a good fit or should we be looking into lxd as well?
- are there any success stories of other after following a similar path?

Thanks

My experience with LXD is that upon install you are now on a regular update plan that is impossible to change.

Ehhmmm we are running some old versions there already (jboss, pgsql), LXD would not differ in this regard.
What do you mean? that the updates for LXD are huge? short spaced/very regular?
Can you pls elaborate some more on that?

IIRC, you can’t really control, which updates are installed for LXD (and snap). You can’t create a local mirror.

IIRC, you can delay snap updates, but you can’t really reject them.

Maybe you can these days, with landscape server?

(insert the usual rant about Enterprise != Ubuntu here)

I don’t know about LXD, but as it’s only running on Ubuntu and is apparently developed by a single guy (who might or might not work for Canonical - sorry, too lazy to check), I wouldn’t hold my breath as to its long-term viability.

Ubuntu will probably morph into a container-only, cloud-only OS sooner than later - the writing is on the wall (IMHO).

This means that your very expensive data connection will be preempted for updates at the whim of the
canonical crew. Suggest not using such (most using such on wireless connections seem to have found
the resultant issues less than wonderful - - cost (on the data connection) being #1 and the inability to achieve
solid reliability crowding it for #2).

Crew has their own paid service. Business connection is for business not crew.

The word „crew“ was meant to say „employees of Canonical“ - I’m sure the allegory was not meant to mess with you...

What I am interested is, could docker be of any use in the above scenario? Containerization in general?
The guys (admins/mgmt) here seem to be dead set on docker, but I have to guarantee some basic data safety requirements.

I know very little about docker, but IMO, for ultimate stability, you could switch to RHEL and use their certified images:

https://catalog.redhat.com/software/containers/search?q=PostgreSQL%2012&amp;p=1

My coworker says, he re-packages all his docker-images himself (with RPMs from his own mirror), so that he understands what’s really in them.

The big problem that I see with your use-case and docker is that docker implies frequent, small updates to the whole stack - including docker itself (unless you pay for the LTS version).

This is not what you do right now, I reckon?

The question is: do you want to get there?
But maybe your developers want to get here, because they don’t want to learn about software-packaging (anymore) - but is that what the business wants?

https://www.docker.com/blog/how-carnival-creates-customized-guest-experiences-with-docker/

(That was pre-pandemic…)

I would make an educated guess that you’d need to have the whole docker-infrastructure on each ship (build-server, repository etc.pp.) to minimize sat-com traffic.

I mean, it looks like it could be done. But this is where the „dev“ part in the „devops" world has to take a step back and the „ops“ guys need to come forward.

Rainer

Hi Rainer,

On 13/12/21 11:50 π.μ., Rainer Duffner wrote:

Am 10.12.2021 um 13:01 schrieb Achilleas Mantzios <achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>>:

On 10/12/21 1:24 μ.μ., o1bigtenor wrote:

On Fri, Dec 10, 2021 at 3:24 AM Achilleas Mantzios <achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>> wrote:

Hi
we are running some 140 remote servers (in the 7 seas via satellite connections), and in each one of them we run:
- jboss
- postgresql
- uucp (not as a daemon)
- gpsd
- samba
- and possibly some other services

Hardware and software upgrades are very hard since there is no physical access to those servers by trained personnel, and also there is a diversity of software versions.

The idea for future upgrades is to containerize certain aspects of the software. The questions are (I am not skilled in docker, only minimal contact with lxd) :
- is this a valid use case for containerization?
- are there any gotchas around postgersql, the reliability of the system ?
- since we are talking about 4+ basic services (pgsqk, jboss, uucp, samba), is docker a good fit or should we be looking into lxd as well?
- are there any success stories of other after following a similar path?

Thanks

My experience with LXD is that upon install you are now on a regular update plan that is impossible to change.

Ehhmmm we are running some old versions there already (jboss, pgsql), LXD would not differ in this regard.
What do you mean? that the updates for LXD are huge? short spaced/very regular?
Can you pls elaborate some more on that?

IIRC, you can’t really control, which updates are installed for LXD (and snap). You can’t create a local mirror.

IIRC, you can delay snap updates, but you can’t really reject them.

Maybe you can these days, with landscape server?

(insert the usual rant about Enterprise != Ubuntu here)

I don’t know about LXD, but as it’s only running on Ubuntu and is  apparently developed by a single guy (who might or might not work for Canonical - sorry, too lazy to check), I wouldn’t hold my
breath as to its long-term viability.

Ubuntu will probably morph into a container-only, cloud-only OS sooner than later - the writing is on the wall (IMHO).

All notes taken, thank you.

This means that your very expensive data connection will be preempted for updates at the whim of the
canonical crew. Suggest not using such (most using such on wireless connections seem to have found
the resultant issues less than wonderful - - cost (on the data connection) being #1 and the inability to achieve
solid reliability crowding it for #2).

Crew has their own paid service. Business connection is for business not crew.

The word „crew“ was meant to say „employees of Canonical“ - I’m sure the allegory was not meant to mess with you...

What I am interested is, could docker be of any use in the above scenario? Containerization in general?
The guys (admins/mgmt) here seem to be dead set on docker, but I have to guarantee some basic data safety requirements.

I know very little about docker, but IMO, for ultimate stability, you could switch to RHEL and use their certified images:

https://catalog.redhat.com/software/containers/search?q=PostgreSQL%2012&amp;p=1

My coworker says, he re-packages all his docker-images himself (with RPMs from his own mirror), so that he understands what’s really in them.

The big problem that I see with your use-case and docker is that docker implies frequent, small updates to the whole stack - including docker itself (unless you pay for the LTS version).

This is not what you do right now, I reckon?

Our setup has been open source since forever. So licenses for something that used to be free for ages would be hard to introduce.
So Docker is NOT free? Please share your thoughts? I am a complete noob.
Those servers I am talking about have no internet connectivity. And the satellite connection costs are high.
(although I think we pay a fixed amount for a certain total data transfer size).

The question is: do you want to get there?
But maybe your developers want to get here, because they don’t want to learn about software-packaging (anymore) - but is that what the business wants?

Those servers live for years, the objective is to facilitate upgrades.

https://www.docker.com/blog/how-carnival-creates-customized-guest-experiences-with-docker/

Thanks for the link, I didn't quite understand what they do with docker (video included).
120 docker containers in two data centers on the ship? Ours will be just a single linux box with limited connectivity (in some seas no connectivity ) to the internet/shore.

(That was pre-pandemic…)

I would make an educated guess that you’d need to have the whole docker-infrastructure on each ship (build-server, repository etc.pp.) to minimize sat-com traffic.

Hmm, I don't know about that. The hardware is given (existing) and limited.
You are like the 2nd person who warned about comms as being an issue with docker/containers.

Can't someone have a free docker system inside a linux server and run the containers (free again) until he/she decides to upgrade either docker or (more frequently) one of the images?
Is Docker upwards compatible? Meaning new Docker versions to run old images ?

I mean, it looks like it could be done. But this is where the „dev“ part in the „devops" world has to take a step back and the „ops“ guys need to come forward.

Can you please explain in more detail?

Rainer

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

Am 13.12.2021 um 12:41 schrieb Achilleas Mantzios <achill@matrix.gatewaynet.com>:

Our setup has been open source since forever. So licenses for something that used to be free for ages would be hard to introduce.

That ist totally understandable.
140x800 for the RHEL license alone is over 100k/year.
Though you might get a volume discount at that point ;-)

So Docker is NOT free? Please share your thoughts? I am a complete noob.

The commercial version is not free.
The „CE“ version is free, but I’m not sure how stable the API is.

Dockerhub, for what it’s worth, isn’t completely free anymore either.

Those servers I am talking about have no internet connectivity. And the satellite connection costs are high.
(although I think we pay a fixed amount for a certain total data transfer size).

Yes. That’s why using a more offline-friendly infrastructure might make more sense.

RHEL seems to be pretty well tuned running in high-secure air-gapped networks - which is what a ship basically is.

The question is: do you want to get there?
But maybe your developers want to get here, because they don’t want to learn about software-packaging (anymore) - but is that what the business wants?

Those servers live for years, the objective is to facilitate upgrades.

Yes, but docker-upgrades aren’t free either.

At least, I cannot imagine running such an infrastructure in the gung-ho style that a typical developer runs his docker-containers.

You’d want to run a tight ship with those ;-)

You will need a lot more tooling around this (continuous integration, continuous deployment) - which is the reason I suggested moving all this infrastructure to the ship itself. If you produce and mirror the artifacts locally, you should have less backhaul traffic (which is what I assume is killing you with sat-com - downstream can probably be had cheap-ish these days - or pretty soon via the likes of StarLink)

https://www.docker.com/blog/how-carnival-creates-customized-guest-experiences-with-docker/ <https://www.docker.com/blog/how-carnival-creates-customized-guest-experiences-with-docker/&gt;Thanks for the link, I didn't quite understand what they do with docker (video included).

It’s passenger entertainment and engagement, from what I can see.

It’s sad and funny to read, as shortly after that huge cruises got out of fashion ;-)

It’s also not mission-critical.

120 docker containers in two data centers on the ship? Ours will be just a single linux box with limited connectivity (in some seas no connectivity ) to the internet/shore.

Yes, and this is IMO where you will have to get management on board to upgrade infrastructure and tooling around docker containers (and likely Kubernetes) - if your developers persist on using it that way.

You will end up running a small server-cluster on each ship - I would guess there are specialized vendors who produce sea-worthy server-equipment.

(That was pre-pandemic…)

I would make an educated guess that you’d need to have the whole docker-infrastructure on each ship (build-server, repository etc.pp.) to minimize sat-com traffic.

Hmm, I don't know about that. The hardware is given (existing) and limited.
You are like the 2nd person who warned about comms as being an issue with docker/containers.

Can't someone have a free docker system inside a linux server and run the containers (free again) until he/she decides to upgrade either docker or (more frequently) one of the images?
Is Docker upwards compatible? Meaning new Docker versions to run old images ?

Yes, likely. But upgrading docker itself sometimes comes with its own challenges.

As you said, all these servers are with limited connectivity and no local help available...

Again, I’m the last person you want to ask how docker actually works (but often, the developers themselves don’t know either, but they know which commands to feed it…)

Also, most of my knowledge of ships is from watching NCIS (and reading Clive Cussler novels...).

I mean, it looks like it could be done. But this is where the „dev“ part in the „devops" world has to take a step back and the „ops“ guys need to come forward.

Can you please explain in more detail?

With 140-ish servers air-gapped on vessels around the world, this is IMO a serious operations-problem and needs to be handled properly, with an air-tight (or water-tight…) operational concept.

Your infrastructure has outgrown its original design limits. Time to talk to management about upping the game (and the budget).

Best Regards

On 12/13/21 03:41, Achilleas Mantzios wrote:

Hi Rainer,

On 13/12/21 11:50 π.μ., Rainer Duffner wrote:

So Docker is NOT free? Please share your thoughts? I am a complete noob.

Take a look here:

https://www.docker.com/pricing

Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

--
Adrian Klaver
adrian.klaver@aklaver.com

Hi Berto,

On 13/12/21 3:03 μ.μ., Bèrto ëd Sèra wrote:

On Mon, 13 Dec 2021 at 12:41, Achilleas Mantzios <achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>> wrote:

Hmm, I don't know about that. The hardware is given (existing) and limited.
You are like the 2nd person who warned about comms as being an issue with docker/containers.

Quick question: do those ships ever touch port? If so, it's not a fact that you have to rely on sat com to do updates. Just keep an on-shore table of updates and do the updates every time a ship is
in a port and can use a normal port wifi com. IMHO this would be the safest option EVEN if satcom was free of cost, as it will never be half as stable as a land based com line.

Yes they do touch port, otherwise they would be of little commercial value!
I am confused here : Are we talking about traffic due to :
a) normal business traffic (designed by me/us old system based on DBMirror)
b) system updates due to some decision of the system itself and/or our team
c) normal app shipping via new images, or new .ear/.war or whatever
?

And since you are not deploying the same thing everywhere (you quote different HW everywhere, so hardly one image could be optimised for everything), you are going to need to test upgrades for each
ship, which is both dangerous and expensive, if done on sat com.

We are deploying the same exact app everywhere (jboss), but of course exim4, etc other services will be specific for every ship.

My 5p
Berto

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

Check out rolling updates with kubernetis and yes, a containerized
postgresql environment is likely the best option. The really nice bit about
containers is that you don't actually care about underlying hardware. You
need a docker daemon. K8 manages clusters and it sounds like your use case
would benefit from that.

On Mon, Dec 13, 2021, 11:03 AM Achilleas Mantzios <
achill@matrix.gatewaynet.com> wrote:

Hi
On 13/12/21 6:11 μ.μ., Benedict Holland wrote:

Check out rolling updates with kubernetis and yes, a containerized postgresql environment is likely the best option. The really nice bit about containers is that you don't actually care about
underlying hardware. You need a docker daemon. K8 manages clusters and it sounds like your use case would benefit from that.

Thanks, postgresql's bigger challenges as we all know will be upgrading OIDs, upgrading the JDBC related java code, and maybe many more related challenges. I won't say which, but we are running an
unsupported version of pgsql on the ships.
So apps are much more frequently upgraded than the system of pgsql.
I will check out rolling upgrades with kubernetes. (are we talking BDR here?)

On Mon, Dec 13, 2021, 11:03 AM Achilleas Mantzios <achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>> wrote:

Hi Berto,

On 13/12/21 3:03 μ.μ., Bèrto ëd Sèra wrote:

On Mon, 13 Dec 2021 at 12:41, Achilleas Mantzios <achill@matrix.gatewaynet.com <mailto:achill@matrix.gatewaynet.com>> wrote:

Hmm, I don't know about that. The hardware is given (existing) and limited.
You are like the 2nd person who warned about comms as being an issue with docker/containers.

Quick question: do those ships ever touch port? If so, it's not a fact that you have to rely on sat com to do updates. Just keep an on-shore table of updates and do the updates every time a
ship is in a port and can use a normal port wifi com. IMHO this would be the safest option EVEN if satcom was free of cost, as it will never be half as stable as a land based com line.

Yes they do touch port, otherwise they would be of little commercial value!
I am confused here : Are we talking about traffic due to :
a) normal business traffic (designed by me/us old system based on DBMirror)
b) system updates due to some decision of the system itself and/or our team
c) normal app shipping via new images, or new .ear/.war or whatever
?

And since you are not deploying the same thing everywhere (you quote different HW everywhere, so hardly one image could be optimised for everything), you are going to need to test upgrades for
each ship, which is both dangerous and expensive, if done on sat com.

We are deploying the same exact app everywhere (jboss), but of course exim4, etc other services will be specific for every ship.

My 5p
Berto

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

On 13/12/21 5:55 μ.μ., Adrian Klaver wrote:

On 12/13/21 03:41, Achilleas Mantzios wrote:

Hi Rainer,

On 13/12/21 11:50 π.μ., Rainer Duffner wrote:

So Docker is NOT free? Please share your thoughts? I am a complete noob.

Take a look here:

https://www.docker.com/pricing

Thank you Adrian!

Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt

--
Achilleas Mantzios
DBA, Analyst, IT Lead
IT DEPT
Dynacom Tankers Mgmt