Virtual machines (VMs) provide good isolation for multiple applications running on
the same server hardware. But VMs are not perfect. They take a long time to boot
up, must be all kept up to date with patches, and the redundant software takes
lots of unnecessary storage and memory (and hence, money). Developers need to
know details such as setting up security and networking for the VM’s operating
system.

Datacenters today are shifting away from VMs. Instead, they run containers on
the physical servers. Containers are self-contained execution environments
(not the OS though), each with its own private resources such as CPU, memory,
block I/O, and NICs (network connections).

Compared to a VM, a container is very lightweight. The physical servers run
the Linux OS instead of a hypervisor. (To run containers on a Windows OS, a
Linux VM is needed to in turn run the containers.) That one OS has the ability
to isolate a set of processes from other such sets. Each set of processes,
together with a private filesystem, is a container. Because they are simple
processes (and some setup and configuration info), containers are fast to
start. There is no redundant storage wasted for the OS. You do not run a
separate OS for each.

While many operating systems supported containers in the past, creating,
deploying, and managing them was a pain. Docker changed that by defining a
Docker (or container) image, which is composed of several immutable sets of
files, overlaid on top of each other, with only the topmost layer writable by
the running container. This clever design means that each container can be
self-contained with all the DLLs, services, and files it needs. Because of
the overlay structure, if you had 100 containers all using the same (for
example) MariaDB files, you only need storage for one copy of that! Docker
assembles the container image at runtime.

That topmost writable layer is ephemeral; any additions or changes to files
done by the running container are lost once the container stops! For
persistent storage, you need to attach a storage volume (commonly, just
“volume”) to write to, or use a remote storage service (the sort provided by
clouds). You need to keep this in mind as you develop software for containers!

Note: You can however build a container image a step at a time: start your
container and add/change whatever files you wish. When you stop the
container, the state is not reset immediately. You can find the just-updated
container and do a commit to update the corresponding container image. You
can repeat this many times to build a container image a bit at a time.

In addition to having tools to create and generally manage Docker images,
Docker maintains a repository of popular prebuilt images that are ready to
use. (Your organization can also use private repositories of images, known as
registries.) Finally, Docker provides simple tools to start containers from
the downloaded or built images and to manage all running containers. All the
lowest level, the virtualization work is handled by Docker so programmers need
not worry about it.

Delivering an application using a container is called a containerized
application. Such applications are self-contained and can work on nearly any
system without porting! Java works well with containers.

Containers are also useful for running untrusted code. (Those web sites that
allow you to upload code and run it generally use containers to do so.)

It is not unusual to have a container with all your build tools and
development environment, including all testing and code review tools. To work
on software, you would launch your IDE in this container. The output is used
to build the containerized app (a separate container). This method ensures
all developers use the exact same development environment, and that this
environment can be recreated exactly, years later if needed, to update old code.

CI/CD setups make such work-flows nearly painless: you fire up your
containerized IDE, work on code, and do a push to the shared repo when ready.
Every other step (except for human code review) is or can be automated.

Note: There are some additional steps, including building the application
container, testing that (as opposed to building and testing the app directly
which you still do), cleaning and minimizing the app’s container image,
digitally signing it, and adding it to a container registry (a repo of
container images).

Sometimes a service is composed of several containers working together and
which depend on either other. A container orchestration tool is used to
manage these sets of containers (called a pod) and to monitor and manage
them. This also needs to be defined and setup. A complex but common tool for
this today is called Kubernetes. (For example, an application service using 3
web servers, 3 application servers, 2 database servers, and a log server, and
maybe a storage volume, might all form a single pod.)

Don’t let all the complexity put you off. You can get started easily by
installing Docker and using a prebuilt container image for plain JDK Java,
Maven or Gradle Java, Java EE (often using Tomcat), Spring, etc.: create and
test a Java app or service in the they way you’ve always done, commit the
change to the container image, and you’re done.

Of course, Docker isn’t the only container system out there, there are lots.
But Docker is by far the most popular and has contributed most of their stuff
to the open container initiative, which is striving to make such containers
portable and not dependent on Linux.

Currently Linux is required. You can use Docker on Windows by running a Linux
VM that in turn runs all the containers. Like datacenters of VMs, datacenters
of containers have the same management and monitoring needs. Only they use
different software for this, often called container orchestration engines
(COEs). Examples include Docker Swarm, Kubernetes, and Mesos.

Hope that was accurate and useful.

--
Wayne

On 2021-12-20 21:10, Wayne wrote:
> It is not unusual to have a container with all your build tools and
> development environment, including all testing and code review tools.
> To work
> on software, you would launch your IDE in this container. The output is
> used
> to build the containerized app (a separate container). This method
ensures
> all developers use the exact same development environment, and that this
> environment can be recreated exactly, years later if needed, to update
> old code.
>
> CI/CD setups make such work-flows nearly painless: you fire up your
> containerized IDE, work on code, and do a push to the shared repo when
> ready.
> Every other step (except for human code review) is or can be automated.
I've seen many cases where the build/test phase would be executed inside
a container. The build might yield an executable or an image for runtime
container.

But I've never seen the case where the development itself ("launch your
IDE in a container") would take place in a container. Can people confirm
that this is an established practice? Docker containers generally don't
have display ports...

> Currently Linux is required. You can use Docker on Windows by running a
> Linux
> VM that in turn runs all the containers.
Docker is supported on Windows 10 and MacOS.

--
DF.

On 12/30/2021 7:41 AM, Daniele Futtorovic wrote:
>
> On 2021-12-20 21:10, Wayne wrote:
> > It is not unusual to have a container with all your build tools and
> > development environment, including all testing and code review tools.
> > To work
> > on software, you would launch your IDE in this container.  The output is
> > used
> > to build the containerized app (a separate container).  This method ensures
> > all developers use the exact same development environment, and that this
> > environment can be recreated exactly, years later if needed, to update
> > old code.
> >
> > CI/CD setups make such work-flows nearly painless: you fire up your
> > containerized IDE, work on code, and do a push to the shared repo when
> > ready.
> > Every other step (except for human code review) is or can be automated.
> I've seen many cases where the build/test phase would be executed inside a container.
> The build might yield an executable or an image for runtime container.
>
> But I've never seen the case where the development itself ("launch your IDE in a
> container") would take place in a container. Can people confirm that this is an
> established practice? Docker containers generally don't have display ports...
>
> > Currently Linux is required.  You can use Docker on Windows by running a
> > Linux
> > VM that in turn runs all the containers.
> Docker is supported on Windows 10 and MacOS.
>

The thing is, Linux containers *do* support running GUIs from containers, but
I don't think native Windows containers do (at least not yet). You could also
use a VM for development instead of a container, and I've heard of that (using
Vagrant), but containers with Eclipse, STS4, or IntelliJ are smaller and
faster to launch, so that makes them a better choice. But to be honest, I
cannot find my references to someone claiming to do this; I just remember
reading about it.

--
Wayne

On Thu, 30 Dec 2021 13:41:50 +0100, Daniele Futtorovic wrote:

> But I've never seen the case where the development itself ("launch your
> IDE in a container") would take place in a container. Can people confirm
> that this is an established practice? Docker containers generally don't
> have display ports...

There are various IDE/dev solutions that are realized via a web interface
(check out Eclipse Che, or the integrated GitLab web IDE), and those
certainly can (and probably should) be run as a container. But I would
agree that those are the exceptions, not the rule.

There is also the option of running something X-ish in a Docker container
and then connecting to it via the remote X protocol, but I think only
madness lies that way.

Liebe Gruesse,
Joerg

--
Ich lese meine Emails nicht, replies to Email bleiben also leider
ungelesen.

Wayne <wayne@nospam.invalid> writes:
> On 12/30/2021 7:41 AM, Daniele Futtorovic wrote:
>> On 2021-12-20 21:10, Wayne wrote:
[...]
>> But I've never seen the case where the development itself ("launch
>> your IDE in a container") would take place in a container. Can
>> people confirm that this is an established practice? Docker
>> containers generally don't have display ports...
[...]
> The thing is, Linux containers *do* support running GUIs from containers, but
> I don't think native Windows containers do (at least not yet). You could also
> use a VM for development instead of a container, and I've heard of that (using
> Vagrant), but containers with Eclipse, STS4, or IntelliJ are smaller and
> faster to launch, so that makes them a better choice. But to be honest, I
> cannot find my references to someone claiming to do this; I just remember
> reading about it.

They are doing it at my current company. Those that I know of use
vscode. The way I know is that I had the same question about whether
the Xserver would allow connecting to the display from a potentially
different userid in the container. It's probably the correct one from
the outside. I don't know any details, it just works.

Michael

On 2022-01-06 20:16, Michael Jung wrote:
> Wayne <wayne@nospam.invalid> writes:
>> On 12/30/2021 7:41 AM, Daniele Futtorovic wrote:
>>> On 2021-12-20 21:10, Wayne wrote:
> [...]
>>> But I've never seen the case where the development itself ("launch
>>> your IDE in a container") would take place in a container. Can
>>> people confirm that this is an established practice? Docker
>>> containers generally don't have display ports...
> [...]
>> The thing is, Linux containers *do* support running GUIs from containers, but
>> I don't think native Windows containers do (at least not yet). You could also
>> use a VM for development instead of a container, and I've heard of that (using
>> Vagrant), but containers with Eclipse, STS4, or IntelliJ are smaller and
>> faster to launch, so that makes them a better choice. But to be honest, I
>> cannot find my references to someone claiming to do this; I just remember
>> reading about it.
>
> They are doing it at my current company. Those that I know of use
> vscode. The way I know is that I had the same question about whether
> the Xserver would allow connecting to the display from a potentially
> different userid in the container. It's probably the correct one from
> the outside. I don't know any details, it just works.

How's the experience? Would you recommend it?

--
DF.

On Mon, 10 Jan 2022 17:47:49 +0100, Daniele Futtorovic wrote:

> How's the experience? Would you recommend it?
>
Equally important:

1) how much time does it take to rebuild/retest/roll out a new version of
the container whenever a new version of any of it's components is
released and how often does this happen?

2) how does the Container Maintainer like doing that job?

Martin Gregorie <martin@mydomain.invalid> writes:
> On Mon, 10 Jan 2022 17:47:49 +0100, Daniele Futtorovic wrote:
>> How's the experience? Would you recommend it?

Mixed experience. It depends on how you handle changes (see below).

> 1) how much time does it take to rebuild/retest/roll out a new version of
> the container whenever a new version of any of it's components is
> released and how often does this happen?

The CI systems are busy, that's clear. They have changed the setup and
extracted the most variable parts into a separate container, so I think
they have two running in tandem now. But to be more precise, they did
the 1-container-dev-env about a year ago and I have lost track since. I
only know that they also used this to "archive" their dev environment
for some regulatory compliance reasons. At the height of this pattern
they created a version every night and the needed storage space went
through the roof. We only kept one internal release per week or so.
The situation is more relaxed now.

> 2) how does the Container Maintainer like doing that job?

That was mainly automated. It's the build servers, network, and storage
space that were aching.

They are (and were) not using Java at all and their environments look
different than the ones that are common for Java, so their problems with
this approach also may be different. (Still hoping that I didn't answer
the original question in an off-topic way.)

Michael