[DISCUSSION] Adopting Docker for OFBiz - what are the objectives?

[Eugen Stan]

Hi,

There has been some discussion regarding Docker and OFBiz, however no
consensus yet.
I'm starting this thread after discussions on Slack with Jacques Le
Roux, Daniel Watford and Michael Brohl.

The the aim to establish some goals / objectives regarding Docker and OFBiz.

Please add your feedback, comments and suggestions.

Prior work regarding this is found

https://issues.apache.org/jira/browse/OFBIZ-10407
https://lists.apache.org/thread.html/r40fd679818a37e113b469add51755b1097a2b02d3961e71a2cfe928d%40%3Cdev.ofbiz.apache.org%3E

and in the links stemming from the links above.

----

== How can we integrate Docker in OFBiz ?

Docker can be used in two distinct ways:

a. Use Docker as a way to build OFBiz components - this will make builds
more portable - as long as people have Docker (or containerd or podman)
installed locally, they will be able to build OFBiz "for sure" (tm).

This aims to solve the issue of people not having  the proper JDK and
required tools installed (gradle, git ?! etc).

b. Use Docker to deploy OFBiz for production purposes (and demos)
IMO this means building a slim Docker image with only JRE and OFBiz + a
custom selection of plugins. IMO this is best achieved with pre-built,
published binaries for ofbiz.

c. Use Docker to develop / debug OFBiz
I'm not sure if this is really a thing since IMO falls into b).


Using docker multi stage image builds is something that could help.

The goals are sometimes at odds with one another in the sense that doing
something to fix a goal will hinder the other.

Do you see other use-cases for Docker and OFBiz?


== My personal take

Personally I would like to focus on b). I did not have good experience
with building sources with Docker - I could not get used to the workflow.

Also when I deploy to production, I don't really care how it's built as
long as I have the binary and I can run just that.

So we might end up with two or more Dockerfiles, each focusing on
specific objectives.



Regards,
--
Eugen Stan
[phone] / netdava.com

[Daniel Watford]

Hi Eugen,

My personal use cases for using Docker and Ofbiz are:
- Development workflow.
- Production and test builds incorporating custom plugins.
- Use as part of immutable infrastructure.

== Development workflow

I often use docker containers when testing PRs, but building a complete
docker image with demo data loaded is time consuming process.

To reduce build time I checkout trunk and then build a base docker image
using my workspace with all demo data loaded into the integrated Derby
database.
I then checkout the PR and build a second docker image which extends from
the first image, incorporating the differences introduced by the PR.
Because the changes from the PR are normally small and don't require a
change to demo data, building this second docker image is a very quick
process.

I can then quickly spin up a container based on the second image and test.
Once testing is complete I can kill the container, knowing that I have an
unmodified set of demo data still resident in the base image.

Using docker for development builds and testing works well for me since I
can isolate any tools needed for an ofbiz build in the container, rather
than needing to install them on my development host.

The approach of keeping development tools in containers has been working
well across a variety of projects for me targeting java,
javascript/typescript, node.js, sass, php, etc.
Docker has been incredibly useful for creating environments for developing
WordPress sites. Here [1] is a docker-compose project used for building a
wordpress theme for a local brewery [2], tested against
a MySQL database, Apache running with TLS certificates, and a gulp process
running to detect changes to and compile SCSS to CSS.

The good thing about the above is that as soon as I stop working on one
project I can tear down any associated resources (such as DB and HTTP
servers) and move on to the next project.


== Production and test builds

I want to be able to create custom ofbiz docker images incorporating
client-specific plugins.

For a recent project I was able to spin up a Virtual Private Server and
deploy to it customised ofbiz images which included my client's plugin and
data. Deployments would be triggered
by check ins to either the ofbiz-framework repository or to the custom
plugin's repository. Automatic deployments to the server meant the client
could get regular updates on the project's
process and support their decision making on were to focus development next.

The test environment was supported by the Traefik reverse-proxy (also
running in a container) which handled certificate updates with LetsEncrypt
and could restrict access to Ofbiz by IP address.

Having a way to more easily create these docker images with Ofbiz would be
very helpful to me. I can then publish images to a private client-specific
registry, allowing the client to pull the image into their on-premises
production infrastructure.


== Immutable infrastructure

I often need to support clients while having limited access to their
production environment. Normally because they are located some distance
from me and remote-access to their on-premises infrastructure is often
difficult to arrange.
Therefore having builds of client-specific software which can be treated as
part of their immutable infrastructure is very helpful.

As mentioned above, I can create client-specific docker images and deploy
them to test environments. Once a test build is approved by a client they
should be able to pull that image and have it useable within their
production environment with the minimum of fuss. This means any local
configuration needs to be externalised from the docker image, using
environment variables or configuration files bind-mounted into the running
ofbiz container.

When new versions of ofbiz are available, the client should be able to pull
the latest docker image and have it operational with existing
configuration, assuming there are no incompatibilities with the latest
version. This might make deploying security updates to previous ofbiz
releases easier for users.

We could package up variants of ofbiz as docker images which contain
everything needed to run, including database drivers that have been tested
by the community and are considered 'supported'. Users of the docker image
can then be confident that they have all components needed to run ofbiz
without further downloads needed. This could be useful for air-gapped
deployments where manual steps are required to bring data onto a network.
Once the relevant docker image is imported, ofbiz can be used on air-gapped
systems.

Docker images containing demo data in a Derby database can be produced and
deployed to demo hosts. This would allow users to run demonstration
versions of ofbiz themselves without building, but might make
administration of our demo environments easier. Couple this with a
reverse-proxy like Traefik and we'll be able to run multiple demo versions
on a single host.

[1] https://github.com/danwatford/wb-wp-docker-compose
[2] https://wantsumbrewery.co.uk/

On Tue, 1 Dec 2020 at 18:28, Eugen Stan <[hidden email]> wrote:

> Hi,
>
> There has been some discussion regarding Docker and OFBiz, however no
> consensus yet.
> I'm starting this thread after discussions on Slack with Jacques Le
> Roux, Daniel Watford and Michael Brohl.
>
> The the aim to establish some goals / objectives regarding Docker and
> OFBiz.
>
> Please add your feedback, comments and suggestions.
>
> Prior work regarding this is found
>
> https://issues.apache.org/jira/browse/OFBIZ-10407
>
> https://lists.apache.org/thread.html/r40fd679818a37e113b469add51755b1097a2b02d3961e71a2cfe928d%40%3Cdev.ofbiz.apache.org%3E
>
> and in the links stemming from the links above.
>
> ----
>
> == How can we integrate Docker in OFBiz ?
>
> Docker can be used in two distinct ways:
>
> a. Use Docker as a way to build OFBiz components - this will make builds
> more portable - as long as people have Docker (or containerd or podman)
> installed locally, they will be able to build OFBiz "for sure" (tm).
>
> This aims to solve the issue of people not having  the proper JDK and
> required tools installed (gradle, git ?! etc).
>
> b. Use Docker to deploy OFBiz for production purposes (and demos)
> IMO this means building a slim Docker image with only JRE and OFBiz + a
> custom selection of plugins. IMO this is best achieved with pre-built,
> published binaries for ofbiz.
>
> c. Use Docker to develop / debug OFBiz
> I'm not sure if this is really a thing since IMO falls into b).
>
>
> Using docker multi stage image builds is something that could help.
>
> The goals are sometimes at odds with one another in the sense that doing
> something to fix a goal will hinder the other.
>
> Do you see other use-cases for Docker and OFBiz?
>
>
> == My personal take
>
> Personally I would like to focus on b). I did not have good experience
> with building sources with Docker - I could not get used to the workflow.
>
> Also when I deploy to production, I don't really care how it's built as
> long as I have the binary and I can run just that.
>
> So we might end up with two or more Dockerfiles, each focusing on
> specific objectives.
>
>
>
> Regards,
> --
> Eugen Stan
> [phone] / netdava.com
>

--
Daniel Watford

[Ean Schuessler]

These days we are mostly working in Kubernetes. Docker only takes you part
way to the horizontal scalability promised land.

On Tue, Dec 1, 2020 at 12:27 PM Eugen Stan <[hidden email]> wrote:

> Hi,
>
> There has been some discussion regarding Docker and OFBiz, however no
> consensus yet.
> I'm starting this thread after discussions on Slack with Jacques Le
> Roux, Daniel Watford and Michael Brohl.
>
> The the aim to establish some goals / objectives regarding Docker and
> OFBiz.
>
> Please add your feedback, comments and suggestions.
>
> Prior work regarding this is found
>
> https://issues.apache.org/jira/browse/OFBIZ-10407
>
> https://lists.apache.org/thread.html/r40fd679818a37e113b469add51755b1097a2b02d3961e71a2cfe928d%40%3Cdev.ofbiz.apache.org%3E
>
> and in the links stemming from the links above.
>
> ----
>
> == How can we integrate Docker in OFBiz ?
>
> Docker can be used in two distinct ways:
>
> a. Use Docker as a way to build OFBiz components - this will make builds
> more portable - as long as people have Docker (or containerd or podman)
> installed locally, they will be able to build OFBiz "for sure" (tm).
>
> This aims to solve the issue of people not having  the proper JDK and
> required tools installed (gradle, git ?! etc).
>
> b. Use Docker to deploy OFBiz for production purposes (and demos)
> IMO this means building a slim Docker image with only JRE and OFBiz + a
> custom selection of plugins. IMO this is best achieved with pre-built,
> published binaries for ofbiz.
>
> c. Use Docker to develop / debug OFBiz
> I'm not sure if this is really a thing since IMO falls into b).
>
>
> Using docker multi stage image builds is something that could help.
>
> The goals are sometimes at odds with one another in the sense that doing
> something to fix a goal will hinder the other.
>
> Do you see other use-cases for Docker and OFBiz?
>
>
> == My personal take
>
> Personally I would like to focus on b). I did not have good experience
> with building sources with Docker - I could not get used to the workflow.
>
> Also when I deploy to production, I don't really care how it's built as
> long as I have the binary and I can run just that.
>
> So we might end up with two or more Dockerfiles, each focusing on
> specific objectives.
>
>
>
> Regards,
> --
> Eugen Stan
> [phone] / netdava.com
>

--
Ean Schuessler, Brainfood Co-Founder
[hidden email]
214-720-0700

[Daniel Watford]

Interesting point about Kubernetes, especially since its use would promotes
use of runtimes other than docker -
https://dev.to/inductor/wait-docker-is-deprecated-in-kubernetes-now-what-do-i-do-e4m

I'd be interested to explore using Kubernetes, but worry it might be a step
too far at the moment. Hopefully some people will have Ofbiz-Kubernetes
success stories to share that could encourage others to give it a try.

I wonder if a benefit of Kubernetes could be how it externalises common
application services such as config (etcd) and logging.

On Fri, 4 Dec 2020 at 19:35, Ean Schuessler <[hidden email]> wrote:

> These days we are mostly working in Kubernetes. Docker only takes you part
> way to the horizontal scalability promised land.
>
> On Tue, Dec 1, 2020 at 12:27 PM Eugen Stan <[hidden email]> wrote:
>
> > Hi,
> >
> > There has been some discussion regarding Docker and OFBiz, however no
> > consensus yet.
> > I'm starting this thread after discussions on Slack with Jacques Le
> > Roux, Daniel Watford and Michael Brohl.
> >
> > The the aim to establish some goals / objectives regarding Docker and
> > OFBiz.
> >
> > Please add your feedback, comments and suggestions.
> >
> > Prior work regarding this is found
> >
> > https://issues.apache.org/jira/browse/OFBIZ-10407
> >
> >
> https://lists.apache.org/thread.html/r40fd679818a37e113b469add51755b1097a2b02d3961e71a2cfe928d%40%3Cdev.ofbiz.apache.org%3E
> >
> > and in the links stemming from the links above.
> >
> > ----
> >
> > == How can we integrate Docker in OFBiz ?
> >
> > Docker can be used in two distinct ways:
> >
> > a. Use Docker as a way to build OFBiz components - this will make builds
> > more portable - as long as people have Docker (or containerd or podman)
> > installed locally, they will be able to build OFBiz "for sure" (tm).
> >
> > This aims to solve the issue of people not having  the proper JDK and
> > required tools installed (gradle, git ?! etc).
> >
> > b. Use Docker to deploy OFBiz for production purposes (and demos)
> > IMO this means building a slim Docker image with only JRE and OFBiz + a
> > custom selection of plugins. IMO this is best achieved with pre-built,
> > published binaries for ofbiz.
> >
> > c. Use Docker to develop / debug OFBiz
> > I'm not sure if this is really a thing since IMO falls into b).
> >
> >
> > Using docker multi stage image builds is something that could help.
> >
> > The goals are sometimes at odds with one another in the sense that doing
> > something to fix a goal will hinder the other.
> >
> > Do you see other use-cases for Docker and OFBiz?
> >
> >
> > == My personal take
> >
> > Personally I would like to focus on b). I did not have good experience
> > with building sources with Docker - I could not get used to the workflow.
> >
> > Also when I deploy to production, I don't really care how it's built as
> > long as I have the binary and I can run just that.
> >
> > So we might end up with two or more Dockerfiles, each focusing on
> > specific objectives.
> >
> >
> >
> > Regards,
> > --
> > Eugen Stan
> > [phone] / netdava.com
> >
>
>
> --
> Ean Schuessler, Brainfood Co-Founder
> [hidden email]
> 214-720-0700
>

--
Daniel Watford

[Eugen Stan]

On 04.12.2020 23:32, Daniel Watford wrote:

> Interesting point about Kubernetes, especially since its use would promotes
> use of runtimes other than docker -
> https://dev.to/inductor/wait-docker-is-deprecated-in-kubernetes-now-what-do-i-do-e4m
>
> I'd be interested to explore using Kubernetes, but worry it might be a step
> too far at the moment. Hopefully some people will have Ofbiz-Kubernetes
> success stories to share that could encourage others to give it a try.
>
> I wonder if a benefit of Kubernetes could be how it externalises common
> application services such as config (etcd) and logging.
>
> On Fri, 4 Dec 2020 at 19:35, Ean Schuessler <[hidden email]> wrote:

Thank you Daniel for the detailed description of your workflow and
everyone else for input.

I also use Kubernetes mostly and it brings a lot to the table but with a
learning curve, especially if you administer your cluster.

For people starting out iwht kubernetes I recommend k3s distribution:
https://k3s.io/
It's a single binary that is easy to deploy.

Regarding OFBiz, I thought about how to approach this and I think I will
publish binaries to a bintray repository.
 From there anyone (including me) can build their own distribution fast.

If all works well I can automate that to do daily builds.

Once I have the docker image I can build a kubernetes helm chart for it.

The alternative would be to make a repo with ofbiz and plugins and build
everything from source.

Maybe gradle multi project builds can help here
https://docs.gradle.org/current/userguide/multi_project_builds.html


Regards,

--
Eugen Stan
[phone] / netdava.com

[John Lee]

Greetings to all,As I am new to your forum, just a few words to establish my
position and objectives before you consider my opinions. *Background:*
Retired research and development engineer (mostly machinery and custom
processes - short on programing / long on mechanical) *Objective:* Create
easier access for non-programmers/developers to OFBiz *Premise:* While I
think the product (OFBiz) is a wonderful collaboration, its complexity and
advanced skill set places it outside of the masses to implement and
maintain. *Opportunity:* Conceive an easy module system to create and deploy
the product (OFBiz) while allowing the product's complex and very important
development.  Whereby placing the product within reach of more
administrators and users. *Method:* With some simple tools, we can leverage
the popularity and acceptance of new approaches to containers (IE docker,
Kubernetes, etc.) *Approach:* As docker development is lengthy (mostly due
to image creation), using tools like ANSIBLE to create simple scripts for
the "assembly" of OFBiz modules.  These scripts can more easily be modified
and maintained to design, assemble, and deploy the customized images of
OFBiz. *Create Value:* To encourage the continued development of OFBiz, a
revenue path should be considered.  While we must listen to the market and
adjust our processes accordingly, establishing clear revenue generation
should be considered when creating these products.  I want to propose the
opinion that many administrators and users will need and be willing to pay
for technical assistance with software operation, not the installation of
the software.  The skillset necessary to maintain and support the software
is very different and perhaps more easily located than the skillset required
to install and configure the software.  Also, the software's maintenance and
support offer a longer relationship and greater revenue stream from the
customer. *Integration:* By allowing a more simple and efficient
installation, a new market can be created to integrate other applications to
OFBiz.  The container concept is powerful because it will enable many
different systems to "play nice" with each other.  For example, I have
manufacturers and who what an accounting system to integrate with their
manufacturing process.  They do not see the accounting system driving the
manufacturing process, but the manufacturing process driving the accounting
system.  By launching the OFBiz in a container, they can integrate their
current manufacturing process via a shared database.  These customers are
focused on manufacturing and its methods, not the accounting system.  This
concept could drive a more untapped niche market. *Closing:* While I am new
to your forum and this lengthy epistle is equivalent to posting a roast me
along with a picture from my high school days, I would like to hear your
thoughts and, or directions.



-----
John Lee
Engineering – Bright Star Controls
Office [phone]
Direct [phone]
Mobile – SMS [phone]


--
Sent from: http://ofbiz.135035.n4.nabble.com/OFBiz-Dev-f165671.html

[John Lee]

hello,

I want you to imagine spinning up six servers in an organized cluster with
VLAN, security, load balancing, raid volumes, and the ability to scale
services, backup data, upgrade individual software packages without the
worry of breaking connections with other running products. Now imagine
spending less than 1 hour to accomplish this.  I am not talking about just
scripts.  I am talking about a system designed for a production environment,
built for simple, robust, and manageable implementation of OFBiz and other
packages customers need.

This is the world of Rancher, Kubernetes, and docker.

*Objective:* Create easier access for non-programmers/developers to OFBiz

*Premise:* While I think the product (OFBiz) is a wonderful collaboration,
its complexity and advanced skill set places it outside of the masses to
implement and maintain.

*Opportunity:* Conceive an easy module system to create and deploy the
product (OFBiz) while allowing the product's complex and very important
development.  Whereby placing the product within reach of more
administrators and users.

*Method:* With some simple tools, we can leverage the popularity and
acceptance of new approaches to containers (IE docker, Kubernetes, etc.)

*Approach:* As docker development is lengthy (mostly due to image creation),
using tools like ANSIBLE to create simple scripts for the "assembly" of
OFBiz modules.  These scripts can more easily be modified and maintained to
design, assemble, and deploy the customized images of OFBiz.

*Create Value:* To encourage the continued development of OFBiz, a revenue
path should be considered.  While we must listen to the market and adjust
our processes accordingly, establishing clear revenue generation should be
considered when creating these products.  I want to propose the opinion that
many administrators and users will need and be willing to pay for technical
assistance with software operation, not the installation of the software.
The skillset necessary to maintain and support the software is very
different and perhaps more easily located than the skillset required to
install and configure the software.  Also, the software's maintenance and
support offer a longer relationship and greater revenue stream from the
customer.

*Integration:* By allowing a more simple and efficient installation, a new
market can be created to integrate other applications to OFBiz.  The
container concept is powerful because it will enable many different systems
to "play nice" with each other.  For example, I have manufacturers who what
an accounting system to integrate with their manufacturing process.  They do
not see the accounting system driving the manufacturing process, but the
manufacturing process driving the accounting system.  By launching the OFBiz
in a container, they can integrate their current manufacturing process via a
shared database.  These customers are focused on manufacturing and its
methods, not the accounting system.  This concept could drive a more
untapped niche market.

As always, I look forward to knowing your thoughts and or directions.




-----
John Lee
Engineering – Bright Star Controls
Office [phone]
Direct [phone]
Mobile – SMS [phone]


--
Sent from: http://ofbiz.135035.n4.nabble.com/OFBiz-Dev-f165671.html

[Eugen Stan]

Hi,


Thank you all for your feedback.
I've started working on my version of docker image / build for OFBiz.

It is still work in progress and I will share my work when I have the
demo running with Docker.

Perhaps we will find a way to have OFBiz docker images.

Regards,

--
Eugen Stan
[phone] / netdava.com

[John Lee]

Eugen Stan wrote
> Hi,Thank you all for your feedback.I've started working on my version of
> docker image / build for OFBiz.It is still work in progress and I will
> share my work when I have the demo running with Docker.Perhaps we will
> find a way to have OFBiz docker images.Regards,-- Eugen Stan+40720 898 747
> / netdava.com

I have created a private OFBiz docker image (huge...) that will deploy in
the cloud (digital ocean) running in a Docker container in Kubernetes.My
current image is running on the database Derby.  I want to create an image
that can use Mysql and or Postgres.To work around [host-header-allowed]
issues in [security.properties] I create a persistent volume at the location
of this file within the container and edit the file after the container is
launched.  Not ideal.  I want to develop an environment variable approach
that could allow users to define host IP and other settings before the
container launch.I would encourage you to install Rancher and see how they
launch products from their catalog.  These products are complicated, but
they have reduced the complexity via their UI to a few questions.  This
approach would put OFBiz within reach of many more administrators and
users.While I would like to see a reliable Docker Image of OFBiz, I believe
we should consider a quantum leap to Kubernetes.  Just being able to launch
from this platform would expose OFBiz to a much larger audience.You have a
great product developed by a very dedicated group, time more people knew
about this fantastic product.



-----
John Lee
Engineering – Bright Star Controls
Office [phone]
Direct [phone]
Mobile – SMS [phone]


--
Sent from: http://ofbiz.135035.n4.nabble.com/OFBiz-Dev-f165671.html