How to run DataMinder on Docker

This article describes how to install DataMinder on Docker. You can either install in your own Docker runtime or release the Docker image on a Cloud based service:

Google Cloud
Microsoft Azure
Amazon Web Services
Or other services, public or private

This makes it possible to get DataMinder up and running without the need to run an entire server.

Use techniques such as Kubernetes to manage and run Docker images.

It also makes it possible to quickly scale up the number of identical DataMinder instances when used as micro services.

Basic steps for Continuous Delivery

We recommend combining these two release flows:

Develop DataMinder plugins [optional] : Develop, test and deploy plugins to be used by DataMinder server.
Build configurations on DataMinder server : Build, test and release new configurations on the DataMinder server.

Step 1 [Optional]: Develop new or modify previous plugins

If you need to build your own plugins you may choose a release flow where you develop -> test -> and deploy your plugins. This can be done without any DataMinder server installed. You can build automated tests using frameworks like TestNG or JUnit.

When plugins are released you make them available to DataMinder by adding them to DataMinder plugin libraries folder.

Step 2: Build a new DataMinder configuration

Use DataMinder to build new processes by combining plugins using Drag’n’Drop. Test and verify you configuration.

More on building services with DataMinder here.

Step 3: Release your new DataMinder server as a Docker image

First install DataMinder to get the configuration file /DataMinder/Server/Config/DataMinder.properties properly configured before creating any Docker image.

Use this DataMinder to install new libraries and test new builds.

When you are finished you create a new Docker image. You no longer modify this "read-only" Docker image instance since your Docker containers should be as ephemeral as possible. Meaning it can be stopped and destroyed and a new one built and put in place with an absolute minimum of set-up and configuration.

Directory structure

Example of a directory structure to get you started. It contains the Dockerfile and the finished and tested DataMinder server installation.

/ROOT_DIR_FOR_IMAGE/
                /Dockerfile
                /DataMinder
                    /Server
                      ....
                    /README.txt
                    /DataMinder.jar

The Dockerfile

The contents of our Dockerfile. We start with a Java JRE 8 runtime image and add DataMinder to it.

#Get Java JRE 8 runtime image
FROM openjdk:8-jre

#Set working directory to DataMinder server root i.e. /DataMinder
WORKDIR /DataMinder

#Add DataMinder server to our Docker image
ADD ./DataMinder /DataMinder

#Expose the DataMinder ports fund in /DataMinder/Server/Config/DataMinder.properties
#dataminder.environment.web.adminHttps=10001
#dataminder.environment.web.webServiceHttp=10002
#dataminder.environment.web.webServiceHttps=10003
#To prevent access to admin ui don't share the port and/or make a random password
EXPOSE 10001
EXPOSE 10002
EXPOSE 10003

#Run DataMinder calling our Java runtime in the image
CMD ["java", "-jar","DataMinder.jar"]

To create our image we run the following command from the directory mentioned above: /ROOT_DIR_FOR_IMAGE:

docker build -t dataminder:latest .

Useful commands

For more information about Docker specifics please see: https://docs.docker.com/reference/

The two documents we refer to here are the "Dockerfile reference" and "Engine (Docker) CLI", the Docker Engine Command Line Interface

Useful Docker commands:

Create our image: docker build -t dataminder:latest .
List all Docker images: docker images
Run our image (which becomes a Container) and share ports: docker run -p 10001-10003:10001-10003/tcp dataminder
List all running Docker containers: docker ps
Stop a Docker Container: docker stop CONTAINER_ID
Open a terminal into your running image: docker exec -it CONTAINER_ID /bin/bash

Access DataMinder at the following (exported) ports:

DataMinder admin interface: https://127.0.0.1:10001/DM
DataMinder web service http port (the default /HelloWorld process): http://127.0.0.1:10002/HelloWorld
DataMinder web service https port (the default /HelloWorld process): https://127.0.0.1:10003/HelloWorld

Summary

You can develop DataMinder plugins and test them without access to a DataMinder server. When you are finished you create and release a DataMinder plugin library and install it on the DataMinder server to make your plugins available to use when building processes.

If you plan to use Docker platform we recommend you create Docker images and containers from a configured and tested DataMinder instance. That means you build and test any processes before you create create Docker images and containers.

When you have your Docker image and container you should not do any changes to configuration since the container should be as ephemeral as possible.