Gun9niR
Posts Tags Categories About
Gun9niR

Contents

Dockerize Bookstore System

 published in Web
   1058 words   5 minutes 

The bookstore project is not officially finished, so a lot of changes might happen. Github link for the entire project will be available after the project is concluded.

0. Why?

As a wealth of components are being added to my bookstore system, it is growing increasingly complex, putting forward a few problems.

  • Multiple middlewares and databases clutter up local storage, with configuration files, log files, data files scattered all over the place.
  • The project becomes unwieldy to deploy. Even if I have most components automatically launch as a service at computer startup, I still have to open 3 Springboot projects, and launch them one by one.
  • Because a locally installed middleware/database can only have one configuration file at a time, it is very difficult to use it for many applications simultaneously.

In light of these shortcomings, I resort to Docker, which provides a convenient solution to them all.

1. Bookstore system components

Just to obtain a big picture, my Bookstore system consist of the following components:

  • Databases
    • PostgreSQL (primary storage, with primary and backup)
    • MongoDB (book description)
    • Neo4j (book tags)
    • ElasticSearch (full-text search)
    • Redis (cache)
  • Middleware
    • RabbitMQ (asynchronous communication)
    • Spring Cloud Gateway (HTTPS and load balancing)
    • Eureka (service discovery)
    • Logstash (synchronize data between PostgreSQL and ES)
  • Application
    • Springboot main service

Some of these components have readily available images, while others warrant the use of Dockerfile, which is basically a file that specifies how an image should be built.

2. Building & Deployment overview

The ultimate goal is to make the building and deployment process a one-or-two-liner, so I want as few files as possible for each stage. There are mainly three stages:

  1. Build any self-written code, which is all in Java, using maven
  2. Build Docker images if necessary, using one Dockerfile
  3. Deploy Docker containers either from self-made images or those from Docker Hub, using **.

The file structure of my project looks like this

├── config
│   └── ...
├── db
│   └── ...
├── eureka
│   └── ...
├── gateway
│   └── ...
├── main-service
│   └── ...
├── shared-data-access
│   └── ...
├── build.sh
├── Dockerfile
├── docker-compose.yml
└── pom.xml

where:

  • config stores configuration file and scripts for containers
  • db stores persistent data
  • eureka, gateway, main-service, shared-data-access are java modules to be built with pom.xml
  • build.sh builds above Java modules and generate images with Dockerfile
  • docker-compose.yml deploys the entire system

3. Building self-written code

This is easily achievable with a one-liner:

mvn clean install -Dmaven.test.skip=true -pl eureka,gateway,main-service -am

-Dmaven.test.skip=true skips the tests, as some modules will not be able to pass the test (database connection failer) after their configuration files are modified for use within Docker container. pl specifies specific modules to build rather than the whole project, -am means also build projects required by building modules. Reference

4. Building images

Had I been on an amd64 machine, this will be another one-liner, as spring-boot-maven-plugin takes care of it all. On an amd64 machine, just run

mvn spring-boot:build-image -f pom.xml

But the images generated by this command do not run on arm64 machine for some jdk incompatibility issues. So we need to define our own base image intended for arm64 machine. Luckily, this page contains various jdk images for arm64. With that in place, a Dockerfile can now be written to build the images. Assue all jar files have been built.

FROM arm64v8/openjdk:17-jdk-oraclelinux7 as eureka
ADD ./eureka/target/*.jar /eureka.jar
EXPOSE 8040
ENTRYPOINT ["java", "-jar", "/eureka.jar"]

FROM arm64v8/openjdk:17-jdk-oraclelinux7 as gateway
ADD ./gateway/src/main/resources/keystore.p12 /keystore.p12
ADD ./gateway/target/*.jar /gateway.jar
EXPOSE 8090
ENTRYPOINT ["java", "-jar", "/gateway.jar"]

FROM arm64v8/openjdk:17-jdk-oraclelinux7 as main-service
ADD ./main-service/target/*.jar /bookstore.jar
EXPOSE 8080
ENTRYPOINT ["java", "-jar", "/bookstore.jar"]

The Dockerfile can be executed with a script to be executed under project root directory.

docker build --target eureka -t gun9nir/bookstore.eureka .
docker build --target gateway -t gun9nir/bookstore.gateway .
docker build --target main-service -t gun9nir/bookstore.main-service .

5. Communication among containers

Reference

An important advantage of Docker Compose over separate docker run commands is:

By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name.

But since containers communicate with each other via conatiner name, the configuration files of each compoenent has to be changed, with ip/localhost changed to container name.

6. Dockerize components

6.1. PostgreSQL

  postgresql-primary:
    image: postgres
    container_name: bookstore-postgresql-primary
    ports:
      - "5432:5432"
    environment:
      - POSTGRES_USER=gun9nir
      - POSTGRES_PASSWORD=guozhidong12
      - POSTGRES_DB=bookstore
    volumes:
      - ./config/postgres-primary/postgresql.conf:/etc/postgresql/postgresql.conf
      - ./config/postgres-primary/pg_hba.conf:/etc/postgresql/pg_hba.conf
      - ./db/postgresql-primary/data:/var/lib/postgresql/data
      - ./config/postgres-primary/create_replicator.sh:/docker-entrypoint-initdb.d/create_replicator.sh
    command: postgres -c config_file=/etc/postgresql/postgresql.conf -c hba_file=/etc/postgresql/pg_hba.conf

  postgresql-backup:
    image: postgres
    container_name: bookstore-postgresql-backup
    ports:
      - "5433:5432"
    restart: on-failure
    environment:
      - POSTGRES_USER=gun9nir
      - POSTGRES_PASSWORD=guozhidong12
      - POSTGRES_DB=bookstore
    volumes:
      - ./config/postgres-backup/postgresql.conf:/etc/postgresql/postgresql.conf
      - ./config/postgres-backup/pg_hba.conf:/etc/postgresql/pg_hba.conf
      - ./db/postgresql-backup/data:/var/lib/postgresql/data
      - ./config/postgres-backup/init_backup.sh:/docker-entrypoint-initdb.d/init_backup.sh
    command: postgres -c config_file=/etc/postgresql/postgresql.conf -c hba_file=/etc/postgresql/pg_hba.conf

A primary-backup configuration is adopted for high availability. To achieve that, in config directory are stored the respective configuration files and scripts for primary and backup. Two scripts are worth mentioning:

#!/bin/bash

psql -v ON_ERROR_STOP=1 --username gun9nir --dbname bookstore <<-EOSQL
    CREATE USER replicator;
    ALTER USER replicator WITH REPLICATION;
EOSQL

This script is run after primary database is created. It creates a replication user which has replication privilege to bookstore. Backups should use this user to initiate backup.

#!/bin/bash

rm -rf /var/lib/postgresql/data/*
pg_basebackup -h postgresql-primary -D /var/lib/postgresql/data -U replicator -v -P -R

This script is run after backup database is created. The data directory is cleard first, then the intial backup command is executed. The meaning of each argument is here. The most vital argument is -R. The containers automatically exit after first backup, thus the restart: on-failure option in Docker Compose file.

6.2. MongoDB

  mongodb:
    image: mongo
    container_name: bookstore-mongodb
    ports:
      - "27017:27017"
    environment:
      - MONGO_INITDB_ROOT_USERNAME=<username>
      - MONGO_INITDB_ROOT_PASSWORD=<pwd>
    volumes:
      - ./db/mongodb/data/db:/data/db

6.3. Neo4j

The official image of Neo4j does not support arm64 architecture, so an experimental version is used.

  neo4j:
    image: neo4j/neo4j-arm64-experimental:4.1.11-arm64
    container_name: bookstore-neo4j
    ports:
      - "7474:7474"
      - "7473:7473"
      - "7687:7687"
    environment:
      - NEO4J_AUTH=<username>/<pwd>
    volumes:
      - ./db/neo4j/data:/data

6.4. ElasticSearch

ES is very demanding in terms of memory. So extra settings are required.

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.15.0
    container_name: bookstore-elasticsearch
    ports: 
      - "9200:9200"
      - "9300:9300"
    restart: on-failure
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=true
      - ELASTIC_PASSWORD=guozhidong12
      - ES_JAVA_OPTS=-Xms512m -Xmx512m -XX:MaxMetaspaceSize=128m
    volumes:
      - ./db/elasticsearch/data:/usr/share/elasticsearch/data
    deploy:
      resources:
        limits:
          memory: 1024M

6.5. Redis

  redis:
    image: redis
    container_name: redis
    ports:
      - "6379:6379"
    volumes:
      - ./db/redis:/data

6.6. RabbitMQ

The management tag downloads rabbitmq with graphics interface enabled.

  rabbitmq:
    image: rabbitmq:management
    container_name: bookstore-rabbitmq
    ports: 
      - "5672:5672"
      - "15672:15672"
    environment:
      - RABBITMQ_DEFAULT_USER=gun9nir
      - RABBITMQ_DEFAULT_PASS=guozhidong12
    volumes:
      - ./db/rabbitmq/data:/var/lib/rabbitmq

6.7. Spring Cloud Gateway

  gateway:
    image: docker.io/gun9nir/bookstore.gateway:latest
    container_name: bookstore-gateway
    ports:
      - "8090:8090"

One pitfall: the location of keystore file used by HTTPS in specified in application.yml, which cannot refer to anything within a jar file. So the keystore file needs to be copied into the image, thus ADD ./gateway/src/main/resources/keystore.p12 /keystore.p12 in Dockerfile.

6.8. Eureka

  eureka:
    image: docker.io/gun9nir/bookstore.eureka:latest
    container_name: bookstore-eureka
    ports:
      - "8040:8040"

6.9. Logstash

  logstash:
    image: docker.elastic.co/logstash/logstash:7.15.2
    container_name: bookstore-logstash
    restart: on-failure
    environment:
      - XPACK_MONITORING_ELASTICSEARCH_HOSTS=http://bookstore-elasticsearch:9200
    volumes:
      - ./config/logstash/pipelines.yml:/usr/share/logstash/config/pipelines.yml
      - ./config/logstash/pipeline:/usr/share/logstash/pipeline
      - ./config/logstash/postgresql-42.2.24.jar:/usr/share/logstash/logstash-core/lib/jars/postgresql-42.2.24.jar

6.10. Main service

There are two instances of main service, just to utilize the load balancing feature provided by gateway. Main service relies on ES to start properly, but ES takes extremely long to start, thus the timeout, and restart: on-failure:3.

  bookstore-1:
    image: gun9nir/bookstore.main-service
    container_name: bookstore-main-service-1
    restart: on-failure:3

  bookstore-2:
    image: gun9nir/bookstore.main-service
    container_name: bookstore-main-service-2
    restart: on-failure:3

7. Deploy

With all the preparations, building and deploying the project boils down to two commands.

bash ./build.sh
docker compose up -d
Docker
Updated on 2021-11-30