Run Elasticsearch 6.2 In Docker

How To Run Elasticsearch 6.2 In Docker
Dockerising Elasticsearch 6.2

Elasticsearch is a very useful databases being used by almost every company. one of the major use-case is observed in ELK or EFK stack for centralised logging. Setting up elasticsearch from scratch with traditional method could be tedious.

First create a bucket "es-configurations" to store elasticsearch configuration file

upload the file elasticearch.yml with the following content:

cluster.name: mycluster

path.data: /usr/share/elasticsearch/data

network.host: 0.0.0.0

http.port: 9200

node.master: true

node.data: true

node.name: "nodename"

bootstrap.memory_lock: true

transport.tcp.compress: true

Also upload jvm.options file with the following content. Make sure to change heap parameter as per your available memory.

-Xms4g
-Xmx4g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+DisableExplicitGC
-XX:+AlwaysPreTouch
-server
-Xss1m
-Djava.awt.headless=true
-Dfile.encoding=UTF-8
-Djna.nosys=true
-Djdk.io.permissionsUseCanonicalPath=true
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Dlog4j.skipJansi=true
-XX:+HeapDumpOnOutOfMemoryError

Create following IAM policy, say "elasticsearch-bucket-access" to access config file in "elasticsearch" bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:GetObject"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::es-configurations/*"
        }
    ]
}

Create an IAM role "elasticsearch-role" and attach above policy to it.

Now go ahead launching an instance, attach above role to it and provide the following userdata to it:

#!/bin/bash

###
# This is the first part which can be used to prepare base-image
###

# output log of userdata to /var/log/user-data.log
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

# Install awscli
apt-get update
apt install awscli -y

# Set max_map_count
echo 262144 | sudo tee /proc/sys/vm/max_map_count

# Install docker
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt-get update
apt-cache policy docker-ce
apt-get install -y docker-ce
service docker restart

# Get official elasticsearch docker image
docker pull docker.elastic.co/elasticsearch/elasticsearch:6.2.3

# Create /etc/elasticsearch directory to hold elasticsearch config files like elasticsearch.yml and jvm.options
mkdir -p /etc/elasticsearch

###
# Second part of script downloads elasticsearch configuration files from S3 and run container
###

# Get elasticsearch config files from S3
aws s3 cp s3://es-configurations/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml --region ap-south-1
aws s3 cp s3://es-configurations/jvm.options /etc/elasticsearch/jvm.options --region ap-south-1

# Replace nodename in elasticsearch.yml file with hostname
sed -i -e "s/nodename/${HOSTNAME}/g" /etc/elasticsearch/elasticsearch.yml

# Mount a secondary Volume for elasticsearch data directory
mkfs.xfs /dev/xvdb
mkdir -p /vol/es
mount /dev/xvdba /vol/es

# change ownership of data directory and config directory to user with 1000 id as in container elasticsearch runs with user 1000
chown -R 1000:1000 /vol
chown -R 1000:1000 /etc/elasticsearch

# Make sure vm.max_map_count is 262144
sysctl -w vm.max_map_count=262144

#start docker container
docker run --net=host -d -p 9200:9200 -e "xpack.security.enabled=false" --restart unless-stopped -v /vol/es:/usr/share/elasticsearch/data -v /etc/elasticsearch/jvm.options:/usr/share/elasticsearch/config/jvm.options -v /etc/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml  --ulimit nofile=65536:65536 --ulimit memlock=-1:-1 docker.elastic.co/elasticsearch/elasticsearch:6.2.3