Skip to content
Search

Elasticsearch

lastmod 2025-11-24created 2017-01-2242 edits

Elasticsearch is a distributed RESTful search and analytics engine capable of solving a growing number of problems. As the heart of the Elastic Stack, it centrally stores your data so you can search for what you expect and uncover the unexpected.

Categories

Shards and Replicas 

DELETE /your_index/your_type/_query
{
  "query": {
    "range": {
      "timestamp": {
        "lte": "now-10y"
      }
    }
  }
}

Install Elasticsearch with Docker

cluster.name=docker-cluster
The ES cluster name. Every ES service must use the same name.
node.name=master-node1
Sets the ES node name.
bootstrap.memory_lock=true
Adds a setting to prevent memory swapping while ES is running.
"ES_JAVA_OPTS=-Xms512m -Xmx512m"
JVM heap memory setting. Always set the Xms/Xmx options to the same value.
ulimits
Options for Linux system resource limits. ES uses many file descriptors and handlers, so the limits need to be lifted.

Starting a single node cluster with Docker 

docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.1

Starting a multi-node cluster with Docker Compose 

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.1
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.1
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
    networks:
      - elastic
  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.1
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
    networks:
      - elastic

volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local

networks:
  elastic:
    driver: bridge

use elasticsearch in gitlab-ci

If you want to use it for testing in gitlab-ci, apply it as follows. 

test-elasticsearch:
  stage: test
  services:
    - name: "docker.elastic.co/elasticsearch/elasticsearch:7.10.1"
      alias: "elasticsearch"
      command: [ "bin/elasticsearch", "-Expack.security.enabled=false", "-Ediscovery.type=single-node" ]
  script:
    - curl "http://elasticsearch:9200/_cat/health"

Docker GELF logging

See Logstash#Docker GELF logging.

ELK Stack

Elasticsearch vs Solr

Troubleshooting

error 137 in docker container

In short, memory may be insufficient. See Docker#error code 137.

max virtual memory areas vm.max_map_count is too low

The log when Elasticsearch shuts down: 

~/elasticsearch-5.6.1/bin$ [2017-10-11T06:39:53,758][DEBUG][o.e.a.ActionModule       ] Using REST 
wrapper from plugin org.elasticsearch.xpack.XPackPlugin
[2017-10-11T06:39:55,926][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/16678] [Main.cc@128] controller (6
4 bit): Version 5.6.1 (Build e81fa9e3f3e0c1) Copyright (c) 2017 Elasticsearch BV
[2017-10-11T06:39:55,972][INFO ][o.e.d.DiscoveryModule    ] [node1] using discovery type [zen]
[2017-10-11T06:39:57,291][INFO ][o.e.n.Node               ] [node1] initialized
[2017-10-11T06:39:57,292][INFO ][o.e.n.Node               ] [node1] starting ...
[2017-10-11T06:39:57,666][INFO ][o.e.t.TransportService   ] [node1] publish_address {10.146.0.5:9300}, bound_addres
ses {[::]:9300}
[2017-10-11T06:39:57,679][INFO ][o.e.b.BootstrapChecks    ] [node1] bound or publishing to a non-loopback or non-li
nk-local address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-10-11T06:39:57,692][INFO ][o.e.n.Node               ] [node1] stopping ...
[2017-10-11T06:39:57,731][INFO ][o.e.n.Node               ] [node1] stopped
[2017-10-11T06:39:57,732][INFO ][o.e.n.Node               ] [node1] closing ...
[2017-10-11T06:39:57,750][INFO ][o.e.n.Node               ] [node1] closed
[2017-10-11T06:39:57,753][INFO ][o.e.x.m.j.p.NativeController] Native controller process has stopped - no new nativ
e processes can be started
[1]+  Exit 78                 ./elasticsearch

Change vm.max_map_count to the suggested value: 

sudo sysctl -w vm.max_map_count=262144

See also

Favorite site

Documentation

Tutorials

Docker

Articles

author Zer0 sitemap