Run the latest version of the Elastic stack with Docker and Docker Compose.
It gives you the ability to analyze any data set by using the searching/aggregation capabilities of Elasticsearch and
the visualization power of Kibana.
tls: TLS encryption enabled in Elasticsearch, Kibana (opt in),
and Fleet
[!IMPORTANT]
Platinum features are enabled by default for a trial duration of 30 days. After
this evaluation period, you will retain access to all the free features included in the Open Basic license seamlessly,
without manual intervention required, and without losing any data. Refer to the How to disable paid
features section to opt out of this behaviour.
tl;dr
docker compose up setup
docker compose up
Philosophy
The main goal of docker-elk is to make the Elastic stack as easy as possible to get into. It is not a blueprint for a
production-ready deployment, but rather a template that promotes tweaking and exploration.
The authors believe in good documentation over elaborate automation. The project's default configuration is purposely
minimal and unopinionated. The initial setup does not rely on any external dependency, and uses as little scripting as
necessary to get things up and running.
[!NOTE]
Especially on Linux, make sure your user has the required permissions to interact with the Docker
daemon.
By default, the stack exposes the following ports:
5044: Logstash Beats input
50000: Logstash TCP input
9600: Logstash monitoring API
9200: Elasticsearch HTTP
9300: Elasticsearch TCP transport
5601: Kibana
[!WARNING]
Elasticsearch's bootstrap checks were purposely disabled to facilitate the setup of the Elastic
stack in development environments. For production setups, we recommend users to set up their host according to the
instructions from the Elasticsearch documentation: Important System Configuration.
Docker Desktop
Windows
If you are using the legacy Hyper-V mode of Docker Desktop for Windows, ensure that File
Sharing is enabled for the C: drive.
macOS
The default configuration of Docker Desktop for Mac allows mounting files from /Users/, /Volume/, /private/,
/tmp and /var/folders exclusively. Make sure the repository is cloned in one of those locations or follow the
instructions from the documentation to add more locations.
Usage
[!WARNING]
You must rebuild the stack images with docker compose build whenever you switch branch or update the
version of an already existing stack.
Bringing up the stack
Clone this repository onto the Docker host that will run the stack with the command below:
Then, initialize the Elasticsearch users and groups required by docker-elk by executing the command:
docker compose up setup
Optionally (but highly recommended), generate encryption keys for Kibana using the following command and copy its output
to the Kibana configuration file (kibana/config/kibana.yml):
docker compose up kibana-genkeys
If everything went well and the setup completed without error, start the other stack components:
docker compose up
[!NOTE]
You can also run all services in the background (detached mode) by appending the -d flag to the above command.
Give Kibana about a minute to initialize, then access the Kibana web UI by opening http://localhost:5601 in a web
browser and use the following (default) credentials to log in:
user: elastic
password: changeme
[!NOTE]
Upon the initial startup, the elastic, logstash_internal and kibana_system Elasticsearch users are initialized
with the values of the passwords defined in the .env file ("changeme" by default). The first one is the
built-in superuser, the other two are used by Kibana and Logstash respectively to communicate with
Elasticsearch. This task is only performed during the initial startup of the stack. To change users' passwords
after they have been initialized, please refer to the instructions in the next section.
[!WARNING]
Starting with Elastic v8.0.0, it is no longer possible to run Kibana using the bootstrapped privileged elastic user.
The "changeme" password set by default for all aforementioned users is unsecure. For increased security, we will
reset the passwords of all aforementioned Elasticsearch users to random secrets.
Reset passwords for default users
The commands below reset the passwords of the elastic, logstash_internal and kibana_system users. Take note
of them.
If the need for it arises (e.g. if you want to collect monitoring information through Beats and
other components), feel free to repeat this operation at any time for the rest of the built-in
users.
Replace usernames and passwords in configuration files
Replace the password of the elastic user inside the .env file with the password generated in the previous step.
Its value isn't used by any core component, but extensions use it to
connect to Elasticsearch.
[!NOTE]
In case you don't plan on using any of the provided extensions, or
prefer to create your own roles and users to authenticate these services, it is safe to remove the
ELASTIC_PASSWORD entry from the .env file altogether after the stack has been initialized.
Replace the password of the logstash_internal user inside the .env file with the password generated in the
previous step. Its value is referenced inside the Logstash pipeline file (logstash/pipeline/logstash.conf).
Replace the password of the kibana_system user inside the .env file with the password generated in the previous
step. Its value is referenced inside the Kibana configuration file (kibana/config/kibana.yml).
See the Configuration section below for more information about these configuration files.
Restart Logstash and Kibana to re-connect to Elasticsearch using the new passwords
Launch the Kibana web UI by opening http://localhost:5601 in a web browser, and use the following credentials to log
in:
user: elastic
password: <your generated elastic password>
Now that the stack is fully configured, you can go ahead and inject some log entries.
The shipped Logstash configuration allows you to send data over the TCP port 50000. For example, you can use one of the
following commands — depending on your installed version of nc (Netcat) — to ingest the content of the log file
/path/to/logfile.log in Elasticsearch, via Logstash:
# Execute `nc -h` to determine your `nc` version
cat /path/to/logfile.log | nc -q0 localhost 50000 # BSD
cat /path/to/logfile.log | nc -c localhost 50000 # GNU
cat /path/to/logfile.log | nc --send-only localhost 50000 # nmap
You can also load the sample data provided by your Kibana installation.
Cleanup
Elasticsearch data is persisted inside a volume by default.
In order to entirely shutdown the stack and remove all persisted data, use the following Docker Compose command:
docker compose --profile=setup down -v
Version selection
This repository stays aligned with the latest version of the Elastic stack. The main branch tracks the current major
version (9.x).
To use a different version of the core Elastic components, simply change the version number inside the .env
file. If you are upgrading an existing stack, remember to rebuild all container images using the docker compose build
command.
[!IMPORTANT]
Always pay attention to the official upgrade instructions for each individual component before performing a
stack upgrade.
Older major versions are also supported on separate branches:
Please refer to the following documentation page for more details about how to configure Elasticsearch inside Docker
containers: Install Elasticsearch with Docker.
You can also specify the options you want to override by setting environment variables inside the Compose file:
logstash:
environment:
LOG_LEVEL: debug
Please refer to the following documentation page for more details about how to configure Logstash inside Docker
containers: Configuring Logstash for Docker.
How to disable paid features
You can cancel an ongoing trial before its expiry date — and thus revert to a basic license — either from the License
Management panel of Kibana, or using Elasticsearch's start_basicLicensing API. Please
note that the second option is the only way to recover access to Kibana if the license isn't either switched to basic
or upgraded before the trial's expiry date.
Changing the license type by switching the value of Elasticsearch's xpack.license.self_generated.type setting from
trial to basic (see License settings) will only work if done prior to the initial setup.
After a trial has been started, the loss of features from trial to basicmust be acknowledged using one of the two
methods described in the first paragraph.
To run the setup container again and re-initialize all users for which a password was defined inside the .env file,
simply "up" the setup Compose service again:
$ docker compose up setup
⠿ Container docker-elk-elasticsearch-1 Running
⠿ Container docker-elk-setup-1 Created
Attaching to docker-elk-setup-1
...
docker-elk-setup-1 | [+] User 'monitoring_internal'
docker-elk-setup-1 | ⠿ User does not exist, creating
docker-elk-setup-1 | [+] User 'beats_system'
docker-elk-setup-1 | ⠿ User exists, setting password
docker-elk-setup-1 exited with code 0
How to reset a password programmatically
If for any reason your are unable to use Kibana to change the password of your users (including built-in
users), you can use the Elasticsearch API instead and achieve the same result.
In the example below, we reset the password of the elastic user (notice "/user/elastic" in the URL):
Add a RUN statement to the corresponding Dockerfile (eg. RUN logstash-plugin install logstash-filter-json)
Add the associated plugin code configuration to the service configuration (eg. Logstash input/output)
Rebuild the images using the docker compose build command
How to enable the provided extensions
A few extensions are available inside the extensions directory. These extensions provide features which
are not part of the standard Elastic stack, but can be used to enrich it with extra integrations.
The documentation for these extensions is provided inside each individual subdirectory, on a per-extension basis. Some
of them require manual changes to the default ELK configuration.
JVM tuning
How to specify the amount of memory used by a service
The startup scripts for Elasticsearch and Logstash can append extra JVM options from the value of an environment
variable, allowing the user to adjust the amount of memory that can be used by each component:
Service
Environment variable
Elasticsearch
ES_JAVA_OPTS
Logstash
LS_JAVA_OPTS
To accommodate environments where memory is scarce (Docker Desktop for Mac has only 2 GB available by default), the Heap
Size allocation is capped by default in the docker-compose.yml file to 512 MB for Elasticsearch and 256 MB for
Logstash. If you want to override the default JVM configuration, edit the matching environment variable(s) in the
docker-compose.yml file.
For example, to increase the maximum JVM Heap Size for Logstash:
Logstash starts with a fixed JVM Heap Size of 1 GB.
How to enable a remote JMX connection to a service
As for the Java Heap memory (see above), you can specify JVM options to enable JMX and map the JMX port on the Docker
host.
Update the {ES,LS}_JAVA_OPTS environment variable with the following content (I've mapped the JMX service on the port
18080, you can change that). Do not forget to update the -Djava.rmi.server.hostname option with the IP address of your
Docker host (replace DOCKER_HOST_IP):
docker-elk 是一套 Docker Compose 配置，用 Elastic 官方镜像把整套 Elastic 组件——Elasticsearch、Logstash 和 Kibana——一键跑起来，让你几乎不用做任何配置就能搜索、汇总和可视化数据。