Cluster Configuration Overview

  • Three node cluster
  • 2 data nodes
  • 2 ingest nodes
  • 1 kibana node
  • 3 master-eligible nodes
Node Data Ingest Kibana Master-Eligible
elasticnodev01 Yes Yes Yes Yes
elasticnodev02 Yes Yes No Yes
elasticnodev03 No No No Yes

Installation

Install the required components for each node as listed below, as per build documentation.

Node 01

  • Java
  • SyslogNG
  • ElasticSearch
  • Kibana
  • Logstash

Node 02

  • Java
  • SyslogNG
  • ElasticSearch
  • Logstash

Node 03

  • Java
  • ElasticSearch

Configuration

All Nodes

Add the following firewall rule for cluster communications:

firewall-cmd --zone=work --add-port=9200/tcp --permanent
firewall-cmd --zone=work --add-port=9300-9400/tcp --permanent

elasticnodev01

# ======================== Elasticsearch Configuration =========================
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: SecurityMetrics
cluster.remote.connect: false
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: elasticnodev01
node.master: true
node.data: true
node.ingest: true
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.1.90
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# --------------------------------- Discovery ----------------------------------
#
discovery.zen.ping.unicast.hosts: ["192.168.1.90","192.168.1.188","192.168.1.199"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
discovery.zen.minimum_master_nodes: 2
#

elasticnodev02

# ======================== Elasticsearch Configuration =========================
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: SecurityMetrics
cluster.remote.connect: false
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: elasticnodev02
node.master: true
node.data: true
node.ingest: true
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.1.188
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["192.168.1.90","192.168.1.188","192.168.1.199"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
discovery.zen.minimum_master_nodes: 2
#

elasticnodev03

# ======================== Elasticsearch Configuration =========================
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: SecurityMetrics
cluster.remote.connect: false
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: elasticnodev03
node.master: true
node.data: false
node.ingest: false
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.1.199
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["192.168.1.90","192.168.1.188","192.168.1.199"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
discovery.zen.minimum_master_nodes: 2
#

Testing

curl http://192.168.1.90:9200
{
  "name" : "elasticnodev01",
  "cluster_name" : "SecurityMetrics",
  "cluster_uuid" : "Y98r9i6MQJmpl1ddQbZq9w",
  "version" : {
    "number" : "6.5.4",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "d2ef93d",
    "build_date" : "2018-12-17T21:17:40.758843Z",
    "build_snapshot" : false,
    "lucene_version" : "7.5.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

curl http://192.168.1.90:9200/_cluster/health?pretty
{
  "cluster_name" : "SecurityMetrics",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 2,
  "active_primary_shards" : 1,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

Useful API calls

Health Check

curl GET http://192.168.1.199/_cluster/health/?pretty