kugarocks

English

简体中文

English

简体中文

Appearance

Shard Recovery ​

Check State ​

bash
curl http://localhost:9200/_cluster/health?pretty
bash
{
  "cluster_name" : "elasticsearch",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 1191,
  "active_shards" : 1191,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 1178,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 50.274377374419586
}

unassigned_shards active_shards_percent_as_number

Check Shard Allocation Info ​

bash
curl -X GET "localhost:9200/_cluster/allocation/explain?pretty"
bash
{
  "index" : "code_projects-20240827",
  "shard" : 0,
  "primary" : false,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "CLUSTER_RECOVERED",
    "at" : "2025-05-08T10:29:17.209Z",
    "last_allocation_status" : "no_attempt"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "2lM5SXRSSNaODGtEVMxMQw",
      "node_name" : "foobar-elasticsearch-0",
      "transport_address" : "100.114.216.147:9300",
      "node_attributes" : {
        "ml.machine_memory" : "8589934592",
        "xpack.installed" : "true",
        "transform.node" : "true",
        "ml.max_open_jobs" : "20"
      },
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "same_shard",
          "decision" : "NO",
          "explanation" : "a copy of this shard is already allocated to this node [[code_projects-20240827][0], node[2lM5SXRSSNaODGtEVMxMQw], [P], s[STARTED], a[id=jySmkoYeQQuj8ivw33fh7w]]"
        }
      ]
    }
  ]
}
  • A copy of this shard is already allocated to this node
  • [P], s[STARTED]: Primary shard is already in this node
  • No more replicas can allocate in this node
bash
      "deciders" : [
        {
          "decider" : "same_shard",
          "decision" : "NO",
          "explanation" : "a copy of this shard is already allocated to this node [[wiki2_test41][0], node[2lM5SXRSSNaODGtEVMxMQw], [P], s[STARTED], a[id=DCBNC494SJGfy5Jechp07w]]"
        },
        {
          "decider" : "throttling",
          "decision" : "THROTTLE",
          "explanation" : "reached the limit of incoming shard recoveries [6], cluster setting [cluster.routing.allocation.node_concurrent_incoming_recoveries=4] (can also be set via [cluster.routing.allocation.node_concurrent_recoveries])"
        }

You can change the recover settings to speed up the shard allocate.

bash
curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d '{
  "persistent": {
    "cluster.routing.allocation.node_concurrent_incoming_recoveries": 4
  }
}'
bash
curl -X PUT "http://<your_elasticsearch_host>:9200/_cluster/settings" -H 'Content-Type: application/json' -d '{
  "persistent": {
    "cluster.routing.allocation.node_initial_primaries_recoveries": 6
  }
}'

Check Cluster Settings ​

bash
curl -X GET "localhost:9200/_cluster/settings?pretty"
bash
{
  "persistent" : {
    "cluster" : {
      "routing" : {
        "allocation" : {
          "node_concurrent_incoming_recoveries" : "4",
          "node_initial_primaries_recoveries" : "6"
        }
      },
      "max_shards_per_node" : "3000"
    }
  },
  "transient" : { }
}

Check Template Settings ​

number_of_shards number_of_replicas

bash
curl -X GET 'localhost:9200/_index_template?pretty'
bash
curl 'localhost:9200/_index_template?pretty' | grep -A 20 index_templates
bash
{
  "index_templates" : [
    {
      "name" : "ilm-history",
      "index_template" : {
        "index_patterns" : [
          "ilm-history-3*"
        ],
        "template" : {
          "settings" : {
            "index" : {
              "format" : "1",
              "lifecycle" : {
                "name" : "ilm-history-ilm-policy",
                "rollover_alias" : "ilm-history-3"
              },
              "hidden" : "true",
              "number_of_shards" : "1",
              "auto_expand_replicas" : "0-1",
              "number_of_replicas" : "0"
            }
          },

Delete Index ​

bash
curl -X DELETE "localhost:9200/.kibana-event-log-7.10.2-000007"

Check Disk Stat ​

bash
curl -X GET 'localhost:9200/_cat/allocation?v'
bash
shards disk.indices disk.used disk.avail disk.total disk.percent host            ip              node
  1191        9.9gb     1.1tb    253.3gb      1.4tb           82 100.114.216.147 100.114.216.147 foobar-elasticsearch-0
  1178                                                                                           UNASSIGNED

Change Replicas Number ​

Only in 1 Node.

bash
curl -X PUT "localhost:9200/_all/_settings" -H 'Content-Type: application/json' -d'
{
  "index" : {
    "number_of_replicas" : 0
  }
}'

Check Recovery Progress ​

Useless sometimes.

bash
curl -X GET "localhost:9200/_cat/recovery?v"