Skip to main content
Notes by Peter Galonza(Пётр Галонза)
GitHub Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

OpenSearch

Requests

Add pipeline with grok

PUT _ingest/pipeline/<pipeline_name>
{
  "description": "<description>",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": [
          "%{MONTHDAY}-%{MONTH}-%{YEAR} %{TIME} %{INT:PID} %{USERNAME} %{LOGLEVEL:loglevel} %{WORD:function} %{URIPATHPARAM:request}"
        ]
      }
    }
  ]
}

Get information

GET /_cat/indices?v
GET /_cluster/health
GET /_cluster/settings?include_defaults=true
GET /_cat/shards/<index name>?v
GET /_cat/indices/<index name>?v

Set shard limit

PUT /_cluster/settings
{
  "persistent": {
    "cluster.max_shards_per_node" <value>
  }
}

Delete indeces by mask

PUT /_cluster/settings
{
  "transient": {
    "action.destructive_requires_name": true
  }
}

DELETE /<mask>

PUT /_cluster/settings
{
  "transient": {
    "action.destructive_requires_name": false
  }
}

Open/Close index

POST /<index name>_close
POST /<index name>_open

Policy assignment

PORST _plugins/_ism/add/<mask>
{
  "policy_id": "<policy name>"
}

Remove the read only lock

GET /<index name>/_settings

PUT /<index name>/_settings
{
  "index": {
    "blocks": {
      "read_only_allow_delete": false
    }
  }
}

PUT /_all/_settings
{
  "index": {
    "blocks": {
      "read_only_allow_delete": false
    }
  }
}

DataStream

Recomendations

  • Java Heap Size <32GB
  • <20 shards on 1GB RAM
  • <40-50GB on 1 shard
  • GB on 1GB of RAM
    • 30GB for HOT
    • 160GB for WARM
    • 500GB for COLD
  • Amount of data
    • total amount of data = * * <transformation factor 1> *
    • shared data storage = * (1 + 0,15 + 0,1)
    • total number of nodes = / / <GB on 1GB of RAM>
  • bandwidth
    • peak value of the threads = * / 1000ms
    • thread pool volume = ( * <number of threads per core * 3 / 2> + 1)
    • number of data nodes = /