OpenSearch
Add pipeline with grok
PUT _ingest/pipeline/<pipeline_name>
{
"description": "<description>",
"processors": [
{
"grok": {
"field": "message",
"patterns": [
"%{MONTHDAY}-%{MONTH}-%{YEAR} %{TIME} %{INT:PID} %{USERNAME} %{LOGLEVEL:loglevel} %{WORD:function} %{URIPATHPARAM:request}"
]
}
}
]
}
Get information
GET /_cat/indices?v
GET /_cluster/health
GET /_cluster/settings?include_defaults=true
GET /_cat/shards/<index name>?v
GET /_cat/indices/<index name>?v
Set shard limit
PUT /_cluster/settings
{
"persistent": {
"cluster.max_shards_per_node" <value>
}
}
Delete indeces by mask
PUT /_cluster/settings
{
"transient": {
"action.destructive_requires_name": true
}
}
DELETE /<mask>
PUT /_cluster/settings
{
"transient": {
"action.destructive_requires_name": false
}
}
Open/Close index
POST /<index name>_close
POST /<index name>_open
Policy assignment
PORST _plugins/_ism/add/<mask>
{
"policy_id": "<policy name>"
}
Remove the read only lock
GET /<index name>/_settings
PUT /<index name>/_settings
{
"index": {
"blocks": {
"read_only_allow_delete": false
}
}
}
PUT /_all/_settings
{
"index": {
"blocks": {
"read_only_allow_delete": false
}
}
}
- Java Heap Size <32GB
- <20 shards on 1GB RAM
- <40-50GB on 1 shard
- GB on 1GB of RAM
- 30GB for HOT
- 160GB for WARM
- 500GB for COLD
- Amount of data
- total amount of data =
* * <transformation factor 1> * - shared data storage =
* (1 + 0,15 + 0,1) - total number of nodes =
/ / <GB on 1GB of RAM>
- total amount of data =
- bandwidth
- peak value of the threads =
* / 1000ms - thread pool volume = (
* <number of threads per core * 3 / 2> + 1) - number of data nodes =
/
- peak value of the threads =