Skip to main content

Backup. Snapshots en Local (desarrollo)

ATENCIÓN ---> Es un articulo en desarrollo.

Introducción

Como todo y más aun un un software que puede contener millones y millones de filas, que sería de nosotros sin tener un sistemas de backups, llamado en el entorno ELK, snapshots En este artíuclo me centro en la snapshot en modo local Shared file system repository , por estar en desarrolloy ser bueno para hacewr algunas cosas como migrar contenidos a local, para pruebas de reindexación, etc,

Snapshots

Creamos el path local

mkdir /snapshot_elk
chown -R elasticsearch:elasticsearch /snapshot_elk

Configuración

  • Parar elasticsearch
systemctl stop elasticsearch
  • Editar /etc/elasticsearch/elasticsearch.yml
path.repo: /snapshot_elk
  • Iniciar elasticsearch
systemctl start elasticsearch 

Registrar el repositorio de backup

Vamos a realizar esta operación via consola, aunque tambien podemos hacerla via kibana Registrar repositorio backup via kibana

Recordar nuestro artículo Guia de comandos útiles para un rápido vistazo a Elasticsearch ya que en una instalación por defecto securizada (como todas desde la versión 8) es necesario la autenticación

ip=localhost
p=puerto
password=contraseña
usuario=usario
  • Registro del repositorio
[email protected]:~#  curl --cacert /etc/elasticsearch/certs/http_ca.crt -u $usuario:$password -X PUT "https://$ip:$p/_snapshot/backup?pretty" -H 'Content-Type: application/json' -d'
{
    "type": "fs",
    "settings": {
        "location": "/snapshot_elk",
        "compress": true
    }
}
'
{
  "acknowledged" : true
}

El json devuelto con el acknowledged nos indica que se registro.

Verificar

[email protected]:~# curl --cacert /etc/elasticsearch/certs/http_ca.crt -u $usuario:$password -X GET "https://$ip:$p/_snapshot/_all?pretty"
{
  "backup" : {
    "type" : "fs",
    "settings" : {
      "compress" : "true",
      "location" : "/snapshot_elk"
    }
  }
}

Tambien podemos verificarlo en kibana Verificar repositorio backup via kibana

Crear un snapshot global

[email protected]:~#  curl --cacert /etc/elasticsearch/certs/http_ca.crt -u $usuario:$password -X PUT "https://$ip:$p/_snapshot/backup/snapshot_001?wait_for_completion=true&pretty"
{
  "snapshot" : {
    "snapshot" : "snapshot_001",
    "uuid" : "MJgQEC3jRMu1WXIH5Rom3w",
    "repository" : "backup",
    "version_id" : 8030199,
    "version" : "8.3.1",
    "indices" : [
      ".kibana_8.3.1_001",
      ".tasks",
      ".kibana_task_manager_8.2.3_001",
      ".kibana-event-log-8.2.3-000001",
      ".apm-custom-link",
      ".kibana_8.2.3_001",
      ".apm-agent-configuration",
      ".kibana-event-log-8.3.1-000001",
      ".geoip_databases",
      ".kibana_task_manager_8.3.1_001",
      ".ds-.logs-deprecation.elasticsearch-default-2022.06.18-000001",
      ".kibana_security_session_1",
      ".ds-ilm-history-5-2022.06.18-000001",
      "analyzers",
      ".security-7"
    ],
    "data_streams" : [
      "ilm-history-5",
      ".logs-deprecation.elasticsearch-default"
    ],
    "include_global_state" : true,
    "state" : "SUCCESS",
    "start_time" : "2022-07-03T16:41:22.668Z",
    "start_time_in_millis" : 1656866482668,
    "end_time" : "2022-07-03T16:41:49.284Z",
    "end_time_in_millis" : 1656866509284,
    "duration_in_millis" : 26616,
    "failures" : [ ],
    "shards" : {
      "total" : 15,
      "failed" : 0,
      "successful" : 15
    },
    "feature_states" : [
      {
        "feature_name" : "geoip",
        "indices" : [
          ".geoip_databases"
        ]
      },
      {
        "feature_name" : "kibana",
        "indices" : [
          ".kibana_8.3.1_001",
          ".kibana_8.2.3_001",
          ".apm-custom-link",
          ".apm-agent-configuration",
          ".kibana_task_manager_8.2.3_001",
          ".kibana_security_session_1",
          ".kibana_task_manager_8.3.1_001"
        ]
      },
      {
        "feature_name" : "tasks",
        "indices" : [
          ".tasks"
        ]
      },
      {
        "feature_name" : "security",
        "indices" : [
          ".security-7"
        ]
      }
    ]
  }
}

Crear un snapshot de un indice concreto

En mi caso tengo varios indices de demo que no quiero en el indice en preprudcción, por lo que opto por eleiminar primero el global, para solo ocuparme del deseado.

Es importante entender que estos backups no son de sistema, sino de aplicación, por lo que hay cosas que deben ser guardadas de otras formas habituales (configuración, claves, llaves, certificados, etc)

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u $usuario:$password -X PUT "https://$ip:$p/_snapshot/backup/snapshot_index_analyzer_001?wait_for_completion=true&pretty" -H 'Content-Type: application/json' -d'
{
  "indices": "analyzers",
  "ignore_unavailable": true,
  "include_global_state": false,
  "metadata": {
    "taken_by": "Abdelkarim",
    "taken_because": "Backup of the index named ANALYZERS"
  }
}
'
# salida
{
  "snapshot" : {
    "snapshot" : "snapshot_index_analyzer_001",
    "uuid" : "F0PpT4wESm-uzMHqWfmaWQ",
    "repository" : "backup",
    "version_id" : 8030199,
    "version" : "8.3.1",
    "indices" : [
      "analyzers"
    ],
    "data_streams" : [ ],
    "include_global_state" : false,
    "metadata" : {
      "taken_by" : "Abdelkarim",
      "taken_because" : "Backup of the index named ANALYZERS"
    },
    "state" : "SUCCESS",
    "start_time" : "2022-07-03T16:54:21.612Z",
    "start_time_in_millis" : 1656867261612,
    "end_time" : "2022-07-03T16:54:46.227Z",
    "end_time_in_millis" : 1656867286227,
    "duration_in_millis" : 24615,
    "failures" : [ ],
    "shards" : {
      "total" : 1,
      "failed" : 0,
      "successful" : 1
    },
    "feature_states" : [ ]
  }
}
'

Listado de snapshots

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u $usuario:$password -X GET "https://$ip:$p/_cat/snapshots/backup?v&s=id&pretty" 
id                          repository  status start_epoch start_time end_epoch  end_time duration indices successful_shards failed_shards total_shards
snapshot_index_analyzer_001 backup     SUCCESS 1656867261  16:54:21   1656867286 16:54:46    24.6s       1                 1             0            1

Información de un snapshot especifico

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u $usuario:$password -X GET "https://$ip:$p/_cat/snapshots/backup/?pretty" 
snapshot_index_analyzer_001 backup SUCCESS 1656867261 16:54:21 1656867286 16:54:46 24.6s 1 1 0 1