Elasticsearch Integration¶
Infrastructure Integration¶
Instructions¶
Follow the installation guide below for your given collector environment.
During installation, use the configuration section below as reference.
After installation, the infrastructure datasources in the table below will be available in the AOC.
Installation Guide¶
Installing this integration consists of creating a yaml
file in the filesystem of your collectors.
Click below for instructions on how to do so for your given collectors environment.
Configuration¶
Edit elastic.yaml
to configure the agent. Please refer elastic.yaml
for all available configuration options.
init_config:
instances:
- url: http://localhost:9200
Infrastructure Datasources¶
Datasource | Available Aggregations | Unit | Description |
---|---|---|---|
elasticsearch.active_primary_shards | avg max min sum |
shard | The number of active primary shards in the cluster. |
elasticsearch.active_shards | avg max min sum |
shard | The number of active shards in the cluster. |
elasticsearch.breakers.fielddata.estimated_size_in_bytes | avg max min sum |
byte | The estimated size in bytes of the field data circuit breaker. |
elasticsearch.breakers.fielddata.overhead | avg max min sum |
The constant multiplier for byte estimations of the field data circuit breaker. | |
elasticsearch.breakers.fielddata.tripped | avg max min sum |
The number of times the field data circuit breaker has tripped. | |
elasticsearch.breakers.parent.estimated_size_in_bytes | avg max min sum |
byte | The estimated size in bytes of the parent circuit breaker. |
elasticsearch.breakers.parent.overhead | avg max min sum |
The constant multiplier for byte estimations of the parent circuit breaker. | |
elasticsearch.breakers.parent.tripped | avg max min sum |
The number of times the parent circuit breaker has tripped. | |
elasticsearch.breakers.request.estimated_size_in_bytes | avg max min sum |
byte | The estimated size in bytes of the request circuit breaker. |
elasticsearch.breakers.request.overhead | avg max min sum |
The constant multiplier for byte estimations of the request circuit breaker. | |
elasticsearch.breakers.request.tripped | avg max min sum |
The number of times the request circuit breaker has tripped. | |
elasticsearch.cache.field.evictions | avg max min sum |
eviction | The total number of evictions from the field data cache. |
elasticsearch.cache.field.size | avg max min sum |
byte | The size of the field cache. |
elasticsearch.cache.filter.count | avg max min sum |
item | The number of items in the filter cache. |
elasticsearch.cache.filter.evictions | avg max min sum |
eviction | The total number of evictions from the filter cache. |
elasticsearch.cache.filter.size | avg max min sum |
byte | The size of the filter cache. |
elasticsearch.cluster_status | avg max min sum |
The elasticsearch cluster health as a number: red = 0, yellow = 1, green = 2 | |
elasticsearch.docs.count | avg max min sum |
document | The total number of documents in the cluster across all shards. |
elasticsearch.docs.deleted | avg max min sum |
document | The total number of documents deleted from the cluster across all shards. |
elasticsearch.fielddata.evictions | avg max min sum |
eviction | The total number of evictions from the fielddata cache. |
elasticsearch.fielddata.size | avg max min sum |
byte | The size of the fielddata cache. |
elasticsearch.flush.total | avg max min sum |
flush | The total number of index flushes to disk since start. |
elasticsearch.flush.total.time | avg max min sum |
second | The total time spent flushing the index to disk. |
elasticsearch.fs.total.available_in_bytes | avg max min sum |
byte | The total number of bytes available to this Java virtual machine on this file store. |
elasticsearch.fs.total.disk_io_op | avg max min sum |
operation | The total I/O operations on the file store. |
elasticsearch.fs.total.disk_io_size_in_bytes | avg max min sum |
byte | Total bytes used for all I/O operations on the file store. |
elasticsearch.fs.total.disk_read_size_in_bytes | avg max min sum |
byte | The total bytes read from the file store. |
elasticsearch.fs.total.disk_reads | avg max min sum |
read | The total number of reads from the file store. |
elasticsearch.fs.total.disk_write_size_in_bytes | avg max min sum |
byte | The total bytes written to the file store. |
elasticsearch.fs.total.disk_writes | avg max min sum |
write | The total number of writes to the file store. |
elasticsearch.fs.total.free_in_bytes | avg max min sum |
byte | The total number of unallocated bytes in the file store. |
elasticsearch.fs.total.total_in_bytes | avg max min sum |
byte | The total size in bytes of the file store. |
elasticsearch.get.current | avg max min sum |
request | The number of get requests currently running. |
elasticsearch.get.exists.time | avg max min sum |
second | The total time spent on get requests where the document existed. |
elasticsearch.get.exists.total | avg max min sum |
request | The total number of get requests where the document existed. |
elasticsearch.get.missing.time | avg max min sum |
second | The total time spent on get requests where the document was missing. |
elasticsearch.get.missing.total | avg max min sum |
request | The total number of get requests where the document was missing. |
elasticsearch.get.time | avg max min sum |
second | The total time spent on get requests. |
elasticsearch.get.total | avg max min sum |
request | The total number of get requests. |
elasticsearch.http.current_open | avg max min sum |
connection | The number of current open HTTP connections. |
elasticsearch.http.total_opened | avg max min sum |
connection | The total number of opened HTTP connections. |
elasticsearch.id_cache.size | avg max min sum |
byte | The size of the id cache |
elasticsearch.indexing.delete.current | avg max min sum |
document | The number of documents currently being deleted from an index. |
elasticsearch.indexing.delete.time | avg max min sum |
second | The total time spent deleting documents from an index. |
elasticsearch.indexing.delete.total | avg max min sum |
document | The total number of documents deleted from an index. |
elasticsearch.indexing.index.current | avg max min sum |
document | The number of documents currently being indexed to an index. |
elasticsearch.indexing.index.time | avg max min sum |
second | The total time spent indexing documents to an index. |
elasticsearch.indexing.index.total | avg max min sum |
document | The total number of documents indexed to an index. |
elasticsearch.indices.indexing.index_failed | avg max min sum |
The number of failed indexing operations. | |
elasticsearch.indices.indexing.throttle_time | avg max min sum |
millisecond | The total time indexing waited due to throttling. |
elasticsearch.indices.query_cache.evictions | avg max min sum |
eviction | The number of query cache evictions. |
elasticsearch.indices.query_cache.hit_count | avg max min sum |
hit | The number of query cache hits. |
elasticsearch.indices.query_cache.memory_size_in_bytes | avg max min sum |
byte | The memory used by the query cache. |
elasticsearch.indices.query_cache.miss_count | avg max min sum |
miss | The number of query cache misses. |
elasticsearch.indices.recovery.current_as_source | avg max min sum |
The number of ongoing recoveries for which a shard serves as a source. | |
elasticsearch.indices.recovery.current_as_target | avg max min sum |
The number of ongoing recoveries for which a shard serves as a target. | |
elasticsearch.indices.recovery.throttle_time | avg max min sum |
millisecond | The total time recoveries waited due to throttling. |
elasticsearch.indices.request_cache.evictions | avg max min sum |
eviction | The number of request cache evictions. |
elasticsearch.indices.request_cache.hit_count | avg max min sum |
hit | The number of request cache hits. |
elasticsearch.indices.request_cache.memory_size_in_bytes | avg max min sum |
byte | The memory used by the request cache. |
elasticsearch.indices.request_cache.miss_count | avg max min sum |
miss | The number of request cache misses. |
elasticsearch.indices.segments.count | avg max min sum |
segment | The number of segments in an index shard. |
elasticsearch.indices.segments.doc_values_memory_in_bytes | avg max min sum |
byte | The memory used by doc values. |
elasticsearch.indices.segments.fixed_bit_set_memory_in_bytes | avg max min sum |
byte | The memory used by fixed bit set. |
elasticsearch.indices.segments.index_writer_max_memory_in_bytes | avg max min sum |
byte | The maximum memory used by the index writer. |
elasticsearch.indices.segments.index_writer_memory_in_bytes | avg max min sum |
byte | The memory used by the index writer. |
elasticsearch.indices.segments.memory_in_bytes | avg max min sum |
byte | The memory used by index segments. |
elasticsearch.indices.segments.norms_memory_in_bytes | avg max min sum |
byte | The memory used by norms. |
elasticsearch.indices.segments.stored_fields_memory_in_bytes | avg max min sum |
byte | The memory used by stored fields. |
elasticsearch.indices.segments.term_vectors_memory_in_bytes | avg max min sum |
byte | The memory used by term vectors. |
elasticsearch.indices.segments.terms_memory_in_bytes | avg max min sum |
byte | The memory used by terms. |
elasticsearch.indices.segments.version_map_memory_in_bytes | avg max min sum |
byte | The memory used by the segment version map. |
elasticsearch.indices.translog.operations | avg max min sum |
operation | The number of operations in the transaction log. |
elasticsearch.indices.translog.size_in_bytes | avg max min sum |
byte | The size of the transaction log. |
elasticsearch.initializing_shards | avg max min sum |
shard | The number of shards that are currently initializing. |
elasticsearch.merges.current | avg max min sum |
merge | The number of currently active segment merges. |
elasticsearch.merges.current.docs | avg max min sum |
document | The number of documents across segments currently being merged. |
elasticsearch.merges.current.size | avg max min sum |
byte | The size of the segments currently being merged. |
elasticsearch.merges.total | avg max min sum |
merge | The total number of segment merges. |
elasticsearch.merges.total.docs | avg max min sum |
document | The total number of documents across all merged segments. |
elasticsearch.merges.total.size | avg max min sum |
byte | The total size of all merged segments. |
elasticsearch.merges.total.time | avg max min sum |
second | The total time spent on segment merging. |
elasticsearch.number_of_data_nodes | avg max min sum |
node | The number of data nodes in the cluster. |
elasticsearch.number_of_nodes | avg max min sum |
node | The total number of nodes in the cluster. |
elasticsearch.pending_tasks_priority_high | avg max min sum |
task | The number of high priority pending tasks. |
elasticsearch.pending_tasks_priority_urgent | avg max min sum |
task | The number of urgent priority pending tasks. |
elasticsearch.pending_tasks_total | avg max min sum |
task | The total number of pending tasks. |
elasticsearch.primaries.docs.count | avg max min sum |
document | The total number of documents in the primary shards. |
elasticsearch.primaries.docs.deleted | avg max min sum |
document | The total number of documents deleted from the primary shards. |
elasticsearch.primaries.flush.total | avg max min sum |
flush | The total number of index flushes to disk from the primary shards since start. |
elasticsearch.primaries.flush.total.time | avg max min sum |
second | The total time spent flushing the index to disk from the primary shards. |
elasticsearch.primaries.get.current | avg max min sum |
request | The number of get requests currently running on the primary shards. |
elasticsearch.primaries.get.exists.time | avg max min sum |
request | The total time spent on get requests from the primary shards where the document existed. |
elasticsearch.primaries.get.exists.total | avg max min sum |
request | The total number of get requests on primary shards where the document existed. |
elasticsearch.primaries.get.missing.time | avg max min sum |
second | The total time spent on get requests from the primary shards where the document was missing. |
elasticsearch.primaries.get.missing.total | avg max min sum |
request | The total number of get requests from the primary shards where the document was missing. |
elasticsearch.primaries.get.time | avg max min sum |
second | The total time spent on get requests from the primary shards. |
elasticsearch.primaries.get.total | avg max min sum |
request | The total number of get requests from the primary shards. |
elasticsearch.primaries.indexing.delete.current | avg max min sum |
document | The number of documents currently being deleted from an index on the primary shards. |
elasticsearch.primaries.indexing.delete.time | avg max min sum |
second | The total time spent deleting documents from an index on the primary shards. |
elasticsearch.primaries.indexing.delete.total | avg max min sum |
document | The total number of documents deleted from an index on the primary shards. |
elasticsearch.primaries.indexing.index.current | avg max min sum |
document | The number of documents currently being indexed to an index on the primary shards. |
elasticsearch.primaries.indexing.index.time | avg max min sum |
second | The total time spent indexing documents to an index on the primary shards. |
elasticsearch.primaries.indexing.index.total | avg max min sum |
document | The total number of documents indexed to an index on the primary shards. |
elasticsearch.primaries.merges.current | avg max min sum |
merge | The number of currently active segment merges on the primary shards. |
elasticsearch.primaries.merges.current.docs | avg max min sum |
document | The number of documents across segments currently being merged on the primary shards. |
elasticsearch.primaries.merges.current.size | avg max min sum |
byte | The size of the segments currently being merged on the primary shards. |
elasticsearch.primaries.merges.total | avg max min sum |
merge | The total number of segment merges on the primary shards. |
elasticsearch.primaries.merges.total.docs | avg max min sum |
document | The total number of documents across all merged segments on the primary shards. |
elasticsearch.primaries.merges.total.size | avg max min sum |
byte | The total size of all merged segments on the primary shards. |
elasticsearch.primaries.merges.total.time | avg max min sum |
second | The total time spent on segment merging on the primary shards. |
elasticsearch.primaries.refresh.total | avg max min sum |
refresh | The total number of index refreshes on the primary shards. |
elasticsearch.primaries.refresh.total.time | avg max min sum |
second | The total time spent on index refreshes on the primary shards. |
elasticsearch.primaries.search.fetch.current | avg max min sum |
fetch | The number of query fetches currently running on the primary shards. |
elasticsearch.primaries.search.fetch.time | avg max min sum |
second | The total time spent on query fetches on the primary shards. |
elasticsearch.primaries.search.fetch.total | avg max min sum |
fetch | The total number of query fetches on the primary shards. |
elasticsearch.primaries.search.query.current | avg max min sum |
query | The number of currently active queries on the primary shards. |
elasticsearch.primaries.search.query.time | avg max min sum |
second | The total time spent querying on the primary shards. |
elasticsearch.primaries.search.query.total | avg max min sum |
query | The total number of queries to the primary shards. |
elasticsearch.primaries.store.size | avg max min sum |
byte | The total size of all the primary shards. |
elasticsearch.process.open_fd | avg max min sum |
file | The number of opened file descriptors associated with the current process, or -1 if not supported. |
elasticsearch.refresh.total | avg max min sum |
refresh | The total number of index refreshes. |
elasticsearch.refresh.total.time | avg max min sum |
second | The total time spent on index refreshes. |
elasticsearch.relocating_shards | avg max min sum |
shard | The number of shards that are reloacting from one node to another. |
elasticsearch.search.fetch.current | avg max min sum |
fetch | The number of search fetches currently running. |
elasticsearch.search.fetch.open_contexts | avg max min sum |
query | The number of active searches. |
elasticsearch.search.fetch.time | avg max min sum |
second | The total time spent on the search fetch. |
elasticsearch.search.fetch.total | avg max min sum |
fetch | The total number of search fetches. |
elasticsearch.search.query.current | avg max min sum |
query | The number of currently active queries. |
elasticsearch.search.query.time | avg max min sum |
second | The total time spent on queries. |
elasticsearch.search.query.total | avg max min sum |
query | The total number of queries. |
elasticsearch.store.size | avg max min sum |
byte | The total size in bytes of the store. |
elasticsearch.thread_pool.bulk.active | avg max min sum |
thread | The number of active threads in the bulk pool. |
elasticsearch.thread_pool.bulk.queue | avg max min sum |
thread | The number of queued threads in the bulk pool. |
elasticsearch.thread_pool.bulk.threads | avg max min sum |
thread | The total number of threads in the bulk pool. |
elasticsearch.thread_pool.bulk.rejected | avg max min sum |
thread | The number of rejected threads in the bulk pool. |
elasticsearch.thread_pool.fetch_shard_started.active | avg max min sum |
thread | The number of active threads in the fetch shard started pool. |
elasticsearch.thread_pool.fetch_shard_started.threads | avg max min sum |
thread | The total number of threads in the fetch shard started pool. |
elasticsearch.thread_pool.fetch_shard_started.queue | avg max min sum |
thread | The number of queued threads in the fetch shard started pool. |
elasticsearch.thread_pool.fetch_shard_started.rejected | avg max min sum |
thread | The number of rejected threads in the fetch shard started pool. |
elasticsearch.thread_pool.fetch_shard_store.active | avg max min sum |
thread | The number of active threads in the fetch shard store pool. |
elasticsearch.thread_pool.fetch_shard_store.threads | avg max min sum |
thread | The total number of threads in the fetch shard store pool. |
elasticsearch.thread_pool.fetch_shard_store.queue | avg max min sum |
thread | The number of queued threads in the fetch shard store pool. |
elasticsearch.thread_pool.fetch_shard_store.rejected | avg max min sum |
thread | The number of rejected threads in the fetch shard store pool. |
elasticsearch.thread_pool.flush.active | avg max min sum |
thread | The number of active threads in the flush queue. |
elasticsearch.thread_pool.flush.queue | avg max min sum |
thread | The number of queued threads in the flush pool. |
elasticsearch.thread_pool.flush.threads | avg max min sum |
thread | The total number of threads in the flush pool. |
elasticsearch.thread_pool.flush.rejected | avg max min sum |
thread | The number of rejected threads in the flush pool. |
elasticsearch.thread_pool.force_merge.active | avg max min sum |
thread | The number of active threads for force merge operations. |
elasticsearch.thread_pool.force_merge.threads | avg max min sum |
thread | The total number of threads for force merge operations. |
elasticsearch.thread_pool.force_merge.queue | avg max min sum |
thread | The number of queued threads for force merge operations. |
elasticsearch.thread_pool.force_merge.rejected | avg max min sum |
thread | The number of rejected threads for force merge operations. |
elasticsearch.thread_pool.generic.active | avg max min sum |
thread | The number of active threads in the generic pool. |
elasticsearch.thread_pool.generic.queue | avg max min sum |
thread | The number of queued threads in the generic pool. |
elasticsearch.thread_pool.generic.threads | avg max min sum |
thread | The total number of threads in the generic pool. |
elasticsearch.thread_pool.generic.rejected | avg max min sum |
thread | The number of rejected threads in the generic pool. |
elasticsearch.thread_pool.get.active | avg max min sum |
thread | The number of active threads in the get pool. |
elasticsearch.thread_pool.get.queue | avg max min sum |
thread | The number of queued threads in the get pool. |
elasticsearch.thread_pool.get.threads | avg max min sum |
thread | The total number of threads in the get pool. |
elasticsearch.thread_pool.get.rejected | avg max min sum |
thread | The number of rejected threads in the get pool. |
elasticsearch.thread_pool.index.active | avg max min sum |
thread | The number of active threads in the index pool. |
elasticsearch.thread_pool.index.queue | avg max min sum |
thread | The number of queued threads in the index pool. |
elasticsearch.thread_pool.index.threads | avg max min sum |
thread | The total number of threads in the index pool. |
elasticsearch.thread_pool.index.rejected | avg max min sum |
thread | The number of rejected threads in the index pool. |
elasticsearch.thread_pool.listener.active | avg max min sum |
thread | The number of active threads in the listener pool. |
elasticsearch.thread_pool.listener.queue | avg max min sum |
thread | The number of queued threads in the listener pool. |
elasticsearch.thread_pool.listener.threads | avg max min sum |
thread | The total number of threads in the listener pool. |
elasticsearch.thread_pool.listener.rejected | avg max min sum |
thread | The number of rejected threads in the listener pool. |
elasticsearch.thread_pool.management.active | avg max min sum |
thread | The number of active threads in the management pool. |
elasticsearch.thread_pool.management.queue | avg max min sum |
thread | The number of queued threads in the management pool. |
elasticsearch.thread_pool.management.threads | avg max min sum |
thread | The total number of threads in the management pool. |
elasticsearch.thread_pool.management.rejected | avg max min sum |
thread | The number of rejected threads in the management pool. |
elasticsearch.thread_pool.merge.active | avg max min sum |
thread | The number of active threads in the merge pool. |
elasticsearch.thread_pool.merge.queue | avg max min sum |
thread | The number of queued threads in the merge pool. |
elasticsearch.thread_pool.merge.threads | avg max min sum |
thread | The total number of threads in the merge pool. |
elasticsearch.thread_pool.merge.rejected | avg max min sum |
thread | The number of rejected threads in the merge pool. |
elasticsearch.thread_pool.percolate.active | avg max min sum |
thread | The number of active threads in the percolate pool. |
elasticsearch.thread_pool.percolate.queue | avg max min sum |
thread | The number of queued threads in the percolate pool. |
elasticsearch.thread_pool.percolate.threads | avg max min sum |
thread | The total number of threads in the percolate pool. |
elasticsearch.thread_pool.percolate.rejected | avg max min sum |
thread | The number of rejected threads in the percolate pool. |
elasticsearch.thread_pool.refresh.active | avg max min sum |
thread | The number of active threads in the refresh pool. |
elasticsearch.thread_pool.refresh.queue | avg max min sum |
thread | The number of queued threads in the refresh pool. |
elasticsearch.thread_pool.refresh.threads | avg max min sum |
thread | The total number of threads in the refresh pool. |
elasticsearch.thread_pool.refresh.rejected | avg max min sum |
thread | The number of rejected threads in the refresh pool. |
elasticsearch.thread_pool.search.active | avg max min sum |
thread | The number of active threads in the search pool. |
elasticsearch.thread_pool.search.queue | avg max min sum |
thread | The number of queued threads in the search pool. |
elasticsearch.thread_pool.search.threads | avg max min sum |
thread | The total number of threads in the search pool. |
elasticsearch.thread_pool.search.rejected | avg max min sum |
thread | The number of rejected threads in the search pool. |
elasticsearch.thread_pool.snapshot.active | avg max min sum |
thread | The number of active threads in the snapshot pool. |
elasticsearch.thread_pool.snapshot.queue | avg max min sum |
thread | The number of queued threads in the snapshot pool. |
elasticsearch.thread_pool.snapshot.threads | avg max min sum |
thread | The total number of threads in the snapshot pool. |
elasticsearch.thread_pool.snapshot.rejected | avg max min sum |
thread | The number of rejected threads in the snapshot pool. |
elasticsearch.transport.rx_count | avg max min sum |
packet | The total number of packets received in cluster communication. |
elasticsearch.transport.rx_size | avg max min sum |
byte | The total size of data received in cluster communication. |
elasticsearch.transport.server_open | avg max min sum |
connection | The number of connections opened for cluster communication. |
elasticsearch.transport.tx_count | avg max min sum |
packet | The total number of packets sent in cluster communication. |
elasticsearch.transport.tx_size | avg max min sum |
byte | The total size of data sent in cluster communication. |
elasticsearch.unassigned_shards | avg max min sum |
shard | The number of shards that are unassigned to a node. |
jvm.gc.collection_count | avg max min sum |
garbage collection | The total number of garbage collections run by the JVM. |
jvm.gc.collection_time | avg max min sum |
second | The total time spent on garbage collection in the JVM. |
jvm.gc.collectors.old.collection_time | avg max min sum |
second | The total time spent in major GCs in the JVM that collect old generation objects. |
jvm.gc.collectors.old.count | avg max min sum |
garbage collection | The total count of major GCs in the JVM that collect old generation objects. |
jvm.gc.collectors.young.collection_time | avg max min sum |
second | The total time spent in minor GCs in the JVM that collects young generation objects. |
jvm.gc.collectors.young.count | avg max min sum |
garbage collection | The total count of minor GCs in the JVM that collects young generation objects. |
jvm.gc.concurrent_mark_sweep.collection_time | avg max min sum |
second | The total time spent on "concurrent mark & sweep" GCs in the JVM. |
jvm.gc.concurrent_mark_sweep.count | avg max min sum |
garbage collection | The total count of "concurrent mark & sweep" GCs in the JVM. |
jvm.gc.par_new.collection_time | avg max min sum |
second | The total time spent on "parallel new" GCs in the JVM. |
jvm.gc.par_new.count | avg max min sum |
garbage collection | The total count of "parallel new" GCs in the JVM. |
jvm.mem.heap_committed | avg max min sum |
byte | The amount of memory guaranteed to be available to the JVM heap. |
jvm.mem.heap_in_use | avg max min sum |
The amount of memory currently used by the JVM heap as a value between 0 and 1. | |
jvm.mem.heap_max | avg max min sum |
byte | The maximum amount of memory that can be used by the JVM heap. |
jvm.mem.heap_used | avg max min sum |
byte | The amount of memory in bytes currently used by the JVM heap. |
jvm.mem.non_heap_committed | avg max min sum |
byte | The amount of memory guaranteed to be available to JVM non-heap. |
jvm.mem.non_heap_used | avg max min sum |
byte | The amount of memory in bytes currently used by the JVM non-heap. |
jvm.threads.count | avg max min sum |
thread | The number of active threads in the JVM. |
jvm.threads.peak_count | avg max min sum |
thread | The peak number of threads used by the JVM. |