The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host.
Examples include Beats. Agents may also run on observers. ECS agent.* fields shall be populated with details of the agent running on the host or observer where the event happened or the measurement was taken.
An autonomous system (AS) is a collection of connected Internet Protocol (IP) routing prefixes under the control of one or more network operators on behalf of a single administrative entity or domain that presents a common, clearly defined routing policy to the internet.
A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records.
For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term “originator” to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields. Client fields are generally not populated for packet-level events.
Client / server representations can add semantic context to an exchange, which is helpful to visualize the data in certain situations. If your context falls in that category, you should still ensure that source and destination are filled appropriately.
Container fields are used for meta information about the specific container that is the source of information.
These fields help correlate data based containers from any runtime.
The data_stream fields take part in defining the new data stream naming scheme.
In the new data stream naming scheme the value of the data stream fields combine to the name of the actual data stream in the following manner: {data_stream.type}-{data_stream.dataset}-{data_stream.namespace}. This means the fields can only contain characters that are valid as part of names of data streams. More details about this can be found in this https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme[blog post].
An Elasticsearch data stream consists of one or more backing indices, and a data stream name forms part of the backing indices names. Due to this convention, data streams must also follow index naming restrictions. For example, data stream names cannot include \, /, *, ?, ", <, >, |, (space character), ,, or #. Please see the Elasticsearch reference for additional https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#indices-create-api-path-params[restrictions].
Destination fields capture details about the receiver of a network exchange/packet. These fields are populated from a network event, packet, or other event containing details of a network transaction.
Destination fields are usually populated in conjunction with source fields. The source and destination fields are considered the baseline and should always be filled if an event contains source and destination details from a network transaction. If the event also contains identification of the client and server roles, then the client and server fields should also be populated.
Fields that describe a device instance and its characteristics. Data collected for applications and processes running on a (mobile) device can be enriched with these fields to describe the identity, type and other characteristics of the device.
This field group definition is based on the Device namespace of the OpenTelemetry Semantic Conventions (https://opentelemetry.io/docs/reference/specification/resource/semantic_conventions/device/).
Fields describing DNS queries and answers.
DNS events should either represent a single DNS query prior to getting answers (dns.type:query) or they should represent a full exchange and contain the query details as well as all of the answers that were provided for this query (dns.type:answer).
Event details relating to an email transaction.
This field set focuses on the email message header, body, and attachments. Network protocols that send and receive email messages such as SMTP are outside the scope of the email.* fields.
These fields can represent errors of any kind.
Use them for errors that happen while fetching events or in cases where the event itself contains an error.
The event fields are used for context information about the log or metric event itself.
A log is defined as an event containing details of something that happened. Log events must include the time at which the thing happened. Examples of log events include a process starting on a host, a network packet being sent from a source to a destination, or a network connection between a client and a server being initiated or closed. A metric is defined as an event containing one or more numerical measurements and the time at which the measurement was taken. Examples of metric events include memory pressure measured on a host and device temperature. See the event.kind definition in this section for additional details about metric and state events.
A file is defined as a set of information that has been created on, or has existed on a filesystem.
File objects can be associated with host events, network events, and/or file events (e.g., those produced by File Integrity Monitoring [FIM] products or services). File fields provide details about the affected file associated with the event or metric.
Geo fields can carry data about a specific location related to an event.
This geolocation information can be derived from techniques such as Geo IP, or be user-supplied.
The hash fields represent different bitwise hash algorithms and their values.
Field names for common hashes (e.g. MD5, SHA1) are predefined. Add fields for other hashes by lowercasing the hash algorithm name and using underscore separators as appropriate (snake case, e.g. sha3_512).
Note that this fieldset is used for common hashes that may be computed over a range of generic bytes. Entity-specific hashes such as ja3 or imphash are placed in the fieldsets to which they relate (tls and pe, respectively).
A host is defined as a general computing instance.
ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes.
The interface fields are used to record ingress and egress interface information when reported by an observer (e.g. firewall, router, load balancer) in the context of the observer handling a network connection. In the case of a single observer interface (e.g. network sensor on a span port) only the observer.ingress information should be populated.
Details about the event’s logging mechanism or logging transport.
The log.* fields are typically populated with details about the logging mechanism used to create and/or transport the event. For example, syslog details belong under log.syslog.*.
The details specific to your event source are typically not logged under log.*, but rather in event.* or in other ECS fields.
The network is defined as the communication path over which a host or network event happens.
The network.* fields should be populated with details about the network activity associated with an event.
An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics.
This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, web proxies, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS.
The organization fields enrich data with information about the company or entity the data is associated with.
These fields help you arrange or filter data stored in an index by one or multiple organizations.
These fields contain information about an installed software package. It contains general information about a package, such as name, version or size. It also contains installation details, such as time or location.
These fields contain information about a process.
These fields can help you correlate metrics information with a process id/name from a log message. The process.pid often stays in the metric itself and is copied to the global field for correlation.
This field set is meant to facilitate pivoting around a piece of data.
Some pieces of information can be seen in many places in an ECS event. To facilitate searching for them, store an array of all seen values to their corresponding field in related..
A concrete example is IP addresses, which can be under host, observer, source, destination, client, server, and network.forwarded_ip. If you append all IPs to related.ip, you can then search for a given IP trivially, no matter where it appeared, by querying related.ip:192.0.2.15.
Fields for describing risk score and risk level of entities such as hosts and users. These fields are not allowed to be nested under event.*. Please continue to use event.risk_score and event.risk_score_norm for event risk.
Rule fields are used to capture the specifics of any observer or agent rules that generate alerts or other notable events.
Examples of data sources that would populate the rule fields include: network admission control platforms, network or host IDS/IPS, network firewalls, web application firewalls, url filters, endpoint detection and response (EDR) systems, etc.
A Server is defined as the responder in a network connection for events regarding sessions, connections, or bidirectional flow records.
For TCP events, the server is the receiver of the initial SYN packet(s) of the TCP connection. For other protocols, the server is generally the responder in the network transaction. Some systems actually use the term “responder” to refer the server in TCP connections. The server fields describe details about the system acting as the server in the network event. Server fields are usually populated in conjunction with client fields. Server fields are generally not populated for packet-level events.
Client / server representations can add semantic context to an exchange, which is helpful to visualize the data in certain situations. If your context falls in that category, you should still ensure that source and destination are filled appropriately.
The service fields describe the service for or from which the data was collected.
These fields help you find and correlate logs for a specific service and version.
Source fields capture details about the sender of a network exchange/packet. These fields are populated from a network event, packet, or other event containing details of a network transaction.
Source fields are usually populated in conjunction with destination fields. The source and destination fields are considered the baseline and should always be filled if an event contains source and destination details from a network transaction. If the event also contains identification of the client and server roles, then the client and server fields should also be populated.
Fields to classify events and alerts according to a threat taxonomy such as the MITRE ATT&CK® framework.
These fields are for users to classify alerts from all of their sources (e.g. IDS, NGFW, etc.) within a common taxonomy. The threat.tactic.* fields are meant to capture the high level category of the threat (e.g. “impact”). The threat.technique.* fields are meant to capture which kind of approach is used by this detected threat, to accomplish the goal (e.g. “endpoint denial of service”).
Fields related to a TLS connection. These fields focus on the TLS protocol itself and intentionally avoids in-depth analysis of the related x.509 certificate files.
Distributed tracing makes it possible to analyze performance throughout a microservice architecture all in one view. This is accomplished by tracing all of the requests - from the initial web request in the front-end service - to queries made through multiple back-end services.
Unlike most field sets in ECS, the tracing fields are not nested under the field set name. In other words, the correct field name is trace.id, not tracing.trace.id, and so on.
The user fields describe information about the user that is relevant to the event.
Fields can have one entry or multiple entries. If a user has more than one id, provide an array that includes all of them.
The VLAN fields are used to identify 802.1q tag(s) of a packet, as well as ingress and egress VLAN associations of an observer in relation to a specific packet or connection.
Network.vlan fields are used to record a single VLAN tag, or the outer tag in the case of q-in-q encapsulations, for a packet or connection as observed, typically provided by a network sensor (e.g. Zeek, Wireshark) passively reporting on traffic.
Network.inner VLAN fields are used to report inner q-in-q 802.1q tags (multiple 802.1q encapsulations) as observed, typically provided by a network sensor (e.g. Zeek, Wireshark) passively reporting on traffic. Network.inner VLAN fields should only be used in addition to network.vlan fields to indicate q-in-q tagging.
Observer.ingress and observer.egress VLAN values are used to record observer specific information when observer events contain discrete ingress and egress VLAN information, typically provided by firewalls, routers, or load balancers.
This implements the common core fields for x509 certificates. This information is likely logged with TLS sessions, digital signatures found in executable binaries, S/MIME information in email bodies, or analysis of files on disk.
When the certificate relates to a file, use the fields at file.x509. When hashes of the DER-encoded certificate are available, the hash data set should be populated as well (e.g. file.hash.sha256).
Events that contain certificate information about network connections, should use the x509 fields under the relevant TLS fields: tls.server.x509 and/or tls.client.x509.
ECS version this event conforms to. ecs.version is a required field and must exist in all events.
When querying across multiple indices – which may conform to slightly different ECS versions – this field lets integrations adjust to the schema version of the events.