+91 70951 67689 datalabs.training@gmail.com

Components of Apache Flume – Understanding the concept of Flume Events


To understand the concept of Events better consider an example: consider a log file; it has two components, 1. Time sent and 2. The actual message. Now, how does our Flume event look like? It will be divided into two parts: header and payload, where the actual message will be in the payload.


This represents our Flume event; this Flume Event travels from the Source to the Sink inside the Agent. It is not always necessary that the header is there; sometimes there can also be zero headers or in other cases there can be more than 1 header. Headers are stored as a key, value pair. Suppose, we want to send our event with information like the time the particular event was generated; so, the time and the value at which the event is generated will be the header part.


The host at which it was generated, that is the server IP etc. can also be the header. This header information can be used either by the Sink or the Interceptors or any of the routing mechanisms. So, that is how the Flume event looks like; it has two parts, the header and the payload. The payload is just a byte array. It is an array of bytes that contain the actual data/information inside it.

Tags:Components of Apache Flume the concept of Flume Events