Summary -
In this topic, we described about the below sections -
The interceptors used to modify/drop events in-flight. Flume has the capability as it uses the interceptors. The interceptor also decides what sort of data should pass through to the Channel.
An interceptor can modify/drop events based on any criteria chosen by the developer of the interceptor. Flume supports binding of interceptors. Interceptors are stated as a whitespace separated list in the source configuration.
The interceptors are specified in the order and in the same order which they are invoked. The list of events returned by one interceptor is passed to the next interceptor in the chain.
If an interceptor drop events, it does not return the event in the list that it should returns. If it is to drop all events, then it simply returns an empty list. The interceptors are themselves configurable and can be passed configuration values.
The below are the list of interceptors available
- Timestamp Interceptor
- Host Interceptor
- Static interceptor
- Regex filtering interceptor
Timestamp Interceptor -
This interceptor inserts the time in milli-seconds into the event headers, at which it processes the event. This interceptor inserts a header with the relevant timestamp. If the event already contains a timestamp header, it will be overwritten with the current time unless configured to preserve the original value by settings.
<agent-name>.sources = <source_name>
<agent-name>.channels = <channel-name>
<agent-name>.sources.<source_name>.channels = <channel-name>
<agent-name>.sources.<source_name>.type = seq
<agent-name>.sources.<source_name>.interceptors = <interceptor_name>
<agent-name>.sources.<source_name>.interceptors.<interceptor_name>.type
= timestamp
Example for agent named agt -
agt.sources = sr1
agt.channels = chn1
agt.sources.sr1.channels = chn1
agt.sources.sr1.type = seq
agt.sources.sr1.interceptors = ic1
agt.sources.sr1.interceptors.ic1.type = timestamp
Below are the properties for Timestamp Interceptor -
| Property Name | Default | Description |
|---|---|---|
| type | – | The component type name (timestamp) |
| preserveExisting | false | If the timestamp already exists, should it be preserved |
Host Interceptor -
This interceptor inserts the host name or IP address into the event headers, at which it processes the event. This interceptor inserts a header with the host name or IP address.
If the event already contains a IP address or host name in header, it will be overwritten with the current IP address or host name unless configured to preserve the original value by settings.
<agent-name>.sources = <source_name>
<agent-name>.channels = <channel-name>
<agent-name>.sources.<source_name>.interceptors = <interceptor_name>
<agent-name>.sources.<source_name>.interceptors.<interceptor_name>.type
= host
<agent-name>.sources.<source_name>.interceptors.<interceptor_name>.hostHeader
= hostname
Example for agent named agt -
agt.sources = sr1
agt.channels = chn1
agt.sources.sr1.interceptors = ic1
agt.sources.sr1.interceptors.ic1.type = host
agt.sources.sr1.interceptors.ic1.hostHeader = hostname
Below are the properties for host Interceptor -
| Property Name | Default | Description |
|---|---|---|
| type | – | The component type name(host) |
| preserveExisting | false | If the host header already exists, should it be preserved |
| useIP | true | IP Address if true, else use hostname. |
| hostHeader | host | The header key to be used. |
Static Interceptor -
Static interceptor allows user to append a static header to all events. Static interceptor allows user to append a static value to all events.
The current implementation does not allow multiple headers at one time. Instead user might chain multiple static interceptors each defining one static header.
<agent-name>.sources = <source_name>
<agent-name>.channels = <channel-name>
<agent-name>.sources.<source_name>.channels = <channel-name>
<agent-name>.sources.<source_name>.type = seq
<agent-name>.sources.<source_name>.interceptors = <interceptor_name>
<agent-name>.sources.<source_name>.interceptors.<interceptor_name>.type
= timestamp
<agent-name>.sources.<source_name>.interceptors.<interceptor_name>.key
= <key-name>
<agent-name>.sources.<source_name>.interceptors.<interceptor_name>.value
= <static-value>
Below are the properties for static Interceptor -
| Property Name | Default | Description |
|---|---|---|
| type | – | The component type name(static) |
| preserveExisting | true | If configured header already exists, should it be preserved |
| key | key | Name of header |
| value | value | Static value |
Regex filtering Interceptor -
This interceptor filters events selectively by understanding the event body as text. This interceptor matches the text against a configured regular expression. The provided regular expression can be used to include events or exclude events.
| Property Name | Default | Description |
|---|---|---|
| type | – | The component type name (regex_filter) |
| regex | ”.*” | Regular expression |
| excludeEvents | false | If true, regex determines events to exclude, otherwise regex determines events to include. |