Plugins¶
The main components, consumers, routers, producers, filters and formatters are build upon a plugin architecture. This allows each component to be exchanged and configured individually with a different sets of options.
Plugin types:
Aggregate plugins¶
To simplify complex pipeline configs you are able to aggregate plugin configurations. That means that all settings witch are defined in the aggregation scope will injected to each defined “sub-plugin”.
To define an aggregation use the keyword Aggregate as plugin type.
Examples¶
In this example both consumers get the streams and modulator injected from the aggregation settings:
AggregatePipeline:
Type: Aggregate
Streams: console
Modulators:
- format.Envelope:
Postfix: "\n"
Plugins:
consumerFoo:
Type: consumer.File
File: /tmp/foo.log
consumerBar:
Type: consumer.File
File: /tmp/bar.log
This example shows a second use case to reuse server settings easier:
consumerConsole:
Type: consumer.Console
Streams: write
kafka:
Type: Aggregate
Servers:
- kafka0:9092
- kafka1:9093
- kafka2:9094
Plugins:
producer:
Type: producer.Kafka
Streams: write
Compression: zip
Topics:
write: test
consumer:
Type: consumer.Kafka
Streams: read
Topic: test
DefaultOffset: Oldest
producerConsole:
Type: producer.Console
Streams: read