Skip to main content

Data Sources

A Data Source connects your live data to the ingestion pipeline. Each source reads messages from a specific protocol, converts them into a common event format — without interpreting it — and forwards them to your mappings.

Because mappings are transport-independent, switching from one source to another only requires changing the transport configuration:

sources: [{ kind: "mqtt", config: { broker_url, topics } }],

Choosing a Data Source

SourceUse it whenEvent ChannelRuns inInstall
MQTTYour data already flows through a broker — the plant-floor normthe topic — a genuine routing keyNode.js and browsernpm install mqtt
OPC UAYou must talk to a PLC or SCADA server directly over opc.tcpthe node's alias, else its idNode.js onlynpm install node-opcua-client
Azure Event HubsYour telemetry lands in Azure — IoT Hub, Fabric eventstreams, or a hubthe partition id — not a topicNode.js and browsernpm install @azure/event-hubs
PlaybackYou are building a mapping, running a demo, or reproducing a problemthe recorded channelNode.js and browsernothing
Your ownAnything else — Kafka, Modbus, a WebSocket feed, a webhook, a serial portwhatever you put in itwherever it runswhatever it needs

Understanding the Event Channel

The value stored in event.channel depends on the transport. Since channel patterns are evaluated against this field, it determines whether a mapping can filter events using channel or whether it should rely on when instead.

For example, MQTT topics make excellent routing keys, whereas Azure Event Hubs partitions are only load-balancing artifacts.

Where each transport runs

The Runs in column is a hard constraint, not a preference: MQTT is web-native over WebSocket, so a page can subscribe to a broker directly, but opc.tcp:// is raw TCP and an OPC UA agent therefore has to be a Node.js process. That is also where it belongs in a real deployment — on the plant network, next to the PLC, with the scene as its only outbound connection.

Optional dependencies

The client libraries in the Install column are optional peer dependencies, imported lazily by the transport that needs them. Install only the ones you use; not installing the rest costs nothing at build or run time.

Two shortcuts worth knowing

Prefer MQTT when available

If your plant already bridges OPC UA to MQTT — OPC UA PubSub over MQTT on recent firmware, Telegraf's inputs.opcua, Kepware, Ignition — point the MQTT transport at that broker rather than holding an OPC UA session next to the scene. There is no OPC UA session to keep alive, and the source can be read straight from a browser instead of requiring a Node.js process on the plant network. The OPC UA transport is for servers that offer no such bridge.

Start with playback

Record a few seconds of your real stream, build the mapping against the dump, then change one line to go live. The recorded channel is preserved, so a mapping behaves identically replayed and live. See playback.

Custom Transports

If your data source isn't supported, implement the Transport interface and push events into the EventSink the ingestion hands you — that is how any protocol at all reaches the same pipeline, including events that do not arrive on a stream: a webhook, a REST handler, a scheduled poll. See write your own transport.