Why integration, not collection, is where projects stall
Most plants can get a number off a machine. What breaks is everything after that: the number arrives in a format nobody agreed on, under a name that means something different in the ERP, at a moment when the network was down, and with no way to tell a missing reading from a zero. Six months later there are three sources of truth for production count and a standing argument about which is right.
That is an integration problem, not a sensor problem. It is worth separating clearly from shop floor data collection, which is about getting a trustworthy signal in the first place. Collection asks "is this number real?" Integration asks "can every system that needs this number get it, name it the same way, and know when it is missing?"
The four integration methods
| Method | When it fits | What it costs you |
|---|---|---|
| Direct protocol read Modbus TCP/RTU, OPC-UA, EtherNet/IP, vendor drivers |
The machine has an accessible controller and you want the least translation between reality and record. | Tag-by-tag mapping work, and a dependency on someone documenting what each tag means. |
| Edge gateway | Almost always. It normalises mixed protocols, buffers through outages, and keeps the plant network separate from anything outbound. | Hardware per area, and a configuration that must be version-controlled rather than living in one engineer's head. |
| Broker / middleware MQTT, often as a unified namespace |
More than a handful of consumers — MES, BI, maintenance, a dashboard — so publishers stop knowing about subscribers. | A naming convention you must actually govern; a broker to keep alive. |
| File / database bridge CSV drops, SQL views, scheduled exports |
Legacy systems that expose nothing else, and ERP imports that expect batch files. | Latency and silent failure — a stopped export looks exactly like a quiet plant. Needs its own heartbeat check. |
A reference architecture that survives local conditions
From the machine outward, in four hops:
- Machine layer. Controllers, meters and retrofit sensors. Read-only, always: no writes to a PLC from an integration layer, and no integration project that requires changing machine logic.
- Edge layer. A gateway per area that normalises protocols, timestamps at source, applies the canonical name, and buffers locally. This is the hop that makes the architecture survivable here — during load shedding or a network drop, data accumulates and back-fills instead of vanishing.
- Transport and namespace. Outbound-only from plant to platform, typically over MQTT with a consistent topic structure. Whether or not you formally adopt a unified namespace, adopt its useful half: one canonical name per signal, organised by site/area/line/asset/measure, so a rename is a governance decision rather than a search-and-replace across five systems.
- Consumption layer. OEE and dashboards, ERP or MES, maintenance systems, and BI tools like Power BI — each subscribing to the same canonical signals rather than each running its own extract. Export in open formats (CSV, SQL) must be part of the design, not a paid feature.
The rule that saves the most pain later: timestamp at the edge, not on arrival. If the cloud stamps the time, every outage silently compresses hours of production into the minute the link came back — and your downtime analysis becomes fiction exactly for the periods you most need to study.
Choosing integration tools without buying a problem
Five questions separate tools that will still be working in year three from those that won't:
- Does it read your actual machines? Not "supports OPC-UA" in the brochure — your machines, including the 1998 filler with a serial port.
- Does it buffer locally, and for how long? Ask for the buffer depth in hours at your data rate. Under South African conditions this is a primary specification, not a detail.
- Can you get your data out in full? Open export, no per-export fees, no proprietary archive you cannot read without a licence. Data lock-in is the most expensive form of vendor lock-in because it grows.
- Who maintains the mapping? If tag mapping lives in a consultant's laptop, your integration has a single point of failure with a car.
- What does it do at the network boundary? Outbound-only, no inbound ports to the plant network, no exposed controllers — see connecting a plant without exposing the PLCs.
Where it lands: analytics that people actually use
Integration exists so that decisions can be made, and the two decisions it feeds most often are simple: what stopped, and what did it cost? Manufacturing analytics has a habit of skipping past those to demand forecasting and machine learning, on data foundations that cannot yet reconcile a shift count. Get the canonical signals right, get OEE and reason-coded downtime trustworthy, and the analytics that follow will at least be built on numbers the plant believes. That sequencing — and why dashboards fail without it — is covered in why production dashboards fail.