OPC UA to the cloud, without a SCADA layer

Why a collector, and why it never listens

An OPC UA server on a plant LAN sits behind NAT and — if the network is set up properly — behind a firewall that admits nothing inbound. Any product that proposes to connect into your control network is asking you to undo that.

The collector connects outward instead: outward to the PLC's endpoint on the control network, outward over HTTPS to the API. No port forwarding, no inbound firewall rule, no PLC exposed to the internet.

1. Get an API key

Create an account and an API key first. The free plan is €0 and takes an email address — no card — and gives you 5 tags and 50,000 readings a day. Then create the key on the Settings page in Tag Historian Explorer — and the scope you give it is not a detail.

The key is shown once. Copy it straight into the container’s environment — if you lose it, generate a new one rather than hunting for the old one.

2. Start it once to get a config file

The collector reads its configuration and keeps its offline buffer in one directory, /config, on a Docker volume. Start it once on a new, empty volume and it writes a complete, commented example configuration there, says what to edit, and stops:

docker run --name tag-collector -v tag-collector:/config ghcr.io/softi-dev/tag-historian-clients/collector:latest

It ends with exit code 78, “configuration required”. That is the expected result here, not a failure: nothing has connected anywhere yet. The stopped container stays behind so the next step can reach the volume through it.

3. Edit the config

Copy the file out of the volume, so you can edit it with any editor:

docker cp tag-collector:/config/appsettings.json appsettings.json

In appsettings.json, fill in your server under OpcServers: its EndpointUrl, and one Tags row per NodeId you want stored. To have the collector list the server's variables for you instead, delete the example Tags row: it then starts with no tags, and browse adds them. Leave UseSecurity at false for now: prove the plumbing on an unsecured endpoint, then turn security on once data is flowing. Then delete the example entry in MqttSources if you are not using it. Every value still in <angle brackets> is a placeholder; everything else is a default, with a comment saying what it does. Trimmed to what matters, the edited file comes down to this:

{
  "TagHistorian": {
    "ApiUrl": "https://api.taghistorian.com",
    "ApiKey": ""
  },
  "MqttSources": [],
  "OpcServers": [
    {
      "Name": "plant",
      "EndpointUrl": "opc.tcp://192.168.1.20:4840",
      "UseSecurity": false,
      "Tags": [
        { "NodeId": "ns=2;s=Boiler.Temperature", "TagName": "boiler.temperature", "SamplingIntervalMs": 1000 },
        { "NodeId": "ns=2;s=Boiler.Pressure",    "TagName": "boiler.pressure",    "SamplingIntervalMs": 1000 }
      ]
    }
  ]
}

Leave ApiKey empty. The key goes into the container's environment in the next step, so it is never stored on the volume. Then copy the file back and remove the first-start container:

docker cp appsettings.json tag-collector:/config/appsettings.json
docker rm tag-collector

If you run docker with sudo on Linux, use it for all of these, and edit the copied file with sudo as well: a file docker copied out as root belongs to root.

4. Run it

docker run -d --name tag-collector --restart unless-stopped -v tag-collector:/config -e TagHistorian__ApiKey="<your-api-key>" ghcr.io/softi-dev/tag-historian-clients/collector:latest

Follow it with docker logs -f tag-collector. Without a key the collector collects nothing and says so in the log, and a key still reading <your-api-key> stops it with exit code 78 and the name of the variable. Both on purpose: a missing key is a mistake to fix now, not an outage to wait out. Left to run, the collector would have every upload refused with a 401, which it treats like a revoked key, holding the readings on disk and retrying, so a forgotten key would look like a running collector until the buffer reached its 7-day / 512 MiB bound and began dropping the oldest readings.

A placeholder left in the file also stops it with exit code 78 and the name of the setting, and so does a file that no longer parses after the edit, with the line and position where the parser gave up; your file is never overwritten. Under --restart unless-stopped such a container shows as Restarting (78) in docker ps, and the log says why. To change the configuration later, copy it out and back in the same way and run docker restart tag-collector.

Or with compose, once steps 2 and 3 have created the volume and put the config in it:

services:
  collector:
    image: ghcr.io/softi-dev/tag-historian-clients/collector:latest
    container_name: tag-collector
    restart: unless-stopped
    volumes:
      - tag-collector:/config
    environment:
      TagHistorian__ApiKey: "<your-api-key>"

volumes:
  tag-collector:
    external: true

Every command here is one line, so it pastes the same into bash, zsh or PowerShell. The image is built for linux/amd64 and linux/arm64: the same commands run on a PC, on Docker Desktop for Windows or macOS, and on an ARM box with a 64-bit OS such as a Raspberry Pi or an industrial gateway.

Updating the collector

A container keeps the image it was created from, so docker restart tag-collector starts the same collector again. To move to the newest, pull it and create the container again on the same volume; the configuration and the readings not yet sent stay where they are:

docker pull ghcr.io/softi-dev/tag-historian-clients/collector:latest
docker stop tag-collector
docker rm tag-collector
docker run -d --name tag-collector --restart unless-stopped -v tag-collector:/config -e TagHistorian__ApiKey="<your-api-key>" ghcr.io/softi-dev/tag-historian-clients/collector:latest

with your key in place of <your-api-key>, as before. With compose, docker compose pull and then docker compose up -d do the same. Tag files, browse and Modbus devices need collector 1.0.47 or later: an older one ignores TagsFile and ModbusDevices without a word, so update a collector you started before then.

On Windows, without Docker

Where IT does not allow Docker, on the SCADA PC or a small plant's Windows server, the same collector runs as a Windows service: one self-contained program for 64-bit Windows 10, 11 or Server 2016 and later, with nothing else to install. It reads the same appsettings.json, from C:\ProgramData\Tag Historian\Collector instead of a volume, and keeps its offline queue and certificates beside it.

Download the Windows zip and check it against the SHA-256 checksum published beside it.

In an elevated PowerShell, in the folder the zip was extracted to:

powershell -ExecutionPolicy Bypass -File .\install.ps1

install.ps1 copies the program to C:\Program Files\Tag Historian\Collector and registers the service TagHistorianCollector: automatic start, restarted a minute after a crash, running as its own virtual account NT SERVICE\TagHistorianCollector, which is granted write access to its data folder and nothing else. The data folder is readable by administrators and the service only, because your API key goes in it. Then it starts the service once, which writes the example configuration and stops, as the container's first start does.

Edit the file from the same elevated prompt, and put your Write key in ApiKey: on Windows the key goes in the file, because a service does not see a system environment variable set after Windows started. Then start the service and follow its log:

notepad "C:\ProgramData\Tag Historian\Collector\appsettings.json"
Start-Service TagHistorianCollector
Get-Content -Wait -Tail 20 "C:\ProgramData\Tag Historian\Collector\logs\collector.log"

logs\collector.log in the data folder carries the lines docker logs would show, and warnings and errors also go to the Application event log under Tag Historian Collector. An empty ApiKey, a placeholder left in, or a file that no longer parses stops the service with the reason in both places, and it stays stopped instead of restarting every minute.

To update, run install.ps1 from a newer zip's folder the same way: it stops the service, replaces the program and starts it again, and keeps the data folder. The zip's name carries the collector's version; tag files, browse and Modbus devices need 1.0.47 or later.

The program and scripts are not signed yet. Started from Explorer, the program gets “Windows protected your PC”: choose More info, then Run anyway. The -ExecutionPolicy Bypass above is what lets the unsigned install script run, for that one run. On Windows 11 with Smart App Control turned on, unsigned programs are blocked outright; use the container there until the collector is signed.

The same volume holds the store-and-forward queue (/config/queue) and the collector's certificate store (/config/pki), so buffered readings and the trust you have established both survive a container recreate. It is the same image that speaks MQTT — one collector, both protocols.

Subscriptions, not polling

The collector does not poll. It creates an OPC UA subscription and asks the server to do the sampling: SamplingIntervalMs is how often the server samples the node, and changed values are pushed to the collector in batches on the publishing interval (SubscriptionPublishingIntervalMs, default 1000 ms).

• An unchanged value costs nothing on the wire. A tag sampled every 100 ms that only moves once a minute sends one notification a minute.
• Values that change faster than the publishing interval are not lost — the server queues them per item (MonitoredItemQueueSize, default 10) and delivers the queue on the next publish.
• Timestamps are the server's source timestamps, not the collector's arrival times, so a reading is stamped when the value changed rather than when it happened to be delivered.

NodeIds become tags

There is no wildcard. Every tag is one explicit row, mapping one NodeId to one tag name: in Tags, or in a tag file.

Browse writes the list for you, but it stays a list you decide on: one row is exactly one tag against your plan's quota, so an OPC UA source can never surprise you with a tag explosion the way a # wildcard on a busy MQTT broker can. What you listed is what you pay for.

Let the collector list the tags: browse

Typing NodeIds is fine for a handful. For more, the collector lists them for you. Its browse command connects to the server in appsettings.json with the endpoint, security settings and certificate the collector itself uses, walks the server's address space, and writes every variable it finds to a CSV file: one row per node, with a suggested tag name made from its path, its data type, and its engineering unit where the server publishes one. You delete the rows you do not want and point the collector at the file. Nothing is subscribed to until you have.

docker run --rm --pull always -v tag-collector:/config ghcr.io/softi-dev/tag-historian-clients/collector:latest browse

It runs beside the collector on the same volume, needs no API key, collects nothing and writes nothing to the server. When it exits, the list is in /config/browse-plant.csv, named after the server's Name, and it has printed how many variables it found and what it left out: text, dates, structures and arrays, which the collector cannot store, and the diagnostics of the standard Server object. On a large server it stops at 10 levels deep and 5000 variables, and says so at the top of the file; --start "ns=3;s=DataBlocksGlobal" lists one part of the server instead (in double quotes, or the shell ends the command at the semicolon), and --help shows the other options.

Copy the list out, and delete every row you do not want stored:

docker cp tag-collector:/config/browse-plant.csv tags.csv

# Variables browsed from OPC UA server 'plant' at opc.tcp://192.168.1.20:4840 ...
# Every row you keep becomes one tag against your plan: delete the rows you do not want stored.
NodeId,TagName,DataType,Units,SamplingIntervalMs
ns=2;s=Boiler.Pressure,Plant.Boiler.Pressure,Double,bar,1000
ns=2;s=Boiler.Temperature,Plant.Boiler.Temperature,Double,°C,1000

Then copy it back as tags.csv, set TagsFile on the server in appsettings.json — copied out and in with docker cp tag-collector:/config/appsettings.json appsettings.json and docker cp appsettings.json tag-collector:/config/appsettings.json, as in step 3 — and restart:

{
  "Name": "plant",
  "EndpointUrl": "opc.tcp://192.168.1.20:4840",
  "UseSecurity": false,
  "TagsFile": "tags.csv",
  "Tags": []
}

docker cp tags.csv tag-collector:/config/tags.csv
docker restart tag-collector

The log then says how many tags came from the file, ahead of the usual Creating subscription with ... tags. browse writes over its own browse-*.csv files when you run it again, never over a file a TagsFile names: browse again later, and compare the new list with the one you kept.

On Windows

From an elevated PowerShell, since the data folder is open to administrators and the service only; it runs while the service does. --output writes the list to your Documents folder, where Excel can open it. Edit it there, then copy it into the data folder, set TagsFile to tags.csv in appsettings.json, and restart the service. The program has to be 1.0.47 or later, which the name of the zip it came from says; an older one answers browse as a second collector, which stops at once because the service holds the folder. Updating is running the newer zip's install.ps1.

& "C:\Program Files\Tag Historian\Collector\TagHistorianCollector.exe" browse --output "$HOME\Documents\browse-plant.csv"
Copy-Item "$HOME\Documents\browse-plant.csv" "C:\ProgramData\Tag Historian\Collector\tags.csv"
Restart-Service TagHistorianCollector

The tag file

Name it in the server's TagsFile; a relative path is read from the folder appsettings.json is in. Write it with browse, export it from your PLC software, or type it: any file with these columns will do.

• Column names in any case, with or without spaces, in any order. Other columns, such as browse's DataType, are not read; a name the collector does not know is named in the log at start, so a misspelt one is noticed.
• Commas, semicolons or tabs between values, as Excel saves CSV in any language. In a semicolon file a NodeId goes in double quotes, "ns=2;s=Boiler.Pressure", and Excel does that itself. Excel in a decimal-comma language such as Swedish or German opens a comma file in a single column: run browse with --delimiter semicolon for one it splits.
• UTF-8, which is Excel's CSV UTF-8 when you save. Blank lines and lines starting with # are skipped, unless the line has a NodeId: with Units or Description as the first column, a value such as #/h is read, not dropped.
• A row the collector cannot use is skipped, and the rest are collected. Each one is named in the log at start with its line and column, lettered as in Excel: a NodeId that is not one, a tag name the API would refuse, a value that is not a number, the same NodeId or tag name on a second row (the first one is kept). The server's health line counts them every minute as tagsSkipped, so a skipped row stays in sight. A row with Enabled false is left out quietly, whatever else is in it. That is collector 1.0.48 and later: an older one stops at start on such a row, with exit code 78 and the same line and column.
• A file the collector cannot read at all, or in which not one row can be used, stops it at start with exit code 78 and the reason for each row: that is the wrong file or the wrong format, and starting without any of its tags would leave them all empty.
• A NodeId in both the file and Tags is taken as Tags has it, and the log says so: collected once, or not at all if Enabled is false there. A setting you give one node in Tags, switching it off included, survives the next browse.
• Tag names are one list for the whole collector: every server in OpcServers, every Modbus device, and the MQTT and Sparkplug B sources. Identical PLCs browsed alike get identical names, and one name from two places would mix both PLCs' readings into one tag, so it is collected from the first server that lists it, and skipped with an error naming both places on the other: put the server's Name in front of one set. Before collector 1.0.48, the list holds the servers only, and a name on two of them stops the collector at start.
• The file is read at start: restart the collector after you change it. A NodeId the server does not know shows up as a per-item error in the log, as in Tags, while the rest keep working.
• From collector 1.0.48, at start the collector also compares the tags it is configured with to what your plan has room for. If some will not fit, one warning says how many the API will refuse and names the tags that are not in your account yet.
• Modbus devices take a tag file too, read the same way with register columns in place of the NodeId, and by the same rule for a row that cannot be used. One difference: there a line starting with # is always skipped. The Modbus guide has the columns.

Values and quality

Integers, floats and booleans are stored — booleans as 1 and 0, so an hour in which a pump ran fifteen minutes averages to 0.25, its duty cycle. A string-valued node stores nothing at all: it is dropped and reported in the log, never turned into an invented zero.

Every reading carries the exact OPC UA StatusCode the server reported — Good, UncertainLastUsableValue, BadSensorFailure, the rest — stored alongside the value as a full 32-bit code rather than a good/bad flag. A sensor reporting garbage with a Bad quality is distinguishable from a sensor reporting garbage confidently, which is the whole point of quality codes.

When a sensor drops out, the Bad or valueless sample is neither discarded nor interpolated across. It is stored as an outage marker — the exact StatusCode, carried alongside the tag's last known good value — so the interruption is visible in the history instead of silently bridged. The one exception is startup: until a tag has produced its first Good sample there is no last-known-good value to carry, so no marker and no fabricated 0.0 is stored. The first value written for any tag is always a real reading.

Security

{
  "Name": "plant",
  "EndpointUrl": "opc.tcp://192.168.1.20:4840",
  "SecurityPolicy": "Basic256Sha256",
  "SecurityMode": "SignAndEncrypt",
  "PkiPath": "pki"
}

• SecurityPolicy — None or Basic256Sha256. When set, the collector requires an endpoint with exactly this policy.
• SecurityMode — None, Sign or SignAndEncrypt. Same rule: set means exact match.
• If the server offers no endpoint matching what you asked for, the collector fails fast and prints the server's actual endpoint list — policy, mode and URL for each — so the fix is a config edit, not a packet capture. It never silently downgrades to a weaker endpoint than the one you named.

UseSecurity is kept for backwards compatibility: true with no policy or mode means “pick the best secured endpoint the server offers”. It demands a strong policy and a Sign/SignAndEncrypt mode, and if the server's unauthenticated endpoint list contains no such endpoint — the shape of a man-in-the-middle that stripped the secure ones — the collector fails loudly instead of quietly connecting in plaintext. Setting the policy and mode explicitly is the precise form and the one to prefer in new configs.

By default the collector does not check that the endpoint hostname appears in the server's certificate (ValidateEndpointDomain: false). PLCs are almost always reached by IP while their certificate names an internal hostname, so a domain check would reject a server whose certificate you have explicitly trusted. Set it true where the subject and the address genuinely line up.

Trust goes both ways

OPC UA security is mutual: the collector must trust the server's certificate and the server must trust the collector's. Both halves are files.

1. On first start the collector generates a self-signed certificate under pki/own/certs/. Copy it to the OPC UA server and mark it trusted — on most PLCs and SCADA servers that is a “trusted clients” list; some instead quarantine the first connection attempt as rejected and let you promote it.
2. Export the server's certificate and drop it in pki/trusted/certs/ on the collector side.

Until both are done, a secured connect fails with a certificate error naming the untrusted side. That is OPC UA working as designed, not the collector being difficult: the alternative — auto-trusting whatever answers — would make the encryption decorative.

What happens when the uplink drops

A backlog drains oldest first in uploads of up to BatchSize readings (default 5000, the most the API takes), at up to about 2 400 readings a second, with new readings queued behind it. A collector whose appsettings.json says "BatchSize": 100 — every install that started on an image from before the default was raised — is held to about 50 readings a second, below a Business plan's daily average: pull the current image, then change the line to 5000. The MQTT guide has the details, which are the same queue's.

Settings

Per server in OpcServers:

Per tag in Tags:

Picking CompressionDeadband

CompressionDeadband is a filter at the PLC and nothing else. The collector asks the server for an OPC UA DataChangeFilter with DeadbandType.Absolute: a distance in the tag's own engineering units. On a tag reading around 50 °C, 0.5 means a move under half a degree never leaves the server. Size it as you would any PLC deadband, against the instrument's noise in its own unit. Empty asks for no filter, and the server reports every change it samples.

What the historian keeps is decided by the tag's deadband, and that is a different kind of number: a fraction of the last stored value. Every tag starts at 0.001, so a reading is stored when it moves more than 0.1% from the last one stored, or when an hour has passed since then. The collector's number never changes it. The two filters run one after the other, so a coarse PLC filter still hides what the tag would have kept: on a tag reading around 0.002, a PLC deadband of 0.001 holds back moves of up to 50%.

The Explorer cannot change a tag's deadband yet. The tags API can, with an Admin key:

curl -X PUT "https://api.taghistorian.com/api/tags/single?name=boiler.temperature" \
  -H "X-Api-Key: <your-admin-key>" \
  -H "Content-Type: application/json" \
  -d '{ "compressionDeadband": 0.005 }'

0.005 stores on a move over 0.5%, 0.0001 over 0.01%. Two things to know before you send it. The range is 0 to 1, and a number outside it is refused with an error that says why: a 2 meant as “2 °C” would be 200% and store almost nothing. And if the tag has a fixed scale in the Explorer, send its scaleMin and scaleMax in the same request: this request clears a scale it is not sent.

Trying it without a PLC

We test the collector against an OPC UA plant simulator of our own, but it is not published — not as source and not as an image — so you cannot run it. Any OPC UA test server will do the same job. Microsoft's open-source OPC PLC runs as a container (mcr.microsoft.com/iotedge/opc-plc) and serves simulated nodes on port 50000. Out of the box it offers only secured endpoints, signed or signed and encrypted, and Basic256Sha256 with SignAndEncrypt from the security example is among them. Start it with --ut to add an unsecured endpoint for a first test with UseSecurity off, and --autoaccept so it trusts the collector's certificate without a manual step. Point browse at it to see what it serves.

What this is not

• We do not write to your PLC. Ingestion only; the collector never writes a value to any node. The subscription is read-only by construction.
• We do not collect what we find. Browsing writes a list for you to edit, and nothing is collected until you keep a row. There is no wildcard subscription and no browse-and-select tree in the dashboard: one row is one tag on your plan, decided by you.
• This is not a SCADA system. No HMI, no control, no alarming on the plant floor. It stores history and alerts on it. If you need supervisory control, you need a SCADA platform, and what those cost is written up here.