A despatch advice left an ERP this month with a receiver name in it, Müller, correctly encoded in UTF-8. It passed through a converter, a queue and an SFTP drop and arrived at the carrier as M?ller. Four systems handled it correctly; the fifth was configured for Latin-1, and nobody had told it otherwise since 2014.
This is the most common defect we find in inherited interfaces, and it hides well, because test data is written by developers, and developers avoid umlauts without noticing that they do it. The interface passes every test and then meets a real address book.
Two habits help. Test with names taken from the client's actual master data; a Frankfurt address book will find the weak system in an afternoon. And log the byte length of every text field next to the string. A name that is six characters and seven bytes is telling you where the multi-byte character sits; when the string looks right in one log and wrong in the next, the byte counts say which hop changed it, and the conversation with the counterparty stops being a matter of opinion.
The fix is one configuration line; the hunt through five systems is what you are paying for.
Retiring an interface nobody owns
Every inherited estate has one: a nightly file, produced since a year nobody can name, read by nobody anyone can point to. Turning it off is one line. Being sure is the work.
Asking around settles nothing. The person who wanted the file has left, the team that read it was reorganised, and the file may feed a spreadsheet that feeds a report that one person opens in the last week of the quarter. Silence in a meeting is not evidence.
So we log instead of asking. Access logging on the file for a full quarter, long enough to catch the month-end and quarter-end readers. If the only reads are the backup job, we announce a retirement date in writing, wait out the objections, and switch the file off on a Tuesday morning in working hours, with the person who can switch it back on sitting at their desk. Not on a Friday. Not at 02:40.
-rw-r----- 1 edi ops 141872 Feb 2 02:41 STOCK_20260202.csv
-rw-r----- 1 edi ops 141598 Feb 1 02:41 STOCK_20260201.csv
-rw-r----- 1 edi ops 141733 Jan 31 02:41 STOCK_20260131.csv
Written every night for nine years. Read, it turned out, by the backup job.
About one in three of these files has a reader nobody predicted. That is why the quarter of logging is not optional, and why the switch-off happens while everyone is awake.
Log the raw message first
Store every message exactly as received, bytes on disk, before any parsing touches it. Not the parsed result. Not the normalised version. The bytes.
The reasons pile up with every incident. The parser will be wrong at some point, and when it is, the only record of what actually arrived is the copy you kept before the parser ran. The sender will deny sending the message, or deny sending it in that form, and a raw file with a transport timestamp ends that discussion in one email rather than a week of them. And when a mapping changes, the raw archive is a free regression test: replay last month's traffic through the new mapping and compare the output line by line.
Raw archives are cheap. A year of EDIFACT for a mid-sized distributor fits on a laptop. The rule costs one directory and one retention job, and it has paid for itself in every engagement we can remember.
Parse, validate, transform, reject, as the interface requires. First, write the bytes down.