Running an annual cadence data pipeline that sits idle for eleven months

The halls open at nine. At 09:20 somebody runs the scan loader for the first time since last September and it fails on the connection. Twenty minutes later it turns out the certificate on the vendor's endpoint is fine and the one on your side is not, and the person who set it up has left.

An annual cadence data pipeline is a strange object. It is production code that executes once, under time pressure, after a year of silence, and the assumption baked into every scheduled job you have ever written, which is that yesterday's successful run tells you something about today's, does not hold here at all.

What expires while nobody is looking

Almost nothing in a modern stack has a lifetime measured in years, and the mismatch is worth quantifying rather than gesturing at.

Let's Encrypt states its position plainly in the policy it publishes in 2026: "Our default certificates are valid for 90 days", with no room for negotiation, since "there is no way to adjust these lifetimes, there are no exceptions". Their guidance is to renew early, recommending "renewing 90 day certificates every 60 days". A show that runs once a year passes through four full certificate lifetimes between editions, because 365 divided by 90 is 4.06. Let's Encrypt also now offers a six day certificate for subscribers who opt in, renewed every three days, and on that option a year is sixty lifetimes.

Automated renewal handles this, when it runs. On a box that exists to serve a pipeline that only fires in September, the renewal timer is one more thing that has to have stayed healthy through eleven months during which nobody would have noticed it failing.

Certificates are the visible case. The same shape applies to API tokens the vendor rotates on their own schedule, to service accounts an IT team disabled during a licence cleanup in February, to an IP allowlist entry removed when the office moved, and to a cloud credential with a maximum age policy attached to it. Each of those has a clock, none of the clocks is a year long, and none of them sends you a message.

What rots when there is no expiry date?

The second category is worse, because there is no date to put in a calendar.

The Python developer's guide, as it stands in 2026, describes the release cycle set out in PEP 602 and states the end point without ambiguity: "Five years after a release, support ends. The release cycle is frozen; no further changes are allowed." It also gives the dates. Python 3.10 reaches end of life in October 2026, 3.11 in October 2027, 3.12 in October 2028.

Put that next to a five edition comparison window. A pipeline written against the current Python at the first edition of a five year history reaches its runtime's end of life at roughly the fifth. The code has run five times in its entire life and it has already outlived the support of the thing it runs on. Nothing failed. Nobody was told.

Around that sit the dependencies, the warehouse's own deprecations, the vendor's export format, and the schema of the source system, which was redesigned in March by a team who told the people who use it daily and had no reason to think of you.

How do you keep an annual pipeline warm?

Run it. Not the tests, the pipeline, on a schedule, against a fixture edition, using the real credentials and the real network path.

Monthly is the right interval for an annual show. It puts a run inside every ninety day certificate lifetime with a comfortable margin, it surfaces a rotated token within a few weeks of the rotation, and it gives eleven opportunities to fix something calmly before the one occasion when it has to work. Weekly is not wrong and buys little extra. Quarterly leaves a gap long enough for a certificate to expire and be replaced without any of it having been observed.

The rehearsal has to be a real execution with a real result that is checked. A run that fires, throws, logs to a file nobody reads and exits zero is a scheduled way of learning nothing, and it is what most of these end up being. The check is the point: assert a row count, assert a checksum on one aggregate, and fail loudly to a channel a human reads.

What the rehearsal has to touch

The temptation is to make the rehearsal cheap by mocking the expensive parts, and mocking is exactly what removes the value.

Four things have to be real. The credential, because credentials are the most common failure and a mocked one proves nothing. The network path, including whatever proxy, allowlist or private link sits between you and the vendor. The schema, meaning the rehearsal writes to tables with the same definitions as production, so a column added by a migration in April breaks the rehearsal in April. And the code path, meaning the rehearsal calls the same entry point the live run calls rather than a simplified wrapper that drifts away from it over time.

What can safely be fake is the data. A fixture edition of a few hundred rows exercises every one of those four while costing nothing and touching no real person's details.

This is a different activity from testing the SQL, which is worth doing separately and more often, and which has its own approach with a small fixture. A unit test tells you the transformation is correct. A rehearsal tells you the pipeline can still reach the things it needs to reach. Both can be green while the other is red.

What the rehearsal costs, against what it saves

Twelve runs a year at 90 seconds each is 1,080 seconds, which is 18 minutes of compute a year. Over a five year show history that is 60 runs and 90 minutes in total.

Set that against the alternative, and be concrete about it. A loader that fails at 09:20 on the first morning costs an engineer the morning, because they are debugging a certificate chain while the show director asks for numbers. If the edition runs four days and live reporting is down for the first of them, a quarter of the on site reporting window is gone and cannot be recovered, since the moment for deciding where to move staff on day one passes at the end of day one. Backfilling the rows afterwards restores the history and restores none of the decisions.

The 18 minutes is also the wrong number to argue about, because the real cost of a rehearsal is the attention it needs when it fails in March. That is the cost worth defending, and it is also the entire benefit, since a March failure is a February problem discovered in March rather than a September emergency.

The fixture edition, and keeping it honest

A fixture edition is a small, permanently available, deliberately fake show that the rehearsal loads. A few hundred registrations, a few thousand scans, a handful of exhibitors and sessions, with the same column shapes as the real thing.

Three properties make it useful. It is checked into the repository, so it changes under review rather than drifting. It has known answers, so the rehearsal can assert them rather than merely completing. And it contains no real personal data, so it can live anywhere and be shared with a vendor when you need to reproduce a problem.

Keeping it honest is the hard part. A fixture built in year one against the vendor's then current export format becomes a fossil, and the rehearsal keeps passing against a shape the vendor stopped producing. The discipline that works is regenerating the fixture from the most recent real edition after every show, by sampling a few hundred rows and replacing the identifying fields. That is an hour of work once a year and it is the difference between a rehearsal and a ritual.

Where a rehearsal cannot help

It cannot rehearse something it has never seen, and the failures that actually take a show week down are often exactly that.

A vendor who redesigns their export between editions produces a file your loader has never encountered, and a rehearsal against last year's shape passes right up until the real file arrives. The same goes for the first edition after a registration platform migration, or the first time a show runs across two venues. Those are handled by validating the incoming file against a declared schema and failing on the difference, rather than by hoping the rehearsal covered it.

The other limit is scale. A green rehearsal on 400 fixture rows says nothing about how the same code behaves on an edition landing 38,900 scans in four days, and it says nothing about what happens when a run is interrupted halfway and repeated, which is a property the load has to be designed for rather than rehearsed into existence.

A final honest limit is organisational. A rehearsal that fails in March lands on somebody's desk in March, when there is no show and no urgency, and the easiest thing in the world is to mute it. A muted rehearsal is worse than no rehearsal, because it produces a green history that somebody will point at in September. If the alert has no owner, the schedule is decoration, and naming that owner in the data platform conventions is the part that actually has to survive a staff change.

Put a recurring entry in the calendar for the first Monday of every month, point it at last edition's data in a scratch schema, and run the loader end to end with the credentials it would really use. The first run of that will fail, which is the point, and it is much better to find out in a month with no show in it.