Getting usable sandbox and test data out of an event platform

The integration passed every test. Registrations flowed from the platform to the warehouse, the field mapping was right, the deduplication behaved. Then the sale opened, 900 people registered before lunch, and the job started timing out on a call that had never taken more than 200 milliseconds in testing.

Sandbox and test data is where this gets decided, and the reason so many event integrations fail their first real day is that the vendor's sandbox contained twelve fake registrations with tidy names and no custom questions answered.

What a sandbox is actually a copy of

Two different things get called a sandbox and they behave nothing alike.

The first is a genuine environment copy, refreshed from production, running the same code with the same configuration and a subset or a full copy of the data. Enterprise platforms in adjacent categories offer this properly. The second is a permanently empty demonstration tenant with a different feature set, seeded by whoever set it up, and never refreshed since. Most event platform sandboxes are the second kind.

The distinction decides what a passing test proves. Against a refreshed copy, a passing test tells you the integration works against your configuration, which is most of what you wanted to know. Against an empty demonstration tenant, it tells you your code compiles and your credentials work.

Ask the vendor three questions before you plan around their sandbox. Is it refreshed from production, and how often. Does it run the same version of the platform. Are the limits the same. The third question is the one with the most surprising answer.

Why does a load test in a sandbox tell you nothing?

Because the limits are usually different, and usually different in the generous direction.

Salesforce publishes the clearest example. A full sandbox is allocated 5,000,000 API requests per 24 hours, while an Enterprise production org's allocation starts at 100,000 plus an allowance per licence, and a Developer Edition org gets 15,000 (Salesforce, 2026). That is a fiftyfold gap between the environment you test in and the one you ship to. A sync that comfortably clears in the sandbox can exhaust a production day's allocation before lunch, and nothing in the test output hints at it. Sizing your own sync against the production numbers is a separate exercise, and the Salesforce API limits post works through it.

The asymmetry runs the other way too. Microsoft notes that trial environments for Dataverse allocate only a single web server, while most production environments have more than one, and its service protection limits are enforced per web server (Microsoft, 2026). So a trial environment throttles sooner than production for the same load. Either direction breaks the inference you were trying to make. The only safe assumption is that sandbox limits and production limits are unrelated numbers until the vendor tells you otherwise in writing.

Build the file yourself

Since the sandbox will not give you a realistic load, generate one. This is an afternoon of work and it is the highest value afternoon in the whole integration.

Take a real registration export from your last edition, strip the personal data, and use its structure as the template. Not a fresh schema written from the documentation. The actual export, because the documentation describes the fields that exist and the export shows you which ones people fill in.

Then size it against your peak rather than your average. Say a deadline day brought 3,100 registrations across six hours. The average is 8.6 a minute, which no integration on earth struggles with. Look at the ten worst minutes and you might find 240 of those registrations landed there, a rate of 24 a minute, nearly three times the daily mean. Test at 24 a minute. Then test at 48, because next year's deadline day might be twice as sharp, and the marginal cost of finding out now is one more run of a script you have already written.

Portfolio timing adds a second multiplier that single show planning misses. If three of your eight shows share a September deadline and all three feed the same CRM through the same credential, the rate the CRM sees is all three added together. Three shows at 24 a minute makes 72 a minute against one allocation. Work out which of your editions overlap before you decide the test rate, because the integration is sized by the calendar rather than by any single event.

What shape does realistic test data need?

Messiness, in the specific ways your registration form produces it.

Reuse the proportions from the real export rather than inventing them. If 9 per cent of last year's rows carried a non-ASCII character in a name, put 9 per cent in the synthetic file, because character encoding failures are the single most common way a load turns into support tickets. If 4 per cent of rows had no company name, keep 4 per cent empty. If the job title field allowed free text and 300 distinct values appeared, generate 300, not 8.

Five categories are worth deliberately including. Rows where a required field is empty because the form made it optional for one registration type. Rows with a comma and a quotation mark inside a free text answer. Rows for the same person registering twice with different email addresses. Rows with a very long value, at or above the platform's stated field length. And cancellations, because a cancelled registration is a state transition and most test files only contain creations.

Generate the file once, keep it in version control next to the integration, and regenerate the proportions each year from the latest export. It becomes the regression suite, and it is the only thing that tells you whether this year's platform upgrade changed behaviour on the awkward rows.

Test environments expire, and other operational surprises

Sandboxes have a lifecycle that nobody puts in the project plan.

HubSpot documents that a standard account can create up to 10 developer test accounts, that a test account expires after 90 days with no API calls made to it, that renewals must happen within 30 days of the expiry date, and that the test accounts include a 90 day trial of many enterprise features (HubSpot, 2026). Read that carefully and the implications stack up. Your test environment has an enterprise feature set your production account may not have, so an integration that depends on one of those features passes in test and fails in production. And the environment quietly dies during any quarter when the integration is stable and nobody calls it.

Put the expiry dates in the same place as the credential expiry dates. Both belong in the integration runbook, and both are discovered at the worst possible moment when they live only in somebody's inbox.

Running the rehearsal

The last step before a sale opens is a rehearsal, and it wants to be boring.

Load the synthetic file at peak rate against whichever environment is closest to production, with the real credentials and the real network path if that is possible without polluting live data. Watch four things: the error rate, the rate limited response count, the wall clock duration, and the row count that actually landed. The fourth is the one people skip, and a rehearsal that reports success while loading 2,940 of 3,100 rows has told you something important that the exit code did not.

Then rehearse the failure. Kill the job halfway and restart it. Revoke the credential and see what the alert looks like. Send the same batch twice. If the answer to any of those is a support ticket rather than a log line, fix it before the sale rather than during it, because show week is exactly when changes are frozen and the fix will have to wait.

Write down the four numbers from the rehearsal and keep them. Next year the same rehearsal against the same file gives you a comparison rather than a reading, and a duration that has grown from 9 minutes to 26 minutes on identical input is the earliest warning you will get that something upstream changed. Nobody notices a gradual slowdown in a job that still finishes before the office opens, right up until the year it does not.

Where this stops

A synthetic file reproduces the shape of your data and the volume of your traffic. It cannot reproduce the platform's behaviour under someone else's load.

Shared multi tenant platforms are slower on the days everyone in the industry is busy, and the vendor's own throttling responds to aggregate demand across tenants. Your rehearsal on a Tuesday in February tells you very little about the same platform on the Monday of a major trade show week. The honest mitigation is a margin: if the rehearsal uses 60 per cent of the available window, you have room. If it uses 95 per cent, you have a plan that works on quiet days only.

The second limit is that synthetic data cannot contain the thing you have not seen. Every year one registration arrives in a form nobody anticipated, and the useful response is to add it to the generator afterwards rather than to pretend the generator was complete. Treat the file as a growing record of everything that has ever broken the integration.

This week, export last year's registration file, count how many rows arrived in the busiest ten minutes, and divide by ten. That number is the rate your next test should run at, and it is almost certainly higher than the one you have been using.