ENGINEERING NOTE 15 / 50 · SMS
Keep SMS test traffic out of customer reliability reports
Test orders should be identifiable before a report is calculated. A display-name guess is not a reliable way to remove internal activity.
Published by OPEN RAMBO. The engineering patterns and pseudocode illustrate designs, not a documented product API. Numerical examples are illustrative.
Declare test identity early
In a controlled environment, attach a test-run identifier and a known test-account flag. For production acceptance, maintain an explicit approved exclusion list. Do not retroactively label unsuccessful customer orders as tests because they look unusual.
Keep two honest totals
Report gross orders, excluded tests and eligible customer orders. Internal acceptance may demonstrate that a workflow works, but it does not establish customer-wide success. Keep the evidence while excluding it from acquisition and customer conversion metrics.
Version the exclusion rules
Record why each exclusion applies and when it became effective. If a test account later becomes a real customer account, classify by activity and dates rather than silently changing historical reports. Aggregate outputs should not expose the test user’s credentials or message contents.
Illustrative implementation
customer_orders = [o for o in orders
if not exclusion_policy.matches(o)]
assert len(orders) == len(customer_orders) + excluded_countAcceptance check
Run the report twice with the same policy version and verify identical totals. Publish the excluded count beside the customer result.