Deduplication, cooldowns, and expiry in operational alerting

A scheduled detector sees whether a condition is true. An alerting system also knows what happened the last time it was true.

Without that memory, the same customer, order, or opportunity returns on every run. The rule may remain analytically correct while the workflow becomes useless. Recipients learn that the system doesn’t remember their work, so they stop granting each message fresh attention.

Stateful alerting needs several controls that solve different problems. Deduplication identifies the case. A cooldown limits repetition. Expiry protects the useful time window. Reminders continue existing work. Suppression records a deliberate decision not to send. Reopening explains why closed work returned.

Collapsing these into one “already sent” flag works only until the second real requirement arrives.

Identity comes first

The dedupe key defines what the business considers the same situation.

Build it from stable business identity: alert family, customer, order, product, location, owner, variant, event window, or whichever fields distinguish the work. Exclude incidental evidence such as run timestamps, formatted text, localized labels, and metrics that may change while the underlying case remains the same.

A key that is too narrow creates duplicates whenever evidence changes. A key that is too broad suppresses separate cases behind one open alert.

Keep the identity ingredients visible before hashing them. Detection should emit the structured identity and its canonical hash. The candidate queue, payload, history, reminders, and writeback should carry the same values rather than re-deriving them independently.

Review identity with examples. Ask which real cases should be grouped, continued, repeated, or treated as new. Hash design is easy. Deciding what the hash means is the work.

Deduplication and cooldown answer different questions

Deduplication asks whether this is the same case. Cooldown asks when the case may interrupt someone again.

A condition can remain true for days while an owner investigates it. Sending the alert on every run punishes normal response time. The cooldown should reflect the business window: hours for urgent operations, days for account review, or no automatic repeat when one open task is enough.

Start the window from an explicit lifecycle event. A detected candidate that never reached the recipient usually shouldn’t consume the cooldown. A successfully delivered alert usually should. A requested reminder may extend or override it because it continues the same workflow.

Repeat policy also needs a reason. Some unresolved conditions should return after a defined period. Others should remain one open case until closure. “Send again after seven days” is a business choice. Code still has to define how it interacts with open work, reminders, retries, and expiry.

Expiry protects actionability

An alert has a useful window.

An order risk may matter before fulfillment. A quote may matter before acceptance. A customer signal may become misleading after newer activity arrives. A candidate that survives a downstream outage can be correct at detection time and harmful at delivery time.

Store expiry separately from closure. Closure says how the workflow ended. Expiry says the opportunity to act has passed. An unhandled alert may expire. A handled alert may close well before expiry. Keep expired cases for audit and tuning, but remove them from current work.

Check expiry before every side effect, including retries. Recovering a failed writer shouldn’t release stale work merely because the queue can finally move again.

Reminders have lineage

A reminder isn’t a duplicate when the system models it as a continuation.

It should point to the original alert, record who requested it, have a due time, and carry its own idempotency key. The scheduler must check both pending reminders and sent history so one request can’t produce several messages after retries or repeated source reads.

Decide whether a reminder reuses the original snapshot, recalculates current context, or presents both. Current context may show that the condition improved or disappeared. The original snapshot still explains why the workflow exists. Hiding that distinction makes the reminder look like a fresh alert.

If the original case closes before the reminder is due, policy should say whether the reminder is cancelled, logged as no longer needed, or allowed to reopen the work. Silence isn’t policy.

Suppression is observable state

A candidate may qualify and still be intentionally withheld because a schedule is closed, a blackout window is active, an owner is unavailable, a route is at capacity, required context is missing, or a runtime gate rejects it.

Record that outcome with a reason and whether it’s temporary or terminal. Silent disappearance makes support impossible: the business knows the condition existed, while the platform has no evidence explaining why no alert appeared.

Suppression also separates policy from failure. “The system chose not to send today” needs a different repair path from “the payload was invalid” or “the writer was unavailable.”

Reopening preserves history

When handled work becomes relevant again, don’t erase the previous state and manufacture a new case.

A reopened alert should point to the earlier alert and state why its closure no longer settles the situation. Materially changed evidence, a new event on the same entity, an expired snooze, or a response-driven return are different reasons and should remain visible.

Reopening gives recipients useful context: this case returned. It also gives rule owners evidence about weak closure criteria, overbroad identity, or conditions that repeatedly resist resolution.

Use reopening for changed state, not as a workaround for poor deduplication. Making the key narrower until every change looks new only destroys lineage.

Keep the lifecycle joinable

Candidate state, alert history, and writer audit answer different questions, but they need common identifiers.

The candidate record should expose identity, status, execution, attempts, next run time, expiry, suppression reason, and last error. Alert history should expose delivery, owner, open or closed state, response, reminders, repeats, reopening, and the payload snapshot. Writer audit should expose the operation, target, request, downstream result, and item-level errors.

Candidate ID, alert ID, dedupe key, execution ID, reminder ID, and writer event ID should form a path. If an operator has to correlate approximate timestamps across five systems, the platform has delegated observability to luck.

These controls reduce alerts without weakening detection. The system still records qualifying conditions, but it interrupts people only when the lifecycle state justifies it. Even a correct signal makes for bad operations when it ignores previous work.