Burn-rate alerts fire spuriously during traffic shifts#312

Open

MIMiguel Arroyocommented 2d ago

Describe the bug

Burn-rate alerts fire spuriously during traffic shifts — surfaces on thresholdctl in the high-priority path. The full reproduction is below, along with the workaround we use in staging.

Reproduce

Clone ln-dev7/thresholdctl at the tip of main.
Run the smoke suite: pnpm test --workspace thresholdctl.
Notice the test for the affected contract flakes within the first 50 iterations.

Expected behavior

The contract should hold deterministically across the full matrix — staging, production, and the local dev runtime.

Environment

node 22.x
pnpm 9.x
OS: macOS 15.2 (also reproduced on Linux 6.x)

Labels: bug

LNLNcommented 3d ago

I've seen this too. I'll fix.

LNLNcommented 3d ago

Can confirm the repro on a clean checkout — the failing case is reliable on macOS 15.2 with Node 22.

LNLNcommented 2d ago

Pulled this locally, the workaround in the description is solid. Happy to land it as a follow-up.

LNLNcommented 2d ago

Could we add a regression test before the patch lands? Otherwise this will resurface as soon as the surrounding cleanup happens.

LNln-dev7self-assigned this2d ago

MImiguel-arroyoadded the labelbug2d ago

MImiguel-arroyoadded this to the milestonev1.4 — Stability2d ago

LNln-dev7added a commit that references this issuefix(thresholdctl): guard the regression surfaced in #3120307a1e1d ago

LNln-dev7mentioned this in#3601d ago

Issues