A synthetic monitor is a robot that visits your site so you find out about breakage before a customer does. The failure mode is almost never "not enough monitoring" — it is a wall of checks nobody tuned, paging people for blips until the pages get muted, at which point you have expensive decoration. The design goal is a system whose every alert is worth waking up for.
What synthetics are for (and not for)
Synthetics answer exactly two questions well: "is it up, from the outside?" and "did a known journey get slower or break, compared to yesterday?" Their strength is the controlled baseline — same request, same location, same interval — which makes them the right tool for availability, certificate expiry, and regression detection on critical paths. Their weakness is representativeness: a robot in a data centre on perfect transit tells you nothing about what a phone on a hillside experiences. That is RUM's job. Teams that ask synthetics to measure user experience end up arguing with a robot; use synthetics for detection, RUM for experience, and benchmarks for one-off comparisons.
Choosing the checks
Start embarrassingly small: one availability check per critical hostname, one transaction check per journey that makes money (home → product → cart, or login → dashboard), and a certificate-expiry check on every certificate you serve. For a CDN-fronted site add two things most setups miss: a check that fetches a known-cacheable asset and asserts on the cache-status header — so you notice when your hit path breaks, not just when your origin does — and a check against the origin directly (guarded appropriately), so you can tell "origin is down" from "edge can't reach origin" the moment an incident starts. Each check asserts on status, on a content string that proves the page actually rendered, and on a response-time threshold set from its own history, not from a wish.
Choosing the locations
Locations should mirror revenue, not the vendor's map. Three to five regions where your users actually are, plus one deliberately far away as an early-warning canary, is enough for most estates; fifty locations mostly buys you fifty ways to catch the same transit hiccup. Two placement rules matter more than count. First, keep at least two locations per region you alert on, so a single vantage point's network trouble can be outvoted. Second, know whether your checks run from cloud data centres or from backbone/last-mile agents, because cloud agents sit unusually close to CDN infrastructure and will flatter your numbers — fine for regression detection, misleading for absolute claims. Whatever you choose, pin it and leave it; a moving vantage point destroys the baseline that is the entire value of the exercise.
Alert rules that survive contact with reality
The internet flickers constantly, and a rule that pages on one failed request will cry wolf nightly. The rules that hold up: alert on consecutive failures (two or three in a row, cutting your false-page rate enormously at the cost of a few minutes' detection), require confirmation from a second location before paging a human, and route by severity — hard failures on money paths page someone, slowness and single-location failures make tickets, everything else is a dashboard. Set response-time alerts as a multiple of that check's own recent baseline rather than one global number, so a naturally-slow journey isn't permanently red and a naturally-fast one can regress badly while staying "green". And test the pipeline itself quarterly by breaking something on purpose in staging; an alert path nobody has fired in a year is a rumour.
Maintaining trust in the system
Cry-wolf systems aren't born, they rot into it. Put a monthly half-hour on the calendar to look at one number: pages fired versus pages that were actionable. Every non-actionable page gets a verdict — tighten the rule, fix the flapping check, or delete it. Deleting checks is a feature: a retired journey's monitor is pure noise, and a check nobody can explain protects nothing. Keep a change log for the monitoring config itself, because "did the site get slower or did the check change?" is a question you will otherwise ask at the worst moment. The steady state to aim for is boring: a small set of checks everyone can name, a page rate near zero, and total confidence that when the phone does buzz, something real is on fire.