You do not need a lab, a budget, or vendor accounts to get a defensible first read on CDN performance. You need one representative test object, measurement points where your users actually are, and about four hours. This guide is the exact procedure.
Pick the test objects, not a test file
Benchmarks fail at object selection, not at measurement. A single 100 KB dummy file tells you almost nothing, because CDNs behave differently by object size and cacheability. Pull three real objects from your own site: your most-requested static asset (usually a hero image or bundled JS, typically 50–300 KB), one small object under 5 KB (an icon or API response — this isolates connection overhead), and one large object over 5 MB (video segment or download — this isolates throughput).
Record each object’s current URL, size, and Cache-Control header before you start. If your objects carry no-store or short TTLs, fix that first; benchmarking uncacheable content measures your origin, not the CDN.
Choose measurement points where your users are
Look at one month of analytics and write down the five cities that produce the most sessions. Those are your measurement points; everything else is noise. Free tiers of synthetic testing services give you agents in most major cities, and a $5 VPS in each region works when you need a location the services don’t cover.
Measure from residential-like networks where possible. Agents sitting inside well-peered datacenters flatter every CDN, because the hard part of delivery is the last mile, not the datacenter-to-datacenter hop.
Measure four numbers per object, per city
For each object from each city, capture: DNS resolution time, TCP+TLS connect time, time to first byte (TTFB), and total download time. Run every test twice back-to-back and record both runs separately — the first is your cache-miss/cold number, the second your cache-hit number. The hit/miss gap is itself a finding: a CDN with fast hits but 800 ms misses will hurt you if your catalog is long-tail.
Repeat the full set at three times of day, including your traffic peak. Fifteen minutes of measurement at 3 a.m. is how vendors produce their marketing numbers.
Score it: p95, not average
Averages hide the users who leave. For each CDN and city, sort the TTFB samples and take the 95th percentile — the experience of your unluckiest one-in-twenty visitor. Then compute three summary numbers per candidate: p95 TTFB in your top city, worst-city p95, and cache-hit ratio implied by the hit/miss gap. Whichever candidate wins two of the three for your traffic mix is your provisional leader.
Put every raw sample in a spreadsheet with columns for provider, city, object, run, and each timing. A benchmark you cannot hand to someone else is an opinion.
Common traps
Three mistakes account for most bad benchmarks. Testing through a corporate VPN (you are measuring the VPN). Comparing a warmed CDN against a cold one (issue a purge, then give every candidate the same two-run treatment). And testing vendor demo URLs instead of your own objects on a trial account — demo assets sit on permanently hot, hand-tuned configurations.
One afternoon of this produces a defensible shortlist. It does not produce a procurement decision: rates, contract terms, and support live outside the stopwatch, which is where the rest of this library picks up.