[dwwoelfel]

Cloudflare's blog post still says, in bold, "Cloudflare Workers is 196% faster than Fastly’s Compute@Edge based on the time to first byte from the tests we ran on 50 nodes using Catchpoint’s data from across the world".

It is unethical to leave that up after Fastly pointed out core issues with the benchmarking, like using a free tier that was rate-limited.

[kentonv]

Cloudflare's test compared the free tier of both services. The post was explicit about this. Workers free tier has limits too, and we would certainly have preferred to use the paid version of Workers in our test, but as the paid version of C@E is only available with an enterprise contract, the only fair test we could run was between free tiers.

Incidentally, this means Fastly's blog post is currently displaying test results that compare the enterprise version of Compute@Edge against the free version of Workers. Granted, our bad for the ToS clause, but still.

Despite the strong language in their post, Fastly has not actually demonstrated that anything was intentionally biased or unfair in Cloudflare's test. They've only laid out their opinions as to what would make a more representative benchmark. That's a debate you can have about any benchmark, but that doesn't somehow make the original benchmark "unethical".

[dwwoelfel]

It's not the benchmark that's unethical. The unethical part is leaving up the original claim without adding a correction or a note that addresses the problems Fastly found with the benchmark.