OTA update benchmarks
HotCodePush, Capgo, Capawesome Cloud, EAS Update and Stallion measured with one open harness on the same app and the same change. Lower is better in every chart.
How we measure
No run has been published yet.
Every vendor runs on the same sample app and publishes the same one-line change. Each number is the median of five runs, shown with the smallest and the largest. The update check is timed on the device clock. The runs use GitHub-hosted runners, every SDK on a pinned version with its default configuration, and the raw results of every run are committed.
Diagram: the harness publishing one change through each vendor to the same device
Benchmark questions
Because bytes can only be counted on Android. The harness routes the Android emulator through a proxy that counts every byte, and the iOS simulator shares the runner’s network, so it cannot be routed. The SDK footprint, binary size and cold start, is measured on Android and iOS. The four metrics
Because one run can be an outlier. Five runs per cell keep one slow network moment from deciding a number, and the median ignores the extremes. The smallest and the largest run are shown beside every median, and the raw samples are committed, so you can pick another statistic. Runs and medians
Because publishing its results needs its permission. RevoPush’s terms forbid publishing a benchmark against it without written permission. The harness supports RevoPush and measures it once that permission exists; until then it stays out of the raw results and this page. Vendor terms
Fork the open harness and dispatch its workflow once. Every SDK, CLI and framework version is pinned in the repository, so your run measures the same versions as ours; you add each vendor’s token as a secret of your fork. The benchmarks repository
Disagree with a number? The methodology is a repo.
Pinned versions, declared region, raw results committed.
Questions? Join our Discord or contact us.