Itamar Turner-Trauring pythonspeed.com

CI for performance: Reliable benchmarking in noisy environments  ↦

Benchmarking is often not done in CI because it’s so hard to get consistent results; there’s a lot of noise in cloud VMs, so you ideally want dedicated hardware. But, it turns out you can use a tool called Cachegrind to get consistent benchmarks results across different computers, allowing you to run benchmarks in GitHub Actions, GitLab CI, etc. and still get consistent results.


Discussion

Sign in or Join to comment or subscribe

Player art
  0:00 / 0:00