Rust for C-Programmers

24.8 Benchmarking

Benchmarking measures the execution speed (latency) or throughput of code snippets. It complements testing by tracking performance characteristics and helping to identify regressions or validate optimizations. Systems programming often requires careful performance management, making benchmarking a valuable tool.

Rust historically had unstable, built-in benchmarking support usable only on the nightly compiler toolchain. However, the ecosystem has largely standardized on powerful third-party crates that work on stable Rust. criterion and divan are two popular choices.

24.8.1 Built-in Benchmarks (Nightly Rust Only - Deprecated)

The built-in harness (#[bench] attribute, test::Bencher) requires the nightly toolchain and the #![feature(test)] flag. Due to its instability and the maturity of external alternatives, it’s generally not recommended for new projects. We mention it here for historical context but advise using crates like criterion or divan.

24.8.2 Benchmarking with criterion (Stable Rust)

criterion is a widely used, statistics-driven benchmarking library. It performs multiple runs, analyzes results statistically to mitigate noise, detects performance changes between runs, and can generate detailed HTML reports.

1. Add Dependency and Configure Harness:

   # Cargo.toml
   [dev-dependencies]
   criterion = { version = "0.5", features = ["html_reports"] } # Check for latest version
   
   # Tell Cargo to use criterion's test harness for benchmarks, not the default one.
   # 'main' corresponds to the benchmark file benches/main.rs
   [[bench]]
   name = "main"
   harness = false
   

2. Create Benchmark File: Create a file like benches/main.rs.

   // benches/main.rs
   use criterion::{black_box, criterion_group, criterion_main, Criterion};
   // Assume fibonacci function is in your library crate 'my_crate'
   use my_crate::fibonacci; // Ensure this function is pub or accessible
   
   fn fibonacci_benchmarks(c: &mut Criterion) {
       // Benchmark fibonacci(20)
       c.bench_function("fib 20", |b| b.iter(|| fibonacci(black_box(20))));
   
       // Benchmark fibonacci(30) with a different ID
       c.bench_function("fib 30", |b| b.iter(|| fibonacci(black_box(30))));
   }
   
   // Group benchmarks together
   criterion_group!(benches, fibonacci_benchmarks);
   // Generate the main function required to run criterion benchmarks
   criterion_main!(benches);

   • black_box: A function that prevents the compiler from optimizing away the code being benchmarked, ensuring the work is actually performed.
   • Criterion::bench_function: Defines a single benchmark.
   • Bencher::iter: Runs the provided closure multiple times to gather statistics.
3. Run: Execute cargo bench. Results and reports are saved in target/criterion/.

24.8.3 Benchmarking with divan (Stable Rust >= 1.80)

divan is a newer benchmarking library (requires Rust 1.80+) focused on simplicity, low overhead, and features like parameterized benchmarking using attributes.

1. Add Dependency and Configure Harness:

   # Cargo.toml
   [dev-dependencies]
   divan = "0.1" # Check for the latest version
   
   [[bench]]
   name = "main" # Corresponds to benches/main.rs
   harness = false
   

2. Create Benchmark File: Create benches/main.rs.

   // benches/main.rs
   use my_crate::fibonacci; // Assume function is in your lib crate
   
   fn main() {
       // Run all benchmarks registered in this crate
       divan::main();
   }
   
   // Simple benchmark for a fixed input
   #[divan::bench]
   fn fib_10() -> u64 {
       fibonacci(divan::black_box(10))
   }
   
   // Parameterized benchmark: runs for each value in `args`
   #[divan::bench(args = [5, 15, 25])]
   fn fib_param(n: u32) -> u64 {
       fibonacci(n) // black_box is often implicitly handled by divan
   }

   • divan::main(): Initializes and runs the benchmarks.
   • #[divan::bench]: Attribute marks a function as a benchmark.
   • args = [...]: Provides input values for parameterized benchmarks.
   • divan::black_box is available if needed, but divan often applies it automatically.
3. Run: Execute cargo bench. Results are printed directly to the console.

Choosing between criterion and divan depends on project needs; criterion offers more detailed statistical analysis and reporting, while divan emphasizes ease of use and lower overhead.