Introduction to iterators - Rust for C-Programmers

13.1 The Essence of Rust Iterators

In programming, processing collections of items—arrays, lists, maps—is fundamental. Iteration is the process of accessing these items sequentially. While C uses explicit loops with index variables or pointers, Rust provides a more abstract and safer mechanism built around two core concepts: iterables and iterators.

Iterable: A type that can produce an iterator. Standard Rust collections (Vec<T>, HashMap<K, V>, String, arrays, slices) are iterable. They provide methods to create iterators over their contents. The IntoIterator trait formalizes this capability.
Iterator: An object responsible for managing the state of the iteration process. It implements the std::iter::Iterator trait, which defines a standard interface for producing a sequence of values. The fundamental method is next(), which attempts to yield the next item, returning Some(item) if available or None when the sequence is exhausted.

Rust collections offer several methods for iteration, each returning a specific iterator object that controls how elements are accessed:

iter(): Yields immutable references (&T). The collection is borrowed immutably.
iter_mut(): Yields mutable references (&mut T). The collection is borrowed mutably, allowing in-place modification.
into_iter(): Consumes the collection and yields elements by value (T). Ownership is transferred out of the collection.

Rust’s for loop seamlessly integrates with this system. It implicitly calls into_iter() on the expression being looped over and then repeatedly calls next() on the resulting iterator until it returns None.

This separation of concerns—the collection holding the data and the iterator managing the traversal—leads to cleaner, more maintainable code.

Fundamental Concepts:

Abstraction: Iterators decouple sequence processing logic from the underlying data source (vector, hash map, file lines, number range). The same iterator methods (map, filter, collect) work on any sequence produced by an iterator.
Laziness: Many iterator operations, known as adapters (map, filter), do not execute immediately. They return a new iterator representing the transformation. Computation is deferred until a consuming method (collect, sum, for_each) is called, which pulls items through the iterator chain. This avoids unnecessary work.
Composability: Iterators can be chained together elegantly, enabling complex data processing pipelines expressed concisely, often in a functional style (e.g., data.iter().filter(...).map(...).sum()).
Safety: Combined with Rust’s ownership and borrowing rules, iterators provide strong compile-time guarantees against common C pitfalls like dangling pointers or modifying a collection while iterating over it (unless using iter_mut explicitly and safely).
Performance (Zero-Cost Abstraction): Rust’s compiler heavily optimizes iterator chains, often generating machine code equivalent to handwritten C loops. This makes iterators an efficient choice even for performance-critical code.

13.1.1 The `Iterator` Trait

The foundation of Rust’s iteration mechanism is the Iterator trait:

#![allow(unused)]
fn main() {
pub trait Iterator {
    // The type of element produced by the iterator.
    type Item;

    // Advances the iterator and returns the next value.
    // Returns `Some(Item)` if a value is available.
    // Returns `None` when the sequence is exhausted.
    // Takes `&mut self` because advancing typically modifies
    // the iterator's internal state.
    fn next(&mut self) -> Option<Self::Item>;

    // Provides numerous other methods (adapters and consumers)
    // with default implementations that utilize `next()`.
    // Examples: map, filter, fold, sum, collect, etc.
}
}

Item Associated Type: Defines the type of value yielded by the iterator (e.g., i32, &String, Result<String, io::Error>).
next() Method: The sole required method. It must advance the iterator’s internal state and return the next item wrapped in Some. Once the sequence ends, it must consistently return None. (This “always None after first None” behavior is formalized by the FusedIterator trait, implemented by most standard iterators).

While you can manually call next() (e.g., while let Some(item) = my_iterator.next() { ... }), idiomatic Rust overwhelmingly favors using for loops or iterator consumer methods, which handle the next() calls implicitly and more readably.

13.1.2 The `IntoIterator` Trait and `for` Loops

Now that we’ve seen what the Iterator trait requires, how do we typically get an iterator object from a collection like a Vec? This is the role of the IntoIterator trait, which is fundamental to how Rust’s for loop operates.

Rust’s for loop is syntactic sugar built upon the IntoIterator trait:

#![allow(unused)]
fn main() {
pub trait IntoIterator {
    // The type of element yielded by the resulting iterator.
    type Item;
    // The specific iterator type returned by `into_iter`.
    type IntoIter: Iterator<Item = Self::Item>;

    // Consumes `self` (or borrows it) to create an iterator.
    fn into_iter(self) -> Self::IntoIter;
}
}

When you write for item in expression, Rust implicitly calls expression.into_iter(). This method returns an actual Iterator, which the for loop then drives by repeatedly calling next() until it receives None.

Standard collections implement IntoIterator in multiple ways (for the collection type itself, for &collection, and for &mut collection) to support the different iteration modes based on ownership and borrowing.

13.1.3 Iteration Modes: `iter()`, `iter_mut()`, `into_iter()`

Most collections provide three common ways to obtain an iterator, reflecting different needs regarding data access and ownership. These are typically exposed via inherent methods (iter, iter_mut, into_iter) and are also triggered implicitly by for loops based on how the collection is referenced:

Immutable Iteration (iter() / &collection)

Yields immutable references (&T).
The original collection is borrowed immutably; it remains accessible after the loop.
Method: .iter()
for loop syntax: for item_ref in &collection { ... } (equivalent to for item_ref in collection.iter() { ... })

fn main() {
    let data = vec!["alpha", "beta", "gamma"];

    // Using the method explicitly: yields &&str
    println!("Using data.iter():");
    for item_ref in data.iter() {
        // item_ref has type &&str
        // println! can format &&str directly because it implements Display
        println!(" - Item: {}", item_ref);
    }

    // Using the for loop sugar with &data: also yields &&str
    println!("Using &data:");
    for item_ref in &data {
        // item_ref also has type &&str
         println!(" - Item: {}", item_ref);
    }

    // data is still valid and usable here
    println!("Original data: {:?}", data);
}

Mutable Iteration (iter_mut() / &mut collection)

Yields mutable references (&mut T).
Allows modifying the collection’s elements in place.
The original collection is borrowed mutably. Cannot be accessed immutably elsewhere during the loop.
Method: .iter_mut()
for loop syntax: for item_mut_ref in &mut collection { ... } (equivalent to for item_mut_ref in collection.iter_mut() { ... })

fn main() {
    let mut numbers = vec![10, 20, 30];
    // Using the method explicitly:
    for num_ref in numbers.iter_mut() {
        // num_ref has type &mut i32
        *num_ref += 5; // Dereference (*) to modify the value
    }
    println!("Modified numbers: {:?}", numbers); // Output: [15, 25, 35]

    // Using the for loop sugar:
    for num_ref in &mut numbers {
        // num_ref also has type &mut i32
         *num_ref *= 2;
    }
    println!("Doubled numbers: {:?}", numbers); // Output: [30, 50, 70]
}

Consuming Iteration (into_iter() / collection)

Yields owned values (T).
Takes ownership of (consumes) the collection. The original collection variable cannot be used after the for statement, as ownership is moved to the iterator created by into_iter(). The elements themselves are moved out of the collection one by one.
Method: .into_iter()
for loop syntax: for item in collection { ... } (equivalent to for item in collection.into_iter() { ... })

fn main() {
    // --- Using the for loop sugar (most common) ---
    let strings1 = vec![String::from("hello"), String::from("world")];
    let mut lengths1 = Vec::new();
    println!("Using `for s in strings` (sugar):");
    // This implicitly calls strings1.into_iter()
    for s in strings1 { // `strings1` is moved here
         // s has type String (owned value, not Copy)
         println!(" - Got owned string: '{}'", s);
         lengths1.push(s.len());
         // s goes out of scope and is dropped here
    }
    // println!("{:?}", strings1); // Error! `strings1` value was moved
    println!("   Lengths: {:?}", lengths1); // Output: [5, 5]

    // --- Using the method explicitly ---
    let strings2 = vec![String::from("hello"), String::from("world")];
    let mut lengths2 = Vec::new();
    println!("\nUsing `for s in strings.into_iter()` (explicit):");
    // This explicitly calls strings2.into_iter()
    for s in strings2.into_iter() { // `strings2` is moved here
         // s also has type String (owned value)
         println!(" - Got owned string: '{}'", s);
         lengths2.push(s.len());
         // s goes out of scope and is dropped here
    }
    // println!("{:?}", strings2); // Error! `strings2` value was moved
    println!("   Lengths: {:?}", lengths2); // Output: [5, 5]
}

Note on Vec<String> vs. Vec<&str>: This example uses Vec<String> deliberately. The goal is to illustrate consuming iteration where owned values (String), which are not Copy, are moved out of the collection. If we had used let strings = vec!["hello", "world"]; (creating a Vec<&str>), the loop for s in strings would still consume the vector, but s inside the loop would be of type &str. Since &str is Copy, the ownership transfer aspect for the elements wouldn’t be as apparent as it is with the non-Copy String type.

It’s a strong convention in Rust to provide these inherent methods (.iter(), .iter_mut(), and a consuming .into_iter(self)) on collection-like types, even though the for loop can work directly with references via the IntoIterator trait implementations. These methods improve discoverability and allow for explicit iterator creation when needed (e.g., for chaining methods before a loop). Typically, their implementation is straightforward: the inherent iter(&self) method simply calls IntoIterator::into_iter on self (which has type &Collection), and similarly for iter_mut and the consuming into_iter.

Choosing the Correct Mode:

Use iter() (&collection) for read-only access when you need the collection afterward.
Use iter_mut() (&mut collection) when you need to modify elements in place.
Use into_iter() (collection) when you want to transfer ownership of the elements out of the collection (e.g., into a new collection or thread, or to consume them).

13.1.4 Understanding References in Closures (`&x`, `&&x`)

When using iterator adapters like map or filter with iter(), the closures often receive references to the items yielded by the iterator. This can sometimes lead to double references (&&T). This occurs naturally:

some_collection.iter() produces an iterator yielding items of type &T.
Adapters like filter pass a reference to the yielded item into the closure. The closure therefore receives a parameter of type &(&T), which simplifies to &&T.

Rust’s pattern matching in closures often handles this gracefully, allowing you to directly access the underlying value:

fn main() {
    let numbers = vec![1, 2, 3, 4];

    // `numbers.iter()` yields `&i32`.
    // `filter`'s closure receives `&(&i32)`, i.e., `&&i32`.

    // Using pattern matching `|&&x|` to automatically dereference twice:
    let evens_refs: Vec<&i32> = numbers.iter()
        .filter(|&&x| x % 2 == 0) // `x` here is `i32` due to pattern matching
        .collect();
    println!("Evens (refs): {:?}", evens_refs); // Output: [&2, &4]

    // If we need owned values, we can copy *after* filtering:
    // Note: `copied()` works because i32 implements the `Copy` trait.
    // For non-`Copy` types, use `.cloned()` if `T` implements `Clone`.
    let evens_owned: Vec<i32> = numbers.iter()
        .filter(|&&x| x % 2 == 0)
        .copied() // Converts the `&i32` yielded by filter into `i32`
        .collect();
    println!("Evens (owned): {:?}", evens_owned); // Output: [2, 4]

    // Alternatively, dereference explicitly inside the closure:
    let odds: Vec<i32> = numbers.iter()
        .filter(|item_ref_ref| (**item_ref_ref) % 2 != 0) // **item_ref_ref gives i32
        .copied() // Convert &i32 to i32
        .collect();
    println!("Odds (owned): {:?}", odds); // Output: [1, 3]

    // Using `into_iter()` avoids the extra reference layer if ownership is intended:
    let squares: Vec<i32> = numbers.into_iter() // yields `i32` directly
        .map(|x| x * x) // closure receives `i32` directly
        .collect();
    println!("Squares: {:?}", squares); // Output: [1, 4, 9, 16]
    // `numbers` is no longer available here
}

Understanding the iteration mode (iter, iter_mut, into_iter) tells you the base type yielded (&T, &mut T, or T), which helps predict the types received by closures in subsequent adapters and whether dereferencing or methods like copied/cloned are needed.

13.1.5 Iterator Adapters vs. Consumers

Iterator methods fall into two main categories:

Adapters (Lazy): These transform an iterator into a new iterator with different behavior (e.g., map, filter, take, skip, enumerate, zip, chain, peekable, cloned, copied). They perform no work until the iterator is consumed. They are chainable, building up a processing pipeline.
Consumers (Eager): These consume the iterator, driving the next() calls and producing a final result or side effect (e.g., collect, sum, product, fold, for_each, count, last, nth, any, all, find, position). Once a consumer is called, the iterator (and the chain built upon it) is used up and cannot be used again.

fn main() {
    let numbers = vec![1, 2, 3, 4, 5];

    // Adapters: map and filter (lazy, no computation happens yet)
    // numbers.iter() -> yields &i32
    // .map(|&x| x * 10) -> yields i32 (deref pattern `|&x|`)
    // .filter(|&val| val > 25) -> `val` is `i32` here
    let adapter_chain = numbers.iter()
        .map(|&x| x * 10) // Needs `Copy` or manual deref `*x * 10`
        .filter(|&val| val > 25);

    // Consumer: collect (eager, executes the chain)
    // `collect` gathers the i32 values yielded by filter into a Vec<i32>.
    let result: Vec<i32> = adapter_chain.collect();

    println!("Result: {:?}", result); // Output: [30, 40, 50]

    // Trying to use adapter_chain again would fail compilation:
    // let count = adapter_chain.count(); // Error: use of moved value `adapter_chain`
}

Rust for C-Programmers

13.1 The Essence of Rust Iterators

13.1.1 The Iterator Trait

13.1.2 The IntoIterator Trait and for Loops

13.1.3 Iteration Modes: iter(), iter_mut(), into_iter()

13.1.4 Understanding References in Closures (&x, &&x)

13.1.5 Iterator Adapters vs. Consumers

13.1.1 The `Iterator` Trait

13.1.2 The `IntoIterator` Trait and `for` Loops

13.1.3 Iteration Modes: `iter()`, `iter_mut()`, `into_iter()`

13.1.4 Understanding References in Closures (`&x`, `&&x`)