#[cold] / #[hot]

Level 15 — Performance & Optimization Compiler attributes (#[cold] in standard Rust, and #[gnu::hot] / likely/unlikely hints) that inform the LLVM optimizer about function execution frequency, guiding branch prediction, instruction placement, and CPU cache line alignment.

Mistake 3: Relying Solely on #[cold] Without #[inline(never)] for Large Slow Paths

The mistake: Annotating a slow-path error handler with #[cold] but omitting #[inline(never)].

Why it's wrong: While #[cold] strongly discourages inlining, LLVM heuristic optimizations can still inline small #[cold] functions into callers. If the slow-path code is large, inlining it increases the caller's stack frame size and machine code footprint.

Fix: Always pair #[cold] with #[inline(never)] for large out-of-line slow paths.

1. Prerequisites

• Release Profile — Cargo release profile driving LLVM optimization passes.
• perf / flamegraph — Profiling tools used to identify hot and cold code paths.

2. Term Category

Rust Optimization Attributes (compiler branch prediction hints): Function temperature attributes — primarily #[cold] built into standard Rust, alongside #[gnu::hot] (and compiler intrinsics likely/unlikely) — provide explicit hints to the LLVM compiler regarding execution frequency. Annotating a function as #[cold] tells the compiler that the function is rarely executed (e.g. rare error handling, panic formatters, or fallback paths). This allows LLVM to optimize hardware CPU Instruction Cache (I-Cache) layout by pushing cold branch instructions out of the hot execution path.

3. Explanation

(1) Design Motivation — "Why did we design this?"

Modern CPUs achieve high performance through Branch Prediction and Instruction Cache (I-Cache) locality:

1. CPUs pre-fetch consecutive assembly instructions into high-speed L1 Instruction Caches ahead of execution.
2. When a conditional branch (if / else) occurs, if the CPU correctly predicts which branch will execute, execution continues at full speed.
3. If a branch misprediction occurs or if the CPU has to jump to an out-of-cache memory address (an I-Cache miss), the CPU pipeline stalls, wasting 10–20 clock cycles.

In high-performance code, functions often have a "hot fast path" executed 99.9% of the time, and a "cold error path" executed 0.1% of the time:

fn process(val: u32) {
    if val < 1000 {
        // Hot Path (99.9% of calls)
        fast_math(val);
    } else {
        // Cold Path (0.1% of calls)
        log_out_of_bounds_error(val);
    }
}

If LLVM places the assembly code for log_out_of_bounds_error directly adjacent to fast_math inside the main function memory layout:

• Cold error string-formatting assembly instructions take up precious L1 Instruction Cache lines.
• The CPU cache fills with unneeded error handling code, eviction of hot loop instructions, and reduced pipeline throughput.

Annotating log_out_of_bounds_error with #[cold] solves this:

• Assembly Relocation: LLVM moves the #[cold] function assembly to a separate, far-away memory block, keeping the hot path instructions tightly packed together inside the CPU L1 cache line.
• Inlining Prevention: LLVM automatically treats #[cold] functions as undesirable for inlining into hot callers.
• Branch Weight Optimization: Conditional branches leading to #[cold] functions are annotated with low execution weights, configuring the hardware CPU branch predictor to assume the fast path by default.

(2) Function Temperature Attributes Summary

(3) Reality Metaphor

Imagine a Fast-Food Drive-Thru Window Setup:

• Without #[cold] Optimization is like placing a massive, 20-foot customer complaint desk right next to the cash register at the drive-thru window:
  • 99.9% of customers just order a burger (hot fast path).
  • But every car must navigate past the bulky complaint desk (cold error path code cluttering CPU I-Cache), slowing down the entire line.
• Applying #[cold] is moving the complaint desk to a quiet office building in the back parking lot:
  • The drive-thru window (hot CPU cache line) is now 100% streamlined: cars grab burgers and leave at maximum speed.
  • On the rare occasion a customer has a complaint (rare error occurs), they are directed to walk over to the back office (jump to cold out-of-line assembly).

(4) Code Examples

Short Snippet (Out-of-Line Cold Error Path with #[cold])

pub fn divide_fast(a: u32, b: u32) -> u32 {
    if b != 0 {
        // Hot fast path: executed 99.9% of the time
        a / b
    } else {
        // Cold error path: branches to `#[cold]` function
        panic_divide_by_zero()
    }
}

/// Cold helper function marked `#[cold]` and `#[inline(never)]`.
/// LLVM moves this assembly out of the main `divide_fast` I-Cache line!
#[cold]
#[inline(never)]
fn panic_divide_by_zero() -> ! {
    panic!("Catastrophic Error: Division by zero is undefined!");
}

fn main() {
    let result = divide_fast(100, 5);
    println!("Fast division result: {}", result); // 20
}

Fuller Example (Optimizing Vector Capacity Reallocation in a Hot Loop)

pub struct CustomVector<T> {
    ptr: *mut T,
    cap: usize,
    len: usize,
}

impl<T> CustomVector<T> {
    pub fn new() -> Self {
        CustomVector { ptr: std::ptr::null_mut(), cap: 0, len: 0 }
    }

    /// Hot push method called millions of times inside loops.
    /// Needs to stay small so LLVM inlines `push` into caller loops!
    #[inline]
    pub fn push(&mut self, val: T) {
        if self.len == self.cap {
            // Cold path: vector is full, needs capacity reallocation!
            self.grow_cold();
        }
        unsafe {
            // Hot path: write value to uninitialized slot
            std::ptr::write(self.ptr.add(self.len), val);
            self.len += 1;
        }
    }

    /// Cold reallocation helper marked `#[cold]`.
    /// Preserves small instruction size for `push()` in caller CPU cache lines!
    #[cold]
    #[inline(never)]
    fn grow_cold(&mut self) {
        let new_cap = if self.cap == 0 { 4 } else { self.cap * 2 };
        println!("[COLD PATH] Reallocating vector capacity to {}", new_cap);
        // ... memory allocation reallocation logic ...
        self.cap = new_cap;
        let layout = std::alloc::Layout::array::<T>(new_cap).unwrap();
        self.ptr = unsafe { std::alloc::alloc(layout) as *mut T };
    }
}

fn main() {
    let mut v = CustomVector::new();
    v.push(10);
    v.push(20);
    println!("CustomVector pushed elements successfully.");
}

4. Common Mistakes & Pitfalls

Mistake 1: Marking Hot Functions as #[cold]

The mistake: Accidentally annotating a function #[cold] that is executed frequently inside a hot loop.

Why it's wrong: Marking a hot function #[cold] misleads LLVM's branch predictor heuristics and prevents beneficial inlining, causing I-Cache misses and severe performance degradation on the hot path.

Incorrect:

// ❌ Anti-pattern: Marking a core loop math helper as cold!
#[cold] 
fn multiply_matrix_cell(a: f32, b: f32) -> f32 { a * b }

Fix:

// Correct: Use `#[inline]` for hot math helpers
#[inline]
fn multiply_matrix_cell(a: f32, b: f32) -> f32 { a * b }

Mistake 2: Inlining Cold Error Handling Logic inside Hot Functions

The mistake: Writing complex error formatting code directly inside a hot function without delegating to a #[cold] #[inline(never)] helper function.

Why it's wrong: If a hot function contains complex format!(), eprintln!(), or string allocations inline inside an else branch, the LLVM compiler includes all that error formatting assembly inside the main function's code layout, swelling the function's byte size and preventing callers from inlining the hot function.

Incorrect:

pub fn process(val: u32) -> Result<u32, String> {
    if val < 100 {
        Ok(val * 2)
    } else {
        // ❌ Inlines large string formatting code into the main function layout!
        Err(format!("Error: received invalid value {} at timestamp {:?}", val, std::time::Instant::now()))
    }
}

Fix:

pub fn process(val: u32) -> Result<u32, String> {
    if val < 100 {
        Ok(val * 2)
    } else {
        Err(make_error_string(val)) // Delegated to out-of-line cold helper
    }
}

#[cold]
#[inline(never)]
fn make_error_string(val: u32) -> String {
    format!("Error: received invalid value {} at timestamp {:?}", val, std::time::Instant::now())
}

5. Practice Exercises

Exercise 1: Embedded Ring Buffer with Out-of-Line Slow-Path Diagnostics

Scenario: Problem Statement: In an embedded telemetry logging system running on a #![no_std] microcontroller, a high-frequency Direct Memory Access (DMA) ring buffer processes incoming sensor data frames. Pushing a data byte into the ring buffer occurs millions of times per second (the hot fast path, executed 99.9% of the time). However, when the buffer reaches full capacity (the cold slow path, 0.1% of calls), the driver must format overflow statistics, update hardware diagnostic registers, and return an overflow error.

Requirements: If the complex error diagnostic logic is written inline inside the push method, LLVM inflates the machine code size of push. This forces extra register allocations (spilling callee-saved registers to the stack) and pollutes the CPU L1 Instruction Cache (I-Cache), degrading hot-path throughput.

Implement a #![no_std]-compatible TelemetryRingBuffer<T, const CAP: usize> featuring:

1. A fast push(&mut self, item: T) -> Result<(), TelemetryError> method annotated with #[inline].
2. An out-of-line handle_overflow_cold(&mut self, dropped_item: T) -> Result<(), TelemetryError> method annotated with #[cold] and #[inline(never)].
3. Complete unit tests with assertions (assert_eq!, assert!) verifying hot-path enqueueing, cold-path overflow tracking, and post-overflow buffer popping.

#![no_std]

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TelemetryError {
    BufferOverflow,
    BufferEmpty,
}

pub struct TelemetryRingBuffer<T, const CAP: usize> {
    storage: [Option<T>; CAP],
    head: usize,
    tail: usize,
    count: usize,
    overflow_events: u64,
}

impl<T: Copy, const CAP: usize> TelemetryRingBuffer<T, CAP> {
    pub const fn new() -> Self {
        Self {
            storage: [None; CAP],
            head: 0,
            tail: 0,
            count: 0,
            overflow_events: 0,
        }
    }

    /// Hot fast path: executed 99.9% of the time.
    /// Kept minimal so LLVM easily inlines this function into calling ISR loops.
    #[inline]
    pub fn push(&mut self, item: T) -> Result<(), TelemetryError> {
        if self.count < CAP {
            self.storage[self.tail] = Some(item);
            self.tail = (self.tail + 1) % CAP;
            self.count += 1;
            Ok(())
        } else {
            self.handle_overflow_cold(item)
        }
    }

    /// Cold slow path: executed only when capacity is exhausted.
    /// Marked `#[cold]` and `#[inline(never)]` so LLVM relocates this assembly block
    /// out of the main I-Cache line and eliminates stack frame overhead in `push`.
    #[cold]
    #[inline(never)]
    fn handle_overflow_cold(&mut self, _dropped_item: T) -> Result<(), TelemetryError> {
        self.overflow_events += 1;
        Err(TelemetryError::BufferOverflow)
    }

    pub fn pop(&mut self) -> Result<T, TelemetryError> {
        if self.count == 0 {
            Err(TelemetryError::BufferEmpty)
        } else {
            let item = self.storage[self.head].take().unwrap();
            self.head = (self.head + 1) % CAP;
            self.count -= 1;
            Ok(item)
        }
    }

    pub fn overflow_count(&self) -> u64 {
        self.overflow_events
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_telemetry_ring_buffer_hot_and_cold_paths() {
        let mut ring = TelemetryRingBuffer::<u32, 3>::new();

        // 1. Verify fast-path pushes (capacity = 3)
        assert_eq!(ring.push(100), Ok(()));
        assert_eq!(ring.push(200), Ok(()));
        assert_eq!(ring.push(300), Ok(()));
        assert_eq!(ring.overflow_count(), 0);

        // 2. Verify cold-path trigger on 4th push (buffer full)
        assert_eq!(ring.push(400), Err(TelemetryError::BufferOverflow));
        assert_eq!(ring.overflow_count(), 1);

        // 3. Pop elements to free capacity and verify fast-path resumes
        assert_eq!(ring.pop(), Ok(100));
        assert_eq!(ring.pop(), Ok(200));

        assert_eq!(ring.push(500), Ok(()));
        assert_eq!(ring.overflow_count(), 1); // Overflow count remains unchanged
        assert_eq!(ring.pop(), Ok(300));
        assert_eq!(ring.pop(), Ok(500));

        // 4. Verify empty buffer error
        assert_eq!(ring.pop(), Err(TelemetryError::BufferEmpty));
    }
}

Exercise 2: High-Throughput Packet Inspector with Out-of-Line Verification Diagnostics

Scenario: Problem Statement: In a high-frequency 10 Gbps network engine, incoming packet headers must be parsed and verified against magic header bytes, expected length thresholds, and XOR checksums. Under normal operation, 99.99% of network packets are valid. When a corrupted packet arrives, detailed diagnostic error metrics must be recorded.

Requirements: Writing inline error variant construction inside validate_and_parse bloats the validation function's assembly code, degrading branch predictor performance and increasing I-Cache misses on the valid packet pipeline.

Implement a NetworkPacketParser with:

1. validate_and_parse(packet: &[u8]) -> Result<PacketHeader, ParseError> where the hot path validates valid packets directly.
2. Out-of-line error handlers (truncated_packet_cold, invalid_magic_cold, corrupted_checksum_cold) marked #[cold] and #[inline(never)].
3. Unit tests with assertions (assert_eq!, assert!, matches!) covering valid packet parsing, magic byte mismatches, checksum errors, and truncated buffers.

#[derive(Debug, PartialEq, Eq)]
pub struct PacketHeader {
    pub magic: u16,
    pub payload_len: u16,
    pub checksum: u8,
}

#[derive(Debug, PartialEq, Eq)]
pub enum ParseError {
    TruncatedPacket { expected: usize, actual: usize },
    InvalidMagic { found: u16 },
    ChecksumMismatch { calculated: u8, expected: u8 },
}

pub struct NetworkPacketParser;

impl NetworkPacketParser {
    pub const MAGIC_BYTES: u16 = 0x4850; // "HP" (High Performance)

    /// Hot fast path: validates packet header format inline.
    #[inline]
    pub fn validate_and_parse(packet: &[u8]) -> Result<PacketHeader, ParseError> {
        if packet.len() < 5 {
            return Self::truncated_packet_cold(5, packet.len());
        }

        let magic = u16::from_be_bytes([packet[0], packet[1]]);
        if magic != Self::MAGIC_BYTES {
            return Self::invalid_magic_cold(magic);
        }

        let payload_len = u16::from_be_bytes([packet[2], packet[3]]);
        let checksum = packet[4];

        let calculated = packet[0] ^ packet[1] ^ packet[2] ^ packet[3];
        if calculated != checksum {
            return Self::corrupted_checksum_cold(calculated, checksum);
        }

        Ok(PacketHeader {
            magic,
            payload_len,
            checksum,
        })
    }

    #[cold]
    #[inline(never)]
    fn truncated_packet_cold(expected: usize, actual: usize) -> Result<PacketHeader, ParseError> {
        Err(ParseError::TruncatedPacket { expected, actual })
    }

    #[cold]
    #[inline(never)]
    fn invalid_magic_cold(found: u16) -> Result<PacketHeader, ParseError> {
        Err(ParseError::InvalidMagic { found })
    }

    #[cold]
    #[inline(never)]
    fn corrupted_checksum_cold(calculated: u8, expected: u8) -> Result<PacketHeader, ParseError> {
        Err(ParseError::ChecksumMismatch { calculated, expected })
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_valid_packet_parsing() {
        let magic = NetworkPacketParser::MAGIC_BYTES.to_be_bytes();
        let payload_len: u16 = 64;
        let len_bytes = payload_len.to_be_bytes();
        let checksum = magic[0] ^ magic[1] ^ len_bytes[0] ^ len_bytes[1];

        let packet = vec![magic[0], magic[1], len_bytes[0], len_bytes[1], checksum, 0xAA, 0xBB];

        let header = NetworkPacketParser::validate_and_parse(&packet).unwrap();
        assert_eq!(header.magic, NetworkPacketParser::MAGIC_BYTES);
        assert_eq!(header.payload_len, 64);
        assert_eq!(header.checksum, checksum);
    }

    #[test]
    fn test_invalid_magic_cold_path() {
        let packet = vec![0x00, 0x00, 0x00, 0x10, 0x10];
        let result = NetworkPacketParser::validate_and_parse(&packet);
        assert_eq!(result, Err(ParseError::InvalidMagic { found: 0x0000 }));
    }

    #[test]
    fn test_corrupted_checksum_cold_path() {
        let magic = NetworkPacketParser::MAGIC_BYTES.to_be_bytes();
        let packet = vec![magic[0], magic[1], 0x00, 0x10, 0xFF]; // Invalid checksum 0xFF
        let result = NetworkPacketParser::validate_and_parse(&packet);
        assert!(matches!(result, Err(ParseError::ChecksumMismatch { .. })));
    }

    #[test]
    fn test_truncated_packet_cold_path() {
        let packet = vec![0x48, 0x50]; // Buffer length < 5
        let result = NetworkPacketParser::validate_and_parse(&packet);
        assert_eq!(result, Err(ParseError::TruncatedPacket { expected: 5, actual: 2 }));
    }
}

Exercise 3: Zero-Allocation Bump Allocator with Out-of-Line Arena Growth

Scenario: Problem Statement: In real-time game engines and financial order-book processors, memory allocation overhead must be strictly bounded. A BumpAllocator provides fast sequential allocation by advancing an offset pointer inside a pre-allocated memory chunk. The fast path (executed 99.99% of allocations) simply verifies that offset + size <= capacity and returns a pointer. When a chunk runs out of space (0.01% of allocations), the allocator must request a new memory block from system alloc, re-link arena descriptors, and log statistics.

Requirements: If arena chunk growth logic is placed directly inside alloc_fast, callers that invoke alloc_fast inside hot loops experience massive code bloat, preventing callers from being inlined.

Implement a BumpAllocator featuring:

1. A fast allocation method alloc_fast(&mut self, layout: std::alloc::Layout) -> Result<*mut u8, AllocError> annotated with #[inline].
2. A cold chunk expansion helper grow_arena_cold(&mut self, required_size: usize, align: usize) -> Result<*mut u8, AllocError> annotated with #[cold] and #[inline(never)].
3. Unit tests with assertions (assert_eq!, assert!, assert_ne!) verifying fast-path allocations within capacity, cold-path chunk growth when exhausted, and memory pointer alignment.

use std::alloc::{alloc, dealloc, Layout};

#[derive(Debug, PartialEq, Eq)]
pub enum AllocError {
    OutOfMemory,
    InvalidLayout,
}

pub struct BumpAllocator {
    chunk_ptr: *mut u8,
    capacity: usize,
    offset: usize,
    growth_count: usize,
}

impl BumpAllocator {
    pub fn new(initial_capacity: usize) -> Result<Self, AllocError> {
        let layout = Layout::from_size_align(initial_capacity, 8)
            .map_err(|_| AllocError::InvalidLayout)?;
        let ptr = unsafe { alloc(layout) };
        if ptr.is_null() {
            return Err(AllocError::OutOfMemory);
        }

        Ok(Self {
            chunk_ptr: ptr,
            capacity: initial_capacity,
            offset: 0,
            growth_count: 0,
        })
    }

    /// Hot fast path: advances offset pointer if capacity permits.
    #[inline]
    pub fn alloc_fast(&mut self, layout: Layout) -> Result<*mut u8, AllocError> {
        let align_offset = (self.offset + layout.align() - 1) & !(layout.align() - 1);
        let new_offset = align_offset + layout.size();

        if new_offset <= self.capacity {
            self.offset = new_offset;
            unsafe { Ok(self.chunk_ptr.add(align_offset)) }
        } else {
            self.grow_arena_cold(layout.size(), layout.align())
        }
    }

    /// Cold slow path: allocates a new, larger memory arena chunk.
    #[cold]
    #[inline(never)]
    fn grow_arena_cold(&mut self, required_size: usize, align: usize) -> Result<*mut u8, AllocError> {
        let new_capacity = (self.capacity * 2).max(self.offset + required_size + align);
        let new_layout = Layout::from_size_align(new_capacity, 8)
            .map_err(|_| AllocError::InvalidLayout)?;

        let new_ptr = unsafe { alloc(new_layout) };
        if new_ptr.is_null() {
            return Err(AllocError::OutOfMemory);
        }

        // Copy old memory region contents to new arena block
        if self.offset > 0 {
            unsafe {
                std::ptr::copy_nonoverlapping(self.chunk_ptr, new_ptr, self.offset);
                let old_layout = Layout::from_size_align(self.capacity, 8).unwrap();
                dealloc(self.chunk_ptr, old_layout);
            }
        }

        self.chunk_ptr = new_ptr;
        self.capacity = new_capacity;
        self.growth_count += 1;

        let align_offset = (self.offset + align - 1) & !(align - 1);
        self.offset = align_offset + required_size;
        unsafe { Ok(self.chunk_ptr.add(align_offset)) }
    }

    pub fn growth_count(&self) -> usize {
        self.growth_count
    }
}

impl Drop for BumpAllocator {
    fn drop(&mut self) {
        if !self.chunk_ptr.is_null() {
            let layout = Layout::from_size_align(self.capacity, 8).unwrap();
            unsafe { dealloc(self.chunk_ptr, layout) };
        }
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_bump_allocator_fast_path_and_cold_growth() {
        let mut allocator = BumpAllocator::new(64).expect("Initial allocation failed");
        assert_eq!(allocator.growth_count(), 0);

        let layout = Layout::from_size_align(16, 8).unwrap();

        // 1. Fast-path allocations within 64-byte initial capacity
        let ptr1 = allocator.alloc_fast(layout).unwrap();
        let ptr2 = allocator.alloc_fast(layout).unwrap();
        assert_ne!(ptr1, ptr2);
        assert_eq!(allocator.growth_count(), 0);

        let _ptr3 = allocator.alloc_fast(layout).unwrap();
        let _ptr4 = allocator.alloc_fast(layout).unwrap();
        assert_eq!(allocator.growth_count(), 0); // 16 * 4 = 64 bytes used

        // 2. 5th allocation exceeds 64-byte capacity -> triggers out-of-line grow_arena_cold
        let ptr5 = allocator.alloc_fast(layout).unwrap();
        assert!(!ptr5.is_null());
        assert_eq!(allocator.growth_count(), 1); // Growth counter incremented
    }
}

6. Related Terms

• perf / flamegraph — Profiling tools used to identify hot and cold code execution paths.
• Release Profile — Cargo build profile where LLVM branch optimizations take effect.

7. Key Takeaways

• #[cold] informs LLVM that a function is rarely called, guiding branch prediction and CPU Instruction Cache (I-Cache) layout.
• LLVM moves #[cold] assembly out of the main hot path code layout, preventing cold error code from cluttering high-speed L1 cache lines.
• #[cold] functions are automatically treated as undesirable for inlining into hot callers.
• Combine #[cold] with #[inline(never)] on rare error handlers, panics, and capacity reallocation helpers.
• Never mark frequently called hot loop functions as #[cold].