Containers and collections

The option and result modules define optional and error-handling types, [Option<T>] and [Result<T, E>]. The iter module defines Rust’s iterator trait, Iterator, which works with the for loop to access collections.

The standard library exposes three common ways to deal with contiguous regions of memory:

• [Vec<T>] - A heap-allocated vector that is resizable at runtime.
• [T; N] - An inline array with a fixed size at compile time.
• [T] - A dynamically sized slice into any other kind of contiguousstorage, whether heap-allocated or not.

Slices can only be handled through some kind of pointer, and as such come in many flavors such as:

• &[T] - shared slice
• &mut [T] - mutable slice
• Box<[T]> - owned slice

[str], a UTF-8 string slice, is a primitive type, and the standard library defines many methods for it. Rust [str]s are typically accessed as immutable references: &str. Use the owned String for building and mutating strings.

For converting to strings use the format macro, and for converting from strings use the [FromStr] trait.

Data may be shared by placing it in a reference-counted box or the [Rc] type, and if further contained in a [Cell] or [RefCell], may be mutated as well as shared. Likewise, in a concurrent setting it is common to pair an atomically-reference-counted box, [Arc], with a [Mutex] to get the same effect.

The collections module defines maps, sets, linked lists and other typical collection types, including the common [HashMap<K, V>].

Platform abstractions and I/O

Besides basic data types, the standard library is largely concerned with abstracting over differences in common platforms, most notably Windows and Unix derivatives.

Common types of I/O, including [files], [TCP], and [UDP], are defined in the io, fs, and net modules.

The thread module contains Rust’s threading abstractions. sync contains further primitive shared memory types, including [atomic], [mpmc] and [mpsc], which contains the channel types for message passing.

Introducing mutability ‘inside’ of something immutable

Many shared smart pointer types, including Rc<T> and Arc<T>, provide containers that can be cloned and shared between multiple parties. Because the contained values may be multiply-aliased, they can only be borrowed with &, not &mut. Without cells it would be impossible to mutate data inside of these smart pointers at all.

It’s very common then to put a RefCell<T> inside shared pointer types to reintroduce mutability:

use std::cell::{RefCell, RefMut};
use std::collections::HashMap;
use std::rc::Rc;

fn main() {
    let shared_map: Rc<RefCell<_>> = Rc::new(RefCell::new(HashMap::new()));
    // Create a new block to limit the scope of the dynamic borrow
    {
        let mut map: RefMut<'_, _> = shared_map.borrow_mut();
        map.insert("africa", 92388);
        map.insert("kyoto", 11837);
        map.insert("piccadilly", 11826);
        map.insert("marbles", 38);
    }

    // Note that if we had not let the previous borrow of the cache fall out
    // of scope then the subsequent borrow would cause a dynamic thread panic.
    // This is the major hazard of using `RefCell`.
    let total: i32 = shared_map.borrow().values().sum();
    println!("{total}");
}

Note that this example uses Rc<T> and not Arc<T>. RefCell<T>s are for single-threaded scenarios. Consider using RwLock<T> or Mutex<T> if you need shared mutability in a multi-threaded situation.

Implementation details of logically-immutable methods

Occasionally it may be desirable not to expose in an API that there is mutation happening “under the hood”. This may be because logically the operation is immutable, but e.g., caching forces the implementation to perform mutation; or because you must employ mutation to implement a trait method that was originally defined to take &self.

use std::cell::OnceCell;

struct Graph {
    edges: Vec<(i32, i32)>,
    span_tree_cache: OnceCell<Vec<(i32, i32)>>
}

impl Graph {
    fn minimum_spanning_tree(&self) -> Vec<(i32, i32)> {
        self.span_tree_cache
            .get_or_init(|| self.calc_span_tree())
            .clone()
    }

    fn calc_span_tree(&self) -> Vec<(i32, i32)> {
        // Expensive computation goes here
        vec![]
    }
}

Mutating implementations of Clone

This is simply a special - but common - case of the previous: hiding mutability for operations that appear to be immutable. The clone method is expected to not change the source value, and is declared to take &self, not &mut self. Therefore, any mutation that happens in the clone method must use cell types. For example, Rc<T> maintains its reference counts within a Cell<T>.

use std::cell::Cell;
use std::ptr::NonNull;
use std::process::abort;
use std::marker::PhantomData;

struct Rc<T: ?Sized> {
    ptr: NonNull<RcInner<T>>,
    phantom: PhantomData<RcInner<T>>,
}

struct RcInner<T: ?Sized> {
    strong: Cell<usize>,
    refcount: Cell<usize>,
    value: T,
}

impl<T: ?Sized> Clone for Rc<T> {
    fn clone(&self) -> Rc<T> {
        self.inc_strong();
        Rc {
            ptr: self.ptr,
            phantom: PhantomData,
        }
    }
}

trait RcInnerPtr<T: ?Sized> {

    fn inner(&self) -> &RcInner<T>;

    fn strong(&self) -> usize {
        self.inner().strong.get()
    }

    fn inc_strong(&self) {
        self.inner()
            .strong
            .set(self.strong()
                     .checked_add(1)
                     .unwrap_or_else(|| abort() ));
    }
}

impl<T: ?Sized> RcInnerPtr<T> for Rc<T> {
   fn inner(&self) -> &RcInner<T> {
       unsafe {
           self.ptr.as_ref()
       }
   }
}

Unused lifetime parameters

Perhaps the most common use case for PhantomData is a struct that has an unused lifetime parameter, typically as part of some unsafe code. For example, here is a struct Slice that has two pointers of type *const T, presumably pointing into an array somewhere:

struct Slice<'a, T> {
    start: *const T,
    end: *const T,
}

The intention is that the underlying data is only valid for the lifetime 'a, so Slice should not outlive 'a. However, this intent is not expressed in the code, since there are no uses of the lifetime 'a and hence it is not clear what data it applies to. We can correct this by telling the compiler to act as if the Slice struct contained a reference &'a T:

use std::marker::PhantomData;

struct Slice<'a, T> {
    start: *const T,
    end: *const T,
    phantom: PhantomData<&'a T>,
}

This also in turn infers the lifetime bound T: 'a, indicating that any references in T are valid over the lifetime 'a.

When initializing a Slice you simply provide the value PhantomData for the field phantom:

fn borrow_vec<T>(vec: &Vec<T>) -> Slice<'_, T> {
    let ptr = vec.as_ptr();
    Slice {
        start: ptr,
        end: unsafe { ptr.add(vec.len()) },
        phantom: PhantomData,
    }
}

Unused type parameters

It sometimes happens that you have unused type parameters which indicate what type of data a struct is “tied” to, even though that data is not actually found in the struct itself. Here is an example where this arises with FFI. The foreign interface uses handles of type *mut () to refer to Rust values of different types. We track the Rust type using a phantom type parameter on the struct ExternalResource which wraps a handle.

use std::marker::PhantomData;

struct ExternalResource<R> {
   resource_handle: *mut (),
   resource_type: PhantomData<R>,
}

impl<R: ResType> ExternalResource<R> {
    fn new() -> Self {
        let size_of_res = size_of::<R>();
        Self {
            resource_handle: foreign_lib::new(size_of_res),
            resource_type: PhantomData,
        }
    }

    fn do_stuff(&self, param: ParamType) {
        let foreign_params = convert_params(param);
        foreign_lib::do_stuff(self.resource_handle, foreign_params);
    }
}

Ownership and the drop check

The exact interaction of PhantomData with drop check may change in the future.

Currently, adding a field of type PhantomData<T> indicates that your type owns data of type T in very rare circumstances. This in turn has effects on the Rust compiler’s [drop check] analysis. For the exact rules, see the [drop check] documentation.

Layout

For all T, the following are guaranteed:

• size_of::<PhantomData<T>>() == 0
• align_of::<PhantomData<T>>() == 1

Alignment

Valid raw pointers as defined above are not necessarily properly aligned (where “proper” alignment is defined by the pointee type, i.e., *const T must be aligned to align_of::<T>()). However, most functions require their arguments to be properly aligned, and will explicitly state this requirement in their documentation. Notable exceptions to this are read_unaligned and write_unaligned.

When a function requires proper alignment, it does so even if the access has size 0, i.e., even if memory is not actually touched. Consider using NonNull::dangling in such cases.

Pointer to reference conversion

When converting a pointer to a reference (e.g. via &*ptr or &mut *ptr), there are several rules that must be followed:

• The pointer must be properly aligned.

• It must be non-null.

• It must be “dereferenceable” in the sense defined above.

• The pointer must point to a valid value of type T.

• You must enforce Rust’s aliasing rules. The exact aliasing rules are not decided yet, so we only give a rough overview here. The rules also depend on whether a mutable or a shared reference is being created.

  • When creating a mutable reference, then while this reference exists, the memory it points tomust not get accessed (read or written) through any other pointer or reference not derivedfrom this reference.
  • When creating a shared reference, then while this reference exists, the memory it points tomust not get mutated (except inside UnsafeCell).

If a pointer follows all of these rules, it is said to be convertible to a (mutable or shared) reference.

These rules apply even if the result is unused! (The part about being initialized is not yet fully decided, but until it is, the only safe approach is to ensure that they are indeed initialized.)

An example of the implications of the above rules is that an expression such as unsafe { &*(0 as *const u8) } is Immediate Undefined Behavior.

Allocation

An allocation is a subset of program memory which is addressable from Rust, and within which pointer arithmetic is possible. Examples of allocations include heap allocations, stack-allocated variables, statics, and consts. The safety preconditions of some Rust operations - such as offset and field projections (expr.field) - are defined in terms of the allocations on which they operate.

An allocation has a base address, a size, and a set of memory addresses. It is possible for an allocation to have zero size, but such an allocation will still have a base address. The base address of an allocation is not necessarily unique. While it is currently the case that an allocation always has a set of memory addresses which is fully contiguous (i.e., has no “holes”), there is no guarantee that this will not change in the future.

Allocations must behave like “normal” memory: in particular, reads must not have side-effects, and writes must become visible to other threads using the usual synchronization primitives.

For any allocation with base address, size, and a set of addresses, the following are guaranteed:

• For all addresses a in addresses, a is in the range base .. (base + size) (note that this requires a < base + size, not a <= base + size)
• base is not equal to [null()] (i.e., the address with the numericalvalue 0)
• base + size <= usize::MAX
• size <= isize::MAX

As a consequence of these guarantees, given any address a within the set of addresses of an allocation:

• It is guaranteed that a - base does not overflow isize
• It is guaranteed that a - base is non-negative
• It is guaranteed that, given o = a - base (i.e., the offset of a withinthe allocation), base + o will not wrap around the address space (inother words, will not overflow usize)

Pointers Vs Integers

From this discussion, it becomes very clear that a usize cannot accurately represent a pointer, and converting from a pointer to a usize is generally an operation which only extracts the address. Converting this address back into pointer requires somehow answering the question: which provenance should the resulting pointer have?

Rust provides two ways of dealing with this situation: Strict Provenance and Exposed Provenance.

Note that a pointer can represent a usize (via without_provenance), so the right type to use in situations where a value is “sometimes a pointer and sometimes a bare usize” is a pointer type.

Strict Provenance

“Strict Provenance” refers to a set of APIs designed to make working with provenance more explicit. They are intended as substitutes for casting a pointer to an integer and back.

Entirely avoiding integer-to-pointer casts successfully side-steps the inherent ambiguity of that operation. This benefits compiler optimizations, and it is pretty much a requirement for using tools like Miri and architectures like CHERI that aim to detect and diagnose pointer misuse.

The key insight to making programming without integer-to-pointer casts at all viable is the [with_addr] method:

    /// Creates a new pointer with the given address.
    ///
    /// This performs the same operation as an `addr as ptr` cast, but copies
    /// the *provenance* of `self` to the new pointer.
    /// This allows us to dynamically preserve and propagate this important
    /// information in a way that is otherwise impossible with a unary cast.
    ///
    /// This is equivalent to using `wrapping_offset` to offset `self` to the
    /// given address, and therefore has all the same capabilities and restrictions.
    pub fn with_addr(self, addr: usize) -> Self;

So you’re still able to drop down to the address representation and do whatever clever bit tricks you want as long as you’re able to keep around a pointer into the allocation you care about that can “reconstitute” the provenance. Usually this is very easy, because you only are taking a pointer, messing with the address, and then immediately converting back to a pointer. To make this use case more ergonomic, we provide the [map_addr] method.

To help make it clear that code is “following” Strict Provenance semantics, we also provide an [addr] method which promises that the returned address is not part of a pointer-integer-pointer roundtrip. In the future we may provide a lint for pointer<->integer casts to help you audit if your code conforms to strict provenance.

unsafe {
    // A flag we want to pack into our pointer
    static HAS_DATA: usize = 0x1;
    static FLAG_MASK: usize = !HAS_DATA;

    // Our value, which must have enough alignment to have spare least-significant-bits.
    let my_precious_data: u32 = 17;
    assert!(align_of::<u32>() > 1);

    // Create a tagged pointer
    let ptr = &my_precious_data as *const u32;
    let tagged = ptr.map_addr(|addr| addr | HAS_DATA);

    // Check the flag:
    if tagged.addr() & HAS_DATA != 0 {
        // Untag and read the pointer
        let data = *tagged.map_addr(|addr| addr & FLAG_MASK);
        assert_eq!(data, 17);
    } else {
        unreachable!()
    }
}

Exposed Provenance

As discussed above, integer-to-pointer casts are not possible with Strict Provenance APIs. This is by design: the goal of Strict Provenance is to provide a clear specification that we are confident can be formalized unambiguously and can be subject to precise formal reasoning. Integer-to-pointer casts do not (currently) have such a clear specification.

However, there exist situations where integer-to-pointer casts cannot be avoided, or where avoiding them would require major refactoring. Legacy platform APIs also regularly assume that usize can capture all the information that makes up a pointer. Bare-metal platforms can also require the synthesis of a pointer “out of thin air” without anywhere to obtain proper provenance from.

Rust’s model for dealing with integer-to-pointer casts is called Exposed Provenance. However, the semantics of Exposed Provenance are on much less solid footing than Strict Provenance, and at this point it is not yet clear whether a satisfying unambiguous semantics can be defined for Exposed Provenance. (If that sounds bad, be reassured that other popular languages that provide integer-to-pointer casts are not faring any better.) Furthermore, Exposed Provenance will not work (well) with tools like Miri and CHERI.

Exposed Provenance is provided by the [expose_provenance] and [with_exposed_provenance] methods, which are equivalent to as casts between pointers and integers.

• [expose_provenance] is a lot like [addr], but additionally adds the provenance of thepointer to a global list of ‘exposed’ provenances. (This list is purely conceptual, it existsfor the purpose of specifying Rust but is not materialized in actual executions, except intools like Miri.)Memory which is outside the control of the Rust abstract machine (MMIO registers, for example)is always considered to be exposed, so long as this memory is disjoint from memory that willbe used by the abstract machine such as the stack, heap, and statics.
• [with_exposed_provenance] can be used to construct a pointer with one of these previously‘exposed’ provenances. [with_exposed_provenance] takes only addr: usize as arguments, sounlike in [with_addr] there is no indication of what the correct provenance for the returnedpointer is – and that is exactly what makes integer-to-pointer casts so tricky to rigorouslyspecify! The compiler will do its best to pick the right provenance for you, but currently wecannot provide any guarantees about which provenance the resulting pointer will have. Only onething is clear: if there is no previously ‘exposed’ provenance that justifies the way thereturned pointer will be used, the program has undefined behavior.

If at all possible, we encourage code to be ported to Strict Provenance APIs, thus avoiding the need for Exposed Provenance. Maximizing the amount of such code is a major win for avoiding specification complexity and to facilitate adoption of tools like CHERI and Miri that can be a big help in increasing the confidence in (unsafe) Rust code. However, we acknowledge that this is not always possible, and offer Exposed Provenance as a way to explicit “opt out” of the well-defined semantics of Strict Provenance, and “opt in” to the unclear semantics of integer-to-pointer casts.

Example

use nom::{
  IResult,
  bytes::complete::{tag, take_while_m_n},
  combinator::map_res,
  sequence::tuple};

#[derive(Debug,PartialEq)]
pub struct Color {
  pub red:     u8,
  pub green:   u8,
  pub blue:    u8,
}

fn from_hex(input: &str) -> Result<u8, std::num::ParseIntError> {
  u8::from_str_radix(input, 16)
}

fn is_hex_digit(c: char) -> bool {
  c.is_digit(16)
}

fn hex_primary(input: &str) -> IResult<&str, u8> {
  map_res(
    take_while_m_n(2, 2, is_hex_digit),
    from_hex
  )(input)
}

fn hex_color(input: &str) -> IResult<&str, Color> {
  let (input, _) = tag("#")(input)?;
  let (input, (red, green, blue)) = tuple((hex_primary, hex_primary, hex_primary))(input)?;

  Ok((input, Color { red, green, blue }))
}

fn main() {
  assert_eq!(hex_color("#2F14DF"), Ok(("", Color {
    red: 47,
    green: 20,
    blue: 223,
  })));
}

The code is available on Github

There are a few guides with more details about how to write parsers, or the error management system. You can also check out the recipes module that contains examples of common patterns.

Looking for a specific combinator? Read the “choose a combinator” guide

If you are upgrading to nom 5.0, please read the migration document.

Parser combinators

Parser combinators are an approach to parsers that is very different from software like lex and yacc. Instead of writing the grammar in a separate syntax and generating the corresponding code, you use very small functions with very specific purposes, like “take 5 bytes”, or “recognize the word ‘HTTP’”, and assemble them in meaningful patterns like “recognize ‘HTTP’, then a space, then a version”. The resulting code is small, and looks like the grammar you would have written with other parser approaches.

This gives us a few advantages:

• The parsers are small and easy to write
• The parsers components are easy to reuse (if they’re general enough, please add them to nom!)
• The parsers components are easy to test separately (unit tests and property-based tests)
• The parser combination code looks close to the grammar you would have written
• You can build partial parsers, specific to the data you need at the moment, and ignore the rest

Here is an example of one such parser, to recognize text between parentheses:

use nom::{
  IResult,
  sequence::delimited,
  // see the "streaming/complete" paragraph lower for an explanation of these submodules
  character::complete::char,
  bytes::complete::is_not
};

fn parens(input: &str) -> IResult<&str, &str> {
  delimited(char('('), is_not(")"), char(')'))(input)
}

It defines a function named parens which will recognize a sequence of the character (, the longest byte array not containing ), then the character ), and will return the byte array in the middle.

Here is another parser, written without using nom’s combinators this time:

use nom::{IResult, Err, Needed};

fn take4(i: &[u8]) -> IResult<&[u8], &[u8]>{
  if i.len() < 4 {
    Err(Err::Incomplete(Needed::new(4)))
  } else {
    Ok((&i[4..], &i[0..4]))
  }
}

This function takes a byte array as input, and tries to consume 4 bytes. Writing all the parsers manually, like this, is dangerous, despite Rust’s safety features. There are still a lot of mistakes one can make. That’s why nom provides a list of functions to help in developing parsers.

With functions, you would write it like this:

use nom::{IResult, bytes::streaming::take};
fn take4(input: &str) -> IResult<&str, &str> {
  take(4u8)(input)
}

A parser in nom is a function which, for an input type I, an output type O and an optional error type E, will have the following signature:

fn parser(input: I) -> IResult<I, O, E>;

Or like this, if you don’t want to specify a custom error type (it will be (I, ErrorKind) by default):

fn parser(input: I) -> IResult<I, O>;

IResult is an alias for the Result type:

use nom::{Needed, error::Error};

type IResult<I, O, E = Error<I>> = Result<(I, O), Err<E>>;

enum Err<E> {
  Incomplete(Needed),
  Error(E),
  Failure(E),
}

It can have the following values:

• A correct result Ok((I,O)) with the first element being the remaining of the input (not parsed yet), and the second the output value;
• An error Err(Err::Error(c)) with c an error that can be built from the input position and a parser specific error
• An error Err(Err::Incomplete(Needed)) indicating that more input is necessary. Needed can indicate how much data is needed
• An error Err(Err::Failure(c)). It works like the Error case, except it indicates an unrecoverable error: We cannot backtrack and test another parser

Please refer to the “choose a combinator” guide for an exhaustive list of parsers. See also the rest of the documentation here.

Making new parsers with function combinators

nom is based on functions that generate parsers, with a signature like this: (arguments) -> impl Fn(Input) -> IResult<Input, Output, Error>. The arguments of a combinator can be direct values (like take which uses a number of bytes or character as argument) or even other parsers (like delimited which takes as argument 3 parsers, and returns the result of the second one if all are successful).

Here are some examples:

use nom::IResult;
use nom::bytes::complete::{tag, take};
fn abcd_parser(i: &str) -> IResult<&str, &str> {
  tag("abcd")(i) // will consume bytes if the input begins with "abcd"
}

fn take_10(i: &[u8]) -> IResult<&[u8], &[u8]> {
  take(10u8)(i) // will consume and return 10 bytes of input
}

Combining parsers

There are higher level patterns, like the alt combinator, which provides a choice between multiple parsers. If one branch fails, it tries the next, and returns the result of the first parser that succeeds:

use nom::IResult;
use nom::branch::alt;
use nom::bytes::complete::tag;

let mut alt_tags = alt((tag("abcd"), tag("efgh")));

assert_eq!(alt_tags(&b"abcdxxx"[..]), Ok((&b"xxx"[..], &b"abcd"[..])));
assert_eq!(alt_tags(&b"efghxxx"[..]), Ok((&b"xxx"[..], &b"efgh"[..])));
assert_eq!(alt_tags(&b"ijklxxx"[..]), Err(nom::Err::Error((&b"ijklxxx"[..], nom::error::ErrorKind::Tag))));

The opt combinator makes a parser optional. If the child parser returns an error, opt will still succeed and return None:

use nom::{IResult, combinator::opt, bytes::complete::tag};
fn abcd_opt(i: &[u8]) -> IResult<&[u8], Option<&[u8]>> {
  opt(tag("abcd"))(i)
}

assert_eq!(abcd_opt(&b"abcdxxx"[..]), Ok((&b"xxx"[..], Some(&b"abcd"[..]))));
assert_eq!(abcd_opt(&b"efghxxx"[..]), Ok((&b"efghxxx"[..], None)));

many0 applies a parser 0 or more times, and returns a vector of the aggregated results:

use nom::{IResult, multi::many0, bytes::complete::tag};
use std::str;

fn multi(i: &str) -> IResult<&str, Vec<&str>> {
  many0(tag("abcd"))(i)
}

let a = "abcdef";
let b = "abcdabcdef";
let c = "azerty";
assert_eq!(multi(a), Ok(("ef",     vec!["abcd"])));
assert_eq!(multi(b), Ok(("ef",     vec!["abcd", "abcd"])));
assert_eq!(multi(c), Ok(("azerty", Vec::new())));

Here are some basic combinators available:

• opt: Will make the parser optional (if it returns the O type, the new parser returns Option<O>)
• many0: Will apply the parser 0 or more times (if it returns the O type, the new parser returns Vec<O>)
• many1: Will apply the parser 1 or more times

There are more complex (and more useful) parsers like tuple, which is used to apply a series of parsers then assemble their results.

Example with tuple:

use nom::{error::ErrorKind, Needed,
number::streaming::be_u16,
bytes::streaming::{tag, take},
sequence::tuple};

let mut tpl = tuple((be_u16, take(3u8), tag("fg")));

assert_eq!(
  tpl(&b"abcdefgh"[..]),
  Ok((
    &b"h"[..],
    (0x6162u16, &b"cde"[..], &b"fg"[..])
  ))
);
assert_eq!(tpl(&b"abcde"[..]), Err(nom::Err::Incomplete(Needed::new(2))));
let input = &b"abcdejk"[..];
assert_eq!(tpl(input), Err(nom::Err::Error((&input[5..], ErrorKind::Tag))));

But you can also use a sequence of combinators written in imperative style, thanks to the ? operator:

use nom::{IResult, bytes::complete::tag};

#[derive(Debug, PartialEq)]
struct A {
  a: u8,
  b: u8
}

fn ret_int1(i:&[u8]) -> IResult<&[u8], u8> { Ok((i,1)) }
fn ret_int2(i:&[u8]) -> IResult<&[u8], u8> { Ok((i,2)) }

fn f(i: &[u8]) -> IResult<&[u8], A> {
  // if successful, the parser returns `Ok((remaining_input, output_value))` that we can destructure
  let (i, _) = tag("abcd")(i)?;
  let (i, a) = ret_int1(i)?;
  let (i, _) = tag("efgh")(i)?;
  let (i, b) = ret_int2(i)?;

  Ok((i, A { a, b }))
}

let r = f(b"abcdefghX");
assert_eq!(r, Ok((&b"X"[..], A{a: 1, b: 2})));

Streaming / Complete

Some of nom’s modules have streaming or complete submodules. They hold different variants of the same combinators.

A streaming parser assumes that we might not have all of the input data. This can happen with some network protocol or large file parsers, where the input buffer can be full and need to be resized or refilled.

A complete parser assumes that we already have all of the input data. This will be the common case with small files that can be read entirely to memory.

Here is how it works in practice:

use nom::{IResult, Err, Needed, error::{Error, ErrorKind}, bytes, character};

fn take_streaming(i: &[u8]) -> IResult<&[u8], &[u8]> {
  bytes::streaming::take(4u8)(i)
}

fn take_complete(i: &[u8]) -> IResult<&[u8], &[u8]> {
  bytes::complete::take(4u8)(i)
}

// both parsers will take 4 bytes as expected
assert_eq!(take_streaming(&b"abcde"[..]), Ok((&b"e"[..], &b"abcd"[..])));
assert_eq!(take_complete(&b"abcde"[..]), Ok((&b"e"[..], &b"abcd"[..])));

// if the input is smaller than 4 bytes, the streaming parser
// will return `Incomplete` to indicate that we need more data
assert_eq!(take_streaming(&b"abc"[..]), Err(Err::Incomplete(Needed::new(1))));

// but the complete parser will return an error
assert_eq!(take_complete(&b"abc"[..]), Err(Err::Error(Error::new(&b"abc"[..], ErrorKind::Eof))));

// the alpha0 function recognizes 0 or more alphabetic characters
fn alpha0_streaming(i: &str) -> IResult<&str, &str> {
  character::streaming::alpha0(i)
}

fn alpha0_complete(i: &str) -> IResult<&str, &str> {
  character::complete::alpha0(i)
}

// if there's a clear limit to the recognized characters, both parsers work the same way
assert_eq!(alpha0_streaming("abcd;"), Ok((";", "abcd")));
assert_eq!(alpha0_complete("abcd;"), Ok((";", "abcd")));

// but when there's no limit, the streaming version returns `Incomplete`, because it cannot
// know if more input data should be recognized. The whole input could be "abcd;", or
// "abcde;"
assert_eq!(alpha0_streaming("abcd"), Err(Err::Incomplete(Needed::new(1))));

// while the complete version knows that all of the data is there
assert_eq!(alpha0_complete("abcd"), Ok(("", "abcd")));

Going further: Read the guides, check out the recipes!