Expand description
Manually manage memory through raw pointers.
See also the pointer primitive types.
§Safety
Many functions in this module take raw pointers as arguments and read from or write to them. For
this to be safe, these pointers must be valid for the given access. Whether a pointer is valid
depends on the operation it is used for (read or write), and the extent of the memory that is
accessed (i.e., how many bytes are read/written) – it makes no sense to ask “is this pointer
valid”; one has to ask “is this pointer valid for a given access”. Most functions use *mut T
and *const T
to access only a single value, in which case the documentation omits the size and
implicitly assumes it to be size_of::<T>()
bytes.
The precise rules for validity are not determined yet. The guarantees that are provided at this point are very minimal:
- For operations of size zero, every pointer is valid, including the null pointer. The following points are only concerned with non-zero-sized accesses.
- A null pointer is never valid.
- For a pointer to be valid, it is necessary, but not always sufficient, that the pointer be dereferenceable. The provenance of the pointer is used to determine which allocated object it is derived from; a pointer is dereferenceable if the memory range of the given size starting at the pointer is entirely contained within the bounds of that allocated object. Note that in Rust, every (stack-allocated) variable is considered a separate allocated object.
- All accesses performed by functions in this module are non-atomic in the sense
of atomic operations used to synchronize between threads. This means it is
undefined behavior to perform two concurrent accesses to the same location from different
threads unless both accesses only read from memory. Notice that this explicitly
includes
read_volatile
andwrite_volatile
: Volatile accesses cannot be used for inter-thread synchronization. - The result of casting a reference to a pointer is valid for as long as the underlying object is live and no reference (just raw pointers) is used to access the same memory. That is, reference and pointer accesses cannot be interleaved.
These axioms, along with careful use of offset
for pointer arithmetic,
are enough to correctly implement many useful things in unsafe code. Stronger guarantees
will be provided eventually, as the aliasing rules are being determined. For more
information, see the book as well as the section in the reference devoted
to undefined behavior.
We say that a pointer is “dangling” if it is not valid for any non-zero-sized accesses. This
means out-of-bounds pointers, pointers to freed memory, null pointers, and pointers created with
NonNull::dangling
are all dangling.
§Alignment
Valid raw pointers as defined above are not necessarily properly aligned (where
“proper” alignment is defined by the pointee type, i.e., *const T
must be
aligned to mem::align_of::<T>()
). However, most functions require their
arguments to be properly aligned, and will explicitly state
this requirement in their documentation. Notable exceptions to this are
read_unaligned
and write_unaligned
.
When a function requires proper alignment, it does so even if the access
has size 0, i.e., even if memory is not actually touched. Consider using
NonNull::dangling
in such cases.
§Pointer to reference conversion
When converting a pointer to a reference (e.g. via &*ptr
or &mut *ptr
),
there are several rules that must be followed:
-
The pointer must be properly aligned.
-
It must be non-null.
-
It must be “dereferenceable” in the sense defined above.
-
The pointer must point to a valid value of type
T
. -
You must enforce Rust’s aliasing rules. The exact aliasing rules are not decided yet, so we only give a rough overview here. The rules also depend on whether a mutable or a shared reference is being created.
- When creating a mutable reference, then while this reference exists, the memory it points to must not get accessed (read or written) through any other pointer or reference not derived from this reference.
- When creating a shared reference, then while this reference exists, the memory it points to
must not get mutated (except inside
UnsafeCell
).
If a pointer follows all of these rules, it is said to be convertible to a (mutable or shared) reference.
These rules apply even if the result is unused! (The part about being initialized is not yet fully decided, but until it is, the only safe approach is to ensure that they are indeed initialized.)
An example of the implications of the above rules is that an expression such
as unsafe { &*(0 as *const u8) }
is Immediate Undefined Behavior.
§Allocated object
An allocated object is a subset of program memory which is addressable
from Rust, and within which pointer arithmetic is possible. Examples of
allocated objects include heap allocations, stack-allocated variables,
statics, and consts. The safety preconditions of some Rust operations -
such as offset
and field projections (expr.field
) - are defined in
terms of the allocated objects on which they operate.
An allocated object has a base address, a size, and a set of memory addresses. It is possible for an allocated object to have zero size, but such an allocated object will still have a base address. The base address of an allocated object is not necessarily unique. While it is currently the case that an allocated object always has a set of memory addresses which is fully contiguous (i.e., has no “holes”), there is no guarantee that this will not change in the future.
For any allocated object with base
address, size
, and a set of
addresses
, the following are guaranteed:
- For all addresses
a
inaddresses
,a
is in the rangebase .. (base + size)
(note that this requiresa < base + size
, nota <= base + size
) base
is not equal tonull()
(i.e., the address with the numerical value 0)base + size <= usize::MAX
size <= isize::MAX
As a consequence of these guarantees, given any address a
within the set
of addresses of an allocated object:
- It is guaranteed that
a - base
does not overflowisize
- It is guaranteed that
a - base
is non-negative - It is guaranteed that, given
o = a - base
(i.e., the offset ofa
within the allocated object),base + o
will not wrap around the address space (in other words, will not overflowusize
)
§Provenance
Pointers are not simply an “integer” or “address”. For instance, it’s uncontroversial
to say that a Use After Free is clearly Undefined Behaviour, even if you “get lucky”
and the freed memory gets reallocated before your read/write (in fact this is the
worst-case scenario, UAFs would be much less concerning if this didn’t happen!).
As another example, consider that wrapping_offset
is documented to “remember”
the allocated object that the original pointer points to, even if it is offset far
outside the memory range occupied by that allocated object.
To rationalize claims like this, pointers need to somehow be more than just their addresses:
they must have provenance.
A pointer value in Rust semantically contains the following information:
- The address it points to, which can be represented by a
usize
. - The provenance it has, defining the memory it has permission to access. Provenance can be absent, in which case the pointer does not have permission to access any memory.
The exact structure of provenance is not yet specified, but the permission defined by a pointer’s provenance have a spatial component, a temporal component, and a mutability component:
- Spatial: The set of memory addresses that the pointer is allowed to access.
- Temporal: The timespan during which the pointer is allowed to access those memory addresses.
- Mutability: Whether the pointer may only access the memory for reads, or also access it for writes. Note that this can interact with the other components, e.g. a pointer might permit mutation only for a subset of addresses, or only for a subset of its maximal timespan.
When an allocated object is created, it has a unique Original Pointer. For alloc APIs this is literally the pointer the call returns, and for local variables and statics, this is the name of the variable/static. (This is mildly overloading the term “pointer” for the sake of brevity/exposition.)
The Original Pointer for an allocated object has provenance that constrains the spatial
permissions of this pointer to the memory range of the allocation, and the temporal
permissions to the lifetime of the allocation. Provenance is implicitly inherited by all
pointers transitively derived from the Original Pointer through operations like offset
,
borrowing, and pointer casts. Some operations may shrink the permissions of the derived
provenance, limiting how much memory it can access or how long it’s valid for (i.e. borrowing a
subfield and subslicing can shrink the spatial component of provenance, and all borrowing can
shrink the temporal component of provenance). However, no operation can ever grow the
permissions of the derived provenance: even if you “know” there is a larger allocation, you
can’t derive a pointer with a larger provenance. Similarly, you cannot “recombine” two
contiguous provenances back into one (i.e. with a fn merge(&[T], &[T]) -> &[T]
).
A reference to a place always has provenance over at least the memory that place occupies. A reference to a slice always has provenance over at least the range that slice describes. Whether and when exactly the provenance of a reference gets “shrunk” to exactly fit the memory it points to is not yet determined.
A shared reference only ever has provenance that permits reading from memory,
and never permits writes, except inside UnsafeCell
.
Provenance can affect whether a program has undefined behavior:
-
It is undefined behavior to access memory through a pointer that does not have provenance over that memory. Note that a pointer “at the end” of its provenance is not actually outside its provenance, it just has 0 bytes it can load/store. Zero-sized accesses do not require any provenance since they access an empty range of memory.
-
It is undefined behavior to
offset
a pointer across a memory range that is not contained in the allocated object it is derived from, or tooffset_from
two pointers not derived from the same allocated object. Provenance is used to say what exactly “derived from” even means: the lineage of a pointer is traced back to the Original Pointer it descends from, and that identifies the relevant allocated object. In particular, it’s always UB to offset a pointer derived from something that is now deallocated, except if the offset is 0.
But it is still sound to:
-
Create a pointer without provenance from just an address (see
ptr::dangling
). Such a pointer cannot be used for memory accesses (except for zero-sized accesses). This can still be useful for sentinel values likenull
or to represent a tagged pointer that will never be dereferenceable. In general, it is always sound for an integer to pretend to be a pointer “for fun” as long as you don’t use operations on it which require it to be valid (non-zero-sized offset, read, write, etc). -
Forge an allocation of size zero at any sufficiently aligned non-null address. i.e. the usual “ZSTs are fake, do what you want” rules apply.
-
wrapping_offset
a pointer outside its provenance. This includes pointers which have “no” provenance. In particular, this makes it sound to do pointer tagging tricks. -
Compare arbitrary pointers by address. Pointer comparison ignores provenance and addresses are just integers, so there is always a coherent answer, even if the pointers are dangling or from different provenances. Note that if you get “lucky” and notice that a pointer at the end of one allocated object is the “same” address as the start of another allocated object, anything you do with that fact is probably going to be gibberish. The scope of that gibberish is kept under control by the fact that the two pointers still aren’t allowed to access the other’s allocation (bytes), because they still have different provenance.
Note that the full definition of provenance in Rust is not decided yet, as this interacts with the as-yet undecided aliasing rules.
§Pointers Vs Integers
From this discussion, it becomes very clear that a usize
cannot accurately represent a pointer,
and converting from a pointer to a usize
is generally an operation which only extracts the
address. Converting this address back into pointer requires somehow answering the question:
which provenance should the resulting pointer have?
Rust provides two ways of dealing with this situation: Strict Provenance and Exposed Provenance.
Note that a pointer can represent a usize
(via without_provenance
), so the right type to
use in situations where a value is “sometimes a pointer and sometimes a bare usize
” is a
pointer type.
§Strict Provenance
“Strict Provenance” refers to a set of APIs designed to make working with provenance more explicit. They are intended as substitutes for casting a pointer to an integer and back.
Entirely avoiding integer-to-pointer casts successfully side-steps the inherent ambiguity of that operation. This benefits compiler optimizations, and it is pretty much a requirement for using tools like Miri and architectures like CHERI that aim to detect and diagnose pointer misuse.
The key insight to making programming without integer-to-pointer casts at all viable is the
with_addr
method:
/// Creates a new pointer with the given address.
///
/// This performs the same operation as an `addr as ptr` cast, but copies
/// the *provenance* of `self` to the new pointer.
/// This allows us to dynamically preserve and propagate this important
/// information in a way that is otherwise impossible with a unary cast.
///
/// This is equivalent to using `wrapping_offset` to offset `self` to the
/// given address, and therefore has all the same capabilities and restrictions.
pub fn with_addr(self, addr: usize) -> Self;
So you’re still able to drop down to the address representation and do whatever
clever bit tricks you want as long as you’re able to keep around a pointer
into the allocation you care about that can “reconstitute” the provenance.
Usually this is very easy, because you only are taking a pointer, messing with the address,
and then immediately converting back to a pointer. To make this use case more ergonomic,
we provide the map_addr
method.
To help make it clear that code is “following” Strict Provenance semantics, we also provide an
addr
method which promises that the returned address is not part of a
pointer-integer-pointer roundtrip. In the future we may provide a lint for pointer<->integer
casts to help you audit if your code conforms to strict provenance.
§Using Strict Provenance
Most code needs no changes to conform to strict provenance, as the only really concerning
operation is casts from usize to a pointer. For code which does cast a usize
to a pointer,
the scope of the change depends on exactly what you’re doing.
In general, you just need to make sure that if you want to convert a usize
address to a
pointer and then use that pointer to read/write memory, you need to keep around a pointer
that has sufficient provenance to perform that read/write itself. In this way all of your
casts from an address to a pointer are essentially just applying offsets/indexing.
This is generally trivial to do for simple cases like tagged pointers as long as you
represent the tagged pointer as an actual pointer and not a usize
. For instance:
unsafe {
// A flag we want to pack into our pointer
static HAS_DATA: usize = 0x1;
static FLAG_MASK: usize = !HAS_DATA;
// Our value, which must have enough alignment to have spare least-significant-bits.
let my_precious_data: u32 = 17;
assert!(core::mem::align_of::<u32>() > 1);
// Create a tagged pointer
let ptr = &my_precious_data as *const u32;
let tagged = ptr.map_addr(|addr| addr | HAS_DATA);
// Check the flag:
if tagged.addr() & HAS_DATA != 0 {
// Untag and read the pointer
let data = *tagged.map_addr(|addr| addr & FLAG_MASK);
assert_eq!(data, 17);
} else {
unreachable!()
}
}
(Yes, if you’ve been using AtomicUsize for pointers in concurrent datastructures, you should be using AtomicPtr instead. If that messes up the way you atomically manipulate pointers, we would like to know why, and what needs to be done to fix it.)
Situations where a valid pointer must be created from just an address, such as baremetal code accessing a memory-mapped interface at a fixed address, cannot currently be handled with strict provenance APIs and should use exposed provenance.
§Exposed Provenance
As discussed above, integer-to-pointer casts are not possible with Strict Provenance APIs. This is by design: the goal of Strict Provenance is to provide a clear specification that we are confident can be formalized unambiguously and can be subject to precise formal reasoning. Integer-to-pointer casts do not (currently) have such a clear specification.
However, there exist situations where integer-to-pointer casts cannot be avoided, or
where avoiding them would require major refactoring. Legacy platform APIs also regularly assume
that usize
can capture all the information that makes up a pointer.
Bare-metal platforms can also require the synthesis of a pointer “out of thin air” without
anywhere to obtain proper provenance from.
Rust’s model for dealing with integer-to-pointer casts is called Exposed Provenance. However, the semantics of Exposed Provenance are on much less solid footing than Strict Provenance, and at this point it is not yet clear whether a satisfying unambiguous semantics can be defined for Exposed Provenance. (If that sounds bad, be reassured that other popular languages that provide integer-to-pointer casts are not faring any better.) Furthermore, Exposed Provenance will not work (well) with tools like Miri and CHERI.
Exposed Provenance is provided by the expose_provenance
and with_exposed_provenance
methods,
which are equivalent to as
casts between pointers and integers.
expose_provenance
is a lot likeaddr
, but additionally adds the provenance of the pointer to a global list of ‘exposed’ provenances. (This list is purely conceptual, it exists for the purpose of specifying Rust but is not materialized in actual executions, except in tools like Miri.) Memory which is outside the control of the Rust abstract machine (MMIO registers, for example) is always considered to be exposed, so long as this memory is disjoint from memory that will be used by the abstract machine such as the stack, heap, and statics.with_exposed_provenance
can be used to construct a pointer with one of these previously ‘exposed’ provenances.with_exposed_provenance
takes onlyaddr: usize
as arguments, so unlike inwith_addr
there is no indication of what the correct provenance for the returned pointer is – and that is exactly what makes integer-to-pointer casts so tricky to rigorously specify! The compiler will do its best to pick the right provenance for you, but currently we cannot provide any guarantees about which provenance the resulting pointer will have. Only one thing is clear: if there is no previously ‘exposed’ provenance that justifies the way the returned pointer will be used, the program has undefined behavior.
If at all possible, we encourage code to be ported to Strict Provenance APIs, thus avoiding the need for Exposed Provenance. Maximizing the amount of such code is a major win for avoiding specification complexity and to facilitate adoption of tools like CHERI and Miri that can be a big help in increasing the confidence in (unsafe) Rust code. However, we acknowledge that this is not always possible, and offer Exposed Provenance as a way to explicit “opt out” of the well-defined semantics of Strict Provenance, and “opt in” to the unclear semantics of integer-to-pointer casts.
Macros§
- Creates a
const
raw pointer to a place, without creating an intermediate reference. - Creates a
mut
raw pointer to a place, without creating an intermediate reference.
Structs§
*mut T
but non-zero and covariant.- Alignment
Experimental A type storing ausize
which is a power of two, and thus represents a possible alignment in the Rust abstract machine. - DynMetadata
Experimental The metadata for aDyn = dyn SomeTrait
trait object type.
Traits§
- Pointee
Experimental Provides the pointer metadata type of any pointed-to type.
Functions§
- Compares the addresses of the two pointers for equality, ignoring any metadata in fat pointers.
- copy⚠Copies
count * size_of::<T>()
bytes fromsrc
todst
. The source and destination may overlap. - Copies
count * size_of::<T>()
bytes fromsrc
todst
. The source and destination must not overlap. - Creates a new pointer that is dangling, but well-aligned.
- Creates a new pointer that is dangling, but well-aligned.
- Executes the destructor (if any) of the pointed-to value.
- Compares raw pointers for equality.
- Converts a mutable reference to a raw pointer.
- Converts a reference to a raw pointer.
- Hash a raw pointer.
- Creates a null raw pointer.
- Creates a null mutable raw pointer.
- read⚠Reads the value from
src
without moving it. This leaves the memory insrc
unchanged. - Reads the value from
src
without moving it. This leaves the memory insrc
unchanged. - Performs a volatile read of the value from
src
without moving it. This leaves the memory insrc
unchanged. - Moves
src
into the pointeddst
, returning the previousdst
value. - Forms a raw slice from a pointer and a length.
- Forms a raw mutable slice from a pointer and a length.
- swap⚠Swaps the values at two mutable locations of the same type, without deinitializing either.
- Swaps
count * size_of::<T>()
bytes between the two regions of memory beginning atx
andy
. The two regions must not overlap. - Converts an address back to a pointer, picking up some previously ‘exposed’ provenance.
- Converts an address back to a mutable pointer, picking up some previously ‘exposed’ provenance.
- Creates a pointer with the given address and no provenance.
- Creates a pointer with the given address and no provenance.
- Overwrites a memory location with the given value without reading or dropping the old value.
- Sets
count * size_of::<T>()
bytes of memory starting atdst
toval
. - Overwrites a memory location with the given value without reading or dropping the old value.
- Performs a volatile write of a memory location with the given value without reading or dropping the old value.
- fn_
addr_ eq Experimental Compares the addresses of the two function pointers for equality. - from_
raw_ parts Experimental Forms a (possibly-wide) raw pointer from a data pointer and metadata. - from_
raw_ parts_ mut Experimental Performs the same functionality asfrom_raw_parts
, except that a raw*mut
pointer is returned, as opposed to a raw*const
pointer. - metadata
Experimental Extracts the metadata component of a pointer.