wasmtime/runtime/gc/enabled/rooting.rs
1//! Garbage collection rooting APIs.
2//!
3//! Rooting prevents GC objects from being collected while they are actively
4//! being used.
5//!
6//! ## Goals
7//!
8//! We have a few sometimes-conflicting goals with our GC rooting APIs:
9//!
10//! 1. Safety: It should never be possible to get a use-after-free bug because
11//! the user misused the rooting APIs, the collector "mistakenly" determined
12//! an object was unreachable and collected it, and then the user tried to
13//! access the object. This is our highest priority.
14//!
15//! 2. Moving GC: Our rooting APIs should moving collectors (such as
16//! generational and compacting collectors) where an object might get
17//! relocated after a collection and we need to update the GC root's pointer
18//! to the moved object. This means we either need cooperation and internal
19//! mutability from individual GC roots as well as the ability to enumerate
20//! all GC roots on the native Rust stack, or we need a level of indirection.
21//!
22//! 3. Performance: Our rooting APIs should generally be as low-overhead as
23//! possible. They definitely shouldn't require synchronization and locking
24//! to create, access, and drop GC roots.
25//!
26//! 4. Ergonomics: Our rooting APIs should be, if not a pleasure, then at least
27//! not a burden for users. Additionally, the API's types should be `Sync`
28//! and `Send` so that they work well with async Rust.
29//!
30//! For example, goals (3) and (4) are in conflict when we think about how to
31//! support (2). Ideally, for ergonomics, a root would automatically unroot
32//! itself when dropped. But in the general case that requires holding a
33//! reference to the store's root set, and that root set needs to be held
34//! simultaneously by all GC roots, and they each need to mutate the set to
35//! unroot themselves. That implies `Rc<RefCell<...>>` or `Arc<Mutex<...>>`! The
36//! former makes the store and GC root types not `Send` and not `Sync`. The
37//! latter imposes synchronization and locking overhead. So we instead make GC
38//! roots indirect and require passing in a store context explicitly to unroot
39//! in the general case. This trades worse ergonomics for better performance and
40//! support for moving GC.
41//!
42//! ## Two Flavors of Rooting API
43//!
44//! Okay, with that out of the way, this module provides two flavors of rooting
45//! API. One for the common, scoped lifetime case, and another for the rare case
46//! where we really need a GC root with an arbitrary, non-LIFO/non-scoped
47//! lifetime:
48//!
49//! 1. `RootScope` and `Rooted<T>`: These are used for temporarily rooting GC
50//! objects for the duration of a scope. The internal implementation takes
51//! advantage of the LIFO property inherent in scopes, making creating and
52//! dropping `Rooted<T>`s and `RootScope`s super fast and roughly equivalent
53//! to bump allocation.
54//!
55//! This type is vaguely similar to V8's [`HandleScope`].
56//!
57//! [`HandleScope`]: https://v8.github.io/api/head/classv8_1_1HandleScope.html
58//!
59//! Note that `Rooted<T>` can't be statically tied to its context scope via a
60//! lifetime parameter, unfortunately, as that would allow the creation of
61//! only one `Rooted<T>` at a time, since the `Rooted<T>` would take a borrow
62//! of the whole context.
63//!
64//! This supports the common use case for rooting and provides good
65//! ergonomics.
66//!
67//! 2. `ManuallyRooted<T>`: This is the fully general rooting API used for
68//! holding onto non-LIFO GC roots with arbitrary lifetimes. However, users
69//! must manually unroot them. Failure to manually unroot a
70//! `ManuallyRooted<T>` before it is dropped will result in the GC object
71//! (and everything it transitively references) leaking for the duration of
72//! the `Store`'s lifetime.
73//!
74//! This type is roughly similar to SpiderMonkey's [`PersistentRooted<T>`],
75//! although they avoid the manual-unrooting with internal mutation and
76//! shared references. (Our constraints mean we can't do those things, as
77//! mentioned explained above.)
78//!
79//! [`PersistentRooted<T>`]: http://devdoc.net/web/developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JS::PersistentRooted.html
80//!
81//! At the end of the day, both `Rooted<T>` and `ManuallyRooted<T>` are just
82//! tagged indices into the store's `RootSet`. This indirection allows working
83//! with Rust's borrowing discipline (we use `&mut Store` to represent mutable
84//! access to the GC heap) while still allowing rooted references to be moved
85//! around without tying up the whole store in borrows. Additionally, and
86//! crucially, this indirection allows us to update the *actual* GC pointers in
87//! the `RootSet` and support moving GCs (again, as mentioned above).
88//!
89//! ## Unrooted References
90//!
91//! We generally don't expose *unrooted* GC references in the Wasmtime API at
92//! this time -- and I expect it will be a very long time before we do, but in
93//! the limit we may want to let users define their own GC-managed types that
94//! participate in GC tracing and all that -- so we don't have to worry about
95//! failure to root an object causing use-after-free bugs or failing to update a
96//! GC root pointer after a moving GC as long as users stick to our safe rooting
97//! APIs. (The one exception is `ValRaw`, which does hold raw GC references. But
98//! with `ValRaw` all bets are off and safety is 100% up to the user.)
99//!
100//! We do, however, have to worry about these things internally. So first of
101//! all, try to avoid ever working with unrooted GC references if you
102//! can. However, if you really must, consider also using an `AutoAssertNoGc`
103//! across the block of code that is manipulating raw GC references.
104
105use crate::runtime::vm::{GcRootsList, GcStore, VMGcRef};
106use crate::vm::VMStore;
107use crate::{prelude::*, ValRaw};
108use crate::{
109 store::{AutoAssertNoGc, StoreId, StoreOpaque},
110 AsContext, AsContextMut, GcRef, Result, RootedGcRef,
111};
112use core::any;
113use core::marker;
114use core::mem::{self, MaybeUninit};
115use core::num::NonZeroU64;
116use core::{
117 fmt::{self, Debug},
118 hash::{Hash, Hasher},
119 ops::{Deref, DerefMut},
120};
121use wasmtime_slab::{Id as SlabId, Slab};
122
123mod sealed {
124 use super::*;
125
126 /// Sealed, `wasmtime`-internal trait for GC references.
127 ///
128 /// # Safety
129 ///
130 /// All types implementing this trait must:
131 ///
132 /// * Be a newtype of a `GcRootIndex`
133 ///
134 /// * Not implement `Copy` or `Clone`
135 ///
136 /// * Only have `&self` methods.
137 pub unsafe trait GcRefImpl: Sized {
138 /// Transmute a `&GcRootIndex` into an `&Self`.
139 fn transmute_ref(index: &GcRootIndex) -> &Self;
140 }
141
142 /// Sealed, `wasmtime`-internal trait for the common methods on rooted GC
143 /// references.
144 pub trait RootedGcRefImpl<T: GcRef> {
145 /// Get this rooted GC reference's raw `VMGcRef` out of the store's GC
146 /// root set.
147 ///
148 /// Returns `None` for objects that have since been unrooted (eg because
149 /// its associated `RootedScope` was dropped).
150 ///
151 /// Panics if this root is not associated with the given store.
152 fn get_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Option<&'a VMGcRef>;
153
154 /// Same as `get_gc_ref` but returns an error instead of `None` for
155 /// objects that have been unrooted.
156 fn try_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Result<&'a VMGcRef> {
157 self.get_gc_ref(store).ok_or_else(|| {
158 anyhow!("attempted to use a garbage-collected object that has been unrooted")
159 })
160 }
161
162 /// Get a clone of this rooted GC reference's raw `VMGcRef` out of the
163 /// store's GC root set.
164 ///
165 /// Returns `None` for objects that have since been unrooted (eg because
166 /// its associated `RootedScope` was dropped).
167 ///
168 /// Panics if this root is not associated with the given store.
169 fn clone_gc_ref(&self, store: &mut AutoAssertNoGc<'_>) -> Option<VMGcRef> {
170 let gc_ref = self.get_gc_ref(store)?.unchecked_copy();
171 Some(store.unwrap_gc_store_mut().clone_gc_ref(&gc_ref))
172 }
173
174 /// Same as `clone_gc_ref` but returns an error instead of `None` for
175 /// objects that have been unrooted.
176 fn try_clone_gc_ref(&self, store: &mut AutoAssertNoGc<'_>) -> Result<VMGcRef> {
177 let gc_ref = self.try_gc_ref(store)?.unchecked_copy();
178 Ok(store.gc_store_mut()?.clone_gc_ref(&gc_ref))
179 }
180 }
181}
182pub(crate) use sealed::*;
183
184/// The index of a GC root inside a particular store's GC root set.
185///
186/// Can be either a LIFO- or manually-rooted object, depending on the
187/// `PackedIndex`.
188///
189/// Every `T` such that `T: GcRef` must be a newtype over this `GcRootIndex`.
190#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
191// Just `pub` to avoid `warn(private_interfaces)` in public APIs, which we can't
192// `allow(...)` on our MSRV yet.
193#[doc(hidden)]
194#[repr(C)] // NB: if this layout changes be sure to change the C API as well
195pub struct GcRootIndex {
196 store_id: StoreId,
197 generation: u32,
198 index: PackedIndex,
199}
200
201const _: () = {
202 // NB: these match the C API which should also be updated if this changes
203 assert!(mem::size_of::<GcRootIndex>() == 16);
204 assert!(mem::align_of::<GcRootIndex>() == mem::align_of::<u64>());
205};
206
207impl GcRootIndex {
208 #[inline]
209 pub(crate) fn comes_from_same_store(&self, store: &StoreOpaque) -> bool {
210 self.store_id == store.id()
211 }
212
213 /// Same as `RootedGcRefImpl::get_gc_ref` but not associated with any
214 /// particular `T: GcRef`.
215 ///
216 /// We must avoid triggering a GC while holding onto the resulting raw
217 /// `VMGcRef` to avoid use-after-free bugs and similar. The `'a` lifetime
218 /// threaded from the `store` to the result will normally prevent GCs
219 /// statically, at compile time, since performing a GC requires a mutable
220 /// borrow of the store. However, if you call `VMGcRef::unchecked_copy` on
221 /// the resulting GC reference, then all bets are off and this invariant is
222 /// up to you to manually uphold. Failure to uphold this invariant is memory
223 /// safe but will lead to general incorrectness such as panics and wrong
224 /// results.
225 ///
226 /// # Panics
227 ///
228 /// Panics if `self` is not associated with the given store.
229 pub(crate) fn get_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Option<&'a VMGcRef> {
230 assert!(
231 self.comes_from_same_store(store),
232 "object used with wrong store"
233 );
234 if let Some(index) = self.index.as_lifo() {
235 let entry = store.gc_roots().lifo_roots.get(index)?;
236 if entry.generation == self.generation {
237 Some(&entry.gc_ref)
238 } else {
239 None
240 }
241 } else if let Some(id) = self.index.as_manual() {
242 let gc_ref = store.gc_roots().manually_rooted.get(id);
243 debug_assert!(gc_ref.is_some());
244 gc_ref
245 } else {
246 unreachable!()
247 }
248 }
249
250 /// Same as `get_gc_ref` but returns an error instead of `None` if
251 /// the GC reference has been unrooted.
252 ///
253 /// # Panics
254 ///
255 /// Panics if `self` is not associated with the given store.
256 pub(crate) fn try_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Result<&'a VMGcRef> {
257 self.get_gc_ref(store).ok_or_else(|| {
258 anyhow!("attempted to use a garbage-collected object that has been unrooted")
259 })
260 }
261
262 /// Same as `RootedGcRefImpl::clone_gc_ref` but not associated with any
263 /// particular `T: GcRef`.
264 pub(crate) fn try_clone_gc_ref(&self, store: &mut AutoAssertNoGc<'_>) -> Result<VMGcRef> {
265 let gc_ref = self.try_gc_ref(store)?.unchecked_copy();
266 Ok(store.gc_store_mut()?.clone_gc_ref(&gc_ref))
267 }
268}
269
270/// This is a bit-packed version of
271///
272/// ```ignore
273/// enema {
274/// Lifo(usize),
275/// Manual(SlabId),
276/// }
277/// ```
278///
279/// where the high bit is the discriminant and the lower 31 bits are the
280/// payload.
281#[derive(Clone, Copy, PartialEq, Eq, Hash)]
282#[repr(transparent)]
283struct PackedIndex(u32);
284
285impl Debug for PackedIndex {
286 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
287 if let Some(index) = self.as_lifo() {
288 f.debug_tuple("PackedIndex::Lifo").field(&index).finish()
289 } else if let Some(id) = self.as_manual() {
290 f.debug_tuple("PackedIndex::Manual").field(&id).finish()
291 } else {
292 unreachable!()
293 }
294 }
295}
296
297impl PackedIndex {
298 const DISCRIMINANT_MASK: u32 = 0b1 << 31;
299 const LIFO_DISCRIMINANT: u32 = 0b0 << 31;
300 const MANUAL_DISCRIMINANT: u32 = 0b1 << 31;
301 const PAYLOAD_MASK: u32 = !Self::DISCRIMINANT_MASK;
302
303 fn new_lifo(index: usize) -> PackedIndex {
304 let index32 = u32::try_from(index).unwrap();
305 assert_eq!(index32 & Self::DISCRIMINANT_MASK, 0);
306 let packed = PackedIndex(Self::LIFO_DISCRIMINANT | index32);
307 debug_assert!(packed.is_lifo());
308 debug_assert_eq!(packed.as_lifo(), Some(index));
309 debug_assert!(!packed.is_manual());
310 debug_assert!(packed.as_manual().is_none());
311 packed
312 }
313
314 fn new_manual(id: SlabId) -> PackedIndex {
315 let raw = id.into_raw();
316 assert_eq!(raw & Self::DISCRIMINANT_MASK, 0);
317 let packed = PackedIndex(Self::MANUAL_DISCRIMINANT | raw);
318 debug_assert!(packed.is_manual());
319 debug_assert_eq!(packed.as_manual(), Some(id));
320 debug_assert!(!packed.is_lifo());
321 debug_assert!(packed.as_lifo().is_none());
322 packed
323 }
324
325 fn discriminant(&self) -> u32 {
326 self.0 & Self::DISCRIMINANT_MASK
327 }
328
329 fn is_lifo(&self) -> bool {
330 self.discriminant() == Self::LIFO_DISCRIMINANT
331 }
332
333 fn is_manual(&self) -> bool {
334 self.discriminant() == Self::MANUAL_DISCRIMINANT
335 }
336
337 fn payload(&self) -> u32 {
338 self.0 & Self::PAYLOAD_MASK
339 }
340
341 fn as_lifo(&self) -> Option<usize> {
342 if self.is_lifo() {
343 Some(usize::try_from(self.payload()).unwrap())
344 } else {
345 None
346 }
347 }
348
349 fn as_manual(&self) -> Option<SlabId> {
350 if self.is_manual() {
351 Some(SlabId::from_raw(self.payload()))
352 } else {
353 None
354 }
355 }
356}
357
358/// The set of all embedder-API GC roots in a single store/heap.
359#[derive(Debug, Default)]
360pub(crate) struct RootSet {
361 /// GC roots with arbitrary lifetime that are manually rooted and unrooted,
362 /// for use with `ManuallyRooted<T>`.
363 manually_rooted: Slab<VMGcRef>,
364
365 /// Strictly LIFO-ordered GC roots, for use with `RootScope` and
366 /// `Rooted<T>`.
367 lifo_roots: Vec<LifoRoot>,
368
369 /// Generation counter for entries to prevent ABA bugs with `RootScope` and
370 /// `Rooted<T>`.
371 lifo_generation: u32,
372}
373
374#[derive(Debug)]
375struct LifoRoot {
376 generation: u32,
377 gc_ref: VMGcRef,
378}
379
380impl RootSet {
381 pub(crate) fn trace_roots(&mut self, gc_roots_list: &mut GcRootsList) {
382 log::trace!("Begin trace user LIFO roots");
383 for root in &mut self.lifo_roots {
384 unsafe {
385 gc_roots_list.add_root((&mut root.gc_ref).into(), "user LIFO root");
386 }
387 }
388 log::trace!("End trace user LIFO roots");
389
390 log::trace!("Begin trace user manual roots");
391 for (_id, root) in self.manually_rooted.iter_mut() {
392 unsafe {
393 gc_roots_list.add_root(root.into(), "user manual root");
394 }
395 }
396 log::trace!("End trace user manual roots");
397 }
398
399 /// Enter a LIFO rooting scope.
400 ///
401 /// Returns an integer that should be passed unmodified to `exit_lifo_scope`
402 /// when the scope is finished.
403 ///
404 /// Calls to `{enter,exit}_lifo_scope` must happen in a strict LIFO order.
405 #[inline]
406 pub(crate) fn enter_lifo_scope(&self) -> usize {
407 self.lifo_roots.len()
408 }
409
410 /// Exit a LIFO rooting scope.
411 ///
412 /// The `scope` argument must be the result of the corresponding
413 /// `enter_lifo_scope` call.
414 ///
415 /// Calls to `{enter,exit}_lifo_scope` must happen in a strict LIFO order.
416 #[inline]
417 pub(crate) fn exit_lifo_scope(&mut self, gc_store: Option<&mut GcStore>, scope: usize) {
418 debug_assert!(self.lifo_roots.len() >= scope);
419
420 // If we actually have roots to unroot, call an out-of-line slow path.
421 if self.lifo_roots.len() > scope {
422 self.exit_lifo_scope_slow(gc_store, scope);
423 }
424 }
425
426 #[inline(never)]
427 #[cold]
428 fn exit_lifo_scope_slow(&mut self, mut gc_store: Option<&mut GcStore>, scope: usize) {
429 self.lifo_generation += 1;
430
431 // TODO: In the case where we have a tracing GC that doesn't need to
432 // drop barriers, this should really be:
433 //
434 // self.lifo_roots.truncate(scope);
435
436 let mut lifo_roots = mem::take(&mut self.lifo_roots);
437 for root in lifo_roots.drain(scope..) {
438 // Only drop the GC reference if we actually have a GC store. How
439 // can we have a GC reference but not a GC store? If we've only
440 // create `i31refs`, we never force a GC store's allocation. This is
441 // fine because `i31ref`s never need drop barriers.
442 if let Some(gc_store) = &mut gc_store {
443 gc_store.drop_gc_ref(root.gc_ref);
444 }
445 }
446 self.lifo_roots = lifo_roots;
447 }
448
449 pub(crate) fn with_lifo_scope<S, T>(store: &mut S, f: impl FnOnce(&mut S) -> T) -> T
450 where
451 S: ?Sized + DerefMut<Target = StoreOpaque>,
452 {
453 let scope = store.gc_roots().enter_lifo_scope();
454 let ret = f(store);
455 store.exit_gc_lifo_scope(scope);
456 ret
457 }
458
459 pub(crate) fn push_lifo_root(&mut self, store_id: StoreId, gc_ref: VMGcRef) -> GcRootIndex {
460 let generation = self.lifo_generation;
461 let index = self.lifo_roots.len();
462 let index = PackedIndex::new_lifo(index);
463 self.lifo_roots.push(LifoRoot { generation, gc_ref });
464 GcRootIndex {
465 store_id,
466 generation,
467 index,
468 }
469 }
470}
471
472/// A scoped, rooted reference to a garbage-collected `T`.
473///
474/// A `Rooted<T>` is a strong handle to a garbage-collected `T`, preventing its
475/// referent (and anything else transitively referenced) from being collected by
476/// the GC during the scope within which this `Rooted<T>` was created.
477///
478/// When the context exits this `Rooted<T>`'s scope, the underlying GC object is
479/// automatically unrooted and any further attempts to use access the underlying
480/// object will return errors or otherwise fail.
481///
482/// `Rooted<T>` dereferences to its underlying `T`, allowing you to call `T`'s
483/// methods.
484///
485/// # Example
486///
487/// ```
488/// # use wasmtime::*;
489/// # fn _foo() -> Result<()> {
490/// let mut store = Store::<()>::default();
491///
492/// // Allocating a GC object returns a `Rooted<T>`.
493/// let hello: Rooted<ExternRef> = ExternRef::new(&mut store, "hello")?;
494///
495/// // Because `Rooted<T>` derefs to `T`, we can call `T` methods on a
496/// // `Rooted<T>`. For example, we can call the `ExternRef::data` method when we
497/// // have a `Rooted<ExternRef>`.
498/// let data = hello
499/// .data(&store)?
500/// .ok_or_else(|| Error::msg("externref has no host data"))?
501/// .downcast_ref::<&str>()
502/// .ok_or_else(|| Error::msg("not a str"))?;
503/// assert_eq!(*data, "hello");
504///
505/// // A `Rooted<T>` roots its underlying GC object for the duration of the
506/// // scope of the store/caller/context that was passed to the method that created
507/// // it. If we only want to keep a GC reference rooted and alive temporarily, we
508/// // can introduce new scopes with `RootScope`.
509/// {
510/// let mut scope = RootScope::new(&mut store);
511///
512/// // This `Rooted<T>` is automatically unrooted after `scope` is dropped,
513/// // allowing the collector to reclaim its GC object in the next GC.
514/// let scoped_ref = ExternRef::new(&mut scope, "goodbye");
515/// }
516///
517/// let module = Module::new(store.engine(), r#"
518/// (module
519/// (global (export "global") (mut externref) (ref.null extern))
520/// (table (export "table") 10 externref)
521/// (func (export "func") (param externref) (result externref)
522/// local.get 0
523/// )
524/// )
525/// "#)?;
526/// let instance = Instance::new(&mut store, &module, &[])?;
527///
528/// // GC references returned from calls into Wasm also return (optional, if the
529/// // Wasm type is nullable) `Rooted<T>`s.
530/// let result: Option<Rooted<_>> = instance
531/// .get_typed_func::<Option<Rooted<ExternRef>>, Option<Rooted<ExternRef>>>(&mut store, "func")?
532/// .call(&mut store, Some(hello))?;
533///
534/// // Similarly, getting a GC reference from a Wasm instance's exported global
535/// // or table yields a `Rooted<T>`.
536///
537/// let global = instance
538/// .get_global(&mut store, "global")
539/// .ok_or_else(|| Error::msg("missing `global` export"))?;
540/// let global_val = global.get(&mut store);
541/// let global_ref: Option<&Rooted<_>> = global_val
542/// .externref()
543/// .ok_or_else(|| Error::msg("not an externref"))?;
544///
545/// let table = instance.get_table(&mut store, "table").unwrap();
546/// let table_elem = table
547/// .get(&mut store, 3)
548/// .ok_or_else(|| Error::msg("table out of bounds"))?;
549/// let table_elem_ref: Option<&Rooted<_>> = table_elem
550/// .as_extern()
551/// .ok_or_else(|| Error::msg("not an externref"))?;
552/// # Ok(())
553/// # }
554/// ```
555///
556/// # Differences Between `Rooted<T>` and `ManuallyRooted<T>`
557///
558/// While `Rooted<T>` is automatically unrooted when its scope is exited, this
559/// means that `Rooted<T>` is only valid for strictly last-in-first-out (LIFO,
560/// aka stack order) lifetimes. This is in contrast to
561/// [`ManuallyRooted<T>`][crate::ManuallyRooted], which supports rooting GC
562/// objects for arbitrary lifetimes, but requires manual unrooting.
563///
564/// | Type | Supported Lifetimes | Unrooting |
565/// |----------------------------------------------|-----------------------------|-----------|
566/// | [`Rooted<T>`][crate::Rooted] | Strictly LIFO / stack order | Automatic |
567/// | [`ManuallyRooted<T>`][crate::ManuallyRooted] | Arbitrary | Manual |
568///
569/// `Rooted<T>` should suffice for most use cases, and provides better
570/// ergonomics, but `ManuallyRooted<T>` exists as a fully-general escape hatch.
571///
572/// # Scopes
573///
574/// Wasmtime automatically creates two kinds of scopes:
575///
576/// 1. A [`Store`][crate::Store] is the outermost rooting scope. Creating a
577/// `Root<T>` directly inside a `Store` permanently roots the underlying
578/// object, similar to dropping a
579/// [`ManuallyRooted<T>`][crate::ManuallyRooted] without unrooting it.
580///
581/// 2. A [`Caller`][crate::Caller] provides a rooting scope for the duration of
582/// a call from Wasm into a host function. Any objects rooted in a `Caller`
583/// will be unrooted after the host function returns. Note that there can be
584/// nested `Caller` scopes in the case where Wasm calls a host function,
585/// creating the first `Caller` and its rooting scope , and then the host
586/// function calls a Wasm function which then calls another host function,
587/// creating a second `Caller` and a second rooting scope. This nesting can
588/// be arbitrarily deep.
589///
590/// Additionally, if you would like to define finer-grained rooting scopes,
591/// Wasmtime provides the [`RootScope`][crate::RootScope] type.
592///
593/// Scopes are always nested in a last-in-first-out (LIFO) order. An outer scope
594/// is never exited (and the `Rooted<T>`s defined within it are never
595/// automatically unrooted) while an inner scope is still active. All inner
596/// scopes are exited before their outer scopes.
597///
598/// The following diagram illustrates various rooting scopes over time, how they
599/// nest, and when their `Rooted<T>`s are automatically unrooted:
600///
601/// ```text
602/// ----- new Store
603/// |
604/// |
605/// | let a: Rooted<T> = ...;
606/// |
607/// |
608/// | ----- call into Wasm
609/// | |
610/// | |
611/// | | ----- Wasm calls host function F
612/// | | |
613/// | | |
614/// | | | let b: Rooted<T> = ...;
615/// | | |
616/// | | |
617/// | | | ----- F calls into Wasm
618/// | | | |
619/// | | | |
620/// | | | | ----- Wasm call host function G
621/// | | | | |
622/// | | | | |
623/// | | | | | let c: Rooted<T> = ...;
624/// | | | | |
625/// | | | | |
626/// | | | | ----- return to Wasm from host function G (unroots `c`)
627/// | | | |
628/// | | | |
629/// | | | ----- Wasm returns to F
630/// | | |
631/// | | |
632/// | | ----- return from host function F (unroots `b`)
633/// | |
634/// | |
635/// | ----- return from Wasm
636/// |
637/// |
638/// | ----- let scope1 = RootScope::new(...);
639/// | |
640/// | |
641/// | | let d: Rooted<T> = ...;
642/// | |
643/// | |
644/// | | ----- let scope2 = RootScope::new(...);
645/// | | |
646/// | | |
647/// | | | let e: Rooted<T> = ...;
648/// | | |
649/// | | |
650/// | | ----- drop `scope2` (unroots `e`)
651/// | |
652/// | |
653/// | ----- drop `scope1` (unroots `d`)
654/// |
655/// |
656/// ----- drop Store (unroots `a`)
657/// ```
658///
659/// A `Rooted<T>` can be used successfully as long as it is still rooted so, in
660/// the above diagram, `d` is valid inside `scope2` because `scope2` is wholly
661/// contained within the scope `d` was rooted within (`scope1`).
662///
663/// See also the documentation for [`RootScope`][crate::RootScope].
664#[repr(transparent)]
665pub struct Rooted<T: GcRef> {
666 inner: GcRootIndex,
667 _phantom: marker::PhantomData<T>,
668}
669
670impl<T: GcRef> Clone for Rooted<T> {
671 fn clone(&self) -> Self {
672 Rooted {
673 inner: self.inner,
674 _phantom: marker::PhantomData,
675 }
676 }
677}
678
679impl<T: GcRef> Copy for Rooted<T> {}
680
681impl<T: GcRef> Debug for Rooted<T> {
682 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
683 let name = format!("Rooted<{}>", any::type_name::<T>());
684 f.debug_struct(&name).field("inner", &self.inner).finish()
685 }
686}
687
688impl<T: GcRef> RootedGcRefImpl<T> for Rooted<T> {
689 fn get_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Option<&'a VMGcRef> {
690 assert!(
691 self.comes_from_same_store(store),
692 "object used with wrong store"
693 );
694 let index = self.inner.index.as_lifo().unwrap();
695 let entry = store.gc_roots().lifo_roots.get(index)?;
696 if entry.generation == self.inner.generation {
697 Some(&entry.gc_ref)
698 } else {
699 None
700 }
701 }
702}
703
704impl<T: GcRef> Deref for Rooted<T> {
705 type Target = T;
706
707 fn deref(&self) -> &Self::Target {
708 T::transmute_ref(&self.inner)
709 }
710}
711
712impl<T: GcRef> Rooted<T> {
713 /// Push the given `VMGcRef` onto our LIFO root set.
714 ///
715 /// `gc_ref` should belong to `store`'s heap; failure to uphold this is
716 /// memory safe but will result in general failures down the line such as
717 /// panics or incorrect results.
718 ///
719 /// `gc_ref` should be a GC reference pointing to an instance of the GC type
720 /// that `T` represents. Failure to uphold this invariant is memory safe but
721 /// will result in general incorrectness such as panics and wrong results.
722 pub(crate) fn new(store: &mut AutoAssertNoGc<'_>, gc_ref: VMGcRef) -> Rooted<T> {
723 let id = store.id();
724 let roots = store.gc_roots_mut();
725 let inner = roots.push_lifo_root(id, gc_ref);
726 Rooted {
727 inner,
728 _phantom: marker::PhantomData,
729 }
730 }
731
732 /// Create a new `Rooted<T>` from a `GcRootIndex`.
733 ///
734 /// Note that `Rooted::from_gc_root_index(my_rooted.index)` is not
735 /// necessarily an identity function, as it allows changing the `T` type
736 /// parameter.
737 ///
738 /// The given index should be a LIFO index of a GC reference pointing to an
739 /// instance of the GC type that `T` represents. Failure to uphold this
740 /// invariant is memory safe but will result in general incorrectness such
741 /// as panics and wrong results.
742 pub(crate) fn from_gc_root_index(inner: GcRootIndex) -> Rooted<T> {
743 debug_assert!(inner.index.is_lifo());
744 Rooted {
745 inner,
746 _phantom: marker::PhantomData,
747 }
748 }
749
750 #[inline]
751 pub(crate) fn comes_from_same_store(&self, store: &StoreOpaque) -> bool {
752 debug_assert!(self.inner.index.is_lifo());
753 self.inner.comes_from_same_store(store)
754 }
755
756 /// Create a [`ManuallyRooted<T>`][crate::ManuallyRooted] holding onto the
757 /// same GC object as `self`.
758 ///
759 /// Returns `None` if `self` is used outside of its scope and has therefore
760 /// been unrooted.
761 ///
762 /// This does not unroot `self`, and `self` remains valid until its
763 /// associated scope is exited.
764 ///
765 /// # Panics
766 ///
767 /// Panics if this object is not associate with the given store.
768 ///
769 /// # Example
770 ///
771 /// ```
772 /// # use wasmtime::*;
773 /// # fn _foo() -> Result<()> {
774 /// let mut store = Store::<()>::default();
775 ///
776 /// let y: ManuallyRooted<_> = {
777 /// // Create a nested rooting scope.
778 /// let mut scope = RootScope::new(&mut store);
779 ///
780 /// // `x` is only rooted within this nested scope.
781 /// let x: Rooted<_> = ExternRef::new(&mut scope, "hello!")?;
782 ///
783 /// // Extend `x`'s rooting past its scope's lifetime by converting it
784 /// // to a `ManuallyRooted`.
785 /// x.to_manually_rooted(&mut scope)?
786 /// };
787 ///
788 /// // Now we can still access the reference outside the scope it was
789 /// // originally defined within.
790 /// let data = y.data(&store)?.expect("should have host data");
791 /// let data = data.downcast_ref::<&str>().expect("host data should be str");
792 /// assert_eq!(*data, "hello!");
793 ///
794 /// // But we have to manually unroot `y`.
795 /// y.unroot(&mut store);
796 /// # Ok(())
797 /// # }
798 /// ```
799 pub fn to_manually_rooted(&self, mut store: impl AsContextMut) -> Result<ManuallyRooted<T>> {
800 self._to_manually_rooted(store.as_context_mut().0)
801 }
802
803 pub(crate) fn _to_manually_rooted(&self, store: &mut StoreOpaque) -> Result<ManuallyRooted<T>> {
804 let mut store = AutoAssertNoGc::new(store);
805 let gc_ref = self.try_clone_gc_ref(&mut store)?;
806 Ok(ManuallyRooted::new(&mut store, gc_ref))
807 }
808
809 /// Are these two `Rooted<T>`s the same GC root?
810 ///
811 /// Note that this function can return `false` even when `a` and `b` are
812 /// rooting the same underlying GC object, but the object was rooted
813 /// multiple times (for example in different scopes). Use
814 /// [`Rooted::ref_eq`][crate::Rooted::ref_eq] to test whether these are
815 /// references to the same underlying GC object or not.
816 ///
817 /// # Example
818 ///
819 /// ```
820 /// # use wasmtime::*;
821 /// # fn foo() -> Result<()> {
822 /// let mut store = Store::<()>::default();
823 ///
824 /// let a = ExternRef::new(&mut store, "hello")?;
825 /// let b = a;
826 ///
827 /// // `a` and `b` are the same GC root.
828 /// assert!(Rooted::rooted_eq(a, b));
829 ///
830 /// {
831 /// let mut scope = RootScope::new(&mut store);
832 ///
833 /// // `c` is a different GC root, in a different scope, even though it
834 /// // is rooting the same object.
835 /// let c = a.to_manually_rooted(&mut scope)?.into_rooted(&mut scope);
836 /// assert!(!Rooted::rooted_eq(a, c));
837 /// }
838 ///
839 /// let x = ExternRef::new(&mut store, "goodbye")?;
840 ///
841 /// // `a` and `x` are different GC roots, rooting different objects.
842 /// assert!(!Rooted::rooted_eq(a, x));
843 /// # Ok(())
844 /// # }
845 /// ```
846 pub fn rooted_eq(a: Self, b: Self) -> bool {
847 a.inner == b.inner
848 }
849
850 /// Are these two GC roots referencing the same underlying GC object?
851 ///
852 /// This function will return `true` even when `a` and `b` are different GC
853 /// roots (for example because they were rooted in different scopes) if they
854 /// are rooting the same underlying GC object. To only test whether they are
855 /// the same GC root, and not whether they are rooting the same GC object,
856 /// use [`Rooted::rooted_eq`][crate::Rooted::rooted_eq].
857 ///
858 /// Returns an error if either `a` or `b` has been unrooted, for example
859 /// because the scope it was rooted within has been exited.
860 ///
861 /// Because this method takes any `impl RootedGcRef<T>` arguments, it can be
862 /// used to compare, for example, a `Rooted<T>` and a `ManuallyRooted<T>`.
863 ///
864 /// # Panics
865 ///
866 /// Panics if either `a` or `b` is not associated with the given `store`.
867 ///
868 /// # Example
869 ///
870 /// ```
871 /// # use wasmtime::*;
872 /// # fn foo() -> Result<()> {
873 /// let mut store = Store::<()>::default();
874 ///
875 /// let a = ExternRef::new(&mut store, "hello")?;
876 /// let b = a;
877 ///
878 /// // `a` and `b` are rooting the same object.
879 /// assert!(Rooted::ref_eq(&store, &a, &b)?);
880 ///
881 /// {
882 /// let mut scope = RootScope::new(&mut store);
883 ///
884 /// // `c` is a different GC root, in a different scope, but still
885 /// // rooting the same object.
886 /// let c = a.to_manually_rooted(&mut scope)?.into_rooted(&mut scope);
887 /// assert!(!Rooted::ref_eq(&scope, &a, &c)?);
888 /// }
889 ///
890 /// let x = ExternRef::new(&mut store, "goodbye")?;
891 ///
892 /// // `a` and `x` are rooting different objects.
893 /// assert!(!Rooted::ref_eq(&store, &a, &x)?);
894 ///
895 /// // You can also compare `Rooted<T>`s and `ManuallyRooted<T>`s with this
896 /// // function.
897 /// let d = a.to_manually_rooted(&mut store)?;
898 /// assert!(Rooted::ref_eq(&store, &a, &d)?);
899 ///
900 /// d.unroot(&mut store);
901 /// # Ok(())
902 /// # }
903 /// ```
904 pub fn ref_eq(
905 store: impl AsContext,
906 a: &impl RootedGcRef<T>,
907 b: &impl RootedGcRef<T>,
908 ) -> Result<bool> {
909 let store = store.as_context().0;
910 Self::_ref_eq(store, a, b)
911 }
912
913 pub(crate) fn _ref_eq(
914 store: &StoreOpaque,
915 a: &impl RootedGcRef<T>,
916 b: &impl RootedGcRef<T>,
917 ) -> Result<bool> {
918 let a = a.try_gc_ref(store)?;
919 let b = b.try_gc_ref(store)?;
920 Ok(a == b)
921 }
922
923 /// Hash this root.
924 ///
925 /// Note that, similar to `Rooted::rooted_eq`, this only operates on the
926 /// root and *not* the underlying GC reference. That means that two
927 /// different rootings of the same object will hash to different values
928 /// (modulo hash collisions). If this is undesirable, use the
929 /// [`ref_hash`][crate::Rooted::ref_hash] method instead.
930 pub fn rooted_hash<H>(&self, state: &mut H)
931 where
932 H: Hasher,
933 {
934 self.inner.hash(state);
935 }
936
937 /// Hash the underlying rooted object reference.
938 ///
939 /// Note that, similar to `Rooted::ref_eq`, and operates on the underlying
940 /// rooted GC object reference, not the root. That means that two
941 /// *different* rootings of the same object will hash to the *same*
942 /// value. If this is undesirable, use the
943 /// [`rooted_hash`][crate::Rooted::rooted_hash] method instead.
944 pub fn ref_hash<H>(&self, store: impl AsContext, state: &mut H) -> Result<()>
945 where
946 H: Hasher,
947 {
948 let gc_ref = self.try_gc_ref(store.as_context().0)?;
949 gc_ref.hash(state);
950 Ok(())
951 }
952
953 /// Cast `self` to a `Rooted<U>`.
954 ///
955 /// It is the caller's responsibility to ensure that `self` is actually a
956 /// `U`. Failure to uphold this invariant will be memory safe but will
957 /// result in general incorrectness such as panics and wrong results.
958 pub(crate) fn unchecked_cast<U: GcRef>(self) -> Rooted<U> {
959 Rooted::from_gc_root_index(self.inner)
960 }
961
962 /// Common implementation of the `WasmTy::store` trait method for all
963 /// `Rooted<T>`s.
964 pub(super) fn wasm_ty_store(
965 self,
966 store: &mut AutoAssertNoGc<'_>,
967 ptr: &mut MaybeUninit<ValRaw>,
968 val_raw: impl Fn(u32) -> ValRaw,
969 ) -> Result<()> {
970 let gc_ref = self.inner.try_clone_gc_ref(store)?;
971 let raw = store.gc_store_mut()?.expose_gc_ref_to_wasm(gc_ref);
972 ptr.write(val_raw(raw));
973 Ok(())
974 }
975
976 /// Common implementation of the `WasmTy::load` trait method for all
977 /// `Rooted<T>`s.
978 pub(super) fn wasm_ty_load(
979 store: &mut AutoAssertNoGc<'_>,
980 raw_gc_ref: u32,
981 from_cloned_gc_ref: impl Fn(&mut AutoAssertNoGc<'_>, VMGcRef) -> Self,
982 ) -> Self {
983 debug_assert_ne!(raw_gc_ref, 0);
984 let gc_ref = VMGcRef::from_raw_u32(raw_gc_ref).expect("non-null");
985 let gc_ref = store.unwrap_gc_store_mut().clone_gc_ref(&gc_ref);
986 from_cloned_gc_ref(store, gc_ref)
987 }
988
989 /// Common implementation of the `WasmTy::store` trait method for all
990 /// `Option<Rooted<T>>`s.
991 pub(super) fn wasm_ty_option_store(
992 me: Option<Self>,
993 store: &mut AutoAssertNoGc<'_>,
994 ptr: &mut MaybeUninit<ValRaw>,
995 val_raw: impl Fn(u32) -> ValRaw,
996 ) -> Result<()> {
997 match me {
998 Some(me) => me.wasm_ty_store(store, ptr, val_raw),
999 None => {
1000 ptr.write(val_raw(0));
1001 Ok(())
1002 }
1003 }
1004 }
1005
1006 /// Common implementation of the `WasmTy::load` trait method for all
1007 /// `Option<Rooted<T>>`s.
1008 pub(super) fn wasm_ty_option_load(
1009 store: &mut AutoAssertNoGc<'_>,
1010 raw_gc_ref: u32,
1011 from_cloned_gc_ref: impl Fn(&mut AutoAssertNoGc<'_>, VMGcRef) -> Self,
1012 ) -> Option<Self> {
1013 let gc_ref = VMGcRef::from_raw_u32(raw_gc_ref)?;
1014 let gc_ref = store.unwrap_gc_store_mut().clone_gc_ref(&gc_ref);
1015 Some(from_cloned_gc_ref(store, gc_ref))
1016 }
1017}
1018
1019/// Nested rooting scopes.
1020///
1021/// `RootScope` allows the creation or nested rooting scopes for use with
1022/// [`Rooted<T>`][crate::Rooted]. This allows for fine-grained control over how
1023/// long a set of [`Rooted<T>`][crate::Rooted]s are strongly held alive, giving
1024/// gives you the tools necessary to avoid holding onto GC objects longer than
1025/// necessary. `Rooted<T>`s created within a `RootScope` are automatically
1026/// unrooted when the `RootScope` is dropped. For more details on
1027/// [`Rooted<T>`][crate::Rooted] lifetimes and their interaction with rooting
1028/// scopes, see [`Rooted<T>`][crate::Rooted]'s documentation.
1029///
1030/// A `RootScope<C>` wraps a `C: AsContextMut` (that is, anything that
1031/// represents exclusive access to a [`Store`][crate::Store]) and in turn
1032/// implements [`AsContext`][crate::AsContext] and
1033/// [`AsContextMut`][crate::AsContextMut] in terms of its underlying
1034/// `C`. Therefore, `RootScope<C>` can be used anywhere you would use the
1035/// underlying `C`, for example in the [`Global::get`][crate::Global::get]
1036/// method. Any `Rooted<T>`s created by a method that a `RootScope<C>` was
1037/// passed as context to are tied to the `RootScope<C>`'s scope and
1038/// automatically unrooted when the scope is dropped.
1039///
1040/// # Example
1041///
1042/// ```
1043/// # use wasmtime::*;
1044/// # fn _foo() -> Result<()> {
1045/// let mut store = Store::<()>::default();
1046///
1047/// let a: Rooted<_>;
1048/// let b: Rooted<_>;
1049/// let c: Rooted<_>;
1050///
1051/// // Root `a` in the store's scope. It will be rooted for the duration of the
1052/// // store's lifetime.
1053/// a = ExternRef::new(&mut store, 42)?;
1054///
1055/// // `a` is rooted, so we can access its data successfully.
1056/// assert!(a.data(&store).is_ok());
1057///
1058/// {
1059/// let mut scope1 = RootScope::new(&mut store);
1060///
1061/// // Root `b` in `scope1`.
1062/// b = ExternRef::new(&mut scope1, 36)?;
1063///
1064/// // Both `a` and `b` are rooted.
1065/// assert!(a.data(&scope1).is_ok());
1066/// assert!(b.data(&scope1).is_ok());
1067///
1068/// {
1069/// let mut scope2 = RootScope::new(&mut scope1);
1070///
1071/// // Root `c` in `scope2`.
1072/// c = ExternRef::new(&mut scope2, 36)?;
1073///
1074/// // All of `a`, `b`, and `c` are rooted.
1075/// assert!(a.data(&scope2).is_ok());
1076/// assert!(b.data(&scope2).is_ok());
1077/// assert!(c.data(&scope2).is_ok());
1078///
1079/// // Drop `scope2`.
1080/// }
1081///
1082/// // Now `a` and `b` are still rooted, but `c` was unrooted when we dropped
1083/// // `scope2`.
1084/// assert!(a.data(&scope1).is_ok());
1085/// assert!(b.data(&scope1).is_ok());
1086/// assert!(c.data(&scope1).is_err());
1087///
1088/// // Drop `scope1`.
1089/// }
1090///
1091/// // And now only `a` is still rooted. Both `b` and `c` were unrooted when we
1092/// // dropped their respective rooting scopes.
1093/// assert!(a.data(&store).is_ok());
1094/// assert!(b.data(&store).is_err());
1095/// assert!(c.data(&store).is_err());
1096/// # Ok(())
1097/// # }
1098/// ```
1099pub struct RootScope<C>
1100where
1101 C: AsContextMut,
1102{
1103 store: C,
1104 scope: usize,
1105}
1106
1107impl<C> Drop for RootScope<C>
1108where
1109 C: AsContextMut,
1110{
1111 fn drop(&mut self) {
1112 self.store.as_context_mut().0.exit_gc_lifo_scope(self.scope);
1113 }
1114}
1115
1116impl<C> RootScope<C>
1117where
1118 C: AsContextMut,
1119{
1120 // NB: we MUST NOT expose a method like
1121 //
1122 // pub fn store(&mut self) -> &mut Store { ... }
1123 //
1124 // because callers could do treacherous things like
1125 //
1126 // let scope1 = RootScope::new(&mut store1);
1127 // let scope2 = RootScope::new(&mut store2);
1128 // std::mem::swap(scope1.store(), scope2.store());
1129 //
1130 // and then we would start truncate the store's GC root set's LIFO roots to
1131 // the wrong lengths.
1132 //
1133 // Instead, we just implement `AsContext[Mut]` for `RootScope`.
1134
1135 /// Construct a new scope for rooting GC objects.
1136 ///
1137 /// # Example
1138 ///
1139 /// ```
1140 /// # use wasmtime::*;
1141 /// let mut store = Store::<()>::default();
1142 ///
1143 /// {
1144 /// let mut scope = RootScope::new(&mut store);
1145 ///
1146 /// // Temporarily root GC objects in this nested rooting scope...
1147 /// }
1148 /// ```
1149 pub fn new(store: C) -> Self {
1150 let scope = store.as_context().0.gc_roots().enter_lifo_scope();
1151 RootScope { store, scope }
1152 }
1153
1154 fn gc_roots(&mut self) -> &mut RootSet {
1155 self.store.as_context_mut().0.gc_roots_mut()
1156 }
1157
1158 fn lifo_roots(&mut self) -> &mut Vec<LifoRoot> {
1159 &mut self.gc_roots().lifo_roots
1160 }
1161
1162 /// Reserve enough capacity for `additional` GC roots in this scope.
1163 ///
1164 /// # Example
1165 ///
1166 /// ```
1167 /// # use wasmtime::*;
1168 /// let mut store = Store::<()>::default();
1169 ///
1170 /// {
1171 /// let mut scope = RootScope::new(&mut store);
1172 ///
1173 /// // Ensure we have enough storage pre-allocated to root five GC
1174 /// // references inside this scope without any underlying reallocation.
1175 /// scope.reserve(5);
1176 ///
1177 /// // ...
1178 /// }
1179 /// ```
1180 pub fn reserve(&mut self, additional: usize) {
1181 self.lifo_roots().reserve(additional);
1182 }
1183}
1184
1185impl<T> AsContext for RootScope<T>
1186where
1187 T: AsContextMut,
1188{
1189 type Data = T::Data;
1190
1191 fn as_context(&self) -> crate::StoreContext<'_, Self::Data> {
1192 self.store.as_context()
1193 }
1194}
1195
1196impl<T> AsContextMut for RootScope<T>
1197where
1198 T: AsContextMut,
1199{
1200 fn as_context_mut(&mut self) -> crate::StoreContextMut<'_, Self::Data> {
1201 self.store.as_context_mut()
1202 }
1203}
1204
1205pub(crate) trait AsStoreOpaqueMut {
1206 fn as_store_opaque_mut(&mut self) -> &mut StoreOpaque;
1207}
1208
1209impl<T> AsStoreOpaqueMut for T
1210where
1211 T: DerefMut<Target = StoreOpaque>,
1212{
1213 fn as_store_opaque_mut(&mut self) -> &mut StoreOpaque {
1214 &mut **self
1215 }
1216}
1217
1218impl AsStoreOpaqueMut for StoreOpaque {
1219 fn as_store_opaque_mut(&mut self) -> &mut StoreOpaque {
1220 self
1221 }
1222}
1223
1224impl<'a> AsStoreOpaqueMut for &'a mut dyn VMStore {
1225 fn as_store_opaque_mut(&mut self) -> &mut StoreOpaque {
1226 self.store_opaque_mut()
1227 }
1228}
1229
1230/// Internal version of `RootScope` that only wraps a `&mut StoreOpaque` rather
1231/// than a whole `impl AsContextMut<Data = T>`.
1232pub(crate) struct OpaqueRootScope<S>
1233where
1234 S: AsStoreOpaqueMut,
1235{
1236 store: S,
1237 scope: usize,
1238}
1239
1240impl<S> Drop for OpaqueRootScope<S>
1241where
1242 S: AsStoreOpaqueMut,
1243{
1244 fn drop(&mut self) {
1245 self.store
1246 .as_store_opaque_mut()
1247 .exit_gc_lifo_scope(self.scope);
1248 }
1249}
1250
1251impl<S> Deref for OpaqueRootScope<S>
1252where
1253 S: AsStoreOpaqueMut,
1254{
1255 type Target = S;
1256
1257 fn deref(&self) -> &Self::Target {
1258 &self.store
1259 }
1260}
1261
1262// XXX: Don't use this `DerefMut` implementation to `mem::{swap,replace}` or
1263// etc... the underlying `StoreOpaque` in a `OpaqueRootScope`! That will result
1264// in truncating the store's GC root set's LIFO roots to the wrong length.
1265//
1266// We don't implement `DerefMut` for `RootScope` for exactly this reason, but
1267// allow it for `OpaqueRootScope` because it is only Wasmtime-internal and not
1268// publicly exported.
1269impl<S> DerefMut for OpaqueRootScope<S>
1270where
1271 S: AsStoreOpaqueMut,
1272{
1273 fn deref_mut(&mut self) -> &mut Self::Target {
1274 &mut self.store
1275 }
1276}
1277
1278impl<S> OpaqueRootScope<S>
1279where
1280 S: AsStoreOpaqueMut,
1281{
1282 pub(crate) fn new(mut store: S) -> Self {
1283 let scope = store.as_store_opaque_mut().gc_roots().enter_lifo_scope();
1284 OpaqueRootScope { store, scope }
1285 }
1286}
1287
1288/// A rooted reference to a garbage-collected `T` with arbitrary lifetime.
1289///
1290/// A `ManuallyRooted<T>` is a strong handle to a garbage-collected `T`,
1291/// preventing its referent (and anything else transitively referenced) from
1292/// being collected by the GC until [`unroot`][crate::ManuallyRooted::unroot] is
1293/// explicitly called.
1294///
1295/// The primary way to create a `ManuallyRooted<T>` is to promote a temporary
1296/// `Rooted<T>` into a `ManuallyRooted<T>` via its
1297/// [`to_manually_rooted`][crate::Rooted::to_manually_rooted] method.
1298///
1299/// `ManuallyRooted<T>` dereferences to its underlying `T`, allowing you to call
1300/// `T`'s methods.
1301///
1302/// # Example
1303///
1304/// ```
1305/// # use wasmtime::*;
1306/// # fn _foo() -> Result<()> {
1307/// let mut store = Store::<Option<ManuallyRooted<ExternRef>>>::default();
1308///
1309/// // Create our `ManuallyRooted` in a nested scope to avoid rooting it for
1310/// // the duration of the store's lifetime.
1311/// let x = {
1312/// let mut scope = RootScope::new(&mut store);
1313/// let x = ExternRef::new(&mut scope, 1234)?;
1314/// x.to_manually_rooted(&mut scope)?
1315/// };
1316///
1317/// // Place `x` into our store.
1318/// *store.data_mut() = Some(x);
1319///
1320/// // Do a bunch stuff that may or may not access, replace, or take `x`...
1321///
1322/// // At any time, in any arbitrary scope, we can remove `x` from the store
1323/// // and unroot it:
1324/// if let Some(x) = store.data_mut().take() {
1325/// x.unroot(&mut store);
1326/// }
1327/// # Ok(())
1328/// # }
1329/// ```
1330///
1331/// # Differences Between `ManuallyRooted<T>` and `Rooted<T>`
1332///
1333/// While `ManuallyRooted<T>` can have arbitrary lifetimes, it requires manual
1334/// unrooting. This is in contrast to [`Rooted<T>`][crate::Rooted] which is
1335/// restricted to strictly last-in-first-out (LIFO, aka stack order) lifetimes,
1336/// but comes with automatic unrooting.
1337///
1338/// | Type | Supported Lifetimes | Unrooting |
1339/// |----------------------------------------------|-----------------------------|-----------|
1340/// | [`Rooted<T>`][crate::Rooted] | Strictly LIFO / stack order | Automatic |
1341/// | [`ManuallyRooted<T>`][crate::ManuallyRooted] | Arbitrary | Manual |
1342///
1343/// `Rooted<T>` should suffice for most use cases, and provides better
1344/// ergonomics, but `ManuallyRooted<T>` exists as a fully-general escape hatch.
1345///
1346/// # Manual Unrooting
1347///
1348/// Failure to explicitly call [`unroot`][crate::ManuallyRooted::unroot] (or
1349/// another method that consumes `self` and unroots the reference, such as
1350/// [`into_rooted`][crate::ManuallyRooted::into_rooted]) will leak the
1351/// underlying GC object, preventing it from being garbage collected until its
1352/// owning [`Store`][crate::Store] is dropped. That means all of the following
1353/// will result in permanently rooting the underlying GC object:
1354///
1355/// * Implicitly dropping a `ManuallyRooted<T>`:
1356///
1357/// ```no_run
1358/// # use wasmtime::*;
1359/// # let get_manually_rooted = || -> ManuallyRooted<ExternRef> { todo!() };
1360/// {
1361/// let perma_root: ManuallyRooted<_> = get_manually_rooted();
1362///
1363/// // `perma_root` is implicitly dropped at the end of its scope,
1364/// // permanently rooting/leaking its referent.
1365/// }
1366/// ```
1367///
1368/// * Explicitly dropping a `ManuallyRooted<T>`: `drop(my_manually_rooted)`.
1369///
1370/// * Forgetting a `ManuallyRooted<T>`: `std::mem::forget(my_manually_rooted)`.
1371///
1372/// * Inserting a `ManuallyRooted<T>` into a `std::sync::Arc` or `std::rc::Rc`
1373/// cycle.
1374///
1375/// * Etc...
1376///
1377/// Wasmtime does *not* assert that a `ManuallyRooted<T>` is unrooted on `Drop`,
1378/// or otherwise raise a panic, log a warning, or etc... on failure to manually
1379/// unroot. Sometimes leaking is intentional and desirable, particularly when
1380/// dealing with short-lived [`Store`][crate::Store]s where unrooting would just
1381/// be busy work since the whole store is about to be dropped.
1382#[repr(transparent)] // NB: the C API relies on this
1383pub struct ManuallyRooted<T>
1384where
1385 T: GcRef,
1386{
1387 inner: GcRootIndex,
1388 _phantom: marker::PhantomData<T>,
1389}
1390
1391const _: () = {
1392 use crate::{AnyRef, ExternRef};
1393
1394 // NB: these match the C API which should also be updated if this changes
1395 assert!(mem::size_of::<ManuallyRooted<AnyRef>>() == 16);
1396 assert!(mem::align_of::<ManuallyRooted<AnyRef>>() == mem::align_of::<u64>());
1397 assert!(mem::size_of::<ManuallyRooted<ExternRef>>() == 16);
1398 assert!(mem::align_of::<ManuallyRooted<ExternRef>>() == mem::align_of::<u64>());
1399};
1400
1401impl<T: GcRef> Debug for ManuallyRooted<T> {
1402 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1403 let name = format!("ManuallyRooted<{}>", any::type_name::<T>());
1404 f.debug_struct(&name).field("inner", &self.inner).finish()
1405 }
1406}
1407
1408impl<T: GcRef> Deref for ManuallyRooted<T> {
1409 type Target = T;
1410
1411 fn deref(&self) -> &Self::Target {
1412 T::transmute_ref(&self.inner)
1413 }
1414}
1415
1416impl<T> ManuallyRooted<T>
1417where
1418 T: GcRef,
1419{
1420 /// Construct a new manually-rooted GC root.
1421 ///
1422 /// `gc_ref` should belong to `store`'s heap; failure to uphold this is
1423 /// memory safe but will result in general failures down the line such as
1424 /// panics or incorrect results.
1425 ///
1426 /// `gc_ref` should be a GC reference pointing to an instance of the GC type
1427 /// that `T` represents. Failure to uphold this invariant is memory safe but
1428 /// will result in general incorrectness such as panics and wrong results.
1429 pub(crate) fn new(store: &mut AutoAssertNoGc<'_>, gc_ref: VMGcRef) -> Self {
1430 let id = store.gc_roots_mut().manually_rooted.alloc(gc_ref);
1431 ManuallyRooted {
1432 inner: GcRootIndex {
1433 store_id: store.id(),
1434 generation: 0,
1435 index: PackedIndex::new_manual(id),
1436 },
1437 _phantom: marker::PhantomData,
1438 }
1439 }
1440
1441 #[inline]
1442 pub(crate) fn comes_from_same_store(&self, store: &StoreOpaque) -> bool {
1443 debug_assert!(self.inner.index.is_manual());
1444 self.inner.comes_from_same_store(store)
1445 }
1446
1447 /// Clone this `ManuallyRooted`.
1448 ///
1449 /// Does not consume or unroot `self`: both `self` and the new
1450 /// `ManuallyRooted` return value will need to be manually unrooted.
1451 ///
1452 /// # Panics
1453 ///
1454 /// Panics if `self` is not associated with the given `store`.
1455 ///
1456 /// # Example
1457 ///
1458 /// ```
1459 /// # use wasmtime::*;
1460 /// # fn _foo() -> Result<()> {
1461 /// let mut store = Store::<Vec<ManuallyRooted<ExternRef>>>::default();
1462 ///
1463 /// // Create our `ManuallyRooted` in a nested scope to avoid rooting it for
1464 /// // the duration of the store's lifetime.
1465 /// let x = {
1466 /// let mut scope = RootScope::new(&mut store);
1467 /// let x = ExternRef::new(&mut scope, 1234)?;
1468 /// x.to_manually_rooted(&mut scope)?
1469 /// };
1470 ///
1471 /// // Push five clones of `x` into our store.
1472 /// for _ in 0..5 {
1473 /// let x_clone = x.clone(&mut store);
1474 /// store.data_mut().push(x_clone);
1475 /// }
1476 /// # Ok(())
1477 /// # }
1478 /// ```
1479 pub fn clone(&self, mut store: impl AsContextMut) -> Self {
1480 self._clone(store.as_context_mut().0)
1481 }
1482
1483 pub(crate) fn _clone(&self, store: &mut StoreOpaque) -> Self {
1484 let mut store = AutoAssertNoGc::new(store);
1485 let gc_ref = self
1486 .clone_gc_ref(&mut store)
1487 .expect("ManuallyRooted always has a gc ref");
1488 Self::new(&mut store, gc_ref)
1489 }
1490
1491 /// Unroot this GC object.
1492 ///
1493 /// Failure to call this method will result in the GC object, and anything
1494 /// it transitively references, being kept alive (aka "leaking") for the
1495 /// entirety of the store's lifetime.
1496 ///
1497 /// See the type-level docs for example usage.
1498 pub fn unroot(self, mut store: impl AsContextMut) {
1499 self._unroot(store.as_context_mut().0)
1500 }
1501
1502 pub(crate) fn _unroot(self, store: &mut StoreOpaque) {
1503 assert!(
1504 self.comes_from_same_store(store),
1505 "object used with wrong store"
1506 );
1507
1508 let mut store = AutoAssertNoGc::new(store);
1509 let id = self.inner.index.as_manual().unwrap();
1510 let roots = store.gc_roots_mut();
1511 let gc_ref = roots.manually_rooted.dealloc(id);
1512 store.unwrap_gc_store_mut().drop_gc_ref(gc_ref);
1513 }
1514
1515 /// Clone this `ManuallyRooted<T>` into a `Rooted<T>`.
1516 ///
1517 /// This operation does not consume or unroot this `ManuallyRooted<T>`.
1518 ///
1519 /// The underlying GC object is re-rooted in the given context's scope. The
1520 /// resulting `Rooted<T>` is only valid during the given context's
1521 /// scope. See the [`Rooted<T>`][crate::Rooted] documentation for more
1522 /// details on rooting scopes.
1523 ///
1524 /// This operation does not consume or unroot this `ManuallyRooted<T>`.
1525 ///
1526 /// # Panics
1527 ///
1528 /// Panics if this object is not associated with the given context's store.
1529 ///
1530 /// # Example
1531 ///
1532 /// ```
1533 /// # use wasmtime::*;
1534 /// # fn _foo() -> Result<()> {
1535 /// let mut store = Store::<()>::default();
1536 ///
1537 /// let root1: Rooted<_>;
1538 ///
1539 /// let manual = {
1540 /// let mut scope = RootScope::new(&mut store);
1541 /// root1 = ExternRef::new(&mut scope, 1234)?;
1542 /// root1.to_manually_rooted(&mut scope)?
1543 /// };
1544 ///
1545 /// // `root1` is no longer accessible because it was unrooted when `scope`
1546 /// // was dropped.
1547 /// assert!(root1.data(&store).is_err());
1548 ///
1549 /// // But we can re-root `manual` into this scope.
1550 /// let root2 = manual.to_rooted(&mut store);
1551 /// assert!(root2.data(&store).is_ok());
1552 ///
1553 /// // And we also still have access to `manual` and we still have to
1554 /// // manually unroot it.
1555 /// assert!(manual.data(&store).is_ok());
1556 /// manual.unroot(&mut store);
1557 /// # Ok(())
1558 /// # }
1559 /// ```
1560 pub fn to_rooted(&self, mut context: impl AsContextMut) -> Rooted<T> {
1561 self._to_rooted(context.as_context_mut().0)
1562 }
1563
1564 pub(crate) fn _to_rooted(&self, store: &mut StoreOpaque) -> Rooted<T> {
1565 assert!(
1566 self.comes_from_same_store(store),
1567 "object used with wrong store"
1568 );
1569 let mut store = AutoAssertNoGc::new(store);
1570 let gc_ref = self.clone_gc_ref(&mut store).unwrap();
1571 Rooted::new(&mut store, gc_ref)
1572 }
1573
1574 /// Convert this `ManuallyRooted<T>` into a `Rooted<T>`.
1575 ///
1576 /// The underlying GC object is re-rooted in the given context's scope. The
1577 /// resulting `Rooted<T>` is only valid during the given context's
1578 /// scope. See the [`Rooted<T>`][crate::Rooted] documentation for more
1579 /// details on rooting scopes.
1580 ///
1581 /// This operation consumes and unroots this `ManuallyRooted<T>`.
1582 ///
1583 /// # Panics
1584 ///
1585 /// Panics if this object is not associate with the given context's store.
1586 ///
1587 /// # Example
1588 ///
1589 /// ```
1590 /// # use wasmtime::*;
1591 /// # fn _foo() -> Result<()> {
1592 /// let mut store = Store::<()>::default();
1593 ///
1594 /// let root1: Rooted<_>;
1595 ///
1596 /// let manual = {
1597 /// let mut scope = RootScope::new(&mut store);
1598 /// root1 = ExternRef::new(&mut scope, 1234)?;
1599 /// root1.to_manually_rooted(&mut scope)?
1600 /// };
1601 ///
1602 /// // `root1` is no longer accessible because it was unrooted when `scope`
1603 /// // was dropped.
1604 /// assert!(root1.data(&store).is_err());
1605 ///
1606 /// // But we can re-root `manual` into this scope.
1607 /// let root2 = manual.into_rooted(&mut store);
1608 /// assert!(root2.data(&store).is_ok());
1609 ///
1610 /// // `manual` was consumed by the `into_rooted` call, and we no longer
1611 /// // have access to it, nor need to manually unroot it.
1612 /// # Ok(())
1613 /// # }
1614 /// ```
1615 pub fn into_rooted(self, mut context: impl AsContextMut) -> Rooted<T> {
1616 self._into_rooted(context.as_context_mut().0)
1617 }
1618
1619 pub(crate) fn _into_rooted(self, store: &mut StoreOpaque) -> Rooted<T> {
1620 assert!(
1621 self.comes_from_same_store(store),
1622 "object used with wrong store"
1623 );
1624 let rooted = self._to_rooted(store);
1625 self._unroot(store);
1626 rooted
1627 }
1628
1629 /// Are these two GC roots referencing the same underlying GC object?
1630 ///
1631 /// This function will return `true` even when `a` and `b` are different GC
1632 /// roots (for example because they were rooted in different scopes) if they
1633 /// are rooting the same underlying GC object.
1634 ///
1635 /// Because this method takes any `impl RootedGcRef<T>` arguments, it can be
1636 /// used to compare, for example, a `Rooted<T>` and a `ManuallyRooted<T>`.
1637 ///
1638 /// # Panics
1639 ///
1640 /// Panics if either `a` or `b` is not associated with the given `store`.
1641 ///
1642 /// # Example
1643 ///
1644 /// ```
1645 /// # use wasmtime::*;
1646 /// # fn foo() -> Result<()> {
1647 /// let mut store = Store::<()>::default();
1648 ///
1649 /// let a = ExternRef::new_manually_rooted(&mut store, "hello")?;
1650 /// let b = a.clone(&mut store);
1651 ///
1652 /// // `a` and `b` are rooting the same object.
1653 /// assert!(ManuallyRooted::ref_eq(&store, &a, &b)?);
1654 ///
1655 /// {
1656 /// let mut scope = RootScope::new(&mut store);
1657 ///
1658 /// // `c` is a different GC root, is in a different scope, and is a
1659 /// // `Rooted<T>` instead of a `ManuallyRooted<T>`, but is still rooting
1660 /// // the same object.
1661 /// let c = a.to_rooted(&mut scope);
1662 /// assert!(ManuallyRooted::ref_eq(&scope, &a, &c)?);
1663 /// }
1664 ///
1665 /// let x = ExternRef::new_manually_rooted(&mut store, "goodbye")?;
1666 ///
1667 /// // `a` and `x` are rooting different objects.
1668 /// assert!(!ManuallyRooted::ref_eq(&store, &a, &x)?);
1669 ///
1670 /// a.unroot(&mut store);
1671 /// b.unroot(&mut store);
1672 /// x.unroot(&mut store);
1673 /// # Ok(())
1674 /// # }
1675 /// ```
1676 pub fn ref_eq(
1677 store: impl AsContext,
1678 a: &impl RootedGcRef<T>,
1679 b: &impl RootedGcRef<T>,
1680 ) -> Result<bool> {
1681 Rooted::ref_eq(store, a, b)
1682 }
1683
1684 /// Hash this root.
1685 ///
1686 /// Note that, similar to `Rooted::rooted_eq`, this only operates on the
1687 /// root and *not* the underlying GC reference. That means that two
1688 /// different rootings of the same object will hash to different values
1689 /// (modulo hash collisions). If this is undesirable, use the
1690 /// [`ref_hash`][crate::ManuallyRooted::ref_hash] method instead.
1691 pub fn rooted_hash<H>(&self, state: &mut H)
1692 where
1693 H: Hasher,
1694 {
1695 self.inner.hash(state);
1696 }
1697
1698 /// Hash the underlying rooted object reference.
1699 ///
1700 /// Note that, similar to `Rooted::ref_eq`, and operates on the underlying
1701 /// rooted GC object reference, not the root. That means that two
1702 /// *different* rootings of the same object will hash to the *same*
1703 /// value. If this is undesirable, use the
1704 /// [`rooted_hash`][crate::Rooted::rooted_hash] method instead.
1705 pub fn ref_hash<H>(&self, store: impl AsContext, state: &mut H)
1706 where
1707 H: Hasher,
1708 {
1709 let gc_ref = self
1710 .get_gc_ref(store.as_context().0)
1711 .expect("ManuallyRooted's get_gc_ref is infallible");
1712 gc_ref.hash(state);
1713 }
1714
1715 #[doc(hidden)]
1716 pub fn into_parts_for_c_api(self) -> (NonZeroU64, u32, u32) {
1717 (
1718 self.inner.store_id.as_raw(),
1719 self.inner.generation,
1720 self.inner.index.0,
1721 )
1722 }
1723
1724 #[doc(hidden)]
1725 pub unsafe fn from_raw_parts_for_c_api(a: NonZeroU64, b: u32, c: u32) -> ManuallyRooted<T> {
1726 ManuallyRooted {
1727 inner: GcRootIndex {
1728 store_id: StoreId::from_raw(a),
1729 generation: b,
1730 index: PackedIndex(c),
1731 },
1732 _phantom: marker::PhantomData,
1733 }
1734 }
1735
1736 /// Cast `self` to a `ManuallyRooted<U>`.
1737 ///
1738 /// It is the caller's responsibility to ensure that `self` is actually a
1739 /// `U`. Failure to uphold this invariant will be memory safe but will
1740 /// result in general incorrectness such as panics and wrong results.
1741 pub(crate) fn unchecked_cast<U: GcRef>(self) -> ManuallyRooted<U> {
1742 let u = ManuallyRooted {
1743 inner: self.inner,
1744 _phantom: core::marker::PhantomData,
1745 };
1746 core::mem::forget(self);
1747 u
1748 }
1749
1750 /// Common implementation of the `WasmTy::store` trait method for all
1751 /// `ManuallyRooted<T>`s.
1752 pub(super) fn wasm_ty_store(
1753 self,
1754 store: &mut AutoAssertNoGc<'_>,
1755 ptr: &mut MaybeUninit<ValRaw>,
1756 val_raw: impl Fn(u32) -> ValRaw,
1757 ) -> Result<()> {
1758 let gc_ref = self.try_clone_gc_ref(store)?;
1759 let raw = store.gc_store_mut()?.expose_gc_ref_to_wasm(gc_ref);
1760 ptr.write(val_raw(raw));
1761 Ok(())
1762 }
1763
1764 /// Common implementation of the `WasmTy::load` trait method for all
1765 /// `ManuallyRooted<T>`s.
1766 pub(super) fn wasm_ty_load(
1767 store: &mut AutoAssertNoGc<'_>,
1768 raw_gc_ref: u32,
1769 from_cloned_gc_ref: impl Fn(&mut AutoAssertNoGc<'_>, VMGcRef) -> Rooted<T>,
1770 ) -> Self {
1771 debug_assert_ne!(raw_gc_ref, 0);
1772 let gc_ref = VMGcRef::from_raw_u32(raw_gc_ref).expect("non-null");
1773 let gc_ref = store.unwrap_gc_store_mut().clone_gc_ref(&gc_ref);
1774 RootSet::with_lifo_scope(store, |store| {
1775 let rooted = from_cloned_gc_ref(store, gc_ref);
1776 rooted
1777 ._to_manually_rooted(store)
1778 .expect("rooted is in scope")
1779 })
1780 }
1781
1782 /// Common implementation of the `WasmTy::store` trait method for all
1783 /// `Option<ManuallyRooted<T>>`s.
1784 pub(super) fn wasm_ty_option_store(
1785 me: Option<Self>,
1786 store: &mut AutoAssertNoGc<'_>,
1787 ptr: &mut MaybeUninit<ValRaw>,
1788 val_raw: impl Fn(u32) -> ValRaw,
1789 ) -> Result<()> {
1790 match me {
1791 Some(me) => me.wasm_ty_store(store, ptr, val_raw),
1792 None => {
1793 ptr.write(val_raw(0));
1794 Ok(())
1795 }
1796 }
1797 }
1798
1799 /// Common implementation of the `WasmTy::load` trait method for all
1800 /// `Option<ManuallyRooted<T>>`s.
1801 pub(super) fn wasm_ty_option_load(
1802 store: &mut AutoAssertNoGc<'_>,
1803 raw_gc_ref: u32,
1804 from_cloned_gc_ref: impl Fn(&mut AutoAssertNoGc<'_>, VMGcRef) -> Rooted<T>,
1805 ) -> Option<Self> {
1806 let gc_ref = VMGcRef::from_raw_u32(raw_gc_ref)?;
1807 let gc_ref = store.unwrap_gc_store_mut().clone_gc_ref(&gc_ref);
1808 RootSet::with_lifo_scope(store, |store| {
1809 let rooted = from_cloned_gc_ref(store, gc_ref);
1810 Some(
1811 rooted
1812 ._to_manually_rooted(store)
1813 .expect("rooted is in scope"),
1814 )
1815 })
1816 }
1817}
1818
1819impl<T: GcRef> RootedGcRefImpl<T> for ManuallyRooted<T> {
1820 fn get_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Option<&'a VMGcRef> {
1821 assert!(
1822 self.comes_from_same_store(store),
1823 "object used with wrong store"
1824 );
1825
1826 let id = self.inner.index.as_manual().unwrap();
1827 store.gc_roots().manually_rooted.get(id)
1828 }
1829}
1830
1831#[cfg(test)]
1832mod tests {
1833 use crate::ExternRef;
1834
1835 use super::*;
1836
1837 #[test]
1838 fn sizes() {
1839 // Try to keep tabs on the size of these things. Don't want them growing
1840 // unintentionally.
1841 assert_eq!(std::mem::size_of::<Rooted<ExternRef>>(), 16);
1842 assert_eq!(std::mem::size_of::<ManuallyRooted<ExternRef>>(), 16);
1843 }
1844}