wasmtime/runtime/gc/enabled/
rooting.rs

1//! Garbage collection rooting APIs.
2//!
3//! Rooting prevents GC objects from being collected while they are actively
4//! being used.
5//!
6//! ## Goals
7//!
8//! We have a few sometimes-conflicting goals with our GC rooting APIs:
9//!
10//! 1. Safety: It should never be possible to get a use-after-free bug because
11//!    the user misused the rooting APIs, the collector "mistakenly" determined
12//!    an object was unreachable and collected it, and then the user tried to
13//!    access the object. This is our highest priority.
14//!
15//! 2. Moving GC: Our rooting APIs should moving collectors (such as
16//!    generational and compacting collectors) where an object might get
17//!    relocated after a collection and we need to update the GC root's pointer
18//!    to the moved object. This means we either need cooperation and internal
19//!    mutability from individual GC roots as well as the ability to enumerate
20//!    all GC roots on the native Rust stack, or we need a level of indirection.
21//!
22//! 3. Performance: Our rooting APIs should generally be as low-overhead as
23//!    possible. They definitely shouldn't require synchronization and locking
24//!    to create, access, and drop GC roots.
25//!
26//! 4. Ergonomics: Our rooting APIs should be, if not a pleasure, then at least
27//!    not a burden for users. Additionally, the API's types should be `Sync`
28//!    and `Send` so that they work well with async Rust.
29//!
30//! For example, goals (3) and (4) are in conflict when we think about how to
31//! support (2). Ideally, for ergonomics, a root would automatically unroot
32//! itself when dropped. But in the general case that requires holding a
33//! reference to the store's root set, and that root set needs to be held
34//! simultaneously by all GC roots, and they each need to mutate the set to
35//! unroot themselves. That implies `Rc<RefCell<...>>` or `Arc<Mutex<...>>`! The
36//! former makes the store and GC root types not `Send` and not `Sync`. The
37//! latter imposes synchronization and locking overhead. So we instead make GC
38//! roots indirect and require passing in a store context explicitly to unroot
39//! in the general case. This trades worse ergonomics for better performance and
40//! support for moving GC.
41//!
42//! ## Two Flavors of Rooting API
43//!
44//! Okay, with that out of the way, this module provides two flavors of rooting
45//! API. One for the common, scoped lifetime case, and another for the rare case
46//! where we really need a GC root with an arbitrary, non-LIFO/non-scoped
47//! lifetime:
48//!
49//! 1. `RootScope` and `Rooted<T>`: These are used for temporarily rooting GC
50//!    objects for the duration of a scope. The internal implementation takes
51//!    advantage of the LIFO property inherent in scopes, making creating and
52//!    dropping `Rooted<T>`s and `RootScope`s super fast and roughly equivalent
53//!    to bump allocation.
54//!
55//!    This type is vaguely similar to V8's [`HandleScope`].
56//!
57//!    [`HandleScope`]: https://v8.github.io/api/head/classv8_1_1HandleScope.html
58//!
59//!    Note that `Rooted<T>` can't be statically tied to its context scope via a
60//!    lifetime parameter, unfortunately, as that would allow the creation of
61//!    only one `Rooted<T>` at a time, since the `Rooted<T>` would take a borrow
62//!    of the whole context.
63//!
64//!    This supports the common use case for rooting and provides good
65//!    ergonomics.
66//!
67//! 2. `ManuallyRooted<T>`: This is the fully general rooting API used for
68//!    holding onto non-LIFO GC roots with arbitrary lifetimes. However, users
69//!    must manually unroot them. Failure to manually unroot a
70//!    `ManuallyRooted<T>` before it is dropped will result in the GC object
71//!    (and everything it transitively references) leaking for the duration of
72//!    the `Store`'s lifetime.
73//!
74//!    This type is roughly similar to SpiderMonkey's [`PersistentRooted<T>`],
75//!    although they avoid the manual-unrooting with internal mutation and
76//!    shared references. (Our constraints mean we can't do those things, as
77//!    mentioned explained above.)
78//!
79//!    [`PersistentRooted<T>`]: http://devdoc.net/web/developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JS::PersistentRooted.html
80//!
81//! At the end of the day, both `Rooted<T>` and `ManuallyRooted<T>` are just
82//! tagged indices into the store's `RootSet`. This indirection allows working
83//! with Rust's borrowing discipline (we use `&mut Store` to represent mutable
84//! access to the GC heap) while still allowing rooted references to be moved
85//! around without tying up the whole store in borrows. Additionally, and
86//! crucially, this indirection allows us to update the *actual* GC pointers in
87//! the `RootSet` and support moving GCs (again, as mentioned above).
88//!
89//! ## Unrooted References
90//!
91//! We generally don't expose *unrooted* GC references in the Wasmtime API at
92//! this time -- and I expect it will be a very long time before we do, but in
93//! the limit we may want to let users define their own GC-managed types that
94//! participate in GC tracing and all that -- so we don't have to worry about
95//! failure to root an object causing use-after-free bugs or failing to update a
96//! GC root pointer after a moving GC as long as users stick to our safe rooting
97//! APIs. (The one exception is `ValRaw`, which does hold raw GC references. But
98//! with `ValRaw` all bets are off and safety is 100% up to the user.)
99//!
100//! We do, however, have to worry about these things internally. So first of
101//! all, try to avoid ever working with unrooted GC references if you
102//! can. However, if you really must, consider also using an `AutoAssertNoGc`
103//! across the block of code that is manipulating raw GC references.
104
105use crate::runtime::vm::{GcRootsList, GcStore, VMGcRef};
106use crate::{
107    AsContext, AsContextMut, GcRef, Result, RootedGcRef,
108    store::{AsStoreOpaque, AutoAssertNoGc, StoreId, StoreOpaque},
109};
110use crate::{ValRaw, prelude::*};
111use core::any;
112use core::marker;
113use core::mem::{self, MaybeUninit};
114use core::num::NonZeroU64;
115use core::{
116    fmt::{self, Debug},
117    hash::{Hash, Hasher},
118    ops::{Deref, DerefMut},
119};
120use wasmtime_slab::{Id as SlabId, Slab};
121
122mod sealed {
123    use super::*;
124
125    /// Sealed, `wasmtime`-internal trait for GC references.
126    ///
127    /// # Safety
128    ///
129    /// All types implementing this trait must:
130    ///
131    /// * Be a newtype of a `GcRootIndex`
132    ///
133    /// * Not implement `Copy` or `Clone`
134    ///
135    /// * Only have `&self` methods.
136    pub unsafe trait GcRefImpl: Sized {
137        /// Transmute a `&GcRootIndex` into an `&Self`.
138        fn transmute_ref(index: &GcRootIndex) -> &Self;
139    }
140
141    /// Sealed, `wasmtime`-internal trait for the common methods on rooted GC
142    /// references.
143    pub trait RootedGcRefImpl<T: GcRef> {
144        /// Get this rooted GC reference's raw `VMGcRef` out of the store's GC
145        /// root set.
146        ///
147        /// Returns `None` for objects that have since been unrooted (eg because
148        /// its associated `RootedScope` was dropped).
149        ///
150        /// Panics if this root is not associated with the given store.
151        fn get_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Option<&'a VMGcRef>;
152
153        /// Same as `get_gc_ref` but returns an error instead of `None` for
154        /// objects that have been unrooted.
155        fn try_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Result<&'a VMGcRef> {
156            self.get_gc_ref(store).ok_or_else(|| {
157                anyhow!("attempted to use a garbage-collected object that has been unrooted")
158            })
159        }
160
161        /// Get a clone of this rooted GC reference's raw `VMGcRef` out of the
162        /// store's GC root set.
163        ///
164        /// Returns `None` for objects that have since been unrooted (eg because
165        /// its associated `RootedScope` was dropped).
166        ///
167        /// Panics if this root is not associated with the given store.
168        fn clone_gc_ref(&self, store: &mut AutoAssertNoGc<'_>) -> Option<VMGcRef> {
169            let gc_ref = self.get_gc_ref(store)?.unchecked_copy();
170            Some(store.clone_gc_ref(&gc_ref))
171        }
172
173        /// Same as `clone_gc_ref` but returns an error instead of `None` for
174        /// objects that have been unrooted.
175        fn try_clone_gc_ref(&self, store: &mut AutoAssertNoGc<'_>) -> Result<VMGcRef> {
176            let gc_ref = self.try_gc_ref(store)?.unchecked_copy();
177            Ok(store.clone_gc_ref(&gc_ref))
178        }
179    }
180}
181pub(crate) use sealed::*;
182
183/// The index of a GC root inside a particular store's GC root set.
184///
185/// Can be either a LIFO- or manually-rooted object, depending on the
186/// `PackedIndex`.
187///
188/// Every `T` such that `T: GcRef` must be a newtype over this `GcRootIndex`.
189#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
190// Just `pub` to avoid `warn(private_interfaces)` in public APIs, which we can't
191// `allow(...)` on our MSRV yet.
192#[doc(hidden)]
193#[repr(C)] // NB: if this layout changes be sure to change the C API as well
194pub struct GcRootIndex {
195    store_id: StoreId,
196    generation: u32,
197    index: PackedIndex,
198}
199
200const _: () = {
201    // NB: these match the C API which should also be updated if this changes
202    assert!(mem::size_of::<GcRootIndex>() == 16);
203    assert!(mem::align_of::<GcRootIndex>() == mem::align_of::<u64>());
204};
205
206impl GcRootIndex {
207    #[inline]
208    pub(crate) fn comes_from_same_store(&self, store: &StoreOpaque) -> bool {
209        self.store_id == store.id()
210    }
211
212    /// Same as `RootedGcRefImpl::get_gc_ref` but not associated with any
213    /// particular `T: GcRef`.
214    ///
215    /// We must avoid triggering a GC while holding onto the resulting raw
216    /// `VMGcRef` to avoid use-after-free bugs and similar. The `'a` lifetime
217    /// threaded from the `store` to the result will normally prevent GCs
218    /// statically, at compile time, since performing a GC requires a mutable
219    /// borrow of the store. However, if you call `VMGcRef::unchecked_copy` on
220    /// the resulting GC reference, then all bets are off and this invariant is
221    /// up to you to manually uphold. Failure to uphold this invariant is memory
222    /// safe but will lead to general incorrectness such as panics and wrong
223    /// results.
224    ///
225    /// # Panics
226    ///
227    /// Panics if `self` is not associated with the given store.
228    pub(crate) fn get_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Option<&'a VMGcRef> {
229        assert!(
230            self.comes_from_same_store(store),
231            "object used with wrong store"
232        );
233        if let Some(index) = self.index.as_lifo() {
234            let entry = store.gc_roots().lifo_roots.get(index)?;
235            if entry.generation == self.generation {
236                Some(&entry.gc_ref)
237            } else {
238                None
239            }
240        } else if let Some(id) = self.index.as_manual() {
241            let gc_ref = store.gc_roots().manually_rooted.get(id);
242            debug_assert!(gc_ref.is_some());
243            gc_ref
244        } else {
245            unreachable!()
246        }
247    }
248
249    /// Same as `get_gc_ref` but returns an error instead of `None` if
250    /// the GC reference has been unrooted.
251    ///
252    /// # Panics
253    ///
254    /// Panics if `self` is not associated with the given store.
255    pub(crate) fn try_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Result<&'a VMGcRef> {
256        self.get_gc_ref(store).ok_or_else(|| {
257            anyhow!("attempted to use a garbage-collected object that has been unrooted")
258        })
259    }
260
261    /// Same as `RootedGcRefImpl::clone_gc_ref` but not associated with any
262    /// particular `T: GcRef`.
263    pub(crate) fn try_clone_gc_ref(&self, store: &mut AutoAssertNoGc<'_>) -> Result<VMGcRef> {
264        let gc_ref = self.try_gc_ref(store)?.unchecked_copy();
265        Ok(store.clone_gc_ref(&gc_ref))
266    }
267}
268
269/// This is a bit-packed version of
270///
271/// ```ignore
272/// enum {
273///     Lifo(usize),
274///     Manual(SlabId),
275/// }
276/// ```
277///
278/// where the high bit is the discriminant and the lower 31 bits are the
279/// payload.
280#[derive(Clone, Copy, PartialEq, Eq, Hash)]
281#[repr(transparent)]
282struct PackedIndex(u32);
283
284impl Debug for PackedIndex {
285    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
286        if let Some(index) = self.as_lifo() {
287            f.debug_tuple("PackedIndex::Lifo").field(&index).finish()
288        } else if let Some(id) = self.as_manual() {
289            f.debug_tuple("PackedIndex::Manual").field(&id).finish()
290        } else {
291            unreachable!()
292        }
293    }
294}
295
296impl PackedIndex {
297    const DISCRIMINANT_MASK: u32 = 0b1 << 31;
298    const LIFO_DISCRIMINANT: u32 = 0b0 << 31;
299    const MANUAL_DISCRIMINANT: u32 = 0b1 << 31;
300    const PAYLOAD_MASK: u32 = !Self::DISCRIMINANT_MASK;
301
302    fn new_lifo(index: usize) -> PackedIndex {
303        let index32 = u32::try_from(index).unwrap();
304        assert_eq!(index32 & Self::DISCRIMINANT_MASK, 0);
305        let packed = PackedIndex(Self::LIFO_DISCRIMINANT | index32);
306        debug_assert!(packed.is_lifo());
307        debug_assert_eq!(packed.as_lifo(), Some(index));
308        debug_assert!(!packed.is_manual());
309        debug_assert!(packed.as_manual().is_none());
310        packed
311    }
312
313    fn new_manual(id: SlabId) -> PackedIndex {
314        let raw = id.into_raw();
315        assert_eq!(raw & Self::DISCRIMINANT_MASK, 0);
316        let packed = PackedIndex(Self::MANUAL_DISCRIMINANT | raw);
317        debug_assert!(packed.is_manual());
318        debug_assert_eq!(packed.as_manual(), Some(id));
319        debug_assert!(!packed.is_lifo());
320        debug_assert!(packed.as_lifo().is_none());
321        packed
322    }
323
324    fn discriminant(&self) -> u32 {
325        self.0 & Self::DISCRIMINANT_MASK
326    }
327
328    fn is_lifo(&self) -> bool {
329        self.discriminant() == Self::LIFO_DISCRIMINANT
330    }
331
332    fn is_manual(&self) -> bool {
333        self.discriminant() == Self::MANUAL_DISCRIMINANT
334    }
335
336    fn payload(&self) -> u32 {
337        self.0 & Self::PAYLOAD_MASK
338    }
339
340    fn as_lifo(&self) -> Option<usize> {
341        if self.is_lifo() {
342            Some(usize::try_from(self.payload()).unwrap())
343        } else {
344            None
345        }
346    }
347
348    fn as_manual(&self) -> Option<SlabId> {
349        if self.is_manual() {
350            Some(SlabId::from_raw(self.payload()))
351        } else {
352            None
353        }
354    }
355}
356
357/// The set of all embedder-API GC roots in a single store/heap.
358#[derive(Debug, Default)]
359pub(crate) struct RootSet {
360    /// GC roots with arbitrary lifetime that are manually rooted and unrooted,
361    /// for use with `ManuallyRooted<T>`.
362    manually_rooted: Slab<VMGcRef>,
363
364    /// Strictly LIFO-ordered GC roots, for use with `RootScope` and
365    /// `Rooted<T>`.
366    lifo_roots: Vec<LifoRoot>,
367
368    /// Generation counter for entries to prevent ABA bugs with `RootScope` and
369    /// `Rooted<T>`.
370    lifo_generation: u32,
371}
372
373#[derive(Debug)]
374struct LifoRoot {
375    generation: u32,
376    gc_ref: VMGcRef,
377}
378
379impl RootSet {
380    pub(crate) fn trace_roots(&mut self, gc_roots_list: &mut GcRootsList) {
381        log::trace!("Begin trace user LIFO roots");
382        for root in &mut self.lifo_roots {
383            unsafe {
384                gc_roots_list.add_root((&mut root.gc_ref).into(), "user LIFO root");
385            }
386        }
387        log::trace!("End trace user LIFO roots");
388
389        log::trace!("Begin trace user manual roots");
390        for (_id, root) in self.manually_rooted.iter_mut() {
391            unsafe {
392                gc_roots_list.add_root(root.into(), "user manual root");
393            }
394        }
395        log::trace!("End trace user manual roots");
396    }
397
398    /// Enter a LIFO rooting scope.
399    ///
400    /// Returns an integer that should be passed unmodified to `exit_lifo_scope`
401    /// when the scope is finished.
402    ///
403    /// Calls to `{enter,exit}_lifo_scope` must happen in a strict LIFO order.
404    #[inline]
405    pub(crate) fn enter_lifo_scope(&self) -> usize {
406        self.lifo_roots.len()
407    }
408
409    /// Exit a LIFO rooting scope.
410    ///
411    /// The `scope` argument must be the result of the corresponding
412    /// `enter_lifo_scope` call.
413    ///
414    /// Calls to `{enter,exit}_lifo_scope` must happen in a strict LIFO order.
415    #[inline]
416    pub(crate) fn exit_lifo_scope(&mut self, gc_store: Option<&mut GcStore>, scope: usize) {
417        debug_assert!(self.lifo_roots.len() >= scope);
418
419        // If we actually have roots to unroot, call an out-of-line slow path.
420        if self.lifo_roots.len() > scope {
421            self.exit_lifo_scope_slow(gc_store, scope);
422        }
423    }
424
425    #[inline(never)]
426    #[cold]
427    fn exit_lifo_scope_slow(&mut self, mut gc_store: Option<&mut GcStore>, scope: usize) {
428        self.lifo_generation += 1;
429
430        // TODO: In the case where we have a tracing GC that doesn't need to
431        // drop barriers, this should really be:
432        //
433        //     self.lifo_roots.truncate(scope);
434
435        let mut lifo_roots = mem::take(&mut self.lifo_roots);
436        for root in lifo_roots.drain(scope..) {
437            // Only drop the GC reference if we actually have a GC store. How
438            // can we have a GC reference but not a GC store? If we've only
439            // created `i31refs`, we never force a GC store's allocation. This
440            // is fine because `i31ref`s never need drop barriers.
441            if let Some(gc_store) = &mut gc_store {
442                gc_store.drop_gc_ref(root.gc_ref);
443            }
444        }
445        self.lifo_roots = lifo_roots;
446    }
447
448    pub(crate) fn with_lifo_scope<S, T>(store: &mut S, f: impl FnOnce(&mut S) -> T) -> T
449    where
450        S: ?Sized + DerefMut<Target = StoreOpaque>,
451    {
452        let scope = store.gc_roots().enter_lifo_scope();
453        let ret = f(store);
454        store.exit_gc_lifo_scope(scope);
455        ret
456    }
457
458    pub(crate) fn push_lifo_root(&mut self, store_id: StoreId, gc_ref: VMGcRef) -> GcRootIndex {
459        let generation = self.lifo_generation;
460        let index = self.lifo_roots.len();
461        let index = PackedIndex::new_lifo(index);
462        self.lifo_roots.push(LifoRoot { generation, gc_ref });
463        GcRootIndex {
464            store_id,
465            generation,
466            index,
467        }
468    }
469}
470
471/// A scoped, rooted reference to a garbage-collected `T`.
472///
473/// A `Rooted<T>` is a strong handle to a garbage-collected `T`, preventing its
474/// referent (and anything else transitively referenced) from being collected by
475/// the GC during the scope within which this `Rooted<T>` was created.
476///
477/// When the context exits this `Rooted<T>`'s scope, the underlying GC object is
478/// automatically unrooted and any further attempts to use access the underlying
479/// object will return errors or otherwise fail.
480///
481/// `Rooted<T>` dereferences to its underlying `T`, allowing you to call `T`'s
482/// methods.
483///
484/// # Example
485///
486/// ```
487/// # use wasmtime::*;
488/// # fn _foo() -> Result<()> {
489/// let mut store = Store::<()>::default();
490///
491/// // Allocating a GC object returns a `Rooted<T>`.
492/// let hello: Rooted<ExternRef> = ExternRef::new(&mut store, "hello")?;
493///
494/// // Because `Rooted<T>` derefs to `T`, we can call `T` methods on a
495/// // `Rooted<T>`. For example, we can call the `ExternRef::data` method when we
496/// // have a `Rooted<ExternRef>`.
497/// let data = hello
498///     .data(&store)?
499///     .ok_or_else(|| Error::msg("externref has no host data"))?
500///     .downcast_ref::<&str>()
501///     .ok_or_else(|| Error::msg("not a str"))?;
502/// assert_eq!(*data, "hello");
503///
504/// // A `Rooted<T>` roots its underlying GC object for the duration of the
505/// // scope of the store/caller/context that was passed to the method that created
506/// // it. If we only want to keep a GC reference rooted and alive temporarily, we
507/// // can introduce new scopes with `RootScope`.
508/// {
509///     let mut scope = RootScope::new(&mut store);
510///
511///     // This `Rooted<T>` is automatically unrooted after `scope` is dropped,
512///     // allowing the collector to reclaim its GC object in the next GC.
513///     let scoped_ref = ExternRef::new(&mut scope, "goodbye");
514/// }
515///
516/// let module = Module::new(store.engine(), r#"
517///     (module
518///         (global (export "global") (mut externref) (ref.null extern))
519///         (table (export "table") 10 externref)
520///         (func (export "func") (param externref) (result externref)
521///             local.get 0
522///         )
523///     )
524/// "#)?;
525/// let instance = Instance::new(&mut store, &module, &[])?;
526///
527/// // GC references returned from calls into Wasm also return (optional, if the
528/// // Wasm type is nullable) `Rooted<T>`s.
529/// let result: Option<Rooted<_>> = instance
530///     .get_typed_func::<Option<Rooted<ExternRef>>, Option<Rooted<ExternRef>>>(&mut store, "func")?
531///     .call(&mut store, Some(hello))?;
532///
533/// // Similarly, getting a GC reference from a Wasm instance's exported global
534/// // or table yields a `Rooted<T>`.
535///
536/// let global = instance
537///     .get_global(&mut store, "global")
538///     .ok_or_else(|| Error::msg("missing `global` export"))?;
539/// let global_val = global.get(&mut store);
540/// let global_ref: Option<&Rooted<_>> = global_val
541///     .externref()
542///     .ok_or_else(|| Error::msg("not an externref"))?;
543///
544/// let table = instance.get_table(&mut store, "table").unwrap();
545/// let table_elem = table
546///     .get(&mut store, 3)
547///     .ok_or_else(|| Error::msg("table out of bounds"))?;
548/// let table_elem_ref: Option<&Rooted<_>> = table_elem
549///     .as_extern()
550///     .ok_or_else(|| Error::msg("not an externref"))?;
551/// # Ok(())
552/// # }
553/// ```
554///
555/// # Differences Between `Rooted<T>` and `ManuallyRooted<T>`
556///
557/// While `Rooted<T>` is automatically unrooted when its scope is exited, this
558/// means that `Rooted<T>` is only valid for strictly last-in-first-out (LIFO,
559/// aka stack order) lifetimes. This is in contrast to
560/// [`ManuallyRooted<T>`][crate::ManuallyRooted], which supports rooting GC
561/// objects for arbitrary lifetimes, but requires manual unrooting.
562///
563/// | Type                                         | Supported Lifetimes         | Unrooting |
564/// |----------------------------------------------|-----------------------------|-----------|
565/// | [`Rooted<T>`][crate::Rooted]                 | Strictly LIFO / stack order | Automatic |
566/// | [`ManuallyRooted<T>`][crate::ManuallyRooted] | Arbitrary                   | Manual    |
567///
568/// `Rooted<T>` should suffice for most use cases, and provides better
569/// ergonomics, but `ManuallyRooted<T>` exists as a fully-general escape hatch.
570///
571/// # Scopes
572///
573/// Wasmtime automatically creates two kinds of scopes:
574///
575/// 1. A [`Store`][crate::Store] is the outermost rooting scope. Creating a
576///    `Root<T>` directly inside a `Store` permanently roots the underlying
577///    object, similar to dropping a
578///    [`ManuallyRooted<T>`][crate::ManuallyRooted] without unrooting it.
579///
580/// 2. A [`Caller`][crate::Caller] provides a rooting scope for the duration of
581///    a call from Wasm into a host function. Any objects rooted in a `Caller`
582///    will be unrooted after the host function returns. Note that there can be
583///    nested `Caller` scopes in the case where Wasm calls a host function,
584///    creating the first `Caller` and its rooting scope , and then the host
585///    function calls a Wasm function which then calls another host function,
586///    creating a second `Caller` and a second rooting scope. This nesting can
587///    be arbitrarily deep.
588///
589/// Additionally, if you would like to define finer-grained rooting scopes,
590/// Wasmtime provides the [`RootScope`][crate::RootScope] type.
591///
592/// Scopes are always nested in a last-in-first-out (LIFO) order. An outer scope
593/// is never exited (and the `Rooted<T>`s defined within it are never
594/// automatically unrooted) while an inner scope is still active. All inner
595/// scopes are exited before their outer scopes.
596///
597/// The following diagram illustrates various rooting scopes over time, how they
598/// nest, and when their `Rooted<T>`s are automatically unrooted:
599///
600/// ```text
601/// ----- new Store
602///   |
603///   |
604///   | let a: Rooted<T> = ...;
605///   |
606///   |
607///   | ----- call into Wasm
608///   |   |
609///   |   |
610///   |   | ----- Wasm calls host function F
611///   |   |   |
612///   |   |   |
613///   |   |   | let b: Rooted<T> = ...;
614///   |   |   |
615///   |   |   |
616///   |   |   | ----- F calls into Wasm
617///   |   |   |   |
618///   |   |   |   |
619///   |   |   |   | ----- Wasm call host function G
620///   |   |   |   |   |
621///   |   |   |   |   |
622///   |   |   |   |   | let c: Rooted<T> = ...;
623///   |   |   |   |   |
624///   |   |   |   |   |
625///   |   |   |   | ----- return to Wasm from host function G (unroots `c`)
626///   |   |   |   |
627///   |   |   |   |
628///   |   |   | ----- Wasm returns to F
629///   |   |   |
630///   |   |   |
631///   |   | ----- return from host function F (unroots `b`)
632///   |   |
633///   |   |
634///   | ----- return from Wasm
635///   |
636///   |
637///   | ----- let scope1 = RootScope::new(...);
638///   |   |
639///   |   |
640///   |   | let d: Rooted<T> = ...;
641///   |   |
642///   |   |
643///   |   | ----- let scope2 = RootScope::new(...);
644///   |   |   |
645///   |   |   |
646///   |   |   | let e: Rooted<T> = ...;
647///   |   |   |
648///   |   |   |
649///   |   | ----- drop `scope2` (unroots `e`)
650///   |   |
651///   |   |
652///   | ----- drop `scope1` (unroots `d`)
653///   |
654///   |
655/// ----- drop Store (unroots `a`)
656/// ```
657///
658/// A `Rooted<T>` can be used successfully as long as it is still rooted so, in
659/// the above diagram, `d` is valid inside `scope2` because `scope2` is wholly
660/// contained within the scope `d` was rooted within (`scope1`).
661///
662/// See also the documentation for [`RootScope`][crate::RootScope].
663#[repr(transparent)]
664pub struct Rooted<T: GcRef> {
665    inner: GcRootIndex,
666    _phantom: marker::PhantomData<T>,
667}
668
669impl<T: GcRef> Clone for Rooted<T> {
670    fn clone(&self) -> Self {
671        Rooted {
672            inner: self.inner,
673            _phantom: marker::PhantomData,
674        }
675    }
676}
677
678impl<T: GcRef> Copy for Rooted<T> {}
679
680impl<T: GcRef> Debug for Rooted<T> {
681    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
682        let name = format!("Rooted<{}>", any::type_name::<T>());
683        f.debug_struct(&name).field("inner", &self.inner).finish()
684    }
685}
686
687impl<T: GcRef> RootedGcRefImpl<T> for Rooted<T> {
688    fn get_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Option<&'a VMGcRef> {
689        assert!(
690            self.comes_from_same_store(store),
691            "object used with wrong store"
692        );
693        let index = self.inner.index.as_lifo().unwrap();
694        let entry = store.gc_roots().lifo_roots.get(index)?;
695        if entry.generation == self.inner.generation {
696            Some(&entry.gc_ref)
697        } else {
698            None
699        }
700    }
701}
702
703impl<T: GcRef> Deref for Rooted<T> {
704    type Target = T;
705
706    fn deref(&self) -> &Self::Target {
707        T::transmute_ref(&self.inner)
708    }
709}
710
711impl<T: GcRef> Rooted<T> {
712    /// Push the given `VMGcRef` onto our LIFO root set.
713    ///
714    /// `gc_ref` should belong to `store`'s heap; failure to uphold this is
715    /// memory safe but will result in general failures down the line such as
716    /// panics or incorrect results.
717    ///
718    /// `gc_ref` should be a GC reference pointing to an instance of the GC type
719    /// that `T` represents. Failure to uphold this invariant is memory safe but
720    /// will result in general incorrectness such as panics and wrong results.
721    pub(crate) fn new(store: &mut AutoAssertNoGc<'_>, gc_ref: VMGcRef) -> Rooted<T> {
722        let id = store.id();
723        let roots = store.gc_roots_mut();
724        let inner = roots.push_lifo_root(id, gc_ref);
725        Rooted {
726            inner,
727            _phantom: marker::PhantomData,
728        }
729    }
730
731    /// Create a new `Rooted<T>` from a `GcRootIndex`.
732    ///
733    /// Note that `Rooted::from_gc_root_index(my_rooted.index)` is not
734    /// necessarily an identity function, as it allows changing the `T` type
735    /// parameter.
736    ///
737    /// The given index should be a LIFO index of a GC reference pointing to an
738    /// instance of the GC type that `T` represents. Failure to uphold this
739    /// invariant is memory safe but will result in general incorrectness such
740    /// as panics and wrong results.
741    pub(crate) fn from_gc_root_index(inner: GcRootIndex) -> Rooted<T> {
742        debug_assert!(inner.index.is_lifo());
743        Rooted {
744            inner,
745            _phantom: marker::PhantomData,
746        }
747    }
748
749    #[inline]
750    pub(crate) fn comes_from_same_store(&self, store: &StoreOpaque) -> bool {
751        debug_assert!(self.inner.index.is_lifo());
752        self.inner.comes_from_same_store(store)
753    }
754
755    /// Create a [`ManuallyRooted<T>`][crate::ManuallyRooted] holding onto the
756    /// same GC object as `self`.
757    ///
758    /// Returns `None` if `self` is used outside of its scope and has therefore
759    /// been unrooted.
760    ///
761    /// This does not unroot `self`, and `self` remains valid until its
762    /// associated scope is exited.
763    ///
764    /// # Panics
765    ///
766    /// Panics if this object is not associate with the given store.
767    ///
768    /// # Example
769    ///
770    /// ```
771    /// # use wasmtime::*;
772    /// # fn _foo() -> Result<()> {
773    /// let mut store = Store::<()>::default();
774    ///
775    /// let y: ManuallyRooted<_> = {
776    ///     // Create a nested rooting scope.
777    ///     let mut scope = RootScope::new(&mut store);
778    ///
779    ///     // `x` is only rooted within this nested scope.
780    ///     let x: Rooted<_> = ExternRef::new(&mut scope, "hello!")?;
781    ///
782    ///     // Extend `x`'s rooting past its scope's lifetime by converting it
783    ///     // to a `ManuallyRooted`.
784    ///     x.to_manually_rooted(&mut scope)?
785    /// };
786    ///
787    /// // Now we can still access the reference outside the scope it was
788    /// // originally defined within.
789    /// let data = y.data(&store)?.expect("should have host data");
790    /// let data = data.downcast_ref::<&str>().expect("host data should be str");
791    /// assert_eq!(*data, "hello!");
792    ///
793    /// // But we have to manually unroot `y`.
794    /// y.unroot(&mut store);
795    /// # Ok(())
796    /// # }
797    /// ```
798    pub fn to_manually_rooted(&self, mut store: impl AsContextMut) -> Result<ManuallyRooted<T>> {
799        self._to_manually_rooted(store.as_context_mut().0)
800    }
801
802    pub(crate) fn _to_manually_rooted(&self, store: &mut StoreOpaque) -> Result<ManuallyRooted<T>> {
803        let mut store = AutoAssertNoGc::new(store);
804        let gc_ref = self.try_clone_gc_ref(&mut store)?;
805        Ok(ManuallyRooted::new(&mut store, gc_ref))
806    }
807
808    /// Are these two `Rooted<T>`s the same GC root?
809    ///
810    /// Note that this function can return `false` even when `a` and `b` are
811    /// rooting the same underlying GC object, but the object was rooted
812    /// multiple times (for example in different scopes). Use
813    /// [`Rooted::ref_eq`][crate::Rooted::ref_eq] to test whether these are
814    /// references to the same underlying GC object or not.
815    ///
816    /// # Example
817    ///
818    /// ```
819    /// # use wasmtime::*;
820    /// # fn foo() -> Result<()> {
821    /// let mut store = Store::<()>::default();
822    ///
823    /// let a = ExternRef::new(&mut store, "hello")?;
824    /// let b = a;
825    ///
826    /// // `a` and `b` are the same GC root.
827    /// assert!(Rooted::rooted_eq(a, b));
828    ///
829    /// {
830    ///     let mut scope = RootScope::new(&mut store);
831    ///
832    ///     // `c` is a different GC root, in a different scope, even though it
833    ///     // is rooting the same object.
834    ///     let c = a.to_manually_rooted(&mut scope)?.into_rooted(&mut scope);
835    ///     assert!(!Rooted::rooted_eq(a, c));
836    /// }
837    ///
838    /// let x = ExternRef::new(&mut store, "goodbye")?;
839    ///
840    /// // `a` and `x` are different GC roots, rooting different objects.
841    /// assert!(!Rooted::rooted_eq(a, x));
842    /// # Ok(())
843    /// # }
844    /// ```
845    pub fn rooted_eq(a: Self, b: Self) -> bool {
846        a.inner == b.inner
847    }
848
849    /// Are these two GC roots referencing the same underlying GC object?
850    ///
851    /// This function will return `true` even when `a` and `b` are different GC
852    /// roots (for example because they were rooted in different scopes) if they
853    /// are rooting the same underlying GC object. To only test whether they are
854    /// the same GC root, and not whether they are rooting the same GC object,
855    /// use [`Rooted::rooted_eq`][crate::Rooted::rooted_eq].
856    ///
857    /// Returns an error if either `a` or `b` has been unrooted, for example
858    /// because the scope it was rooted within has been exited.
859    ///
860    /// Because this method takes any `impl RootedGcRef<T>` arguments, it can be
861    /// used to compare, for example, a `Rooted<T>` and a `ManuallyRooted<T>`.
862    ///
863    /// # Panics
864    ///
865    /// Panics if either `a` or `b` is not associated with the given `store`.
866    ///
867    /// # Example
868    ///
869    /// ```
870    /// # use wasmtime::*;
871    /// # fn foo() -> Result<()> {
872    /// let mut store = Store::<()>::default();
873    ///
874    /// let a = ExternRef::new(&mut store, "hello")?;
875    /// let b = a;
876    ///
877    /// // `a` and `b` are rooting the same object.
878    /// assert!(Rooted::ref_eq(&store, &a, &b)?);
879    ///
880    /// {
881    ///     let mut scope = RootScope::new(&mut store);
882    ///
883    ///     // `c` is a different GC root, in a different scope, but still
884    ///     // rooting the same object.
885    ///     let c = a.to_manually_rooted(&mut scope)?.into_rooted(&mut scope);
886    ///     assert!(!Rooted::ref_eq(&scope, &a, &c)?);
887    /// }
888    ///
889    /// let x = ExternRef::new(&mut store, "goodbye")?;
890    ///
891    /// // `a` and `x` are rooting different objects.
892    /// assert!(!Rooted::ref_eq(&store, &a, &x)?);
893    ///
894    /// // You can also compare `Rooted<T>`s and `ManuallyRooted<T>`s with this
895    /// // function.
896    /// let d = a.to_manually_rooted(&mut store)?;
897    /// assert!(Rooted::ref_eq(&store, &a, &d)?);
898    ///
899    /// d.unroot(&mut store);
900    /// # Ok(())
901    /// # }
902    /// ```
903    pub fn ref_eq(
904        store: impl AsContext,
905        a: &impl RootedGcRef<T>,
906        b: &impl RootedGcRef<T>,
907    ) -> Result<bool> {
908        let store = store.as_context().0;
909        Self::_ref_eq(store, a, b)
910    }
911
912    pub(crate) fn _ref_eq(
913        store: &StoreOpaque,
914        a: &impl RootedGcRef<T>,
915        b: &impl RootedGcRef<T>,
916    ) -> Result<bool> {
917        let a = a.try_gc_ref(store)?;
918        let b = b.try_gc_ref(store)?;
919        Ok(a == b)
920    }
921
922    /// Hash this root.
923    ///
924    /// Note that, similar to `Rooted::rooted_eq`, this only operates on the
925    /// root and *not* the underlying GC reference. That means that two
926    /// different rootings of the same object will hash to different values
927    /// (modulo hash collisions). If this is undesirable, use the
928    /// [`ref_hash`][crate::Rooted::ref_hash] method instead.
929    pub fn rooted_hash<H>(&self, state: &mut H)
930    where
931        H: Hasher,
932    {
933        self.inner.hash(state);
934    }
935
936    /// Hash the underlying rooted object reference.
937    ///
938    /// Note that, similar to `Rooted::ref_eq`, and operates on the underlying
939    /// rooted GC object reference, not the root. That means that two
940    /// *different* rootings of the same object will hash to the *same*
941    /// value. If this is undesirable, use the
942    /// [`rooted_hash`][crate::Rooted::rooted_hash] method instead.
943    pub fn ref_hash<H>(&self, store: impl AsContext, state: &mut H) -> Result<()>
944    where
945        H: Hasher,
946    {
947        let gc_ref = self.try_gc_ref(store.as_context().0)?;
948        gc_ref.hash(state);
949        Ok(())
950    }
951
952    /// Cast `self` to a `Rooted<U>`.
953    ///
954    /// It is the caller's responsibility to ensure that `self` is actually a
955    /// `U`. Failure to uphold this invariant will be memory safe but will
956    /// result in general incorrectness such as panics and wrong results.
957    pub(crate) fn unchecked_cast<U: GcRef>(self) -> Rooted<U> {
958        Rooted::from_gc_root_index(self.inner)
959    }
960
961    /// Common implementation of the `WasmTy::store` trait method for all
962    /// `Rooted<T>`s.
963    pub(super) fn wasm_ty_store(
964        self,
965        store: &mut AutoAssertNoGc<'_>,
966        ptr: &mut MaybeUninit<ValRaw>,
967        val_raw: impl Fn(u32) -> ValRaw,
968    ) -> Result<()> {
969        let gc_ref = self.inner.try_clone_gc_ref(store)?;
970
971        let raw = match store.optional_gc_store_mut() {
972            Some(s) => s.expose_gc_ref_to_wasm(gc_ref),
973            None => {
974                // NB: do not force the allocation of a GC heap just because the
975                // program is using `i31ref`s.
976                debug_assert!(gc_ref.is_i31());
977                gc_ref.as_raw_non_zero_u32()
978            }
979        };
980
981        ptr.write(val_raw(raw.get()));
982        Ok(())
983    }
984
985    /// Common implementation of the `WasmTy::load` trait method for all
986    /// `Rooted<T>`s.
987    pub(super) fn wasm_ty_load(
988        store: &mut AutoAssertNoGc<'_>,
989        raw_gc_ref: u32,
990        from_cloned_gc_ref: impl Fn(&mut AutoAssertNoGc<'_>, VMGcRef) -> Self,
991    ) -> Self {
992        debug_assert_ne!(raw_gc_ref, 0);
993        let gc_ref = VMGcRef::from_raw_u32(raw_gc_ref).expect("non-null");
994
995        let gc_ref = match store.optional_gc_store_mut() {
996            Some(s) => s.clone_gc_ref(&gc_ref),
997            None => {
998                // NB: do not force the allocation of a GC heap just because the
999                // program is using `i31ref`s.
1000                debug_assert!(gc_ref.is_i31());
1001                gc_ref.unchecked_copy()
1002            }
1003        };
1004
1005        from_cloned_gc_ref(store, gc_ref)
1006    }
1007
1008    /// Common implementation of the `WasmTy::store` trait method for all
1009    /// `Option<Rooted<T>>`s.
1010    pub(super) fn wasm_ty_option_store(
1011        me: Option<Self>,
1012        store: &mut AutoAssertNoGc<'_>,
1013        ptr: &mut MaybeUninit<ValRaw>,
1014        val_raw: impl Fn(u32) -> ValRaw,
1015    ) -> Result<()> {
1016        match me {
1017            Some(me) => me.wasm_ty_store(store, ptr, val_raw),
1018            None => {
1019                ptr.write(val_raw(0));
1020                Ok(())
1021            }
1022        }
1023    }
1024
1025    /// Common implementation of the `WasmTy::load` trait method for all
1026    /// `Option<Rooted<T>>`s.
1027    pub(super) fn wasm_ty_option_load(
1028        store: &mut AutoAssertNoGc<'_>,
1029        raw_gc_ref: u32,
1030        from_cloned_gc_ref: impl Fn(&mut AutoAssertNoGc<'_>, VMGcRef) -> Self,
1031    ) -> Option<Self> {
1032        let gc_ref = VMGcRef::from_raw_u32(raw_gc_ref)?;
1033        let gc_ref = store.clone_gc_ref(&gc_ref);
1034        Some(from_cloned_gc_ref(store, gc_ref))
1035    }
1036}
1037
1038/// Nested rooting scopes.
1039///
1040/// `RootScope` allows the creation or nested rooting scopes for use with
1041/// [`Rooted<T>`][crate::Rooted]. This allows for fine-grained control over how
1042/// long a set of [`Rooted<T>`][crate::Rooted]s are strongly held alive, giving
1043/// gives you the tools necessary to avoid holding onto GC objects longer than
1044/// necessary. `Rooted<T>`s created within a `RootScope` are automatically
1045/// unrooted when the `RootScope` is dropped. For more details on
1046/// [`Rooted<T>`][crate::Rooted] lifetimes and their interaction with rooting
1047/// scopes, see [`Rooted<T>`][crate::Rooted]'s documentation.
1048///
1049/// A `RootScope<C>` wraps a `C: AsContextMut` (that is, anything that
1050/// represents exclusive access to a [`Store`][crate::Store]) and in turn
1051/// implements [`AsContext`][crate::AsContext] and
1052/// [`AsContextMut`][crate::AsContextMut] in terms of its underlying
1053/// `C`. Therefore, `RootScope<C>` can be used anywhere you would use the
1054/// underlying `C`, for example in the [`Global::get`][crate::Global::get]
1055/// method. Any `Rooted<T>`s created by a method that a `RootScope<C>` was
1056/// passed as context to are tied to the `RootScope<C>`'s scope and
1057/// automatically unrooted when the scope is dropped.
1058///
1059/// # Example
1060///
1061/// ```
1062/// # use wasmtime::*;
1063/// # fn _foo() -> Result<()> {
1064/// let mut store = Store::<()>::default();
1065///
1066/// let a: Rooted<_>;
1067/// let b: Rooted<_>;
1068/// let c: Rooted<_>;
1069///
1070/// // Root `a` in the store's scope. It will be rooted for the duration of the
1071/// // store's lifetime.
1072/// a = ExternRef::new(&mut store, 42)?;
1073///
1074/// // `a` is rooted, so we can access its data successfully.
1075/// assert!(a.data(&store).is_ok());
1076///
1077/// {
1078///     let mut scope1 = RootScope::new(&mut store);
1079///
1080///     // Root `b` in `scope1`.
1081///     b = ExternRef::new(&mut scope1, 36)?;
1082///
1083///     // Both `a` and `b` are rooted.
1084///     assert!(a.data(&scope1).is_ok());
1085///     assert!(b.data(&scope1).is_ok());
1086///
1087///     {
1088///         let mut scope2 = RootScope::new(&mut scope1);
1089///
1090///         // Root `c` in `scope2`.
1091///         c = ExternRef::new(&mut scope2, 36)?;
1092///
1093///         // All of `a`, `b`, and `c` are rooted.
1094///         assert!(a.data(&scope2).is_ok());
1095///         assert!(b.data(&scope2).is_ok());
1096///         assert!(c.data(&scope2).is_ok());
1097///
1098///         // Drop `scope2`.
1099///     }
1100///
1101///     // Now `a` and `b` are still rooted, but `c` was unrooted when we dropped
1102///     // `scope2`.
1103///     assert!(a.data(&scope1).is_ok());
1104///     assert!(b.data(&scope1).is_ok());
1105///     assert!(c.data(&scope1).is_err());
1106///
1107///     // Drop `scope1`.
1108/// }
1109///
1110/// // And now only `a` is still rooted. Both `b` and `c` were unrooted when we
1111/// // dropped their respective rooting scopes.
1112/// assert!(a.data(&store).is_ok());
1113/// assert!(b.data(&store).is_err());
1114/// assert!(c.data(&store).is_err());
1115/// # Ok(())
1116/// # }
1117/// ```
1118pub struct RootScope<C>
1119where
1120    C: AsContextMut,
1121{
1122    store: C,
1123    scope: usize,
1124}
1125
1126impl<C> Drop for RootScope<C>
1127where
1128    C: AsContextMut,
1129{
1130    fn drop(&mut self) {
1131        self.store.as_context_mut().0.exit_gc_lifo_scope(self.scope);
1132    }
1133}
1134
1135impl<C> RootScope<C>
1136where
1137    C: AsContextMut,
1138{
1139    // NB: we MUST NOT expose a method like
1140    //
1141    //     pub fn store(&mut self) -> &mut Store { ... }
1142    //
1143    // because callers could do treacherous things like
1144    //
1145    //     let scope1 = RootScope::new(&mut store1);
1146    //     let scope2 = RootScope::new(&mut store2);
1147    //     std::mem::swap(scope1.store(), scope2.store());
1148    //
1149    // and then we would start truncate the store's GC root set's LIFO roots to
1150    // the wrong lengths.
1151    //
1152    // Instead, we just implement `AsContext[Mut]` for `RootScope`.
1153
1154    /// Construct a new scope for rooting GC objects.
1155    ///
1156    /// # Example
1157    ///
1158    /// ```
1159    /// # use wasmtime::*;
1160    /// let mut store = Store::<()>::default();
1161    ///
1162    /// {
1163    ///     let mut scope = RootScope::new(&mut store);
1164    ///
1165    ///     // Temporarily root GC objects in this nested rooting scope...
1166    /// }
1167    /// ```
1168    pub fn new(store: C) -> Self {
1169        let scope = store.as_context().0.gc_roots().enter_lifo_scope();
1170        RootScope { store, scope }
1171    }
1172
1173    fn gc_roots(&mut self) -> &mut RootSet {
1174        self.store.as_context_mut().0.gc_roots_mut()
1175    }
1176
1177    fn lifo_roots(&mut self) -> &mut Vec<LifoRoot> {
1178        &mut self.gc_roots().lifo_roots
1179    }
1180
1181    /// Reserve enough capacity for `additional` GC roots in this scope.
1182    ///
1183    /// # Example
1184    ///
1185    /// ```
1186    /// # use wasmtime::*;
1187    /// let mut store = Store::<()>::default();
1188    ///
1189    /// {
1190    ///     let mut scope = RootScope::new(&mut store);
1191    ///
1192    ///     // Ensure we have enough storage pre-allocated to root five GC
1193    ///     // references inside this scope without any underlying reallocation.
1194    ///     scope.reserve(5);
1195    ///
1196    ///     // ...
1197    /// }
1198    /// ```
1199    pub fn reserve(&mut self, additional: usize) {
1200        self.lifo_roots().reserve(additional);
1201    }
1202}
1203
1204impl<T> AsContext for RootScope<T>
1205where
1206    T: AsContextMut,
1207{
1208    type Data = T::Data;
1209
1210    fn as_context(&self) -> crate::StoreContext<'_, Self::Data> {
1211        self.store.as_context()
1212    }
1213}
1214
1215impl<T> AsContextMut for RootScope<T>
1216where
1217    T: AsContextMut,
1218{
1219    fn as_context_mut(&mut self) -> crate::StoreContextMut<'_, Self::Data> {
1220        self.store.as_context_mut()
1221    }
1222}
1223
1224/// Internal version of `RootScope` that only wraps a `&mut StoreOpaque` rather
1225/// than a whole `impl AsContextMut<Data = T>`.
1226pub(crate) struct OpaqueRootScope<S>
1227where
1228    S: AsStoreOpaque,
1229{
1230    store: S,
1231    scope: usize,
1232}
1233
1234impl<S> Drop for OpaqueRootScope<S>
1235where
1236    S: AsStoreOpaque,
1237{
1238    fn drop(&mut self) {
1239        self.store.as_store_opaque().exit_gc_lifo_scope(self.scope);
1240    }
1241}
1242
1243impl<S> Deref for OpaqueRootScope<S>
1244where
1245    S: AsStoreOpaque,
1246{
1247    type Target = S;
1248
1249    fn deref(&self) -> &Self::Target {
1250        &self.store
1251    }
1252}
1253
1254// XXX: Don't use this `DerefMut` implementation to `mem::{swap,replace}` or
1255// etc... the underlying `StoreOpaque` in a `OpaqueRootScope`! That will result
1256// in truncating the store's GC root set's LIFO roots to the wrong length.
1257//
1258// We don't implement `DerefMut` for `RootScope` for exactly this reason, but
1259// allow it for `OpaqueRootScope` because it is only Wasmtime-internal and not
1260// publicly exported.
1261impl<S> DerefMut for OpaqueRootScope<S>
1262where
1263    S: AsStoreOpaque,
1264{
1265    fn deref_mut(&mut self) -> &mut Self::Target {
1266        &mut self.store
1267    }
1268}
1269
1270impl<S> OpaqueRootScope<S>
1271where
1272    S: AsStoreOpaque,
1273{
1274    pub(crate) fn new(mut store: S) -> Self {
1275        let scope = store.as_store_opaque().gc_roots().enter_lifo_scope();
1276        OpaqueRootScope { store, scope }
1277    }
1278}
1279
1280/// A rooted reference to a garbage-collected `T` with arbitrary lifetime.
1281///
1282/// A `ManuallyRooted<T>` is a strong handle to a garbage-collected `T`,
1283/// preventing its referent (and anything else transitively referenced) from
1284/// being collected by the GC until [`unroot`][crate::ManuallyRooted::unroot] is
1285/// explicitly called.
1286///
1287/// The primary way to create a `ManuallyRooted<T>` is to promote a temporary
1288/// `Rooted<T>` into a `ManuallyRooted<T>` via its
1289/// [`to_manually_rooted`][crate::Rooted::to_manually_rooted] method.
1290///
1291/// `ManuallyRooted<T>` dereferences to its underlying `T`, allowing you to call
1292/// `T`'s methods.
1293///
1294/// # Example
1295///
1296/// ```
1297/// # use wasmtime::*;
1298/// # fn _foo() -> Result<()> {
1299/// let mut store = Store::<Option<ManuallyRooted<ExternRef>>>::default();
1300///
1301/// // Create our `ManuallyRooted` in a nested scope to avoid rooting it for
1302/// // the duration of the store's lifetime.
1303/// let x = {
1304///     let mut scope = RootScope::new(&mut store);
1305///     let x = ExternRef::new(&mut scope, 1234)?;
1306///     x.to_manually_rooted(&mut scope)?
1307/// };
1308///
1309/// // Place `x` into our store.
1310/// *store.data_mut() = Some(x);
1311///
1312/// // Do a bunch stuff that may or may not access, replace, or take `x`...
1313///
1314/// // At any time, in any arbitrary scope, we can remove `x` from the store
1315/// // and unroot it:
1316/// if let Some(x) = store.data_mut().take() {
1317///     x.unroot(&mut store);
1318/// }
1319/// # Ok(())
1320/// # }
1321/// ```
1322///
1323/// # Differences Between `ManuallyRooted<T>` and `Rooted<T>`
1324///
1325/// While `ManuallyRooted<T>` can have arbitrary lifetimes, it requires manual
1326/// unrooting. This is in contrast to [`Rooted<T>`][crate::Rooted] which is
1327/// restricted to strictly last-in-first-out (LIFO, aka stack order) lifetimes,
1328/// but comes with automatic unrooting.
1329///
1330/// | Type                                         | Supported Lifetimes         | Unrooting |
1331/// |----------------------------------------------|-----------------------------|-----------|
1332/// | [`Rooted<T>`][crate::Rooted]                 | Strictly LIFO / stack order | Automatic |
1333/// | [`ManuallyRooted<T>`][crate::ManuallyRooted] | Arbitrary                   | Manual    |
1334///
1335/// `Rooted<T>` should suffice for most use cases, and provides better
1336/// ergonomics, but `ManuallyRooted<T>` exists as a fully-general escape hatch.
1337///
1338/// # Manual Unrooting
1339///
1340/// Failure to explicitly call [`unroot`][crate::ManuallyRooted::unroot] (or
1341/// another method that consumes `self` and unroots the reference, such as
1342/// [`into_rooted`][crate::ManuallyRooted::into_rooted]) will leak the
1343/// underlying GC object, preventing it from being garbage collected until its
1344/// owning [`Store`][crate::Store] is dropped. That means all of the following
1345/// will result in permanently rooting the underlying GC object:
1346///
1347/// * Implicitly dropping a `ManuallyRooted<T>`:
1348///
1349///   ```no_run
1350///   # use wasmtime::*;
1351///   # let get_manually_rooted = || -> ManuallyRooted<ExternRef> { todo!() };
1352///   {
1353///       let perma_root: ManuallyRooted<_> = get_manually_rooted();
1354///
1355///       // `perma_root` is implicitly dropped at the end of its scope,
1356///       // permanently rooting/leaking its referent.
1357///   }
1358///   ```
1359///
1360/// * Explicitly dropping a `ManuallyRooted<T>`: `drop(my_manually_rooted)`.
1361///
1362/// * Forgetting a `ManuallyRooted<T>`: `std::mem::forget(my_manually_rooted)`.
1363///
1364/// * Inserting a `ManuallyRooted<T>` into a `std::sync::Arc` or `std::rc::Rc`
1365///   cycle.
1366///
1367/// * Etc...
1368///
1369/// Wasmtime does *not* assert that a `ManuallyRooted<T>` is unrooted on `Drop`,
1370/// or otherwise raise a panic, log a warning, or etc... on failure to manually
1371/// unroot. Sometimes leaking is intentional and desirable, particularly when
1372/// dealing with short-lived [`Store`][crate::Store]s where unrooting would just
1373/// be busy work since the whole store is about to be dropped.
1374#[repr(transparent)] // NB: the C API relies on this
1375pub struct ManuallyRooted<T>
1376where
1377    T: GcRef,
1378{
1379    inner: GcRootIndex,
1380    _phantom: marker::PhantomData<T>,
1381}
1382
1383const _: () = {
1384    use crate::{AnyRef, ExternRef};
1385
1386    // NB: these match the C API which should also be updated if this changes
1387    assert!(mem::size_of::<ManuallyRooted<AnyRef>>() == 16);
1388    assert!(mem::align_of::<ManuallyRooted<AnyRef>>() == mem::align_of::<u64>());
1389    assert!(mem::size_of::<ManuallyRooted<ExternRef>>() == 16);
1390    assert!(mem::align_of::<ManuallyRooted<ExternRef>>() == mem::align_of::<u64>());
1391};
1392
1393impl<T: GcRef> Debug for ManuallyRooted<T> {
1394    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1395        let name = format!("ManuallyRooted<{}>", any::type_name::<T>());
1396        f.debug_struct(&name).field("inner", &self.inner).finish()
1397    }
1398}
1399
1400impl<T: GcRef> Deref for ManuallyRooted<T> {
1401    type Target = T;
1402
1403    fn deref(&self) -> &Self::Target {
1404        T::transmute_ref(&self.inner)
1405    }
1406}
1407
1408impl<T> ManuallyRooted<T>
1409where
1410    T: GcRef,
1411{
1412    /// Construct a new manually-rooted GC root.
1413    ///
1414    /// `gc_ref` should belong to `store`'s heap; failure to uphold this is
1415    /// memory safe but will result in general failures down the line such as
1416    /// panics or incorrect results.
1417    ///
1418    /// `gc_ref` should be a GC reference pointing to an instance of the GC type
1419    /// that `T` represents. Failure to uphold this invariant is memory safe but
1420    /// will result in general incorrectness such as panics and wrong results.
1421    pub(crate) fn new(store: &mut AutoAssertNoGc<'_>, gc_ref: VMGcRef) -> Self {
1422        let id = store.gc_roots_mut().manually_rooted.alloc(gc_ref);
1423        ManuallyRooted {
1424            inner: GcRootIndex {
1425                store_id: store.id(),
1426                generation: 0,
1427                index: PackedIndex::new_manual(id),
1428            },
1429            _phantom: marker::PhantomData,
1430        }
1431    }
1432
1433    #[inline]
1434    pub(crate) fn comes_from_same_store(&self, store: &StoreOpaque) -> bool {
1435        debug_assert!(self.inner.index.is_manual());
1436        self.inner.comes_from_same_store(store)
1437    }
1438
1439    /// Clone this `ManuallyRooted`.
1440    ///
1441    /// Does not consume or unroot `self`: both `self` and the new
1442    /// `ManuallyRooted` return value will need to be manually unrooted.
1443    ///
1444    /// # Panics
1445    ///
1446    /// Panics if `self` is not associated with the given `store`.
1447    ///
1448    /// # Example
1449    ///
1450    /// ```
1451    /// # use wasmtime::*;
1452    /// # fn _foo() -> Result<()> {
1453    /// let mut store = Store::<Vec<ManuallyRooted<ExternRef>>>::default();
1454    ///
1455    /// // Create our `ManuallyRooted` in a nested scope to avoid rooting it for
1456    /// // the duration of the store's lifetime.
1457    /// let x = {
1458    ///     let mut scope = RootScope::new(&mut store);
1459    ///     let x = ExternRef::new(&mut scope, 1234)?;
1460    ///     x.to_manually_rooted(&mut scope)?
1461    /// };
1462    ///
1463    /// // Push five clones of `x` into our store.
1464    /// for _ in 0..5 {
1465    ///     let x_clone = x.clone(&mut store);
1466    ///     store.data_mut().push(x_clone);
1467    /// }
1468    /// # Ok(())
1469    /// # }
1470    /// ```
1471    pub fn clone(&self, mut store: impl AsContextMut) -> Self {
1472        self._clone(store.as_context_mut().0)
1473    }
1474
1475    pub(crate) fn _clone(&self, store: &mut StoreOpaque) -> Self {
1476        let mut store = AutoAssertNoGc::new(store);
1477        let gc_ref = self
1478            .clone_gc_ref(&mut store)
1479            .expect("ManuallyRooted always has a gc ref");
1480        Self::new(&mut store, gc_ref)
1481    }
1482
1483    /// Unroot this GC object.
1484    ///
1485    /// Failure to call this method will result in the GC object, and anything
1486    /// it transitively references, being kept alive (aka "leaking") for the
1487    /// entirety of the store's lifetime.
1488    ///
1489    /// See the type-level docs for example usage.
1490    pub fn unroot(self, mut store: impl AsContextMut) {
1491        self._unroot(store.as_context_mut().0)
1492    }
1493
1494    pub(crate) fn _unroot(self, store: &mut StoreOpaque) {
1495        assert!(
1496            self.comes_from_same_store(store),
1497            "object used with wrong store"
1498        );
1499
1500        let mut store = AutoAssertNoGc::new(store);
1501        let id = self.inner.index.as_manual().unwrap();
1502        let roots = store.gc_roots_mut();
1503        let gc_ref = roots.manually_rooted.dealloc(id);
1504        store.unwrap_gc_store_mut().drop_gc_ref(gc_ref);
1505    }
1506
1507    /// Clone this `ManuallyRooted<T>` into a `Rooted<T>`.
1508    ///
1509    /// This operation does not consume or unroot this `ManuallyRooted<T>`.
1510    ///
1511    /// The underlying GC object is re-rooted in the given context's scope. The
1512    /// resulting `Rooted<T>` is only valid during the given context's
1513    /// scope. See the [`Rooted<T>`][crate::Rooted] documentation for more
1514    /// details on rooting scopes.
1515    ///
1516    /// This operation does not consume or unroot this `ManuallyRooted<T>`.
1517    ///
1518    /// # Panics
1519    ///
1520    /// Panics if this object is not associated with the given context's store.
1521    ///
1522    /// # Example
1523    ///
1524    /// ```
1525    /// # use wasmtime::*;
1526    /// # fn _foo() -> Result<()> {
1527    /// let mut store = Store::<()>::default();
1528    ///
1529    /// let root1: Rooted<_>;
1530    ///
1531    /// let manual = {
1532    ///     let mut scope = RootScope::new(&mut store);
1533    ///     root1 = ExternRef::new(&mut scope, 1234)?;
1534    ///     root1.to_manually_rooted(&mut scope)?
1535    /// };
1536    ///
1537    /// // `root1` is no longer accessible because it was unrooted when `scope`
1538    /// // was dropped.
1539    /// assert!(root1.data(&store).is_err());
1540    ///
1541    /// // But we can re-root `manual` into this scope.
1542    /// let root2 = manual.to_rooted(&mut store);
1543    /// assert!(root2.data(&store).is_ok());
1544    ///
1545    /// // And we also still have access to `manual` and we still have to
1546    /// // manually unroot it.
1547    /// assert!(manual.data(&store).is_ok());
1548    /// manual.unroot(&mut store);
1549    /// # Ok(())
1550    /// # }
1551    /// ```
1552    pub fn to_rooted(&self, mut context: impl AsContextMut) -> Rooted<T> {
1553        self._to_rooted(context.as_context_mut().0)
1554    }
1555
1556    pub(crate) fn _to_rooted(&self, store: &mut StoreOpaque) -> Rooted<T> {
1557        assert!(
1558            self.comes_from_same_store(store),
1559            "object used with wrong store"
1560        );
1561        let mut store = AutoAssertNoGc::new(store);
1562        let gc_ref = self.clone_gc_ref(&mut store).unwrap();
1563        Rooted::new(&mut store, gc_ref)
1564    }
1565
1566    /// Convert this `ManuallyRooted<T>` into a `Rooted<T>`.
1567    ///
1568    /// The underlying GC object is re-rooted in the given context's scope. The
1569    /// resulting `Rooted<T>` is only valid during the given context's
1570    /// scope. See the [`Rooted<T>`][crate::Rooted] documentation for more
1571    /// details on rooting scopes.
1572    ///
1573    /// This operation consumes and unroots this `ManuallyRooted<T>`.
1574    ///
1575    /// # Panics
1576    ///
1577    /// Panics if this object is not associate with the given context's store.
1578    ///
1579    /// # Example
1580    ///
1581    /// ```
1582    /// # use wasmtime::*;
1583    /// # fn _foo() -> Result<()> {
1584    /// let mut store = Store::<()>::default();
1585    ///
1586    /// let root1: Rooted<_>;
1587    ///
1588    /// let manual = {
1589    ///     let mut scope = RootScope::new(&mut store);
1590    ///     root1 = ExternRef::new(&mut scope, 1234)?;
1591    ///     root1.to_manually_rooted(&mut scope)?
1592    /// };
1593    ///
1594    /// // `root1` is no longer accessible because it was unrooted when `scope`
1595    /// // was dropped.
1596    /// assert!(root1.data(&store).is_err());
1597    ///
1598    /// // But we can re-root `manual` into this scope.
1599    /// let root2 = manual.into_rooted(&mut store);
1600    /// assert!(root2.data(&store).is_ok());
1601    ///
1602    /// // `manual` was consumed by the `into_rooted` call, and we no longer
1603    /// // have access to it, nor need to manually unroot it.
1604    /// # Ok(())
1605    /// # }
1606    /// ```
1607    pub fn into_rooted(self, mut context: impl AsContextMut) -> Rooted<T> {
1608        self._into_rooted(context.as_context_mut().0)
1609    }
1610
1611    pub(crate) fn _into_rooted(self, store: &mut StoreOpaque) -> Rooted<T> {
1612        assert!(
1613            self.comes_from_same_store(store),
1614            "object used with wrong store"
1615        );
1616        let rooted = self._to_rooted(store);
1617        self._unroot(store);
1618        rooted
1619    }
1620
1621    /// Are these two GC roots referencing the same underlying GC object?
1622    ///
1623    /// This function will return `true` even when `a` and `b` are different GC
1624    /// roots (for example because they were rooted in different scopes) if they
1625    /// are rooting the same underlying GC object.
1626    ///
1627    /// Because this method takes any `impl RootedGcRef<T>` arguments, it can be
1628    /// used to compare, for example, a `Rooted<T>` and a `ManuallyRooted<T>`.
1629    ///
1630    /// # Panics
1631    ///
1632    /// Panics if either `a` or `b` is not associated with the given `store`.
1633    ///
1634    /// # Example
1635    ///
1636    /// ```
1637    /// # use wasmtime::*;
1638    /// # fn foo() -> Result<()> {
1639    /// let mut store = Store::<()>::default();
1640    ///
1641    /// let a;
1642    /// let b;
1643    /// let x;
1644    ///
1645    /// {
1646    ///     let mut scope = RootScope::new(&mut store);
1647    ///
1648    ///     a = ExternRef::new(&mut scope, "hello")?.to_manually_rooted(&mut scope)?;
1649    ///     b = a.clone(&mut scope);
1650    ///
1651    ///     // `a` and `b` are rooting the same object.
1652    ///     assert!(ManuallyRooted::ref_eq(&scope, &a, &b)?);
1653    ///
1654    ///     // `c` is a different GC root, is in a different scope, and is a
1655    ///     // `Rooted<T>` instead of a `ManuallyRooted<T>`, but is still rooting
1656    ///     // the same object.
1657    ///     let c = a.to_rooted(&mut scope);
1658    ///     assert!(ManuallyRooted::ref_eq(&scope, &a, &c)?);
1659    ///
1660    ///     x = ExternRef::new(&mut scope, "goodbye")?.to_manually_rooted(&mut scope)?;
1661    ///
1662    ///     // `a` and `x` are rooting different objects.
1663    ///     assert!(!ManuallyRooted::ref_eq(&scope, &a, &x)?);
1664    /// }
1665    ///
1666    /// // Unroot our manually-rooted GC references.
1667    /// a.unroot(&mut store);
1668    /// b.unroot(&mut store);
1669    /// x.unroot(&mut store);
1670    /// # Ok(())
1671    /// # }
1672    /// ```
1673    pub fn ref_eq(
1674        store: impl AsContext,
1675        a: &impl RootedGcRef<T>,
1676        b: &impl RootedGcRef<T>,
1677    ) -> Result<bool> {
1678        Rooted::ref_eq(store, a, b)
1679    }
1680
1681    /// Hash this root.
1682    ///
1683    /// Note that, similar to `Rooted::rooted_eq`, this only operates on the
1684    /// root and *not* the underlying GC reference. That means that two
1685    /// different rootings of the same object will hash to different values
1686    /// (modulo hash collisions). If this is undesirable, use the
1687    /// [`ref_hash`][crate::ManuallyRooted::ref_hash] method instead.
1688    pub fn rooted_hash<H>(&self, state: &mut H)
1689    where
1690        H: Hasher,
1691    {
1692        self.inner.hash(state);
1693    }
1694
1695    /// Hash the underlying rooted object reference.
1696    ///
1697    /// Note that, similar to `Rooted::ref_eq`, and operates on the underlying
1698    /// rooted GC object reference, not the root. That means that two
1699    /// *different* rootings of the same object will hash to the *same*
1700    /// value. If this is undesirable, use the
1701    /// [`rooted_hash`][crate::Rooted::rooted_hash] method instead.
1702    pub fn ref_hash<H>(&self, store: impl AsContext, state: &mut H)
1703    where
1704        H: Hasher,
1705    {
1706        let gc_ref = self
1707            .get_gc_ref(store.as_context().0)
1708            .expect("ManuallyRooted's get_gc_ref is infallible");
1709        gc_ref.hash(state);
1710    }
1711
1712    #[doc(hidden)]
1713    pub fn into_parts_for_c_api(self) -> (NonZeroU64, u32, u32) {
1714        (
1715            self.inner.store_id.as_raw(),
1716            self.inner.generation,
1717            self.inner.index.0,
1718        )
1719    }
1720
1721    #[doc(hidden)]
1722    pub unsafe fn from_raw_parts_for_c_api(a: NonZeroU64, b: u32, c: u32) -> ManuallyRooted<T> {
1723        ManuallyRooted {
1724            inner: GcRootIndex {
1725                store_id: StoreId::from_raw(a),
1726                generation: b,
1727                index: PackedIndex(c),
1728            },
1729            _phantom: marker::PhantomData,
1730        }
1731    }
1732
1733    /// Cast `self` to a `ManuallyRooted<U>`.
1734    ///
1735    /// It is the caller's responsibility to ensure that `self` is actually a
1736    /// `U`. Failure to uphold this invariant will be memory safe but will
1737    /// result in general incorrectness such as panics and wrong results.
1738    pub(crate) fn unchecked_cast<U: GcRef>(self) -> ManuallyRooted<U> {
1739        let u = ManuallyRooted {
1740            inner: self.inner,
1741            _phantom: core::marker::PhantomData,
1742        };
1743        core::mem::forget(self);
1744        u
1745    }
1746
1747    /// Common implementation of the `WasmTy::store` trait method for all
1748    /// `ManuallyRooted<T>`s.
1749    pub(super) fn wasm_ty_store(
1750        self,
1751        store: &mut AutoAssertNoGc<'_>,
1752        ptr: &mut MaybeUninit<ValRaw>,
1753        val_raw: impl Fn(u32) -> ValRaw,
1754    ) -> Result<()> {
1755        let gc_ref = self.try_clone_gc_ref(store)?;
1756        let raw = store.require_gc_store_mut()?.expose_gc_ref_to_wasm(gc_ref);
1757        ptr.write(val_raw(raw.get()));
1758        Ok(())
1759    }
1760
1761    /// Common implementation of the `WasmTy::load` trait method for all
1762    /// `ManuallyRooted<T>`s.
1763    pub(super) fn wasm_ty_load(
1764        store: &mut AutoAssertNoGc<'_>,
1765        raw_gc_ref: u32,
1766        from_cloned_gc_ref: impl Fn(&mut AutoAssertNoGc<'_>, VMGcRef) -> Rooted<T>,
1767    ) -> Self {
1768        debug_assert_ne!(raw_gc_ref, 0);
1769        let gc_ref = VMGcRef::from_raw_u32(raw_gc_ref).expect("non-null");
1770        let gc_ref = store.clone_gc_ref(&gc_ref);
1771        RootSet::with_lifo_scope(store, |store| {
1772            let rooted = from_cloned_gc_ref(store, gc_ref);
1773            rooted
1774                ._to_manually_rooted(store)
1775                .expect("rooted is in scope")
1776        })
1777    }
1778
1779    /// Common implementation of the `WasmTy::store` trait method for all
1780    /// `Option<ManuallyRooted<T>>`s.
1781    pub(super) fn wasm_ty_option_store(
1782        me: Option<Self>,
1783        store: &mut AutoAssertNoGc<'_>,
1784        ptr: &mut MaybeUninit<ValRaw>,
1785        val_raw: impl Fn(u32) -> ValRaw,
1786    ) -> Result<()> {
1787        match me {
1788            Some(me) => me.wasm_ty_store(store, ptr, val_raw),
1789            None => {
1790                ptr.write(val_raw(0));
1791                Ok(())
1792            }
1793        }
1794    }
1795
1796    /// Common implementation of the `WasmTy::load` trait method for all
1797    /// `Option<ManuallyRooted<T>>`s.
1798    pub(super) fn wasm_ty_option_load(
1799        store: &mut AutoAssertNoGc<'_>,
1800        raw_gc_ref: u32,
1801        from_cloned_gc_ref: impl Fn(&mut AutoAssertNoGc<'_>, VMGcRef) -> Rooted<T>,
1802    ) -> Option<Self> {
1803        let gc_ref = VMGcRef::from_raw_u32(raw_gc_ref)?;
1804        let gc_ref = store.clone_gc_ref(&gc_ref);
1805        RootSet::with_lifo_scope(store, |store| {
1806            let rooted = from_cloned_gc_ref(store, gc_ref);
1807            Some(
1808                rooted
1809                    ._to_manually_rooted(store)
1810                    .expect("rooted is in scope"),
1811            )
1812        })
1813    }
1814}
1815
1816impl<T: GcRef> RootedGcRefImpl<T> for ManuallyRooted<T> {
1817    fn get_gc_ref<'a>(&self, store: &'a StoreOpaque) -> Option<&'a VMGcRef> {
1818        assert!(
1819            self.comes_from_same_store(store),
1820            "object used with wrong store"
1821        );
1822
1823        let id = self.inner.index.as_manual().unwrap();
1824        store.gc_roots().manually_rooted.get(id)
1825    }
1826}
1827
1828#[cfg(test)]
1829mod tests {
1830    use crate::ExternRef;
1831
1832    use super::*;
1833
1834    #[test]
1835    fn sizes() {
1836        // Try to keep tabs on the size of these things. Don't want them growing
1837        // unintentionally.
1838        assert_eq!(std::mem::size_of::<Rooted<ExternRef>>(), 16);
1839        assert_eq!(std::mem::size_of::<ManuallyRooted<ExternRef>>(), 16);
1840    }
1841}