1 //! User-defined stack maps.
2 //!
3 //! This module provides types allowing users to define stack maps and associate
4 //! them with safepoints.
5 //!
6 //! A **safepoint** is a program point (i.e. CLIF instruction) where it must be
7 //! safe to run GC. Currently all non-tail call instructions are considered
8 //! safepoints. (This does *not* allow, for example, skipping safepoints for
9 //! calls that are statically known not to trigger collections, or to have a
10 //! safepoint on a volatile load to a page that gets protected when it is time
11 //! to GC, triggering a fault that pauses the mutator and lets the collector do
12 //! its work before resuming the mutator. We can lift this restriction in the
13 //! future, if necessary.)
14 //!
15 //! A **stack map** is a description of where to find all the GC-managed values
16 //! that are live at a particular safepoint. Stack maps let the collector find
17 //! on-stack roots. Each stack map is logically a set of offsets into the stack
18 //! frame and the type of value at that associated offset. However, because the
19 //! stack layout isn't defined until much later in the compiler's pipeline, each
20 //! stack map entry instead includes both an `ir::StackSlot` and an offset
21 //! within that slot.
22 //!
23 //! These stack maps are **user-defined** in that it is the CLIF producer's
24 //! responsibility to identify and spill the live GC-managed values and attach
25 //! the associated stack map entries to each safepoint themselves (see
26 //! `cranelift_frontend::Function::declare_needs_stack_map` and
27 //! `cranelift_codegen::ir::DataFlowGraph::append_user_stack_map_entry`). Cranelift
28 //! will not insert spills and record these stack map entries automatically (in
29 //! contrast to the old system and its `r64` values).
30 //!
31 //! Logically, a set of stack maps for a function record a table of the form:
32 //!
33 //! ```text
34 //! +---------------------+-------------------------------------------+
35 //! | Instruction Pointer | SP-Relative Offsets of Live GC References |
36 //! +---------------------+-------------------------------------------+
37 //! | 0x12345678          | 2, 6, 12                                  |
38 //! | 0x1234abcd          | 2, 6                                      |
39 //! | ...                 | ...                                       |
40 //! +---------------------+-------------------------------------------+
41 //! ```
42 //!
43 //! Where "instruction pointer" is an instruction pointer within the function,
44 //! and "offsets of live GC references" contains the offsets (in units of words)
45 //! from the frame's stack pointer where live GC references are stored on the
46 //! stack. Instruction pointers within the function that do not have an entry in
47 //! this table are not GC safepoints.
48 //!
49 //! Because
50 //!
51 //! * offsets of live GC references are relative from the stack pointer, and
52 //! * stack frames grow down from higher addresses to lower addresses,
53 //!
54 //! to get a pointer to a live reference at offset `x` within a stack frame, you
55 //! add `x` to the frame's stack pointer.
56 //!
57 //! For example, to calculate the pointer to the live GC reference inside "frame
58 //! 1" below, you would do `frame_1_sp + x`:
59 //!
60 //! ```text
61 //!           Stack
62 //!         +-------------------+
63 //!         | Frame 0           |
64 //!         |                   |
65 //!    |    |                   |
66 //!    |    +-------------------+ <--- Frame 0's SP
67 //!    |    | Frame 1           |
68 //!  Grows  |                   |
69 //!  down   |                   |
70 //!    |    | Live GC reference | --+--
71 //!    |    |                   |   |
72 //!    |    |                   |   |
73 //!    V    |                   |   x = offset of live GC reference
74 //!         |                   |   |
75 //!         |                   |   |
76 //!         +-------------------+ --+--  <--- Frame 1's SP
77 //!         | Frame 2           |
78 //!         | ...               |
79 //! ```
80 //!
81 //! An individual `UserStackMap` is associated with just one instruction pointer
82 //! within the function, contains the size of the stack frame, and represents
83 //! the stack frame as a bitmap. There is one bit per word in the stack frame,
84 //! and if the bit is set, then the word contains a live GC reference.
85 //!
86 //! Note that a caller's outgoing argument stack slots (if any) and callee's
87 //! incoming argument stack slots (if any) overlap, so we must choose which
88 //! function's stack maps record live GC references in these slots. We record
89 //! the incoming arguments in the callee's stack map. This choice plays nice
90 //! with tail calls, where by the time we transfer control to the callee, the
91 //! caller no longer exists.
92 
93 use crate::ir;
94 use cranelift_bitset::CompoundBitSet;
95 use cranelift_entity::PrimaryMap;
96 use smallvec::SmallVec;
97 
98 pub(crate) type UserStackMapEntryVec = SmallVec<[UserStackMapEntry; 4]>;
99 
100 /// A stack map entry describes a single GC-managed value and its location on
101 /// the stack.
102 ///
103 /// A stack map entry is associated with a particular instruction, and that
104 /// instruction must be a safepoint. The GC-managed value must be stored in the
105 /// described location across this entry's instruction.
106 #[derive(Clone, Debug, PartialEq, Hash)]
107 #[cfg_attr(
108     feature = "enable-serde",
109     derive(serde_derive::Serialize, serde_derive::Deserialize)
110 )]
111 pub struct UserStackMapEntry {
112     /// The type of the value stored in this stack map entry.
113     pub ty: ir::Type,
114 
115     /// The stack slot that this stack map entry is within.
116     pub slot: ir::StackSlot,
117 
118     /// The offset within the stack slot where this entry's value can be found.
119     pub offset: u32,
120 }
121 
122 /// A compiled stack map, describing the location of many GC-managed values.
123 ///
124 /// A stack map is associated with a particular instruction, and that
125 /// instruction is a safepoint.
126 #[derive(Clone, Debug, PartialEq)]
127 #[cfg_attr(
128     feature = "enable-serde",
129     derive(serde_derive::Deserialize, serde_derive::Serialize)
130 )]
131 pub struct UserStackMap {
132     // Offsets into the frame's sized stack slots that are GC references, by type.
133     by_type: SmallVec<[(ir::Type, CompoundBitSet); 1]>,
134 
135     // The offset of the sized stack slots, from SP, for this stack map's
136     // associated PC.
137     //
138     // This is initially `None` upon construction during lowering, but filled in
139     // after regalloc during emission when we have the precise frame layout.
140     sp_to_sized_stack_slots: Option<u32>,
141 }
142 
143 impl UserStackMap {
144     /// Coalesce the given entries into a new `UserStackMap`.
145     pub(crate) fn new(
146         entries: &[UserStackMapEntry],
147         stack_slot_offsets: &PrimaryMap<ir::StackSlot, u32>,
148     ) -> Self {
149         let mut by_type = SmallVec::<[(ir::Type, CompoundBitSet); 1]>::default();
150 
151         for entry in entries {
152             let offset = stack_slot_offsets[entry.slot] + entry.offset;
153             let offset = usize::try_from(offset).unwrap();
154 
155             // Don't bother trying to avoid an `O(n)` search here: `n` is
156             // basically always one in practice; even if it isn't, there aren't
157             // that many different CLIF types.
158             let index = by_type
159                 .iter()
160                 .position(|(ty, _)| *ty == entry.ty)
161                 .unwrap_or_else(|| {
162                     by_type.push((entry.ty, CompoundBitSet::with_capacity(offset + 1)));
163                     by_type.len() - 1
164                 });
165 
166             by_type[index].1.insert(offset);
167         }
168 
169         UserStackMap {
170             by_type,
171             sp_to_sized_stack_slots: None,
172         }
173     }
174 
175     /// Finalize this stack map by filling in the SP-to-stack-slots offset.
176     pub(crate) fn finalize(&mut self, sp_to_sized_stack_slots: u32) {
177         debug_assert!(self.sp_to_sized_stack_slots.is_none());
178         self.sp_to_sized_stack_slots = Some(sp_to_sized_stack_slots);
179     }
180 
181     /// Iterate over the entries in this stack map.
182     ///
183     /// Yields pairs of the type of GC reference that is at the offset, and the
184     /// offset from SP. If a pair `(i64, 0x42)` is yielded, for example, then
185     /// when execution is at this stack map's associated PC, `SP + 0x42` is a
186     /// pointer to an `i64`, and that `i64` is a live GC reference.
187     pub fn entries(&self) -> impl Iterator<Item = (ir::Type, u32)> + '_ {
188         let sp_to_sized_stack_slots = self.sp_to_sized_stack_slots.expect(
189             "`sp_to_sized_stack_slots` should have been filled in before this stack map was used",
190         );
191         self.by_type.iter().flat_map(move |(ty, bitset)| {
192             bitset.iter().map(move |slot_offset| {
193                 (
194                     *ty,
195                     sp_to_sized_stack_slots + u32::try_from(slot_offset).unwrap(),
196                 )
197             })
198         })
199     }
200 }
201