1# What is XNU? 2 3XNU kernel is part of the Darwin operating system for use in macOS and iOS operating systems. XNU is an acronym for X is Not Unix. 4XNU is a hybrid kernel combining the Mach kernel developed at Carnegie Mellon University with components from FreeBSD and a C++ API for writing drivers called IOKit. 5XNU runs on x86_64 and ARM64 for both single processor and multi-processor configurations. 6 7## The XNU Source Tree 8 9* `config` - configurations for exported apis for supported architecture and platform 10* `SETUP` - Basic set of tools used for configuring the kernel, versioning and kextsymbol management. 11* `EXTERNAL_HEADERS` - Headers sourced from other projects to avoid dependency cycles when building. These headers should be regularly synced when source is updated. 12* `libkern` - C++ IOKit library code for handling of drivers and kexts. 13* `libsa` - kernel bootstrap code for startup 14* `libsyscall` - syscall library interface for userspace programs 15* `libkdd` - source for user library for parsing kernel data like kernel chunked data. 16* `makedefs` - top level rules and defines for kernel build. 17* `osfmk` - Mach kernel based subsystems 18* `pexpert` - Platform specific code like interrupt handling, atomics etc. 19* `security` - Mandatory Access Check policy interfaces and related implementation. 20* `bsd` - BSD subsystems code 21* `tools` - A set of utilities for testing, debugging and profiling kernel. 22 23## How to Build XNU 24 25### Building a `DEVELOPMENT` Kernel 26 27The xnu make system can build kernel based on `KERNEL_CONFIGS` & `ARCH_CONFIGS` variables as arguments. 28Here is the syntax: 29 30```text 31make SDKROOT=<sdkroot> ARCH_CONFIGS=<arch> KERNEL_CONFIGS=<variant> 32``` 33 34Where: 35 36* `<sdkroot>`: path to macOS SDK on disk. (defaults to `/`) 37* `<variant>`: can be `debug`, `development`, `release`, `profile` and configures compilation flags and asserts throughout kernel code. 38* `<arch>`: can be valid arch to build for. (E.g. `X86_64`) 39 40To build a kernel for the same architecture as running OS, just type 41 42```text 43make SDKROOT=macosx.internal 44``` 45 46Additionally, there is support for configuring architectures through `ARCH_CONFIGS` and kernel configurations with `KERNEL_CONFIGS`. 47 48```text 49make SDKROOT=macosx.internal ARCH_CONFIGS=X86_64 KERNEL_CONFIGS=DEVELOPMENT 50make SDKROOT=macosx.internal ARCH_CONFIGS=X86_64 KERNEL_CONFIGS="RELEASE DEVELOPMENT DEBUG" 51``` 52 53> Note: By default, the architecture is set to the build machine's architecture, and the default kernel config is set to build for `DEVELOPMENT`. 54 55This will also create a bootable image, kernel.[config], and a kernel binary 56with symbols, kernel.[config].unstripped. 57 58To install the kernel into a DSTROOT, use the `install_kernels` target: 59 60```text 61make install_kernels DSTROOT=/tmp/xnu-dst 62``` 63 64For a more satisfying kernel debugging experience, with access to all 65local variables and arguments, but without all the extra check of the 66DEBUG kernel, add something like the following to your make command: 67 68```text 69CFLAGS_DEVELOPMENTARM64="-O0 -g -DKERNEL_STACK_MULTIPLIER=2" 70CXXFLAGS_DEVELOPMENTARM64="-O0 -g -DKERNEL_STACK_MULTIPLIER=2" 71``` 72 73Remember to replace `DEVELOPMENT` and `ARM64` with the appropriate build and platform. 74 75> Extra Flags: You can pass additional flags to the C compiler at the command line with the `EXTRA_CFLAGS` build setting. These flags are appended to the base `CFLAGS`, and the default value for the setting is an empty string. 76> 77> This setting allows you to e.g. selectively turn on debugging code that is guarded by a preprocessor macro. Example usage... 78> 79> ```text 80> make SDKROOT=macosx.internal PRODUCT_CONFIGS=j314s 81> EXTRA_CFLAGS='-DKERNEL_STACK_MULTIPLIER=2' 82> ``` 83 84 85* To build with RELEASE kernel configuration 86 87 ```text 88 make KERNEL_CONFIGS=RELEASE SDKROOT=/path/to/SDK 89 ``` 90 91### Building FAT Kernel Binary 92 93Define architectures in your environment or when running a make command. 94 95```text 96make ARCH_CONFIGS="X86_64" exporthdrs all 97``` 98 99 100 101### Other Makefile Options 102 103* $ make MAKEJOBS=-j8 # this will use 8 processes during the build. The default is 2x the number of active CPUS. 104* $ make -j8 # the standard command-line option is also accepted 105* $ make -w # trace recursive make invocations. Useful in combination with VERBOSE=YES 106* $ make BUILD_LTO=0 # build without LLVM Link Time Optimization 107* $ make BOUND_CHECKS=0 # disable -fbound-attributes for this build 108* $ make REMOTEBUILD=user@remotehost # perform build on remote host 109* $ make BUILD_CODE_COVERAGE=1 # build with support for collecting code coverage information 110 111The XNU build system can optionally output color-formatted build output. To enable this, you can either 112set the `XNU_LOGCOLORS` environment variable to `y`, or you can pass `LOGCOLORS=y` to the make command. 113 114### Customize the XNU Version 115 116The xnu version is derived from the SDK or KDK by reading the `CFBundleVersion` 117of their `System/Library/Extensions/System.kext/Info.plist` file. 118This can be customized by setting the `RC_DARWIN_KERNEL_VERSION` variable in 119the environment or on the `make` command line. 120 121 122See doc/xnu_version.md for more details. 123 124### Debug Information Formats 125 126By default, a DWARF debug information repository is created during the install phase; this is a "bundle" named kernel.development.\<variant>.dSYM 127To select the older STABS debug information format (where debug information is embedded in the kernel.development.unstripped image), set the BUILD_STABS environment variable. 128 129```sh 130export BUILD_STABS=1 131make 132``` 133 134 135## Building KernelCaches 136 137To test the xnu kernel, you need to build a kernelcache that links the kexts and 138kernel together into a single bootable image. 139To build a kernelcache you can use the following mechanisms: 140 141* Using automatic kernelcache generation with `kextd`. 142 The kextd daemon keeps watching for changing in `/System/Library/Extensions` directory. 143 So you can setup new kernel as 144 145 ```text 146 cp BUILD/obj/DEVELOPMENT/X86_64/kernel.development /System/Library/Kernels/ 147 touch /System/Library/Extensions 148 ps -e | grep kextd 149 ``` 150 151* Manually invoking `kextcache` to build new kernelcache. 152 153 ```text 154 kextcache -q -z -a x86_64 -l -n -c /var/tmp/kernelcache.test -K /var/tmp/kernel.test /System/Library/Extensions 155 ``` 156 157 158## Booting a KernelCache on a Target machine 159 160The development kernel and iBoot supports configuring boot arguments so that we can safely boot into test kernel and, if things go wrong, safely fall back to previously used kernelcache. 161Following are the steps to get such a setup: 162 1631. Create kernel cache using the kextcache command as `/kernelcache.test` 1642. Copy exiting boot configurations to alternate file 165 166 ```sh 167 cp /Library/Preferences/SystemConfiguration/com.apple.Boot.plist /next_boot.plist 168 ``` 169 1703. Update the kernelcache and boot-args for your setup 171 172 ```sh 173 plutil -insert "Kernel Cache" -string "kernelcache.test" /next_boot.plist 174 plutil -replace "Kernel Flags" -string "debug=0x144 -v kernelsuffix=test " /next_boot.plist 175 ``` 176 1774. Copy the new config to `/Library/Preferences/SystemConfiguration/` 178 179 ```sh 180 cp /next_boot.plist /Library/Preferences/SystemConfiguration/boot.plist 181 ``` 182 1835. Bless the volume with new configs. 184 185 ```text 186 sudo -n bless --mount / --setBoot --nextonly --options "config=boot" 187 ``` 188 189 The `--nextonly` flag specifies that use the `boot.plist` configs only for one boot. 190 So if the kernel panic's you can easily power reboot and recover back to original kernel. 191 192 193## Creating tags and cscope 194 195Set up your build environment and from the top directory, run: 196 197 make tags # this will build ctags and etags on a case-sensitive volume, only ctags on case-insensitive 198 make TAGS # this will build etags 199 make cscope # this will build cscope database 200 201## Installing New Header Files from XNU 202 203XNU installs header files at the following locations - 204 205 a. $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers 206 b. $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders 207 c. $(DSTROOT)/usr/include/ 208 d. $(DSTROOT)/usr/local/include/ 209 e. $(DSTROOT)/System/DriverKit/usr/include/ 210 f. $(DSTROOT)/System/Library/Frameworks/IOKit.framework/Headers 211 g. $(DSTROOT)/System/Library/Frameworks/IOKit.framework/PrivateHeaders 212 h. $(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders 213 214`Kernel.framework` is used by kernel extensions.\ 215The `System.framework`, `/usr/include` and `/usr/local/include` are used by user level applications. \ 216`IOKit.framework` is used by IOKit userspace clients. \ 217`/System/DriverKit/usr/include` is used by userspace drivers. \ 218The header files in framework's `PrivateHeaders` are only available for **Apple Internal Development**. 219 220The directory containing the header file should have a Makefile that 221creates the list of files that should be installed at different locations. 222If you are adding the first header file in a directory, you will need to 223create Makefile similar to `xnu/bsd/sys/Makefile`. 224 225Add your header file to the correct file list depending on where you want 226to install it. The default locations where the header files are installed 227from each file list are - 228 229 a. `DATAFILES` : To make header file available in user level - 230 `$(DSTROOT)/usr/include` 231 `$(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders` 232 233 b. `DRIVERKIT_DATAFILES` : To make header file available to DriverKit userspace drivers - 234 `$(DSTROOT)/System/DriverKit/usr/include` 235 236 c. `PRIVATE_DATAFILES` : To make header file available to Apple internal in 237 user level - 238 `$(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders` 239 240 d. `EMBEDDED_PRIVATE_DATAFILES` : To make header file available in user 241 level for macOS as `EXTRA_DATAFILES`, but Apple internal in user level 242 for embedded OSes as `EXTRA_PRIVATE_DATAFILES` - 243 `$(DSTROOT)/usr/include` (`EXTRA_DATAFILES`) 244 `$(DSTROOT)/usr/local/include` (`EXTRA_PRIVATE_DATAFILES`) 245 246 e. `KERNELFILES` : To make header file available in kernel level - 247 `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers` 248 `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders` 249 250 f. `PRIVATE_KERNELFILES` : To make header file available to Apple internal 251 for kernel extensions - 252 `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders` 253 254 g. `MODULEMAPFILES` : To make module map file available in user level - 255 `$(DSTROOT)/usr/include` 256 257 h. `PRIVATE_MODULEMAPFILES` : To make module map file available to Apple 258 internal in user level - 259 `$(DSTROOT)/usr/local/include` 260 261 i. `LIBCXX_DATAFILES` : To make header file available to in-kernel libcxx clients: 262 `$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders/kernel_sdkroot` 263 264 j. `EXCLAVEKIT_DATAFILES` : To make header file available to Apple internal 265 ExclaveKit SDK - 266 `$(DSTROOT)/System/ExclaveKit/usr/include` 267 268 k. `EXCLAVECORE_DATAFILES` : To make header file available to Apple internal 269 ExclaveCore SDK - 270 `$(DSTROOT)/System/ExclaveCore/usr/include` 271 272The Makefile combines the file lists mentioned above into different 273install lists which are used by build system to install the header files. There 274are two types of install lists: machine-dependent and machine-independent. 275These lists are indicated by the presence of `MD` and `MI` in the build 276setting, respectively. If your header is architecture-specific, then you should 277use a machine-dependent install list (e.g. `INSTALL_MD_LIST`). If your header 278should be installed for all architectures, then you should use a 279machine-independent install list (e.g. `INSTALL_MI_LIST`). 280 281If the install list that you are interested does not exist, create it 282by adding the appropriate file lists. The default install lists, its 283member file lists and their default location are described below - 284 285a. `INSTALL_MI_LIST`, `INSTALL_MODULEMAP_MI_LIST` : Installs header and module map 286 files to a location that is available to everyone in user level. 287 Locations - 288 $(DSTROOT)/usr/include 289 Definition - 290 INSTALL_MI_LIST = ${DATAFILES} 291 INSTALL_MODULEMAP_MI_LIST = ${MODULEMAPFILES} 292 293b. `INSTALL_DRIVERKIT_MI_LIST` : Installs header file to a location that is 294 available to DriverKit userspace drivers. 295 Locations - 296 $(DSTROOT)/System/DriverKit/usr/include 297 Definition - 298 INSTALL_DRIVERKIT_MI_LIST = ${DRIVERKIT_DATAFILES} 299 300c. `INSTALL_MI_LCL_LIST`, `INSTALL_MODULEMAP_MI_LCL_LIST` : Installs header and 301 module map files to a location that is available for Apple internal in user level. 302 Locations - 303 $(DSTROOT)/usr/local/include 304 Definition - 305 INSTALL_MI_LCL_LIST = 306 INSTALL_MODULEMAP_MI_LCL_LIST = ${PRIVATE_MODULEMAPFILES} 307 308d. `INSTALL_IF_MI_LIST` : Installs header file to location that is available 309 to everyone for IOKit userspace clients. 310 Locations - 311 $(DSTROOT)/System/Library/Frameworks/IOKit.framework/Headers 312 Definition - 313 INSTALL_IF_MI_LIST = ${DATAFILES} 314 315e. `INSTALL_IF_MI_LCL_LIST` : Installs header file to location that is 316 available to Apple internal for IOKit userspace clients. 317 Locations - 318 $(DSTROOT)/System/Library/Frameworks/IOKit.framework/PrivateHeaders 319 Definition - 320 INSTALL_IF_MI_LCL_LIST = ${DATAFILES} ${PRIVATE_DATAFILES} 321 322f. `INSTALL_SF_MI_LCL_LIST` : Installs header file to a location that is available 323 for Apple internal in user level. 324 Locations - 325 $(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders 326 Definition - 327 INSTALL_SF_MI_LCL_LIST = ${DATAFILES} ${PRIVATE_DATAFILES} 328 329g. `INSTALL_KF_MI_LIST` : Installs header file to location that is available 330 to everyone for kernel extensions. 331 Locations - 332 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers 333 Definition - 334 INSTALL_KF_MI_LIST = ${KERNELFILES} 335 336h. `INSTALL_KF_MI_LCL_LIST` : Installs header file to location that is 337 available for Apple internal for kernel extensions. 338 Locations - 339 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders 340 Definition - 341 INSTALL_KF_MI_LCL_LIST = ${KERNELFILES} ${PRIVATE_KERNELFILES} 342 343i. `EXPORT_MI_LIST` : Exports header file to all of xnu (bsd/, osfmk/, etc.) 344 for compilation only. Does not install anything into the SDK. 345 Definition - 346 EXPORT_MI_LIST = ${KERNELFILES} ${PRIVATE_KERNELFILES} 347 348j. `INSTALL_KF_LIBCXX_MI_LIST` : Installs header file for in-kernel libc++ support. 349 Locations - 350 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders/kernel_sdkroot 351 Definition - 352 INSTALL_KF_LIBCXX_MI_LIST = ${LIBCXX_DATAFILES} 353 354k. `INSTALL_EXCLAVEKIT_MI_LIST` : Installs header file to location that is 355 available for Apple internal for ExclaveKit. 356 Locations - 357 $(DSTROOT)/System/ExclaveKit/usr/include 358 Definition - 359 INSTALL_EXCLAVEKIT_MI_LIST = ${EXCLAVEKIT_DATAFILES} 360 361l. `INSTALL_EXCLAVECORE_MI_LIST` : Installs header file to location that is 362 available for Apple internal for ExclaveCore. 363 Locations - 364 $(DSTROOT)/System/ExclaveCore/usr/include 365 Definition - 366 INSTALL_EXCLAVECORE_MI_LIST = ${EXCLAVECORE_DATAFILES} 367 368If you want to install the header file in a sub-directory of the paths 369described in (1), specify the directory name using two variables 370`INSTALL_MI_DIR` and `EXPORT_MI_DIR` as follows - 371 372```text 373INSTALL_MI_DIR = dirname 374EXPORT_MI_DIR = dirname 375``` 376 377If you want to install the module map file in a sub-directory, specify the 378directory name using the variable `INSTALL_MODULEMAP_MI_DIR` as follows - 379 380```text 381INSTALL_MODULEMAP_MI_DIR = dirname 382``` 383 384A single header file can exist at different locations using the steps 385mentioned above. However it might not be desirable to make all the code 386in the header file available at all the locations. For example, you 387want to export a function only to kernel level but not user level. 388 389 You can use C language's pre-processor directive (#ifdef, #endif, #ifndef) 390 to control the text generated before a header file is installed. The kernel 391 only includes the code if the conditional macro is TRUE and strips out 392 code for FALSE conditions from the header file. 393 394 Some pre-defined macros and their descriptions are - 395 3961. `PRIVATE` : If defined, enclosed definitions are considered System 397Private Interfaces. These are visible within xnu and 398exposed in user/kernel headers installed within the AppleInternal 399"PrivateHeaders" sections of the System and Kernel frameworks. 4002. `KERNEL_PRIVATE` : If defined, enclosed code is available to all of xnu 401kernel and Apple internal kernel extensions and omitted from user 402headers. 4033. `BSD_KERNEL_PRIVATE` : If defined, enclosed code is visible exclusively 404within the xnu/bsd module. 4054. `MACH_KERNEL_PRIVATE`: If defined, enclosed code is visible exclusively 406within the xnu/osfmk module. 4075. `XNU_KERNEL_PRIVATE`: If defined, enclosed code is visible exclusively 408within xnu. 4096. `KERNEL` : If defined, enclosed code is available within xnu and kernel 410 extensions and is not visible in user level header files. Only the 411 header files installed in following paths will have the code - 412 413 ```text 414 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers 415 $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders 416 ``` 417 4187. `DRIVERKIT`: If defined, enclosed code is visible exclusively in the 419DriverKit SDK headers used by userspace drivers. 4208. `EXCLAVEKIT`: If defined, enclosed code is visible exclusively in the 421ExclaveKit SDK headers. 4229. `EXCLAVECORE`: If defined, enclosed code is visible exclusively in the 423ExclaveCore SDK headers. 424 425## VM header file name convention 426The VM headers follow the following naming conventions: 427* `*_internal.h` headers contain components of the VM subsystem only for use by VM code. 428* `*_xnu.h` headers contain components of the VM subsystem only for use by other xnu code. 429* `*.h` headers contain components of the VM subsystem exported to kexts. 430* `vm_iokit.h` header contains components of the VM subsystem exported to the iokit subsystem. 431* `vm_ubc.h` header contains components of the VM subsystem exported to the ubc subsystem. 432 433 434## Module map file name convention 435 436In the simple case, a subdirectory of `usr/include` or `usr/local/include` 437can be represented by a standalone module. Where this is the case, set 438`INSTALL_MODULEMAP_MI_DIR` to `INSTALL_MI_DIR` and install a `module.modulemap` 439file there. `module.modulemap` is used even for private modules in 440`usr/local/include`; `module.private.modulemap` is not used. Caveat: in order 441to stay in the simple case, the module name needs to be exactly the same as 442the directory name. If that's not possible, then the following method will 443need to be applied. 444 445`xnu` contributes to the modules defined in CoreOSModuleMaps by installing 446module map files that are sourced from `usr/include/module.modulemap` and 447`usr/local/include/module.modulemap`. The naming convention for the `xnu` 448module map files are as follows. 449 450a. Ideally the module map file covers an entire directory. A module map 451 file covering `usr/include/a/b/c` would be named `a_b_c.modulemap`. 452 `usr/local/include/a/b/c` would be `a_b_c_private.modulemap`. 453b. Some headers are special and require their own module. In that case, 454 the module map file would be named after the module it defines. 455 A module map file defining the module `One.Two.Three` would be named 456 `one_two_three.modulemap`. 457 458## Conditional Compilation 459 460`xnu` offers the following mechanisms for conditionally compiling code: 461 4621. *CPU Characteristics* If the code you are guarding has specific 463 characterstics that will vary only based on the CPU architecture being 464 targeted, use this option. Prefer checking for features of the 465 architecture (e.g. `__LP64__`, `__LITTLE_ENDIAN__`, etc.). 4662. *New Features* If the code you are guarding, when taken together, 467 implements a feature, you should define a new feature in `config/MASTER` 468 and use the resulting `CONFIG` preprocessor token (e.g. for a feature 469 named `config_virtual_memory`, check for `#if CONFIG_VIRTUAL_MEMORY`). 470 This practice ensures that existing features may be brought to other 471 platforms by simply changing a feature switch. 4723. *Existing Features* You can use existing features if your code is 473 strongly tied to them (e.g. use `SECURE_KERNEL` if your code implements 474 new functionality that is exclusively relevant to the trusted kernel and 475 updates the definition/understanding of what being a trusted kernel means). 476 477It is recommended that you avoid compiling based on the target platform. `xnu` 478does not define the platform macros from `TargetConditionals.h` 479(`TARGET_OS_OSX`, `TARGET_OS_IOS`, etc.). 480 481 482## Debugging XNU 483 484By default, the kernel reboots in the event of a panic. 485This behavior can be overriden by the `debug` boot-arg -- `debug=0x14e` will cause a panic to wait for a debugger to attach. 486To boot a kernel so it can be debugged by an attached machine, override the `kdp_match_name` boot-arg with the appropriate `ifconfig` interface. 487Ethernet, Thunderbolt, and serial debugging are supported, depending on the hardware. 488 489Use LLDB to debug the kernel: 490 491```text 492xcrun -sdk macosx lldb <path-to-unstripped-kernel> 493(lldb) gdb-remote [<host-ip>:]<port> 494``` 495 496The debug info for the kernel (dSYM) comes with a set of macros to support kernel debugging. 497To load these macros automatically when attaching to the kernel, add the following to `~/.lldbinit`: 498 499```text 500settings set target.load-script-from-symbol-file true 501``` 502 503`tools/lldbmacros` contains the source for these commands. 504See the README in that directory for their usage, or use the built-in LLDB help with: 505 506```text 507(lldb) help showcurrentstacks 508``` 509 510