1Kernel Data Descriptors 2======================= 3 4This project allows for dynamic data to be passed from the kernel to userspace tools without binding them to particular version of 5struct definition. The `libkdd` library provides convenient API for parsing and interpreting `kernel chunked data`. 6 7The libkdd APIs are defined in [kdd.h](./kdd.h) 8 9The `KCDATA` format 10=================== 11 12The format for data is setup in a generic format as follows 13 14Layout of data structure 15------------------------ 16 17 | 8 - bytes | 18 |---------------------------| ------ offset = 00 19 | type = MAGIC | LENGTH | # BEGIN Header 20 | 0 | 21 |---------------------------| ------ offset = 16 22 | type | size | # chunk header 23 | flags | 24 |---------------------------| ------ offset = 32 25 | data | # arbitrary data (len=16) 26 |___________data____________| 27 |---------------------------| ------ offset = 48 28 | type | size | # chunk header 29 | flags | 30 |---------------------------| ------ offset = 64 31 | data | # arbitrary data (len=32) 32 | data | 33 | data | 34 |___________data____________| 35 |---------------------------| ------ offset = 96 36 | type = END | size=0 | # chunk header 37 | 0 | 38 39 40The type field describes what kind of data is passed. For example type = `TASK_CRASHINFO_UUID` means the following data is a uuid. 41These types need to be defined in task_corpses.h for easy consumption by userspace inspection tools. 42 43Some range of types is reserved for special types like ints, longs etc. A cool new functionality made possible with this 44extensible data format is that kernel can decide to put more information as required without requiring user space tools to 45re-compile to be compatible. The case of `rusage` struct versions could be introduced without breaking existing tools. 46 47Feature description: Generic data with description 48------------------- 49Further more generic data with description is very much possible now. For example 50 51 - kcdata_add_uint64_with_description(cdatainfo, 0x700, "NUM MACH PORTS"); 52 - and more functions that allow adding description. 53 54The userspace tools can then look at the description and print the data even if they are not compiled with knowledge of the field apriori. 55 56 Example data: 57 0000 57 f1 ad de 00 00 00 00 00 00 00 00 00 00 00 00 W............... 58 0010 01 00 00 00 00 00 00 00 30 00 00 00 00 00 00 00 ........0....... 59 0020 50 49 44 00 00 00 00 00 00 00 00 00 00 00 00 00 PID............. 60 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 61 0040 9c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 62 0050 01 00 00 00 00 00 00 00 30 00 00 00 00 00 00 00 ........0....... 63 0060 50 41 52 45 4e 54 20 50 49 44 00 00 00 00 00 00 PARENT PID...... 64 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 65 0080 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 66 0090 ed 58 91 f1 67 68 69Feature description: Container markers for compound data 70------------------ 71 72If a given kernel data type is complex and requires adding multiple optional fields inside a container 73object for a consumer to understand arbitrary data, we package it using container markers. 74 75For example, the stackshot code gathers information and describes the state of a given task with respect 76to many subsystems. It includes data such as io stats, vm counters, process names/flags and syscall counts. 77 78 kcdata_add_container_marker(kcdata_p, KCDATA_TYPE_CONTAINER_BEGIN, STACKSHOT_KCCONTAINER_TASK, task_uniqueid); 79 // add multiple data, or add_<type>_with_description()s here 80 81 kcdata_add_container_marker(kcdata_p, KCDATA_TYPE_CONTAINER_END, STACKSHOT_KCCONTAINER_TASK, task_uniqueid); 82 83 84Feature description: Custom Data formats on demand 85-------------------- 86 87With the self describing nature of format, the kernel provider can describe a data type (uniquely identified by a number) and use 88it in the buffer for sending data. The consumer can parse the type information and have knowledge of describing incoming data. 89Following is an example of how we can describe a kernel specific struct sample_disk_io_stats in buffer. 90 91 struct sample_disk_io_stats { 92 uint64_t disk_reads_count; 93 uint64_t disk_reads_size; 94 uint64_t io_priority_count[4]; 95 uint64_t io_priority_size; 96 } __attribute__ ((packed)); 97 98 99 struct kcdata_subtype_descriptor disk_io_stats_def[] = { 100 {KCS_SUBTYPE_FLAGS_NONE, KC_ST_UINT64, 0 * sizeof(uint64_t), sizeof(uint64_t), "disk_reads_count"}, 101 {KCS_SUBTYPE_FLAGS_NONE, KC_ST_UINT64, 1 * sizeof(uint64_t), sizeof(uint64_t), "disk_reads_size"}, 102 {KCS_SUBTYPE_FLAGS_ARRAY, KC_ST_UINT64, 2 * sizeof(uint64_t), KCS_SUBTYPE_PACK_SIZE(4, sizeof(uint64_t)), "io_priority_count"}, 103 {KCS_SUBTYPE_FLAGS_ARRAY, KC_ST_UINT64, (2 + 4) * sizeof(uint64_t), sizeof(uint64_t), "io_priority_size"}, 104 }; 105 106Now you can add this custom type definition into the buffer as 107 kcdata_add_type_definition(kcdata_p, KCTYPE_SAMPLE_DISK_IO_STATS, "sample_disk_io_stats", 108 &disk_io_stats_def[0], sizeof(disk_io_stats_def)/sizeof(struct kcdata_subtype_descriptor)); 109 110