Changes in Makefile

ebeed8d4 - tracing/probes: Move finding func-proto API and getting func-param API to trace_btf

Tue, 22 Aug 2023 16:25:52 +0000

tracing/probes: Move finding func-proto API and getting func-param API to trace_btfMove generic function-proto find API and getting function parameter APIto BTF library code from trace_probe.c. This will avoid redundant effortson different feature.Link: https://lore.kernel.org/all/169272155255.160970.719426926348706349.stgit@devnote2/Signed-off-by: Masami Hiramatsu (Google) Acked-by: Steven Rostedt (Google) List of files: /linux-6.15/kernel/trace/Makefile

334e5519 - tracing/probes: Add fprobe events for tracing function entry and exit.

Tue, 06 Jun 2023 12:39:55 +0000

tracing/probes: Add fprobe events for tracing function entry and exit.Add fprobe events for tracing function entry and exit instead of kprobeevents. With this change, we can continue to trace function entry/exiteven if the CONFIG_KPROBES_ON_FTRACE is not available. SinceCONFIG_KPROBES_ON_FTRACE requires the CONFIG_DYNAMIC_FTRACE_WITH_REGS,it is not available if the architecture only supportsCONFIG_DYNAMIC_FTRACE_WITH_ARGS. And that means kprobe events can notprobe function entry/exit effectively on such architecture.But this can be solved if the dynamic events supports fprobe events.The fprobe event is a new dynamic events which is only for the function(symbol) entry and exit. This event accepts non register fetch argumentsso that user can trace the function arguments and return values.The fprobe events syntax is here; f[:[GRP/][EVENT]] FUNCTION [FETCHARGS] f[MAXACTIVE][:[GRP/][EVENT]] FUNCTION%return [FETCHARGS]E.g. # echo 'f vfs_read $arg1' >> dynamic_events # echo 'f vfs_read%return $retval' >> dynamic_events # cat dynamic_events f:fprobes/vfs_read__entry vfs_read arg1=$arg1 f:fprobes/vfs_read__exit vfs_read%return arg1=$retval # echo 1 > events/fprobes/enable # head -n 20 trace | tail # TASK-PID CPU# ||||| TIMESTAMP FUNCTION # | | | ||||| | | sh-142 [005] ...1. 448.386420: vfs_read__entry: (vfs_read+0x4/0x340) arg1=0xffff888007f7c540 sh-142 [005] ..... 448.386436: vfs_read__exit: (ksys_read+0x75/0x100 <- vfs_read) arg1=0x1 sh-142 [005] ...1. 448.386451: vfs_read__entry: (vfs_read+0x4/0x340) arg1=0xffff888007f7c540 sh-142 [005] ..... 448.386458: vfs_read__exit: (ksys_read+0x75/0x100 <- vfs_read) arg1=0x1 sh-142 [005] ...1. 448.386469: vfs_read__entry: (vfs_read+0x4/0x340) arg1=0xffff888007f7c540 sh-142 [005] ..... 448.386476: vfs_read__exit: (ksys_read+0x75/0x100 <- vfs_read) arg1=0x1 sh-142 [005] ...1. 448.602073: vfs_read__entry: (vfs_read+0x4/0x340) arg1=0xffff888007f7c540 sh-142 [005] ..... 448.602089: vfs_read__exit: (ksys_read+0x75/0x100 <- vfs_read) arg1=0x1Link: https://lore.kernel.org/all/168507469754.913472.6112857614708350210.stgit@mhiramat.roam.corp.google.com/Reported-by: kernel test robot Link: https://lore.kernel.org/all/202302011530.7vm4O8Ro-lkp@intel.com/Signed-off-by: Masami Hiramatsu (Google) List of files: /linux-6.15/kernel/trace/Makefile

102227b9 - rv: Add Runtime Verification (RV) interface

Fri, 29 Jul 2022 09:38:40 +0000

rv: Add Runtime Verification (RV) interfaceRV is a lightweight (yet rigorous) method that complements classicalexhaustive verification techniques (such as model checking andtheorem proving) with a more practical approach to complex systems.RV works by analyzing the trace of the system's actual execution,comparing it against a formal specification of the system behavior.RV can give precise information on the runtime behavior of themonitored system while enabling the reaction for unexpectedevents, avoiding, for example, the propagation of a failure onsafety-critical systems.The development of this interface roots in the development of thepaper:De Oliveira, Daniel Bristot; Cucinotta, Tommaso; De Oliveira, RomuloSilva. Efficient formal verification for the Linux kernel. In:International Conference on Software Engineering and Formal Methods.Springer, Cham, 2019. p. 315-332.And:De Oliveira, Daniel Bristot. Automata-based formal analysisand verification of the real-time Linux kernel. PhD Thesis, 2020.The RV interface resembles the tracing/ interface on purpose. The currentpath for the RV interface is /sys/kernel/tracing/rv/.It presents these files: "available_monitors" - List the available monitors, one per line. For example: # cat available_monitors wip wwnr "enabled_monitors" - Lists the enabled monitors, one per line; - Writing to it enables a given monitor; - Writing a monitor name with a '!' prefix disables it; - Truncating the file disables all enabled monitors. For example: # cat enabled_monitors # echo wip > enabled_monitors # echo wwnr >> enabled_monitors # cat enabled_monitors wip wwnr # echo '!wip' >> enabled_monitors # cat enabled_monitors wwnr # echo > enabled_monitors # cat enabled_monitors # Note that more than one monitor can be enabled concurrently. "monitoring_on" - It is an on/off general switcher for monitoring. Note that it does not disable enabled monitors or detach events, but stop the per-entity monitors of monitoring the events received from the system. It resembles the "tracing_on" switcher. "monitors/" Each monitor will have its one directory inside "monitors/". There the monitor specific files will be presented. The "monitors/" directory resembles the "events" directory on tracefs. For example: # cd monitors/wip/ # ls desc enable # cat desc wakeup in preemptive per-cpu testing monitor. # cat enable 0For further information, see the comments in the header ofkernel/trace/rv/rv.c from this patch.Link: https://lkml.kernel.org/r/a4bfe038f50cb047bfb343ad0e12b0e646ab308b.1659052063.git.bristot@kernel.orgCc: Wim Van Sebroeck Cc: Guenter Roeck Cc: Jonathan Corbet Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Peter Zijlstra Cc: Will Deacon Cc: Catalin Marinas Cc: Marco Elver Cc: Dmitry Vyukov Cc: "Paul E. McKenney" Cc: Shuah Khan Cc: Gabriele Paoloni Cc: Juri Lelli Cc: Clark Williams Cc: Tao Zhou Cc: Randy Dunlap Cc: linux-doc@vger.kernel.orgCc: linux-kernel@vger.kernel.orgCc: linux-trace-devel@vger.kernel.orgSigned-off-by: Daniel Bristot de Oliveira Signed-off-by: Steven Rostedt (Google) List of files: /linux-6.15/kernel/trace/Makefile

bb5eb8f3 - tracing: Disable kcov on trace_preemptirq.c

Mon, 23 May 2022 06:30:33 +0000

tracing: Disable kcov on trace_preemptirq.cFunctions in trace_preemptirq.c could be invoked from early interruptcode that bypasses kcov trace function's in_task() check. Disable kcovon this file to reduce random code coverage.Link: https://lkml.kernel.org/r/20220523063033.1778974-1-liu3101@purdue.eduAcked-by: Dmitry Vyukov Signed-off-by: Congyu Liu Signed-off-by: Steven Rostedt (Google) List of files: /linux-6.15/kernel/trace/Makefile

54ecbe6f - rethook: Add a generic return hook

Tue, 15 Mar 2022 14:00:50 +0000

rethook: Add a generic return hookAdd a return hook framework which hooks the function return. Most of thelogic came from the kretprobe, but this is independent from kretprobe.Note that this is expected to be used with other function entry hookingfeature, like ftrace, fprobe, adn kprobes. Eventually this will replacethe kretprobe (e.g. kprobe + rethook = kretprobe), but at this moment,this is just an additional hook.Signed-off-by: Masami Hiramatsu Signed-off-by: Steven Rostedt (Google) Tested-by: Steven Rostedt (Google) Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/164735285066.1084943.9259661137330166643.stgit@devnote2 List of files: /linux-6.15/kernel/trace/Makefile

cad9931f - fprobe: Add ftrace based probe APIs

Tue, 15 Mar 2022 14:00:38 +0000

fprobe: Add ftrace based probe APIsThe fprobe is a wrapper API for ftrace function tracer.Unlike kprobes, this probes only supports the function entry, but thiscan probe multiple functions by one fprobe. The usage is similar, userwill set their callback to fprobe::entry_handler and callregister_fprobe*() with probed functions.There are 3 registration interfaces, - register_fprobe() takes filtering patterns of the functin names. - register_fprobe_ips() takes an array of ftrace-location addresses. - register_fprobe_syms() takes an array of function names.The registered fprobes can be unregistered with unregister_fprobe().e.g.struct fprobe fp = { .entry_handler = user_handler };const char *targets[] = { "func1", "func2", "func3"};...ret = register_fprobe_syms(&fp, targets, ARRAY_SIZE(targets));...unregister_fprobe(&fp);Signed-off-by: Masami Hiramatsu Signed-off-by: Steven Rostedt (Google) Tested-by: Steven Rostedt (Google) Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/164735283857.1084943.1154436951479395551.stgit@devnote2 List of files: /linux-6.15/kernel/trace/Makefile

7f5a08c7 - user_events: Add minimal support for trace_event into ftrace

Tue, 18 Jan 2022 20:43:15 +0000

user_events: Add minimal support for trace_event into ftraceMinimal support for interacting with dynamic events, trace_event andftrace. Core outline of flow between user process, ioctl and trace_eventAPIs.User mode processes that wish to use trace events to get data intoftrace, perf, eBPF, etc are limited to uprobes today. The user eventsfeatures enables an ABI for user mode processes to create and write totrace events that are isolated from kernel level trace events. Thisenables a faster path for tracing from user mode data as well as opensmanaged code to participate in trace events, where stub locations aredynamic.User processes often want to trace only when it's useful. To enable thisa set of pages are mapped into the user process space that indicate thecurrent state of the user events that have been registered. Userprocesses can check if their event is hooked to a trace/probe, and if itis, emit the event data out via the write() syscall.Two new files are introduced into tracefs to accomplish this:user_events_status - This file is mmap'd into participating user modeprocesses to indicate event status.user_events_data - This file is opened and register/delete ioctl's areissued to create/open/delete trace events that can be used for tracing.The typical scenario is on process start to mmap user_events_status. Processesthen register the events they plan to use via the REG ioctl. The ioctl readsand updates the passed in user_reg struct. The status_index of the struct isused to know the byte in the status page to check for that event. Thewrite_index of the struct is used to describe that event when writing out tothe fd that was used for the ioctl call. The data must always include thisindex first when writing out data for an event. Data can be written either bywrite() or by writev().For example, in memory:int index;char data[];Psuedo code example of typical usage:struct user_reg reg;int page_fd = open("user_events_status", O_RDWR);char *page_data = mmap(NULL, PAGE_SIZE, PROT_READ, MAP_SHARED, page_fd, 0);close(page_fd);int data_fd = open("user_events_data", O_RDWR);reg.size = sizeof(reg);reg.name_args = (__u64)"test";ioctl(data_fd, DIAG_IOCSREG, ®);int status_id = reg.status_index;int write_id = reg.write_index;struct iovec io[2];io[0].iov_base = &write_id;io[0].iov_len = sizeof(write_id);io[1].iov_base = payload;io[1].iov_len = sizeof(payload);if (page_data[status_id]) writev(data_fd, io, 2);User events are also exposed via the dynamic_events tracefs file forboth create and delete. Current status is exposed via the user_events_statustracefs file.Simple example to register a user event via dynamic_events: echo u:test >> dynamic_events cat dynamic_events u:testIf an event is hooked to a probe, the probe hooked shows up: echo 1 > events/user_events/test/enable cat user_events_status 1:test # Used by ftrace Active: 1 Busy: 1 Max: 4096If an event is not hooked to a probe, no probe status shows up: echo 0 > events/user_events/test/enable cat user_events_status 1:test Active: 1 Busy: 0 Max: 4096Users can describe the trace event format via the following format: name[:FLAG1[,FLAG2...] [field1[;field2...]]Each field has the following format: type nameExample for char array with a size of 20 named msg: echo 'u:detailed char[20] msg' >> dynamic_events cat dynamic_events u:detailed char[20] msgData offsets are based on the data written out via write() and will beupdated to reflect the correct offset in the trace_event fields. For dynamicdata it is recommended to use the new __rel_loc data type. This type will bethe same as __data_loc, but the offset is relative to this entry. This allowsuser_events to not worry about what common fields are being inserted beforethe data.The above format is valid for both the ioctl and the dynamic_events file.Link: https://lkml.kernel.org/r/20220118204326.2169-2-beaub@linux.microsoft.comAcked-by: Masami Hiramatsu Signed-off-by: Beau Belgrave Signed-off-by: Steven Rostedt (Google) List of files: /linux-6.15/kernel/trace/Makefile

6954e415 - tracing: Place trace_pid_list logic into abstract functions

Fri, 24 Sep 2021 01:03:49 +0000

tracing: Place trace_pid_list logic into abstract functionsInstead of having the logic that does trace_pid_list open coded, wrap it inabstract functions. This will allow a rewrite of the logic that implementsthe trace_pid_list without affecting the users.Note, this causes a change in behavior. Every time a pid is written intothe set_*_pid file, it creates a new list and uses RCU to update it. Ifpid_max is lowered, but there was a pid currently in the list that washigher than pid_max, those pids will now be removed on updating the list.The old behavior kept that from happening.The rewrite of the pid_list logic will no longer depend on pid_max,and will return the old behavior.Signed-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile

7491e2c4 - tracing: Add a probe that attaches to trace events

Thu, 19 Aug 2021 15:26:06 +0000

tracing: Add a probe that attaches to trace eventsA new dynamic event is introduced: event probe. The event is attachedto an existing tracepoint and uses its fields as arguments. The usercan specify custom format string of the new event, select what tracepointarguments will be printed and how to print them.An event probe is created by writing configuration string in'dynamic_events' ftrace file: e[:[SNAME/]ENAME] SYSTEM/EVENT [FETCHARGS] - Set an event probe -:SNAME/ENAME - Delete an event probeWhere: SNAME - System name, if omitted 'eprobes' is used. ENAME - Name of the new event in SNAME, if omitted the SYSTEM_EVENT is used. SYSTEM - Name of the system, where the tracepoint is defined, mandatory. EVENT - Name of the tracepoint event in SYSTEM, mandatory. FETCHARGS - Arguments: =$[:TYPE] - Fetch given filed of the tracepoint and print it as given TYPE with given name. Supported types are: (u8/u16/u32/u64/s8/s16/s32/s64), basic type (x8/x16/x32/x64), hexadecimal types "string", "ustring" and bitfield.Example, attach an event probe on openat system call and print name of thefile that will be opened: echo "e:esys/eopen syscalls/sys_enter_openat file=\$filename:string" >> dynamic_eventsA new dynamic event is created in events/esys/eopen/ directory. Itcan be deleted with: echo "-:esys/eopen" >> dynamic_eventsFilters, triggers and histograms can be attached to the new event, it canbe matched in synthetic events. There is one limitation - an event probecan not be attached to kprobe, uprobe or another event probe.Link: https://lkml.kernel.org/r/20210812145805.2292326-1-tz.stoyanov@gmail.comLink: https://lkml.kernel.org/r/20210819152825.142428383@goodmis.orgAcked-by: Masami Hiramatsu Co-developed-by: Steven Rostedt (VMware) Signed-off-by: Tzvetomir Stoyanov (VMware) Signed-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile

bce29ac9 - trace: Add osnoise tracer

Tue, 22 Jun 2021 14:42:27 +0000

trace: Add osnoise tracerIn the context of high-performance computing (HPC), the Operating SystemNoise (*osnoise*) refers to the interference experienced by an applicationdue to activities inside the operating system. In the context of Linux,NMIs, IRQs, SoftIRQs, and any other system thread can cause noise to thesystem. Moreover, hardware-related jobs can also cause noise, for example,via SMIs.The osnoise tracer leverages the hwlat_detector by running a similarloop with preemption, SoftIRQs and IRQs enabled, thus allowing allthe sources of *osnoise* during its execution. Using the same approachof hwlat, osnoise takes note of the entry and exit point of anysource of interferences, increasing a per-cpu interference counter. Theosnoise tracer also saves an interference counter for each source ofinterference. The interference counter for NMI, IRQs, SoftIRQs, andthreads is increased anytime the tool observes these interferences' entryevents. When a noise happens without any interference from the operatingsystem level, the hardware noise counter increases, pointing to ahardware-related noise. In this way, osnoise can account for anysource of interference. At the end of the period, the osnoise tracerprints the sum of all noise, the max single noise, the percentage of CPUavailable for the thread, and the counters for the noise sources.UsageWrite the ASCII text "osnoise" into the current_tracer file of thetracing system (generally mounted at /sys/kernel/tracing).For example:: [root@f32 ~]# cd /sys/kernel/tracing/ [root@f32 tracing]# echo osnoise > current_tracerIt is possible to follow the trace by reading the trace trace file:: [root@f32 tracing]# cat trace # tracer: osnoise # # _-----=> irqs-off # / _----=> need-resched # | / _---=> hardirq/softirq # || / _--=> preempt-depth MAX # || / SINGLE Interference counters: # |||| RUNTIME NOISE % OF CPU NOISE +-----------------------------+ # TASK-PID CPU# |||| TIMESTAMP IN US IN US AVAILABLE IN US HW NMI IRQ SIRQ THREAD # | | | |||| | | | | | | | | | | <...>-859 [000] .... 81.637220: 1000000 190 99.98100 9 18 0 1007 18 1 <...>-860 [001] .... 81.638154: 1000000 656 99.93440 74 23 0 1006 16 3 <...>-861 [002] .... 81.638193: 1000000 5675 99.43250 202 6 0 1013 25 21 <...>-862 [003] .... 81.638242: 1000000 125 99.98750 45 1 0 1011 23 0 <...>-863 [004] .... 81.638260: 1000000 1721 99.82790 168 7 0 1002 49 41 <...>-864 [005] .... 81.638286: 1000000 263 99.97370 57 6 0 1006 26 2 <...>-865 [006] .... 81.638302: 1000000 109 99.98910 21 3 0 1006 18 1 <...>-866 [007] .... 81.638326: 1000000 7816 99.21840 107 8 0 1016 39 19In addition to the regular trace fields (from TASK-PID to TIMESTAMP), thetracer prints a message at the end of each period for each CPU that isrunning an osnoise/CPU thread. The osnoise specific fields report: - The RUNTIME IN USE reports the amount of time in microseconds that the osnoise thread kept looping reading the time. - The NOISE IN US reports the sum of noise in microseconds observed by the osnoise tracer during the associated runtime. - The % OF CPU AVAILABLE reports the percentage of CPU available for the osnoise thread during the runtime window. - The MAX SINGLE NOISE IN US reports the maximum single noise observed during the runtime window. - The Interference counters display how many each of the respective interference happened during the runtime window.Note that the example above shows a high number of HW noise samples.The reason being is that this sample was taken on a virtual machine,and the host interference is detected as a hardware interference.Tracer optionsThe tracer has a set of options inside the osnoise directory, they are: - osnoise/cpus: CPUs at which a osnoise thread will execute. - osnoise/period_us: the period of the osnoise thread. - osnoise/runtime_us: how long an osnoise thread will look for noise. - osnoise/stop_tracing_us: stop the system tracing if a single noise higher than the configured value happens. Writing 0 disables this option. - osnoise/stop_tracing_total_us: stop the system tracing if total noise higher than the configured value happens. Writing 0 disables this option. - tracing_threshold: the minimum delta between two time() reads to be considered as noise, in us. When set to 0, the default value will be used, which is currently 5 us.Additional TracingIn addition to the tracer, a set of tracepoints were added tofacilitate the identification of the osnoise source. - osnoise:sample_threshold: printed anytime a noise is higher than the configurable tolerance_ns. - osnoise:nmi_noise: noise from NMI, including the duration. - osnoise:irq_noise: noise from an IRQ, including the duration. - osnoise:softirq_noise: noise from a SoftIRQ, including the duration. - osnoise:thread_noise: noise from a thread, including the duration.Note that all the values are *net values*. For example, if while osnoiseis running, another thread preempts the osnoise thread, it will start athread_noise duration at the start. Then, an IRQ takes place, preemptingthe thread_noise, starting a irq_noise. When the IRQ ends its execution,it will compute its duration, and this duration will be subtracted fromthe thread_noise, in such a way as to avoid the double accounting of theIRQ execution. This logic is valid for all sources of noise.Here is one example of the usage of these tracepoints:: osnoise/8-961 [008] d.h. 5789.857532: irq_noise: local_timer:236 start 5789.857529929 duration 1845 ns osnoise/8-961 [008] dNh. 5789.858408: irq_noise: local_timer:236 start 5789.858404871 duration 2848 ns migration/8-54 [008] d... 5789.858413: thread_noise: migration/8:54 start 5789.858409300 duration 3068 ns osnoise/8-961 [008] .... 5789.858413: sample_threshold: start 5789.858404555 duration 8723 ns interferences 2In this example, a noise sample of 8 microseconds was reported in the lastline, pointing to two interferences. Looking backward in the trace, thetwo previous entries were about the migration thread running after atimer IRQ execution. The first event is not part of the noise becauseit took place one millisecond before.It is worth noticing that the sum of the duration reported in thetracepoints is smaller than eight us reported in the sample_threshold.The reason roots in the overhead of the entry and exit code that happensbefore and after any interference execution. This justifies the dualapproach: measuring thread and tracing.Link: https://lkml.kernel.org/r/e649467042d60e7b62714c9c6751a56299d15119.1624372313.git.bristot@redhat.comCc: Phil Auld Cc: Sebastian Andrzej Siewior Cc: Kate Carcia Cc: Jonathan Corbet Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Alexandre Chartre Cc: Clark Willaims Cc: John Kacur Cc: Juri Lelli Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: x86@kernel.orgCc: linux-doc@vger.kernel.orgCc: linux-kernel@vger.kernel.orgSigned-off-by: Daniel Bristot de Oliveira [ Made the following functions static: trace_irqentry_callback() trace_irqexit_callback() trace_intel_irqentry_callback() trace_intel_irqexit_callback() Added to include/trace.h: osnoise_arch_register() osnoise_arch_unregister() Fixed define logic for LATENCY_FS_NOTIFY Reported-by: kernel test robot ]Signed-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile

9c0dee54 - tracing: add error_report_end trace point

Fri, 26 Feb 2021 01:19:44 +0000

tracing: add error_report_end trace pointPatch series "Add error_report_end tracepoint to KFENCE and KASAN", v3.This patchset adds a tracepoint, error_repor_end, that is to be used byKFENCE, KASAN, and potentially other bug detection tools, when they printan error report. One of the possible use cases is userspace collection ofkernel error reports: interested parties can subscribe to the tracingevent via tracefs, and get notified when an error report occurs.This patch (of 3):Introduce error_report_end tracepoint. It can be used in debugging toolslike KASAN, KFENCE, etc. to provide extensions to the error reportingmechanisms (e.g. allow tests hook into error reporting, ease error reportcollection from production kernels). Another benefit would be making useof ftrace for debugging or benchmarking the tools themselves.Should we need it, the tracepoint name leaves us with the possibility tointroduce a complementary error_report_start tracepoint in the future.Link: https://lkml.kernel.org/r/20210121131915.1331302-1-glider@google.comLink: https://lkml.kernel.org/r/20210121131915.1331302-2-glider@google.comSigned-off-by: Alexander Potapenko Suggested-by: Marco Elver Cc: Andrey Konovalov Cc: Dmitry Vyukov Cc: Ingo Molnar Cc: Petr Mladek Cc: Steven Rostedt Cc: Sergey Senozhatsky Cc: Greg Kroah-Hartman Cc: Vlastimil Babka Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/kernel/trace/Makefile

773c1670 - ftrace: Add recording of functions that caused recursion

Fri, 06 Nov 2020 02:32:46 +0000

ftrace: Add recording of functions that caused recursionThis adds CONFIG_FTRACE_RECORD_RECURSION that will record to a file"recursed_functions" all the functions that caused recursion while acallback to the function tracer was running.Link: https://lkml.kernel.org/r/20201106023548.102375687@goodmis.orgCc: Masami Hiramatsu Cc: Andrew Morton Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Jonathan Corbet Cc: Guo Ren Cc: "James E.J. Bottomley" Cc: Helge Deller Cc: Michael Ellerman Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Heiko Carstens Cc: Vasily Gorbik Cc: Christian Borntraeger Cc: Thomas Gleixner Cc: Borislav Petkov Cc: x86@kernel.orgCc: "H. Peter Anvin" Cc: Kees Cook Cc: Anton Vorontsov Cc: Colin Cross Cc: Tony Luck Cc: Josh Poimboeuf Cc: Jiri Kosina Cc: Miroslav Benes Cc: Petr Mladek Cc: Joe Lawrence Cc: Kamalesh Babulal Cc: Mauro Carvalho Chehab Cc: Sebastian Andrzej Siewior Cc: linux-doc@vger.kernel.orgCc: linux-kernel@vger.kernel.orgCc: linux-csky@vger.kernel.orgCc: linux-parisc@vger.kernel.orgCc: linuxppc-dev@lists.ozlabs.orgCc: linux-s390@vger.kernel.orgCc: live-patching@vger.kernel.orgSigned-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile

15d5761a - kbuild: introduce ccflags-remove-y and asflags-remove-y

Tue, 07 Jul 2020 09:21:16 +0000

kbuild: introduce ccflags-remove-y and asflags-remove-yCFLAGS_REMOVE_.o filters out flags when compiling a particularobject, but there is no convenient way to do that for every object ina directory.Add ccflags-remove-y and asflags-remove-y to make it easily.Use ccflags-remove-y to clean up some Makefiles.The add/remove order works as follows: [1] KBUILD_CFLAGS specifies compiler flags used globally [2] ccflags-y adds compiler flags for all objects in the current Makefile [3] ccflags-remove-y removes compiler flags for all objects in the current Makefile (New feature) [4] CFLAGS_ adds compiler flags per file. [5] CFLAGS_REMOVE_ removes compiler flags per file.Having [3] before [4] allows us to remove flags from most (but not all)objects in the current Makefile.For example, kernel/trace/Makefile removes $(CC_FLAGS_FTRACE)from all objects in the directory, then adds it back totrace_selftest_dynamic.o and CFLAGS_trace_kprobe_selftest.oThe same applies to lib/livepatch/Makefile.Please note ccflags-remove-y has no effect to the sub-directories.In contrast, the previous notation got rid of compiler flags also fromall the sub-directories.The following are not affected because they have no sub-directories: arch/arm/boot/compressed/ arch/powerpc/xmon/ arch/sh/ kernel/trace/However, lib/ has several sub-directories.To keep the behavior, I added ccflags-remove-y to all Makefilesin subdirectories of lib/, except the following: lib/vdso/Makefile - Kbuild does not descend into this Makefile lib/raid/test/Makefile - This is not used for the kernel buildI think commit 2464a609ded0 ("ftrace: do not trace library functions")excluded too much. In the next commit, I will remove ccflags-remove-yfrom the sub-directories of lib/.Suggested-by: Sami Tolvanen Signed-off-by: Masahiro Yamada Acked-by: Steven Rostedt (VMware) Acked-by: Michael Ellerman (powerpc)Acked-by: Brendan Higgins (KUnit)Tested-by: Anders Roxell List of files: /linux-6.15/kernel/trace/Makefile

ac5a72ea - bpf: Use dedicated bpf_trace_printk event instead of trace_printk()

Mon, 13 Jul 2020 11:52:33 +0000

bpf: Use dedicated bpf_trace_printk event instead of trace_printk()The bpf helper bpf_trace_printk() uses trace_printk() under the hood.This leads to an alarming warning message originating from tracebuffer allocation which occurs the first time a program usingbpf_trace_printk() is loaded.We can instead create a trace event for bpf_trace_printk() and enableit in-kernel when/if we encounter a program using thebpf_trace_printk() helper. With this approach, trace_printk()is not used directly and no warning message appears.This work was started by Steven (see Link) and finished by Alan; addedSteven's Signed-off-by with his permission.Signed-off-by: Steven Rostedt (VMware) Signed-off-by: Alan Maguire Signed-off-by: Alexei Starovoitov Acked-by: Andrii Nakryiko Link: https://lore.kernel.org/r/20200628194334.6238b933@oasis.local.homeLink: https://lore.kernel.org/bpf/1594641154-18897-2-git-send-email-alan.maguire@oracle.com List of files: /linux-6.15/kernel/trace/Makefile

726721a5 - tracing: Move synthetic events to a separate file

Thu, 28 May 2020 19:32:37 +0000

tracing: Move synthetic events to a separate fileWith the addition of the in-kernel synthetic event API, syntheticevents are no longer specifically tied to the histogram triggers.The synthetic event code is also making trace_event_hist.c verybloated, so for those reasons, move it to a separate file,trace_events_synth.c, along with a new trace_synth.h header file.Because synthetic events are now independent from hist triggers, add anew CONFIG_SYNTH_EVENTS config option, and have CONFIG_HIST_TRIGGERSselect it, and have CONFIG_SYNTH_EVENT_GEN_TEST depend on it.Link: http://lkml.kernel.org/r/4d1fa1f85ed5982706ac44844ac92451dcb04715.1590693308.git.zanussi@kernel.orgSigned-off-by: Tom Zanussi Signed-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile

f5d2313b - kcsan, trace: Make KCSAN compatible with tracing

Fri, 14 Feb 2020 21:10:35 +0000

kcsan, trace: Make KCSAN compatible with tracingPreviously the system would lock up if ftrace was enabled together withKCSAN. This is due to recursion on reporting if the tracer code isinstrumented with KCSAN.To avoid this for all types of tracing, disable KCSAN instrumentationfor all of kernel/trace.Furthermore, since KCSAN relies on udelay() to introduce delay, we haveto disable ftrace for udelay() (currently done for x86) in case KCSAN isused together with lockdep and ftrace. The reason is that it may corruptlockdep IRQ flags tracing state due to a peculiar case of recursion(details in Makefile comment).Reported-by: Qian Cai Tested-by: Qian Cai Acked-by: Steven Rostedt (VMware) Signed-off-by: Marco Elver Signed-off-by: Paul E. McKenney Signed-off-by: Ingo Molnar List of files: /linux-6.15/kernel/trace/Makefile

64836248 - tracing: Add kprobe event command generation test module

Wed, 29 Jan 2020 18:59:31 +0000

tracing: Add kprobe event command generation test moduleAdd a test module that checks the basic functionality of the in-kernelkprobe event command generation API by creating kprobe events from amodule.Link: http://lkml.kernel.org/r/97e502b204f9dba948e3fa3a4315448298218787.1580323897.git.zanussi@kernel.orgAcked-by: Masami Hiramatsu Signed-off-by: Tom Zanussi Signed-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile

9fe41efa - tracing: Add synth event generation test module

Wed, 29 Jan 2020 18:59:28 +0000

tracing: Add synth event generation test moduleAdd a test module that checks the basic functionality of the in-kernelsynthetic event generation API by generating and tracing syntheticevents from a module.Link: http://lkml.kernel.org/r/fcb4dd9eb9eefb70ab20538d3529d51642389664.1580323897.git.zanussi@kernel.orgAcked-by: Masami Hiramatsu Signed-off-by: Tom Zanussi Signed-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile

9c5b9d3d - tracing/boot: Add boot-time tracing

Fri, 10 Jan 2020 16:06:17 +0000

tracing/boot: Add boot-time tracingSetup tracing options via extra boot config in addition to kernelcommand line.This adds following commands support. These are applied tothe global trace instance. - ftrace.options = OPT1[,OPT2...] Enable given ftrace options. - ftrace.trace_clock = CLOCK Set given CLOCK to ftrace's trace_clock. - ftrace.buffer_size = SIZE Configure ftrace buffer size to SIZE. You can use "KB" or "MB" for that SIZE. - ftrace.events = EVENT[, EVENT2...] Enable given events on boot. You can use a wild card in EVENT. - ftrace.tracer = TRACER Set TRACER to current tracer on boot. (e.g. function)Note that this is NOT replacing the kernel parameters, becausethis boot config based setting is later than that. If you want totrace earlier boot events, you still need kernel parameters.Link: http://lkml.kernel.org/r/157867237723.17873.17494943526320587488.stgit@devnote2Signed-off-by: Masami Hiramatsu Signed-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile

6c3edaf9 - tracing: Introduce trace event injection

Sat, 30 Nov 2019 04:52:18 +0000

tracing: Introduce trace event injectionWe have been trying to use rasdaemon to monitor hardware errors likecorrectable memory errors. rasdaemon uses trace events to monitorvarious hardware errors. In order to test it, we have to inject somehardware errors, unfortunately not all of them provide errorinjections. MCE does provide a way to inject MCE errors, but errorslike PCI error and devlink error don't, it is not easy to add errorinjection to each of them. Instead, it is relatively easier to justallow users to inject trace events in a generic way so that all traceevents can be injected.This patch introduces trace event injection, where a new 'inject' isadded to each tracepoint directory. Users could write into this filewith key=value pairs to specify the value of each fields of the traceevent, all unspecified fields are set to zero values by default.For example, for the net/net_dev_queue tracepoint, we can inject: INJECT=/sys/kernel/debug/tracing/events/net/net_dev_queue/inject echo "" > $INJECT echo "name='test'" > $INJECT echo "name='test' len=1024" > $INJECT cat /sys/kernel/debug/tracing/trace ... <...>-614 [000] .... 36.571483: net_dev_queue: dev= skbaddr=00000000fbf338c2 len=0 <...>-614 [001] .... 136.588252: net_dev_queue: dev=test skbaddr=00000000fbf338c2 len=0 <...>-614 [001] .N.. 208.431878: net_dev_queue: dev=test skbaddr=00000000fbf338c2 len=1024Triggers could be triggered as usual too: echo "stacktrace if len == 1025" > /sys/kernel/debug/tracing/events/net/net_dev_queue/trigger echo "len=1025" > $INJECT cat /sys/kernel/debug/tracing/trace ... bash-614 [000] .... 36.571483: net_dev_queue: dev= skbaddr=00000000fbf338c2 len=0 bash-614 [001] .... 136.588252: net_dev_queue: dev=test skbaddr=00000000fbf338c2 len=0 bash-614 [001] .N.. 208.431878: net_dev_queue: dev=test skbaddr=00000000fbf338c2 len=1024 bash-614 [001] .N.1 284.236349: => event_inject_write => vfs_write => ksys_write => do_syscall_64 => entry_SYSCALL_64_after_hwframeThe only thing that can't be injected is string pointers as theyrequire constant string pointers, this can't be done at run time.Link: http://lkml.kernel.org/r/20191130045218.18979-1-xiyou.wangcong@gmail.comCc: Ingo Molnar Signed-off-by: Cong Wang Signed-off-by: Steven Rostedt (VMware) List of files: /linux-6.15/kernel/trace/Makefile