dtrace: Avoid including dtrace_isa.c directly into dtrace.cThis was done in the original DTrace import, presumably because thatmade it a bit easier to handle includes. However, this can causedtr
dtrace: Avoid including dtrace_isa.c directly into dtrace.cThis was done in the original DTrace import, presumably because thatmade it a bit easier to handle includes. However, this can causedtrace_getpcstack() to be inlined into dtrace_probe(), resulting in amissing frame in stack traces since dtrace_getpcstack() takes care tobump "aframes" to account for its own stack frame.To avoid this, compile dtrace_isa.c separately on all platforms. Addrequisite includes.MFC after: 2 weeksSponsored by: Innovate UK(cherry picked from commit 82283cad12a417abfb1469d899b2d7cfb1d38f77)
show more ...
sys: Remove $FreeBSD$: one-line .h patternRemove /^\s*\*+\s*\$FreeBSD\$.*$\n/
sys: Remove $FreeBSD$: two-line .h patternRemove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
dtrace: rename rp to frame in dtrace_getreg()Reviewed by: mhorne, markjApproved by: markj (mentor)Sponsored by: The FreeBSD FoundationDifferential Revision: https://reviews.freebsd.org/D40231
dtrace/powerpc: Remove accidental commented out lineFixes: 3e1155ade1b
dtrace/powerpc: "Fix" stack traces across trap framesIn function boundary tracing the link register is not yet saved to thesave stack location, so the save point contains whatever the previous'lr
dtrace/powerpc: "Fix" stack traces across trap framesIn function boundary tracing the link register is not yet saved to thesave stack location, so the save point contains whatever the previous'lr' save was, or even garbage, at the time the trap is taken. Addressthis by explicitly loading the link register from the trap frame insteadof the stack, and propagate that out.
dtrace: Add a "regs" variableThis allows invop-based providers (i.e., fbt and kinst) to expose theregister file of the CPU at the point where the probe fired. It doesnot work for SDT providers b
dtrace: Add a "regs" variableThis allows invop-based providers (i.e., fbt and kinst) to expose theregister file of the CPU at the point where the probe fired. It doesnot work for SDT providers because their probes are implemented as plainfunction calls and so don't save registers. It's not clear whatsemantics "regs" should have for them anyway.This is akin to "uregs", which nominally provides access to theuserspace registers. In fact, DIF already had a DIF_VAR_REGS variabledefined, it was simply unimplemented.Usage example: print the contents of %rdi upon each call toamd64_syscall(): fbt::amd64_syscall:entry {printf("%x", regs[R_RDI]);}Note that the R_* constants are defined in /usr/lib/dtrace/regs_x86.d.Currently there are no similar definitions for non-x86 platforms.Reviewed by: christosMFC after: 2 weeksDifferential Revision: https://reviews.freebsd.org/D36799
Adjust function definition in powerpc's dtrace_subr.c to avoid clang 15 warningWith clang 15, the following -Werror warning is produced: sys/cddl/dev/dtrace/powerpc/dtrace_subr.c:237:17: error
Adjust function definition in powerpc's dtrace_subr.c to avoid clang 15 warningWith clang 15, the following -Werror warning is produced: sys/cddl/dev/dtrace/powerpc/dtrace_subr.c:237:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes] dtrace_gethrtime() ^ voidThis is because dtrace_gethrtime() is declared with a (void) argumentlist, but defined with an empty argument list. Make the definition matchthe declaration.MFC after: 3 days
Create sys/reg.h for the common code previously in machine/reg.hMove the common kernel function signatures from machine/reg.h to a newsys/reg.h. This is in preperation for adding PT_GETREGSET to p
Create sys/reg.h for the common code previously in machine/reg.hMove the common kernel function signatures from machine/reg.h to a newsys/reg.h. This is in preperation for adding PT_GETREGSET to ptrace(2).Reviewed by: imp, markjSponsored by: DARPA, AFRL (original work)Sponsored by: The FreeBSD FoundationDifferential Revision: https://reviews.freebsd.org/D19830
powerpc/dtrace: Actually fix stack tracesFix stack unwinding such that requesting N stack frames in lockstat willactually give you N frames, not anywhere from 0-3 as had been before.lockstat pri
powerpc/dtrace: Actually fix stack tracesFix stack unwinding such that requesting N stack frames in lockstat willactually give you N frames, not anywhere from 0-3 as had been before.lockstat prints the mutex function instead of the caller as the reportedlocker, but the stack frame is detailed enough to find the real caller.MFC after: 2 weeks
powerpc/dtrace: Fix dtrace powerpc asm, and simplify stack walkingFix some execution bugs in the dtrace powerpc asm. addme pulls in the carryflag which we don't want, and the result wasn't record
powerpc/dtrace: Fix dtrace powerpc asm, and simplify stack walkingFix some execution bugs in the dtrace powerpc asm. addme pulls in the carryflag which we don't want, and the result wasn't recorded anyways, so thefollowing beq to check for exit condition wasn't checking the rightcondition.Simplify the stack walking in dtrace_isa.c, so there's only a single walkerthat handles both pc and sp. This should make it easier to follow, and anybugfix that may be needed for walking only needs to be made in one placeinstead of two now.MFC after: 2 weeks
powerpc/dtrace: Use explicit bit numbers to mask out PSL_EEThere seems to be a race in CI, such that dtrace_asm.S might be assembledbefore the genassym is completed. This causes a build failure w
powerpc/dtrace: Use explicit bit numbers to mask out PSL_EEThere seems to be a race in CI, such that dtrace_asm.S might be assembledbefore the genassym is completed. This causes a build failure when PSL_EEdoesn't exist, and is read as 0. Get around this by explicitly specifyingthe bits in the mask instead.
dtrace/powerpc: Correct register indices for non-indexed registers in the trapframeFix an off-by-one error, LR starts at index 32, not index 33, and the othersfollow suit.
Protect dtrace_getpcstack() from a NULL stack pointer in a trap frameFound when trying to use lockstat on a POWER9, the stack pointer (r1) couldbe NULL, and result in a NULL pointer dereference, c
Protect dtrace_getpcstack() from a NULL stack pointer in a trap frameFound when trying to use lockstat on a POWER9, the stack pointer (r1) couldbe NULL, and result in a NULL pointer dereference, crashing the kernel.
Rename assym.s to assym.incassym is only to be included by other .s files, and should neveractually be assembled by itself.Reviewed by: imp, bdrewery (earlier)Sponsored by: The FreeBSD Foundati
Rename assym.s to assym.incassym is only to be included by other .s files, and should neveractually be assembled by itself.Reviewed by: imp, bdrewery (earlier)Sponsored by: The FreeBSD FoundationDifferential Revision: https://reviews.freebsd.org/D14180
Follow up r313841 on powerpcClose a potential race in reading the CPU dtrace flags, where a thread canstart on one CPU, and partway through retrieving the flags be swapped out,while another threa
Follow up r313841 on powerpcClose a potential race in reading the CPU dtrace flags, where a thread canstart on one CPU, and partway through retrieving the flags be swapped out,while another thread traps and sets the CPU_DTRACE_NOFAULT. This couldcause the first thread to return without handling the fault.Discussed with: markj@
Fix stack tracing in dtrace for powerpcThe current method only sort of works, and usually doesn't work reliably.Also, on Book-E the return address from DEBUG exceptions is not the sentineladdress
Fix stack tracing in dtrace for powerpcThe current method only sort of works, and usually doesn't work reliably.Also, on Book-E the return address from DEBUG exceptions is not the sentineladdresses, so it won't exit the loop correctly.Fix this by better handling trap frames during unwinding, and using thecommon trap handler for debug traps, as the code in that segment isidentical between the two.MFC after: 1 week
Corrected misspelled versions of rendezvous.The MFC will include a compat definition of smp_no_rendevous_barrier()that calls smp_no_rendezvous_barrier().Reviewed by: gnn, kibMFC after: 1 weekD
Corrected misspelled versions of rendezvous.The MFC will include a compat definition of smp_no_rendevous_barrier()that calls smp_no_rendezvous_barrier().Reviewed by: gnn, kibMFC after: 1 weekDifferential Revision: https://reviews.freebsd.org/D10313
Two fixups for dtrace* Use the right incantation to get the next stack pointer. Since powerpc uses special frames for traps, dereferencing the stack pointer straight up won't get us the next s
Two fixups for dtrace* Use the right incantation to get the next stack pointer. Since powerpc uses special frames for traps, dereferencing the stack pointer straight up won't get us the next stack pointer in every case.* Clear EE using the correct instruction sequence. The PowerISA states that 'andi.' ANDs the register with 0||<imm>, instead of sign extending or filling out the unavailable bits with 1. Even if it did sign extend, PSL_EE is 0x8000, so ~PSL_EE is 0x7fff, and the upper bits would be cleared. Use rlwinm in the 32-bit case, and a two-rotate sequence in the 64-bit case, the latter chosen to follow the output generated by gcc.MFC after: 1 week
Add an EARLY_AP_STARTUP option to start APs earlier during boot.Currently, Application Processors (non-boot CPUs) are started byMD code at SI_SUB_CPU, but they are kept waiting in a "pen" untilSI
Add an EARLY_AP_STARTUP option to start APs earlier during boot.Currently, Application Processors (non-boot CPUs) are started byMD code at SI_SUB_CPU, but they are kept waiting in a "pen" untilSI_SUB_SMP at which point they are released to run kernel threads.SI_SUB_SMP is one of the last SYSINIT levels, so APs don't enterthe scheduler and start running threads until fairly late in theboot.This change moves SI_SUB_SMP up to just before software interruptthreads are created allowing the APs to start executing kernelthreads much sooner (before any devices are probed). This allowsseveral initialization routines that need to perform initializationon all CPUs to now perform that initialization in one step ratherthan having to defer the AP initialization to a second SYSINIT runat SI_SUB_SMP. It also permits all CPUs to be available forhandling interrupts before any devices are probed.This last feature fixes a problem on with interrupt vector exhaustion.Specifically, in the old model all device interrupts were routedonto the boot CPU during boot. Later after the APs were released atSI_SUB_SMP, interrupts were redistributed across all CPUs.However, several drivers for multiqueue hardware allocate N interruptsper CPU in the system. In a system with many CPUs, just a few driversdoing this could exhaust the available pool of interrupt vectors onthe boot CPU as each driver was allocating N * mp_ncpu vectors on theboot CPU. Now, drivers will allocate interrupts on their desired CPUsduring boot meaning that only N interrupts are allocated from the bootCPU instead of N * mp_ncpu.Some other bits of code can also be simplified as smp_started isnow true much earlier and will now always be true for these bits ofcode. This removes the need to treat the single-CPU boot environmentas a special case.As a transition aid, the new behavior is available under a new kerneloption (EARLY_AP_STARTUP). This will allow the option to be turned offif need be during initial testing. I plan to enable this on x86 bydefault in a followup commit in the next few days and to have allplatforms moved over before 11.0. Once the transition is complete,the option will be removed along with the !EARLY_AP_STARTUP code.These changes have only been tested on x86. Other platform maintainersare encouraged to port their architectures over as well. The mainthings to check for are any uses of smp_started in MD code that can besimplified and SI_SUB_SMP SYSINITs in MD code that can be removed inthe EARLY_AP_STARTUP case (e.g. the interrupt shuffling).PR: kern/199321Reviewed by: markj, gnn, kibSponsored by: Netflix
Make the second argument of dtrace_invop() a trapframe pointer.Currently this argument is a pointer into the stack which is used by FBTto fetch the first five probe arguments. On all non-x86 archi
Make the second argument of dtrace_invop() a trapframe pointer.Currently this argument is a pointer into the stack which is used by FBTto fetch the first five probe arguments. On all non-x86 architectures it'ssimply the trapframe address, so this change has no functional impact. Onamd64 it's a pointer into the trapframe such that stack[1 .. 5] gives thefirst five argument registers, which are deliberately grouped together inthe amd64 trapframe definition.A trapframe argument simplifies the invop handlers on !x86 and makes thex86 FBT invop handler easier to understand. Moreover, it allows for invophandlers that may want to modify the register set of the interrupted thread.
Remove unused variables dtrace_in_probe and dtrace_in_probe_addr.
dtrace_getarg: remove stray return statement on amd64, powerpcMFC after: 10 days
Fix DTrace's panic() action.It would previously call into some unfinished Solaris compatibility code andreturn without actually calling panic(9). The compatibility code isunneeded, however, so ju
Fix DTrace's panic() action.It would previously call into some unfinished Solaris compatibility code andreturn without actually calling panic(9). The compatibility code isunneeded, however, so just remove it and have dtrace_panic() call vpanic(9)directly.Differential Revision: https://reviews.freebsd.org/D2349Reviewed by: avgMFC after: 2 weeksSponsored by: EMC / Isilon Storage Division
Fix build after unifying DAR/DEAR storage in trap frame.
12