[BPF] handle unsigned icmp ops in BPFAdjustOpt pass

When investigating an issue with bcc tool inject.py, I found
a verifier failure with latest clang. The portion of code
can be illustrated as below

[BPF] handle unsigned icmp ops in BPFAdjustOpt pass

When investigating an issue with bcc tool inject.py, I found
a verifier failure with latest clang. The portion of code
can be illustrated as below:
struct pid_struct {
u64 curr_call;
u64 conds_met;
u64 stack[2];
};
struct pid_struct *bpf_map_lookup_elem();
int foo() {
struct pid_struct *p = bpf_map_lookup_elem();
if (!p) return 0;
p->curr_call--;
if (p->conds_met < 1 || p->conds_met >= 3)
return 0;
if (p->stack[p->conds_met - 1] == p->curr_call)
p->conds_met--;
...
}

The verifier failure looks like:
...
8: (79) r1 = *(u64 *)(r0 +0)
R0_w=map_value(id=0,off=0,ks=4,vs=32,imm=0) R10=fp0 fp-8=mmmm????
9: (07) r1 += -1
10: (7b) *(u64 *)(r0 +0) = r1
R0_w=map_value(id=0,off=0,ks=4,vs=32,imm=0) R1_w=inv(id=0) R10=fp0 fp-8=mmmm????
11: (79) r2 = *(u64 *)(r0 +8)
R0_w=map_value(id=0,off=0,ks=4,vs=32,imm=0) R1_w=inv(id=0) R10=fp0 fp-8=mmmm????
12: (bf) r3 = r2
13: (07) r3 += -3
14: (b7) r4 = -2
15: (2d) if r4 > r3 goto pc+13
R0=map_value(id=0,off=0,ks=4,vs=32,imm=0) R1=inv(id=0) R2=inv(id=2)
R3=inv(id=0,umin_value=18446744073709551614,var_off=(0xffffffff00000000; 0xffffffff))
R4=inv-2 R10=fp0 fp-8=mmmm????
16: (07) r2 += -1
17: (bf) r3 = r2
18: (67) r3 <<= 3
19: (bf) r4 = r0
20: (0f) r4 += r3
math between map_value pointer and register with unbounded min value is not allowed

Here the compiler optimized "p->conds_met < 1 || p->conds_met >= 3" to
r2 = p->conds_met
r3 = r2
r3 += -3
r4 = -2
if (r3 < r4) return 0
r2 += -1
r3 = r2
...
In the above, r3 is initially equal to r2, but is modified used by the comparison.
But later on r2 is used again. This caused verification failure.

BPF backend has a pass, AdjustOpt, to prevent such transformation, but only
focused on signed integers since typical bpf helper returns signed integers.
To fix this case, let us handle unsigned integers as well.

Differential Revision: https://reviews.llvm.org/D121937

show more ...

[BPF] Fix a bug in BPFAdjustOpt pass for icmp transformation

When checking a bcc issue related to bcc tool inject.py,
I found a bug in BPFAdjustOpt pass for icmp transformation,
caused by typo's. Fo

[BPF] Fix a bug in BPFAdjustOpt pass for icmp transformation

When checking a bcc issue related to bcc tool inject.py,
I found a bug in BPFAdjustOpt pass for icmp transformation,
caused by typo's. For the following condition:
Cond2Op != ICmpInst::ICMP_SLT && Cond1Op != ICmpInst::ICMP_SLE
it should be
Cond2Op != ICmpInst::ICMP_SLT && Cond2Op != ICmpInst::ICMP_SLE

This patch fixed the problem and a test case is added.

Differential Revision: https://reviews.llvm.org/D121883

show more ...

BPF: Workaround InstCombine trunc+icmp => mask+icmp Optimization

Patch [1] added further InstCombine trunc+icmp => mask+icmp
optimization and this caused a couple of bpf selftest failure.
Previous l

BPF: Workaround InstCombine trunc+icmp => mask+icmp Optimization

Patch [1] added further InstCombine trunc+icmp => mask+icmp
optimization and this caused a couple of bpf selftest failure.
Previous llvm BPF backend patch [2] introduced llvm.bpf.compare
builtin to handle such situations.

This patch further added support ">" and ">=" icmp opcodes.
Tested with bpf selftests and all tests are passed including two
previously failed ones.

Note Patch [1] also added optimization if the to-be-compared
constant is negative-power-of-2 (-C) or not-of-power-of-2 (~C).
This patch didn't implement these two cases as typical bpf
program compares a scalar to a positive length or boundary value,
and this scalar later is used as a index into an array buffer
or packet buffer.

[1] https://reviews.llvm.org/D112634
[2] https://reviews.llvm.org/D112938

Differential Revision: https://reviews.llvm.org/D114215

show more ...

BPF: Workaround an InstCombine ICmp transformation with llvm.bpf.compare builtin

Commit acabad9ff6bf ("[InstCombine] try to canonicalize icmp with
trunc op into mask and cmp") added a transformation

BPF: Workaround an InstCombine ICmp transformation with llvm.bpf.compare builtin

Commit acabad9ff6bf ("[InstCombine] try to canonicalize icmp with
trunc op into mask and cmp") added a transformation to
convert "(conv)a < power_2_const" to "a & <const>" in certain
cases and bpf kernel verifier has to handle the resulted code
conservatively and this may reject otherwise legitimate program.

This commit tries to prevent such a transformation. A bpf backend
builtin llvm.bpf.compare is added. The ICMP insn, which is subject to
above InstCombine transformation, is converted to the builtin
function. The builtin function is later lowered to original ICMP insn,
certainly after InstCombine pass.

With this change, all affected bpf strobemeta* selftests are
passed now.

Differential Revision: https://reviews.llvm.org/D112938

show more ...

Update BPFAdjustOpt.cpp to accept select form of or as well

This is a minor pattern-match update to BPFAdjustOpt.cpp to accept
not only 'or i1 a, b' but also 'select i1 a, i1 true, i1 b'.
This resol

Update BPFAdjustOpt.cpp to accept select form of or as well

This is a minor pattern-match update to BPFAdjustOpt.cpp to accept
not only 'or i1 a, b' but also 'select i1 a, i1 true, i1 b'.
This resolves regression after SimplifyCFG's creating select form
of and/or instead (https://reviews.llvm.org/D95026).
This is a small change, and currently such select form isn't created
or doesn't reach to the late pipeline (because InstCombine eagerly
folds it into and/or i1), so I chose to commit without a review process.

show more ...

[llvm] Use isa instead of dyn_cast (NFC)

[Target] Use llvm::append_range (NFC)

[MemorySSA, BPF] Use isa instead of dyn_cast (NFC)

[BPF][NewPM] Port bpf-adjust-opt to NPM and add it to pipeline

Reviewed By: yonghong-song

Differential Revision: https://reviews.llvm.org/D91990

BPF: add AdjustOpt IR pass to generate verifier friendly codes

Add an IR phase right before main module optimization.
This is to modify IR to restrict certain downward optimizations
in order to gene

BPF: add AdjustOpt IR pass to generate verifier friendly codes

Add an IR phase right before main module optimization.
This is to modify IR to restrict certain downward optimizations
in order to generate verifier friendly code.
> prevent certain instcombine optimizations, handling both
in-block/cross-block instcombines.
> avoid speculative code motion if the variable used in
condition is also used in the later blocks.

Internally, a bpf IR builtin
result = __builtin_bpf_passthrough(seq_num, result)
is used to enforce ordering. This builtin is only used
during target independent IR optimizations and it will
be removed at the beginning of target dependent IR
optimizations.

For example, removing the following workaround,
--- a/tools/testing/selftests/bpf/progs/test_sysctl_loop1.c
+++ b/tools/testing/selftests/bpf/progs/test_sysctl_loop1.c
@@ -47,7 +47,7 @@ int sysctl_tcp_mem(struct bpf_sysctl *ctx)
/* a workaround to prevent compiler from generating
* codes verifier cannot handle yet.
*/
- volatile int ret;
+ int ret;
this patch is able to generate code which passed the verifier.

To disable optimization, users need to use "opt" command like below:
clang -target bpf -O2 -S -emit-llvm -Xclang -disable-llvm-passes test.c
// disable icmp serialization
opt -O2 -bpf-disable-serialize-icmp test.ll | llvm-dis > t.ll
// disable avoid-speculation
opt -O2 -bpf-disable-avoid-speculation test.ll | llvm-dis > t.ll
llc t.ll

Differential Revision: https://reviews.llvm.org/D85570

show more ...