drm/amdgpu: Fix missing drain retry fault the last entry

While the entry get in svm_range_unmap_from_cpu is the last entry, and
the entry is page fault, it also need to be dropped. So for equal case

drm/amdgpu: Fix missing drain retry fault the last entry

While the entry get in svm_range_unmap_from_cpu is the last entry, and
the entry is page fault, it also need to be dropped. So for equal case,
it also need to be dropped.

v2:
Only modify the svm_range_restore_pages.

Signed-off-by: Emily Deng <[email protected]>
Reviewed-by: Xiaogang Chen<[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Show warning message if IH ring overflow

If IH primary ring and KFD ih fifo overflows, we may miss CP, SDMA
interrupts and cause application soft hang. Show warning message with
ring nam

drm/amdgpu: Show warning message if IH ring overflow

If IH primary ring and KFD ih fifo overflows, we may miss CP, SDMA
interrupts and cause application soft hang. Show warning message with
ring name if overflow happens.

Add function to get ih ring name to avoid duplicating it. To keep
warning message consistent between GPU generations, change all
*_ih.c except ASICs older than Vega which has only one ih ring.

Signed-off-by: Philip Yang <[email protected]>
Reviewed-by: Christian König <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Increase IH soft ring size for GFX v9.4.3 dGPU

On GFX v9.4.3 dGPU, applications have random timeout failure when XNACK
on, dmesg log has "amdgpu: IH soft ring buffer overflow 0x900, 0x90

drm/amdgpu: Increase IH soft ring size for GFX v9.4.3 dGPU

On GFX v9.4.3 dGPU, applications have random timeout failure when XNACK
on, dmesg log has "amdgpu: IH soft ring buffer overflow 0x900, 0x900",
because dGPU mode has 272 cam entries. After increasing IH soft ring
to 512 entries, no more IH soft ring overflow message and application
passed.

Fixes: bf80d34b6c58 ("drm/amdgpu: Increase soft IH ring size")
Signed-off-by: Philip Yang <[email protected]>
Reviewed-by: Christian König <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Increase soft IH ring size

Retry faults are delegated to soft IH ring and then processed by
deferred worker. Current soft IH ring size PAGE_SIZE can store 128
entries, which may overflow

drm/amdgpu: Increase soft IH ring size

Retry faults are delegated to soft IH ring and then processed by
deferred worker. Current soft IH ring size PAGE_SIZE can store 128
entries, which may overflow and drop retry faults, causes HW stucks
because the retry fault is not recovered.

Increase soft IH ring size to 8KB, enough to store 256 CAM entries
because we clear the CAM entry after handling the retry fault from soft
ring.

Define macro IH_RING_SIZE and IH_SW_RING_SIZE to remove duplicate
constant.

Show warning message if soft IH ring overflows with CAM enabled because
this should not happen.

Signed-off-by: Philip Yang <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: handle IH ring1 overflow

IH ring1 is used to process GPU retry fault, overflow is enabled to
drain retry fault because we want receive other interrupts while
handling retry fault to reco

drm/amdgpu: handle IH ring1 overflow

IH ring1 is used to process GPU retry fault, overflow is enabled to
drain retry fault because we want receive other interrupts while
handling retry fault to recover range. There is no overflow flag set
when wptr pass rptr. Use timestamp of rptr and wptr to handle overflow
and drain retry fault.

If fault timestamp goes backward, the fault is filtered and should not
be processed. Drain fault is finished if processed_timestamp is equal to
or larger than checkpoint timestamp.

Add amdgpu_ih_functions interface decode_iv_ts for different chips to
get timestamp from IV entry with different iv size and timestamp offset.
amdgpu_ih_decode_iv_ts_helper is used for vega10, vega20, navi10.

Signed-off-by: Philip Yang <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Acked-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: nuke the ih reentrant lock

Interrupts on are non-reentrant on linux. This is just an ancient
leftover from radeon where irq processing was kicked of from different
places.

Signed-off-by

drm/amdgpu: nuke the ih reentrant lock

Interrupts on are non-reentrant on linux. This is just an ancient
leftover from radeon where irq processing was kicked of from different
places.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Daniel Vetter <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add ih waiter on process until checkpoint

Add IH function to allow caller to wait until ring entries are processed
until the checkpoint write pointer.

This will be primarily used by HMM

drm/amdgpu: add ih waiter on process until checkpoint

Add IH function to allow caller to wait until ring entries are processed
until the checkpoint write pointer.

This will be primarily used by HMM to drain pending page fault interrupts
before memory unmap to prevent HMM from handling stale interrupts.

Suggested-by: Christian König <[email protected]>
Signed-off-by: Jonathan Kim <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add a helper function to decode iv

since from soc15, all the chips share the same
iv format. create a common helper to decode iv

Signed-off-by: Hawking Zhang <[email protected]>
Ack

drm/amdgpu: add a helper function to decode iv

since from soc15, all the chips share the same
iv format. create a common helper to decode iv

Signed-off-by: Hawking Zhang <[email protected]>
Acked-by: Christian König <[email protected]>
Acked-by: Felix Kuehling <[email protected]>
Reviewed-by: Dennis Li <[email protected]>
Reviewed-by: Feifei Xu <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add amdgpu_ih_regs structure

amdgpu_ih_regs holds all the registers for
an ih ring

Signed-off-by: Hawking Zhang <[email protected]>
Reviewed-by: Christian König <christian.koenig@am

drm/amdgpu: add amdgpu_ih_regs structure

amdgpu_ih_regs holds all the registers for
an ih ring

Signed-off-by: Hawking Zhang <[email protected]>
Reviewed-by: Christian König <[email protected]>
Acked-by: Felix Kuehling <[email protected]>
Reviewed-by: Dennis Li <[email protected]>
Reviewed-by: Feifei Xu <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add infrastructure for soft IH ring

Add a soft IH ring implementation similar to the hardware IH1/2.

This can be used if the hardware delegation of interrupts to IH1/2
doesn't work for

drm/amdgpu: add infrastructure for soft IH ring

Add a soft IH ring implementation similar to the hardware IH1/2.

This can be used if the hardware delegation of interrupts to IH1/2
doesn't work for some reason.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: limit the number of IVs processed at once

Only process a maximum of 32 IVs before writing back the RPTR. This improves
hw handling when we get close to an overflow in the ring buffer.

S

drm/amdgpu: limit the number of IVs processed at once

Only process a maximum of 32 IVs before writing back the RPTR. This improves
hw handling when we get close to an overflow in the ring buffer.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Michel Dänzer <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: cleanup amdgpu_ih_process a bit more

Remove the callback and call the dispatcher directly.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Felix Kuehling <Felix.K

drm/amdgpu: cleanup amdgpu_ih_process a bit more

Remove the callback and call the dispatcher directly.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: simplify IH programming

Calculate all the addresses and pointers in amdgpu_ih.c

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Alex Deucher <alexander.deucher@am

drm/amdgpu: simplify IH programming

Calculate all the addresses and pointers in amdgpu_ih.c

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add IH ring to ih_get_wptr/ih_set_rptr v2

Let's start to support multiple rings.

v2: decode IV is needed as well

Signed-off-by: Christian König <[email protected]>
Acked-by: Ale

drm/amdgpu: add IH ring to ih_get_wptr/ih_set_rptr v2

Let's start to support multiple rings.

v2: decode IV is needed as well

Signed-off-by: Christian König <[email protected]>
Acked-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: move IV prescreening into the GMC code

The GMC/VM subsystem is causing the faults, so move the handling here as
well.

Signed-off-by: Christian König <[email protected]>
Reviewed-

drm/amdgpu: move IV prescreening into the GMC code

The GMC/VM subsystem is causing the faults, so move the handling here as
well.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Acked-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: move more defines into amdgpu_irq.h

Everything that isn't related to the IH ring.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Huang Rui <[email protected]>
Si

drm/amdgpu: move more defines into amdgpu_irq.h

Everything that isn't related to the IH ring.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Huang Rui <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: move more interrupt processing into amdgpu_irq.c

Add a callback to amdgpu_ih_process to remove most of the IV logic.

Signed-off-by: Christian König <[email protected]>
Acked-by:

drm/amdgpu: move more interrupt processing into amdgpu_irq.c

Add a callback to amdgpu_ih_process to remove most of the IV logic.

Signed-off-by: Christian König <[email protected]>
Acked-by: Huang Rui <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: cleanup amdgpu_ih.c

Cleanup amdgpu_ih.c to be able to handle multiple interrupt rings.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Huang Rui <[email protected]

drm/amdgpu: cleanup amdgpu_ih.c

Cleanup amdgpu_ih.c to be able to handle multiple interrupt rings.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Huang Rui <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Move fault hash table to amdgpu vm

In stead of share one fault hash table per device, make it
per vm. This can avoid inter-process lock issue when fault
hash table is full.

Change-Id: I

drm/amdgpu: Move fault hash table to amdgpu vm

In stead of share one fault hash table per device, make it
per vm. This can avoid inter-process lock issue when fault
hash table is full.

Change-Id: I5d1281b7c41eddc8e26113e010516557588d3708
Signed-off-by: Oak Zeng <[email protected]>
Suggested-by: Christian Konig <[email protected]>
Suggested-by: Felix Kuehling <[email protected]>
Reviewed-by: Christian Konig <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: move ih definitions into amdgpu_ih header

Demangle amdgpu.h

Signed-off-by: Huang Rui <[email protected]>
Acked-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher

drm/amdgpu: move ih definitions into amdgpu_ih header

Demangle amdgpu.h

Signed-off-by: Huang Rui <[email protected]>
Acked-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Move IH clientid defs to separate file

This is preparation for sharing client ID definitions
between amdgpu and amdkfd

Signed-off-by: Oak Zeng <[email protected]>
Reviewed-by: Chunming Z

drm/amdgpu: Move IH clientid defs to separate file

This is preparation for sharing client ID definitions
between amdgpu and amdkfd

Signed-off-by: Oak Zeng <[email protected]>
Reviewed-by: Chunming Zhou <[email protected]>
Acked-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: rename pas_id to pasid

sed -i "s/pas_id/pasid/g" drivers/gpu/drm/amd/amdgpu/*.c
sed -i "s/pas_id/pasid/g" drivers/gpu/drm/amd/amdgpu/*.h

Signed-off-by: Christian König <christian.koenig

drm/amdgpu: rename pas_id to pasid

sed -i "s/pas_id/pasid/g" drivers/gpu/drm/amd/amdgpu/*.c
sed -i "s/pas_id/pasid/g" drivers/gpu/drm/amd/amdgpu/*.h

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Chunming Zhou <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: rename vm_id to vmid

sed -i "s/vm_id/vmid/g" drivers/gpu/drm/amd/amdgpu/*.c
sed -i "s/vm_id/vmid/g" drivers/gpu/drm/amd/amdgpu/*.h

Signed-off-by: Christian König <[email protected]

drm/amdgpu: rename vm_id to vmid

sed -i "s/vm_id/vmid/g" drivers/gpu/drm/amd/amdgpu/*.c
sed -i "s/vm_id/vmid/g" drivers/gpu/drm/amd/amdgpu/*.h

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Chunming Zhou <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Track pending retry faults in IH and VM (v2)

IH tracks pending retry faults in a hash table for fast lookup in
interrupt context. Each VM has a short FIFO of pending VM faults for
proces

drm/amdgpu: Track pending retry faults in IH and VM (v2)

IH tracks pending retry faults in a hash table for fast lookup in
interrupt context. Each VM has a short FIFO of pending VM faults for
processing in a bottom half.

The IH prescreening stage adds retry faults and filters out repeated
retry interrupts to minimize the impact of interrupt storms.

It's the VM's responsibility remove pending faults once they are
handled. For now this is only done when the VM is destroyed.

v2:
- Made the hash table smaller and the FIFO longer. I never want the
FIFO to fill up, because that would make prescreen take longer.
128 pending page faults should be enough to keep migrations busy.

Signed-off-by: Felix Kuehling <[email protected]>
Acked-by: Christian König <[email protected]> (v1)
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add vcn ip block functions (v2)

Fill in the core VCN 1.0 setup functionality.

v2: squash in fixup (Alex)

Signed-off-by: Leo Liu <[email protected]>
Acked-by: Chunming Zhou <david1.zhou@a

drm/amdgpu: add vcn ip block functions (v2)

Fill in the core VCN 1.0 setup functionality.

v2: squash in fixup (Alex)

Signed-off-by: Leo Liu <[email protected]>
Acked-by: Chunming Zhou <[email protected]>
Acked-by: Hawking Zhang <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...