drm/amdgpu: immediately use GTT for new allocations

Only use GTT as a fallback if we already have a backing store. This
prevents evictions when an application constantly allocates and frees new
memo

drm/amdgpu: immediately use GTT for new allocations

Only use GTT as a fallback if we already have a backing store. This
prevents evictions when an application constantly allocates and frees new
memory.

Partially fixes
https://gitlab.freedesktop.org/drm/amd/-/issues/3844#note_2833985.

Signed-off-by: Christian König <[email protected]>
Fixes: 216c1282dde3 ("drm/amdgpu: use GTT only as fallback for VRAM|GTT")
Acked-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
Cc: [email protected]

show more ...

drm/amdgpu: remove all KFD fences from the BO on release

Remove all KFD BOs from the private dma_resv object.

This prevents the KFD from being evict unecessarily when an exported BO
is released.

S

drm/amdgpu: remove all KFD fences from the BO on release

Remove all KFD BOs from the private dma_resv object.

This prevents the KFD from being evict unecessarily when an exported BO
is released.

Signed-off-by: Christian König <[email protected]>
Signed-off-by: James Zhu <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Reviewed-and-tested-by: James Zhu <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: track bo memory stats at runtime

Before, every time fdinfo is queried we try to lock all the BOs in the
VM and calculate memory usage from scratch. This works okay if the
fdinfo is rarel

drm/amdgpu: track bo memory stats at runtime

Before, every time fdinfo is queried we try to lock all the BOs in the
VM and calculate memory usage from scratch. This works okay if the
fdinfo is rarely read and the VMs don't have a ton of BOs. If either of
these conditions is not true, we get a massive performance hit.

In this new revision, we track the BOs as they change states. This way
when the fdinfo is queried we only need to take the status lock and copy
out the usage stats with minimal impact to the runtime performance. With
this new approach however, we would no longer be able to track active
buffers.

Signed-off-by: Yunxiang Li <[email protected]>
Reviewed-by: Christian König <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Christian König <[email protected]>

show more ...

drm/amdgpu: remove unused function parameter

amdgpu_vm_bo_invalidate doesn't use the adev parameter and not all
callers have a reference to adev handy, so remove it for cleanliness.

Signed-off-by:

drm/amdgpu: remove unused function parameter

amdgpu_vm_bo_invalidate doesn't use the adev parameter and not all
callers have a reference to adev handy, so remove it for cleanliness.

Signed-off-by: Yunxiang Li <[email protected]>
Reviewed-by: Christian König <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Christian König <[email protected]>

show more ...

drm/amdgpu: Update the variable name to dma_buf

Instead of fixing the warning for missing variable
its better to update the variable name to match
with the style followed in the code.

This will fix

drm/amdgpu: Update the variable name to dma_buf

Instead of fixing the warning for missing variable
its better to update the variable name to match
with the style followed in the code.

This will fix the below mentioned warning:
warning: Function parameter or struct member 'dbuf' not described in 'amdgpu_bo_create_isp_user'
warning: Excess function parameter 'dma_buf' description in 'amdgpu_bo_create_isp_user'

Signed-off-by: Sunil Khatri <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amd/amdgpu: Add support for isp buffers

Add support to create user BOs with MC address for isp using the dma-buf handle
exported for the buffers allocated from system memory in isp driver.

Expo

drm/amd/amdgpu: Add support for isp buffers

Add support to create user BOs with MC address for isp using the dma-buf handle
exported for the buffers allocated from system memory in isp driver.

Export amdgpu_bo_create_kernel() and amdgpu_bo_free_kernel() as well for isp to
allocate GTT internal buffers required for fw to run.

Signed-off-by: Pratap Nirujogi <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: enable GTT fallback handling for dGPUs only

That is just a waste of time on APUs.

Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/3704
Fixes: 216c1282dde3 ("drm/amdgpu: use GTT

drm/amdgpu: enable GTT fallback handling for dGPUs only

That is just a waste of time on APUs.

Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/3704
Fixes: 216c1282dde3 ("drm/amdgpu: use GTT only as fallback for VRAM|GTT")
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
(cherry picked from commit e8fc090d322346e5ce4c4cfe03a8100e31f61c3c)
Cc: [email protected]

show more ...

drm/amdgpu: enable GTT fallback handling for dGPUs only

That is just a waste of time on APUs.

Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/3704
Fixes: 216c1282dde3 ("drm/amdgpu: use GTT

drm/amdgpu: enable GTT fallback handling for dGPUs only

That is just a waste of time on APUs.

Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/3704
Fixes: 216c1282dde3 ("drm/amdgpu: use GTT only as fallback for VRAM|GTT")
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Stop reporting special chip memory pools as CPU memory in fdinfo

So far these specialized on chip memory pools were reported as system
memory (aka 'cpu') which is not correct and mislead

drm/amdgpu: Stop reporting special chip memory pools as CPU memory in fdinfo

So far these specialized on chip memory pools were reported as system
memory (aka 'cpu') which is not correct and misleading. Lets remove that
and consider later making them visible as their own thing.

Signed-off-by: Tvrtko Ursulin <[email protected]>
Suggested-by: Christian König <[email protected]>
Cc: Yunxiang Li <[email protected]>
Cc: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: stop tracking visible memory stats

Since on modern systems all of vram can be made visible anyways, to
simplify the new implementation, drops tracking how much memory is
visible for now.

drm/amdgpu: stop tracking visible memory stats

Since on modern systems all of vram can be made visible anyways, to
simplify the new implementation, drops tracking how much memory is
visible for now. If this is really needed we can add it back on top of
the new implementation, or just report all the BOs as visible.

Signed-off-by: Yunxiang Li <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Tvrtko Ursulin <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: make drm-memory-* report resident memory

The old behavior reports the resident memory usage for this key and the
documentation say so as well. However this was accidentally changed to
in

drm/amdgpu: make drm-memory-* report resident memory

The old behavior reports the resident memory usage for this key and the
documentation say so as well. However this was accidentally changed to
include buffers that was evicted.

Fixes: 04bdba46542c ("drm/amdgpu: Use drm_print_memory_stats helper from fdinfo")
Signed-off-by: Yunxiang Li <[email protected]>
Reviewed-by: Tvrtko Ursulin <[email protected]>
Acked-by: Christian König <[email protected]>
Signed-off-by: Tvrtko Ursulin <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Use drm_print_memory_stats helper from fdinfo

Convert fdinfo memory stats to use the common drm_print_memory_stats
helper.

This achieves alignment with the common keys as documented in

drm/amdgpu: Use drm_print_memory_stats helper from fdinfo

Convert fdinfo memory stats to use the common drm_print_memory_stats
helper.

This achieves alignment with the common keys as documented in
drm-usage-stats.rst, adding specifically drm-total- key the driver was
missing until now.

Additionally I made the code stop skipping total size for objects which
currently do not have a backing store, and I added resident, active and
purgeable reporting.

Legacy keys have been preserved, with the outlook of only potentially
removing only the drm-memory- when the time gets right.

The example output now looks like this:

pos: 0
flags: 02100002
mnt_id: 24
ino: 1239
drm-driver: amdgpu
drm-client-id: 4
drm-pdev: 0000:04:00.0
pasid: 32771
drm-total-cpu: 0
drm-shared-cpu: 0
drm-active-cpu: 0
drm-resident-cpu: 0
drm-purgeable-cpu: 0
drm-total-gtt: 2392 KiB
drm-shared-gtt: 0
drm-active-gtt: 0
drm-resident-gtt: 2392 KiB
drm-purgeable-gtt: 0
drm-total-vram: 44564 KiB
drm-shared-vram: 31952 KiB
drm-active-vram: 0
drm-resident-vram: 44564 KiB
drm-purgeable-vram: 0
drm-memory-vram: 44564 KiB
drm-memory-gtt: 2392 KiB
drm-memory-cpu: 0 KiB
amd-memory-visible-vram: 44564 KiB
amd-evicted-vram: 0 KiB
amd-evicted-visible-vram: 0 KiB
amd-requested-vram: 44564 KiB
amd-requested-visible-vram: 11952 KiB
amd-requested-gtt: 2392 KiB
drm-engine-compute: 46464671 ns

v2:
* Track purgeable via AMDGPU_GEM_CREATE_DISCARDABLE.

Acked-by: Daniel Vetter <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Tvrtko Ursulin <[email protected]>
Cc: Alex Deucher <[email protected]>
Cc: Christian König <[email protected]>
Cc: Daniel Vetter <[email protected]>
Cc: Rob Clark <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: use GEM references instead of TTMs v2

Instead of a TTM reference grab a GEM reference whenever necessary.

v2: fix typo in amdgpu_bo_unref pointed out by Vitaly,
initialize the GEM f

drm/amdgpu: use GEM references instead of TTMs v2

Instead of a TTM reference grab a GEM reference whenever necessary.

v2: fix typo in amdgpu_bo_unref pointed out by Vitaly,
initialize the GEM funcs for kernel allocations as well.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Daniel Vetter <[email protected]> (v1)
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: remove amdgpu_pin_restricted()

We haven't used the functionality to pin BOs in a certain range at all
while the driver existed. Just nuke it.

Signed-off-by: Christian König <christian.k

drm/amdgpu: remove amdgpu_pin_restricted()

We haven't used the functionality to pin BOs in a certain range at all
while the driver existed. Just nuke it.

Signed-off-by: Christian König <[email protected]>
Acked-by: Lijo Lazar <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: explicitely set the AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS flag

Instead of having that in the amdgpu_bo_pin() function applied for all
pinned BOs.

Signed-off-by: Christian König <christian.k

drm/amdgpu: explicitely set the AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS flag

Instead of having that in the amdgpu_bo_pin() function applied for all
pinned BOs.

Signed-off-by: Christian König <[email protected]>
Acked-by: Lijo Lazar <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: nuke the VM PD/PT shadow handling

This was only used as workaround for recovering the page tables after
VRAM was lost and is no longer necessary after the function
amdgpu_vm_bo_reset_sta

drm/amdgpu: nuke the VM PD/PT shadow handling

This was only used as workaround for recovering the page tables after
VRAM was lost and is no longer necessary after the function
amdgpu_vm_bo_reset_state_machine() started to do the same.

Compute never used shadows either, so the only proplematic case left is
SVM and that is most likely not recoverable in any way when VRAM is
lost.

Signed-off-by: Christian König <[email protected]>
Acked-by: Lijo Lazar <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

amdgpu: don't dereference a NULL resource in sysfs code

dma_resv_trylock being successful doesn't guarantee that bo->tbo.base.resv
is not NULL, so check its validity before using it.

Signed-off-by:

amdgpu: don't dereference a NULL resource in sysfs code

dma_resv_trylock being successful doesn't guarantee that bo->tbo.base.resv
is not NULL, so check its validity before using it.

Signed-off-by: Pierre-Eric Pelloux-Prayer <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Describe all object placements in debugfs

Accurately show all placements when describing objects in debugfs, instead
of bunching them up under the 'CPU' placement.

Signed-off-by: Tvrtko

drm/amdgpu: Describe all object placements in debugfs

Accurately show all placements when describing objects in debugfs, instead
of bunching them up under the 'CPU' placement.

Signed-off-by: Tvrtko Ursulin <[email protected]>
Cc: Christian König <[email protected]>
Cc: Felix Kuehling <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Reduce mem_type to domain double indirection

All apart from AMDGPU_GEM_DOMAIN_GTT memory domains map 1:1 to TTM
placements. And the former be either AMDGPU_PL_PREEMPT or TTM_PL_TT,
depen

drm/amdgpu: Reduce mem_type to domain double indirection

All apart from AMDGPU_GEM_DOMAIN_GTT memory domains map 1:1 to TTM
placements. And the former be either AMDGPU_PL_PREEMPT or TTM_PL_TT,
depending on AMDGPU_GEM_CREATE_PREEMPTIBLE.

Simplify a few places in the code which convert the TTM placement into
a domain by checking against the current placement directly.

In the conversion AMDGPU_PL_PREEMPT either does not have to be handled
because amdgpu_mem_type_to_domain() cannot return that value anyway.

v2:
* Remove AMDGPU_PL_PREEMPT handling.

v3:
* Rebase.

Signed-off-by: Tvrtko Ursulin <[email protected]>
Reviewed-by: Christian König <[email protected]> # v1
Reviewed-by: Felix Kuehling <[email protected]> # v2
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: once more fix the call oder in amdgpu_ttm_move() v2

This reverts drm/amdgpu: fix ftrace event amdgpu_bo_move always move
on same heap. The basic problem here is that after the move the o

drm/amdgpu: once more fix the call oder in amdgpu_ttm_move() v2

This reverts drm/amdgpu: fix ftrace event amdgpu_bo_move always move
on same heap. The basic problem here is that after the move the old
location is simply not available any more.

Some fixes were suggested, but essentially we should call the move
notification before actually moving things because only this way we have
the correct order for DMA-buf and VM move notifications as well.

Also rework the statistic handling so that we don't update the eviction
counter before the move.

v2: add missing NULL check

Signed-off-by: Christian König <[email protected]>
Fixes: 94aeb4117343 ("drm/amdgpu: fix ftrace event amdgpu_bo_move always move on same heap")
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/3171
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: Fix the BO release clear memory warning

This happens when the amdgpu_bo_release_notify running
before amdgpu_ttm_set_buffer_funcs_status set the buffer
funcs to enabled.

check the buffe

drm/amdgpu: Fix the BO release clear memory warning

This happens when the amdgpu_bo_release_notify running
before amdgpu_ttm_set_buffer_funcs_status set the buffer
funcs to enabled.

check the buffer funcs enablement before calling the fill
buffer memory.

v2:(Christian)
- Apply it only for GEM buffers and since GEM buffers are only
allocated/freed while the driver is loaded we never run into
the issue to clear with buffer funcs disabled.

v3:(Mario)
- drop the stable tag as this will presumably go into a
-fixes PR for 6.10

Log snip:
*ERROR* Trying to clear memory with ring turned off.
RIP: 0010:amdgpu_bo_release_notify+0x201/0x220 [amdgpu]

Fixes: a68c7eaa7a8f ("drm/amdgpu: Enable clear page functionality")
Signed-off-by: Arunpravin Paneer Selvam <[email protected]>
Reviewed-by: Christian König <[email protected]>
Tested-by: Mikhail Gavrilov <[email protected]>
Tested-by: Richard Gong <[email protected]>
Suggested-by: Christian König <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

drm/amdgpu: once more fix the call oder in amdgpu_ttm_move() v2

This reverts drm/amdgpu: fix ftrace event amdgpu_bo_move always move
on same heap. The basic problem here is that after the move the o

drm/amdgpu: once more fix the call oder in amdgpu_ttm_move() v2

This reverts drm/amdgpu: fix ftrace event amdgpu_bo_move always move
on same heap. The basic problem here is that after the move the old
location is simply not available any more.

Some fixes were suggested, but essentially we should call the move
notification before actually moving things because only this way we have
the correct order for DMA-buf and VM move notifications as well.

Also rework the statistic handling so that we don't update the eviction
counter before the move.

v2: add missing NULL check

Signed-off-by: Christian König <[email protected]>
Fixes: 94aeb4117343 ("drm/amdgpu: fix ftrace event amdgpu_bo_move always move on same heap")
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/3171
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: Modify the contiguous flags behaviour

Now we have two flags for contiguous VRAM buffer allocation.
If the application request for AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS,
it would set the ttm

drm/amdgpu: Modify the contiguous flags behaviour

Now we have two flags for contiguous VRAM buffer allocation.
If the application request for AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS,
it would set the ttm place TTM_PL_FLAG_CONTIGUOUS flag in the
buffer's placement function.

This patch will change the default behaviour of the two flags.

When we set AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS
- This means contiguous is not mandatory.
- we will try to allocate the contiguous buffer. Say if the
allocation fails, we fallback to allocate the individual pages.

When we setTTM_PL_FLAG_CONTIGUOUS
- This means contiguous allocation is mandatory.
- we are setting this in amdgpu_bo_pin_restricted() before bo validation
and check this flag in the vram manager file.
- if this is set, we should allocate the buffer pages contiguously.
the allocation fails, we return -ENOSPC.

v2:
- keep the mem_flags and bo->flags check as is(Christian)
- place the TTM_PL_FLAG_CONTIGUOUS flag setting into the
amdgpu_bo_pin_restricted function placement range iteration
loop(Christian)
- rename find_pages with amdgpu_vram_mgr_calculate_pages_per_block
(Christian)
- Keep the kernel BO allocation as is(Christain)
- If BO pin vram allocation failed, we need to return -ENOSPC as
RDMA cannot work with scattered VRAM pages(Philip)

v3(Christian):
- keep contiguous flag handling outside of pages_per_block
calculation
- remove the hacky implementation in contiguous flag error
handling code

v4(Christian):
- use any variable and return value for non-contiguous
fallback

v5: rebase to amd-staging-drm-next branch

Signed-off-by: Arunpravin Paneer Selvam <[email protected]>
Suggested-by: Christian König <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: replace tmz flag into buffer flag

Replace tmz flag into buffer flag to make it easier to understand
and extend

Signed-off-by: Likun Gao <[email protected]>
Signed-off-by: Frank Min <Fra

drm/amdgpu: replace tmz flag into buffer flag

Replace tmz flag into buffer flag to make it easier to understand
and extend

Signed-off-by: Likun Gao <[email protected]>
Signed-off-by: Frank Min <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Update BO eviction priorities

Make SVM BOs more likely to get evicted than other BOs. These BOs
opportunistically use available VRAM, but can fall back relatively
seamlessly to system me

drm/amdgpu: Update BO eviction priorities

Make SVM BOs more likely to get evicted than other BOs. These BOs
opportunistically use available VRAM, but can fall back relatively
seamlessly to system memory. It also avoids SVM migrations evicting
other, more important BOs as they will evict other SVM allocations
first.

Signed-off-by: Felix Kuehling <[email protected]>
Acked-by: Mukul Joshi <[email protected]>
Tested-by: Mukul Joshi <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...