drm/amdgpu: track bo memory stats at runtime

Before, every time fdinfo is queried we try to lock all the BOs in the
VM and calculate memory usage from scratch. This works okay if the
fdinfo is rarel

drm/amdgpu: track bo memory stats at runtime

Before, every time fdinfo is queried we try to lock all the BOs in the
VM and calculate memory usage from scratch. This works okay if the
fdinfo is rarely read and the VMs don't have a ton of BOs. If either of
these conditions is not true, we get a massive performance hit.

In this new revision, we track the BOs as they change states. This way
when the fdinfo is queried we only need to take the status lock and copy
out the usage stats with minimal impact to the runtime performance. With
this new approach however, we would no longer be able to track active
buffers.

Signed-off-by: Yunxiang Li <[email protected]>
Reviewed-by: Christian König <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Christian König <[email protected]>

show more ...

drm/amdgpu: drop the amdgpu_device argument from amdgpu_ib_free

It's unused.

Signed-off-by: Pierre-Eric Pelloux-Prayer <[email protected]>
Reviewed-by: Alex Deucher <alexander.deuc

drm/amdgpu: drop the amdgpu_device argument from amdgpu_ib_free

It's unused.

Signed-off-by: Pierre-Eric Pelloux-Prayer <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amd/amdgpu: Add support for isp buffers

Add support to create user BOs with MC address for isp using the dma-buf handle
exported for the buffers allocated from system memory in isp driver.

Expo

drm/amd/amdgpu: Add support for isp buffers

Add support to create user BOs with MC address for isp using the dma-buf handle
exported for the buffers allocated from system memory in isp driver.

Export amdgpu_bo_create_kernel() and amdgpu_bo_free_kernel() as well for isp to
allocate GTT internal buffers required for fw to run.

Signed-off-by: Pratap Nirujogi <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: stop tracking visible memory stats

Since on modern systems all of vram can be made visible anyways, to
simplify the new implementation, drops tracking how much memory is
visible for now.

drm/amdgpu: stop tracking visible memory stats

Since on modern systems all of vram can be made visible anyways, to
simplify the new implementation, drops tracking how much memory is
visible for now. If this is really needed we can add it back on top of
the new implementation, or just report all the BOs as visible.

Signed-off-by: Yunxiang Li <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Tvrtko Ursulin <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: make drm-memory-* report resident memory

The old behavior reports the resident memory usage for this key and the
documentation say so as well. However this was accidentally changed to
in

drm/amdgpu: make drm-memory-* report resident memory

The old behavior reports the resident memory usage for this key and the
documentation say so as well. However this was accidentally changed to
include buffers that was evicted.

Fixes: 04bdba46542c ("drm/amdgpu: Use drm_print_memory_stats helper from fdinfo")
Signed-off-by: Yunxiang Li <[email protected]>
Reviewed-by: Tvrtko Ursulin <[email protected]>
Acked-by: Christian König <[email protected]>
Signed-off-by: Tvrtko Ursulin <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Use drm_print_memory_stats helper from fdinfo

Convert fdinfo memory stats to use the common drm_print_memory_stats
helper.

This achieves alignment with the common keys as documented in

drm/amdgpu: Use drm_print_memory_stats helper from fdinfo

Convert fdinfo memory stats to use the common drm_print_memory_stats
helper.

This achieves alignment with the common keys as documented in
drm-usage-stats.rst, adding specifically drm-total- key the driver was
missing until now.

Additionally I made the code stop skipping total size for objects which
currently do not have a backing store, and I added resident, active and
purgeable reporting.

Legacy keys have been preserved, with the outlook of only potentially
removing only the drm-memory- when the time gets right.

The example output now looks like this:

pos: 0
flags: 02100002
mnt_id: 24
ino: 1239
drm-driver: amdgpu
drm-client-id: 4
drm-pdev: 0000:04:00.0
pasid: 32771
drm-total-cpu: 0
drm-shared-cpu: 0
drm-active-cpu: 0
drm-resident-cpu: 0
drm-purgeable-cpu: 0
drm-total-gtt: 2392 KiB
drm-shared-gtt: 0
drm-active-gtt: 0
drm-resident-gtt: 2392 KiB
drm-purgeable-gtt: 0
drm-total-vram: 44564 KiB
drm-shared-vram: 31952 KiB
drm-active-vram: 0
drm-resident-vram: 44564 KiB
drm-purgeable-vram: 0
drm-memory-vram: 44564 KiB
drm-memory-gtt: 2392 KiB
drm-memory-cpu: 0 KiB
amd-memory-visible-vram: 44564 KiB
amd-evicted-vram: 0 KiB
amd-evicted-visible-vram: 0 KiB
amd-requested-vram: 44564 KiB
amd-requested-visible-vram: 11952 KiB
amd-requested-gtt: 2392 KiB
drm-engine-compute: 46464671 ns

v2:
* Track purgeable via AMDGPU_GEM_CREATE_DISCARDABLE.

Acked-by: Daniel Vetter <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Tvrtko Ursulin <[email protected]>
Cc: Alex Deucher <[email protected]>
Cc: Christian König <[email protected]>
Cc: Daniel Vetter <[email protected]>
Cc: Rob Clark <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: remove amdgpu_pin_restricted()

We haven't used the functionality to pin BOs in a certain range at all
while the driver existed. Just nuke it.

Signed-off-by: Christian König <christian.k

drm/amdgpu: remove amdgpu_pin_restricted()

We haven't used the functionality to pin BOs in a certain range at all
while the driver existed. Just nuke it.

Signed-off-by: Christian König <[email protected]>
Acked-by: Lijo Lazar <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: nuke the VM PD/PT shadow handling

This was only used as workaround for recovering the page tables after
VRAM was lost and is no longer necessary after the function
amdgpu_vm_bo_reset_sta

drm/amdgpu: nuke the VM PD/PT shadow handling

This was only used as workaround for recovering the page tables after
VRAM was lost and is no longer necessary after the function
amdgpu_vm_bo_reset_state_machine() started to do the same.

Compute never used shadows either, so the only proplematic case left is
SVM and that is most likely not recoverable in any way when VRAM is
lost.

Signed-off-by: Christian König <[email protected]>
Acked-by: Lijo Lazar <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: Ensure user queue buffers residency

Add atomic queue_refcount to struct bo_va, return -EBUSY to fail unmap
BO from the GPU if the bo_va queue_refcount is not zero.

Create queue to incre

drm/amdkfd: Ensure user queue buffers residency

Add atomic queue_refcount to struct bo_va, return -EBUSY to fail unmap
BO from the GPU if the bo_va queue_refcount is not zero.

Create queue to increase the bo_va queue_refcount, destroy queue to
decrease the bo_va queue_refcount, to ensure the queue buffers mapped on
the GPU when queue is active.

Signed-off-by: Philip Yang <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Acked-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: once more fix the call oder in amdgpu_ttm_move() v2

This reverts drm/amdgpu: fix ftrace event amdgpu_bo_move always move
on same heap. The basic problem here is that after the move the o

drm/amdgpu: once more fix the call oder in amdgpu_ttm_move() v2

This reverts drm/amdgpu: fix ftrace event amdgpu_bo_move always move
on same heap. The basic problem here is that after the move the old
location is simply not available any more.

Some fixes were suggested, but essentially we should call the move
notification before actually moving things because only this way we have
the correct order for DMA-buf and VM move notifications as well.

Also rework the statistic handling so that we don't update the eviction
counter before the move.

v2: add missing NULL check

Signed-off-by: Christian König <[email protected]>
Fixes: 94aeb4117343 ("drm/amdgpu: fix ftrace event amdgpu_bo_move always move on same heap")
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/3171
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: once more fix the call oder in amdgpu_ttm_move() v2

This reverts drm/amdgpu: fix ftrace event amdgpu_bo_move always move
on same heap. The basic problem here is that after the move the o

drm/amdgpu: once more fix the call oder in amdgpu_ttm_move() v2

This reverts drm/amdgpu: fix ftrace event amdgpu_bo_move always move
on same heap. The basic problem here is that after the move the old
location is simply not available any more.

Some fixes were suggested, but essentially we should call the move
notification before actually moving things because only this way we have
the correct order for DMA-buf and VM move notifications as well.

Also rework the statistic handling so that we don't update the eviction
counter before the move.

v2: add missing NULL check

Signed-off-by: Christian König <[email protected]>
Fixes: 94aeb4117343 ("drm/amdgpu: fix ftrace event amdgpu_bo_move always move on same heap")
Closes: https://gitlab.freedesktop.org/drm/amd/-/issues/3171
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: fix visible VRAM handling during faults

When we removed the hacky start code check we actually didn't took into
account that *all* VRAM pages needs to be CPU accessible.

Clean up the co

drm/amdgpu: fix visible VRAM handling during faults

When we removed the hacky start code check we actually didn't took into
account that *all* VRAM pages needs to be CPU accessible.

Clean up the code and unify the handling into a single helper which
checks if the whole resource is CPU accessible.

The only place where a partial check would make sense is during
eviction, but that is neglitible.

Signed-off-by: Christian König <[email protected]>
Fixes: aed01a68047b ("drm/amdgpu: Remove TTM resource->start visible VRAM condition v2")
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: fix visible VRAM handling during faults

When we removed the hacky start code check we actually didn't took into
account that *all* VRAM pages needs to be CPU accessible.

Clean up the co

drm/amdgpu: fix visible VRAM handling during faults

When we removed the hacky start code check we actually didn't took into
account that *all* VRAM pages needs to be CPU accessible.

Clean up the code and unify the handling into a single helper which
checks if the whole resource is CPU accessible.

The only place where a partial check would make sense is during
eviction, but that is neglitible.

Signed-off-by: Christian König <[email protected]>
Fixes: aed01a68047b ("drm/amdgpu: Remove TTM resource->start visible VRAM condition v2")
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: add shared fdinfo stats

Add shared stats. Useful for seeing shared memory.

v2: take dma-buf into account as well
v3: use the new gem helper

Link: https://lore.kernel.org/all/202312071

drm/amdgpu: add shared fdinfo stats

Add shared stats. Useful for seeing shared memory.

v2: take dma-buf into account as well
v3: use the new gem helper

Link: https://lore.kernel.org/all/[email protected]/
Signed-off-by: Alex Deucher <[email protected]>
Cc: Rob Clark <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Christian König <[email protected]>

show more ...

drm/amdgpu: fix ftrace event amdgpu_bo_move always move on same heap

Issue: during evict or validate happened on amdgpu_bo, the 'from' and
'to' is always same in ftrace event of amdgpu_bo_move

wher

drm/amdgpu: fix ftrace event amdgpu_bo_move always move on same heap

Issue: during evict or validate happened on amdgpu_bo, the 'from' and
'to' is always same in ftrace event of amdgpu_bo_move

where calling the 'trace_amdgpu_bo_move', the comment says move_notify
is called before move happens, but actually it is called after move
happens, here the new_mem is same as bo->resource

Fix: move trace_amdgpu_bo_move from move_notify to amdgpu_bo_move

Signed-off-by: Wang, Beyond <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add missing NULL check

bo->tbo.resource can easily be NULL here.

Link: https://gitlab.freedesktop.org/drm/amd/-/issues/2902
Signed-off-by: Christian König <[email protected]>
Rev

drm/amdgpu: add missing NULL check

bo->tbo.resource can easily be NULL here.

Link: https://gitlab.freedesktop.org/drm/amd/-/issues/2902
Signed-off-by: Christian König <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: add missing NULL check

bo->tbo.resource can easily be NULL here.

Link: https://gitlab.freedesktop.org/drm/amd/-/issues/2902
Signed-off-by: Christian König <[email protected]>
Rev

drm/amdgpu: add missing NULL check

bo->tbo.resource can easily be NULL here.

Link: https://gitlab.freedesktop.org/drm/amd/-/issues/2902
Signed-off-by: Christian König <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: accommodate DOMAIN/PL_DOORBELL

This patch adds changes:
- to accommodate the new GEM domain for DOORBELLs
- to accommodate the new TTM PL for DOORBELLs

in order to manage doorbell pages

drm/amdgpu: accommodate DOMAIN/PL_DOORBELL

This patch adds changes:
- to accommodate the new GEM domain for DOORBELLs
- to accommodate the new TTM PL for DOORBELLs

in order to manage doorbell pages as GEM object.

V2: Addressed reviwe comments from Christian
- drop the doorbell changes for pinning/unpinning
- drop the doorbell changes for dma-buf map
- drop the doorbell changes for sgt
- no need to handle TTM_PL_FLAG_CONTIGUOUS for doorbell
- add caching type for doorbell

V3: - Removed unrelated empty line (Christian)
- Add PL_DOORBELL in mem_type_to_domain() as well (Alex)

Cc: Alex Deucher <[email protected]>
Cc: Christian Koenig <[email protected]>
Reviewed-by: Christian Koenig <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
Signed-off-by: Shashank Sharma <[email protected]>

show more ...

drm/amdkfd: Store xcp partition id to amdgpu bo

For memory accounting per compute partition and export drm amdgpu bo and
then import to KFD, we need the xcp id to account the memory usage or
find th

drm/amdkfd: Store xcp partition id to amdgpu bo

For memory accounting per compute partition and export drm amdgpu bo and
then import to KFD, we need the xcp id to account the memory usage or
find the KFD node of the original amdgpu bo to create the KFD bo on the
correct adev KFD node.

Set xcp_id_plus1 of amdgpu_bo_param to create bo and store xcp_id to
amddgpu bo. Add helper macro to get the mem_id from adev and xcp_id.

v2: squash in fix ("drm/amdgpu: Fix BO creation failure on GFX 9.4.3 dGPU")

Signed-off-by: Philip Yang <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Add memory partition mem_id to amdgpu_bo

Add mem_id_plus1 parameter to amdgpu_gem_object_create and pass it to
amdgpu_bo_create. For dGPU mode allocation, mem_id is used by VRAM
manager

drm/amdgpu: Add memory partition mem_id to amdgpu_bo

Add mem_id_plus1 parameter to amdgpu_gem_object_create and pass it to
amdgpu_bo_create. For dGPU mode allocation, mem_id is used by VRAM
manager to get the memory partition fpfn, lpfn from xcp manager. For APU
native mode allocation, mem_id is used to get NUMA node id from xcp
manager, then pass to TTM as numa pool id to alloc memory from the
specific NUMA node. mem_id -1 means for entire VRAM or any NUMA nodes.

Signed-off-by: Philip Yang <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: expose more memory stats in fdinfo

This will be used for performance investigations.

Signed-off-by: Marek Olšák <[email protected]>
Reviewed-by: Christian König <christian.koenig@amd.

drm/amdgpu: expose more memory stats in fdinfo

This will be used for performance investigations.

Signed-off-by: Marek Olšák <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amd: Convert amdgpu to use suballocation helper.

Now that we have a generic suballocation helper, Use it in amdgpu.
For lines that get moved or changed, also fix up pre-existing style issues.

S

drm/amd: Convert amdgpu to use suballocation helper.

Now that we have a generic suballocation helper, Use it in amdgpu.
For lines that get moved or changed, also fix up pre-existing style issues.

Signed-off-by: Maarten Lankhorst <[email protected]>
Co-developed-by: Thomas Hellström <[email protected]>
Signed-off-by: Thomas Hellström <[email protected]>
Reviewed-by: Christian König <[email protected]>
Acked-by: Maarten Lankhorst <[email protected]>
Acked-by: Christian König <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

drm/amdgpu: Remove unnecessary domain argument

Remove the "domain" argument to amdgpu_bo_create_kernel_at() since this
function takes an "offset" argument which is the offset off of VRAM, and as
suc

drm/amdgpu: Remove unnecessary domain argument

Remove the "domain" argument to amdgpu_bo_create_kernel_at() since this
function takes an "offset" argument which is the offset off of VRAM, and as
such allocation always takes place in VRAM. Thus, the "domain" argument is
unnecessary.

Cc: Alex Deucher <[email protected]>
Cc: Christian König <[email protected]>
Cc: AMD Graphics <[email protected]>
Signed-off-by: Luben Tuikov <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add AMDGPU_GEM_CREATE_DISCARDABLE

Add a AMDGPU_GEM_CREATE_DISCARDABLE flag to note that the content of a BO
doesn't needs to be preserved during eviction.

KFD was already using a simila

drm/amdgpu: add AMDGPU_GEM_CREATE_DISCARDABLE

Add a AMDGPU_GEM_CREATE_DISCARDABLE flag to note that the content of a BO
doesn't needs to be preserved during eviction.

KFD was already using a similar functionality for SVM BOs so replace the
internal flag with the new UAPI.

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Reviewed-by: Marek Olšák <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: unify BO evicting method in amdgpu_ttm

Unify BO evicting functionality for possible memory
types in amdgpu_ttm.c.

Signed-off-by: Nirmoy Das <[email protected]>
Reviewed-by: Christian K

drm/amdgpu: unify BO evicting method in amdgpu_ttm

Unify BO evicting functionality for possible memory
types in amdgpu_ttm.c.

Signed-off-by: Nirmoy Das <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...