drm/amdgpu: add a BO metadata flag to disable write compression for Vulkan

Vulkan can't support DCC and Z/S compression on GFX12 without
WRITE_COMPRESS_DISABLE in this commit or a completely differe

drm/amdgpu: add a BO metadata flag to disable write compression for Vulkan

Vulkan can't support DCC and Z/S compression on GFX12 without
WRITE_COMPRESS_DISABLE in this commit or a completely different DCC
interface.

AMDGPU_TILING_GFX12_SCANOUT is added because it's already used by userspace.

Cc: [email protected] # 6.12.x
Signed-off-by: Marek Olšák <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: track bo memory stats at runtime

Before, every time fdinfo is queried we try to lock all the BOs in the
VM and calculate memory usage from scratch. This works okay if the
fdinfo is rarel

drm/amdgpu: track bo memory stats at runtime

Before, every time fdinfo is queried we try to lock all the BOs in the
VM and calculate memory usage from scratch. This works okay if the
fdinfo is rarely read and the VMs don't have a ton of BOs. If either of
these conditions is not true, we get a massive performance hit.

In this new revision, we track the BOs as they change states. This way
when the fdinfo is queried we only need to take the status lock and copy
out the usage stats with minimal impact to the runtime performance. With
this new approach however, we would no longer be able to track active
buffers.

Signed-off-by: Yunxiang Li <[email protected]>
Reviewed-by: Christian König <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Christian König <[email protected]>

show more ...

drm/amdgpu: Use drm_print_memory_stats helper from fdinfo

Convert fdinfo memory stats to use the common drm_print_memory_stats
helper.

This achieves alignment with the common keys as documented in

drm/amdgpu: Use drm_print_memory_stats helper from fdinfo

Convert fdinfo memory stats to use the common drm_print_memory_stats
helper.

This achieves alignment with the common keys as documented in
drm-usage-stats.rst, adding specifically drm-total- key the driver was
missing until now.

Additionally I made the code stop skipping total size for objects which
currently do not have a backing store, and I added resident, active and
purgeable reporting.

Legacy keys have been preserved, with the outlook of only potentially
removing only the drm-memory- when the time gets right.

The example output now looks like this:

pos: 0
flags: 02100002
mnt_id: 24
ino: 1239
drm-driver: amdgpu
drm-client-id: 4
drm-pdev: 0000:04:00.0
pasid: 32771
drm-total-cpu: 0
drm-shared-cpu: 0
drm-active-cpu: 0
drm-resident-cpu: 0
drm-purgeable-cpu: 0
drm-total-gtt: 2392 KiB
drm-shared-gtt: 0
drm-active-gtt: 0
drm-resident-gtt: 2392 KiB
drm-purgeable-gtt: 0
drm-total-vram: 44564 KiB
drm-shared-vram: 31952 KiB
drm-active-vram: 0
drm-resident-vram: 44564 KiB
drm-purgeable-vram: 0
drm-memory-vram: 44564 KiB
drm-memory-gtt: 2392 KiB
drm-memory-cpu: 0 KiB
amd-memory-visible-vram: 44564 KiB
amd-evicted-vram: 0 KiB
amd-evicted-visible-vram: 0 KiB
amd-requested-vram: 44564 KiB
amd-requested-visible-vram: 11952 KiB
amd-requested-gtt: 2392 KiB
drm-engine-compute: 46464671 ns

v2:
* Track purgeable via AMDGPU_GEM_CREATE_DISCARDABLE.

Acked-by: Daniel Vetter <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Tvrtko Ursulin <[email protected]>
Cc: Alex Deucher <[email protected]>
Cc: Christian König <[email protected]>
Cc: Daniel Vetter <[email protected]>
Cc: Rob Clark <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: restore dcc bo tilling configs while moving

While moving buffer which has dcc tiling config, it is needed to restore
its original dcc tiling.

1. extend copy flag to cover tiling bits
2.

drm/amdgpu: restore dcc bo tilling configs while moving

While moving buffer which has dcc tiling config, it is needed to restore
its original dcc tiling.

1. extend copy flag to cover tiling bits
2. add logic to restore original dcc tiling config

Signed-off-by: Frank Min <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add sdma 7.0 support for copy dcc buffer

1. Add dcc buffer flag for copy buffer
2. Add sdma 7.0 support copy dcc buffer

Signed-off-by: Likun Gao <[email protected]>
Signed-off-by: Frank

drm/amdgpu: add sdma 7.0 support for copy dcc buffer

1. Add dcc buffer flag for copy buffer
2. Add sdma 7.0 support copy dcc buffer

Signed-off-by: Likun Gao <[email protected]>
Signed-off-by: Frank Min <[email protected]>
Reviewed-by: Hawking Zhang <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: replace tmz flag into buffer flag

Replace tmz flag into buffer flag to make it easier to understand
and extend

Signed-off-by: Likun Gao <[email protected]>
Signed-off-by: Frank Min <Fra

drm/amdgpu: replace tmz flag into buffer flag

Replace tmz flag into buffer flag to make it easier to understand
and extend

Signed-off-by: Likun Gao <[email protected]>
Signed-off-by: Frank Min <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Enable clear page functionality

Add clear page support in vram memory region.

v1(Christian):
- Dont handle clear page as TTM flag since when moving the BO back
in from GTT again w

drm/amdgpu: Enable clear page functionality

Add clear page support in vram memory region.

v1(Christian):
- Dont handle clear page as TTM flag since when moving the BO back
in from GTT again we don't need that.
- Make a specialized version of amdgpu_fill_buffer() which only
clears the VRAM areas which are not already cleared
- Drop the TTM_PL_FLAG_WIPE_ON_RELEASE check in
amdgpu_object.c

v2:
- Modify the function name amdgpu_ttm_* (Alex)
- Drop the delayed parameter (Christian)
- handle amdgpu_res_cleared(&cursor) just above the size
calculation (Christian)
- Use AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE for clearing the buffers
in the free path to properly wait for fences etc.. (Christian)

v3(Christian):
- Remove buffer clear code in VRAM manager instead change the
AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE handling to set
the DRM_BUDDY_CLEARED flag.
- Remove ! from amdgpu_res_cleared(&cursor) check.

v4(Christian):
- vres flag setting move to vram manager file
- use dma_fence_get_stub in amdgpu_ttm_clear_buffer function
- make fence a mandatory parameter and drop the if and the get/put dance

Signed-off-by: Arunpravin Paneer Selvam <[email protected]>
Suggested-by: Christian König <[email protected]>
Acked-by: Felix Kuehling <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Christian König <[email protected]>

show more ...

drm/amdgpu: fix visible VRAM handling during faults

When we removed the hacky start code check we actually didn't took into
account that *all* VRAM pages needs to be CPU accessible.

Clean up the co

drm/amdgpu: fix visible VRAM handling during faults

When we removed the hacky start code check we actually didn't took into
account that *all* VRAM pages needs to be CPU accessible.

Clean up the code and unify the handling into a single helper which
checks if the whole resource is CPU accessible.

The only place where a partial check would make sense is during
eviction, but that is neglitible.

Signed-off-by: Christian König <[email protected]>
Fixes: aed01a68047b ("drm/amdgpu: Remove TTM resource->start visible VRAM condition v2")
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: fix visible VRAM handling during faults

When we removed the hacky start code check we actually didn't took into
account that *all* VRAM pages needs to be CPU accessible.

Clean up the co

drm/amdgpu: fix visible VRAM handling during faults

When we removed the hacky start code check we actually didn't took into
account that *all* VRAM pages needs to be CPU accessible.

Clean up the code and unify the handling into a single helper which
checks if the whole resource is CPU accessible.

The only place where a partial check would make sense is during
eviction, but that is neglitible.

Signed-off-by: Christian König <[email protected]>
Fixes: aed01a68047b ("drm/amdgpu: Remove TTM resource->start visible VRAM condition v2")
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
CC: [email protected]

show more ...

drm/amdgpu: accommodate DOMAIN/PL_DOORBELL

This patch adds changes:
- to accommodate the new GEM domain for DOORBELLs
- to accommodate the new TTM PL for DOORBELLs

in order to manage doorbell pages

drm/amdgpu: accommodate DOMAIN/PL_DOORBELL

This patch adds changes:
- to accommodate the new GEM domain for DOORBELLs
- to accommodate the new TTM PL for DOORBELLs

in order to manage doorbell pages as GEM object.

V2: Addressed reviwe comments from Christian
- drop the doorbell changes for pinning/unpinning
- drop the doorbell changes for dma-buf map
- drop the doorbell changes for sgt
- no need to handle TTM_PL_FLAG_CONTIGUOUS for doorbell
- add caching type for doorbell

V3: - Removed unrelated empty line (Christian)
- Add PL_DOORBELL in mem_type_to_domain() as well (Alex)

Cc: Alex Deucher <[email protected]>
Cc: Christian Koenig <[email protected]>
Reviewed-by: Christian Koenig <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
Signed-off-by: Shashank Sharma <[email protected]>

show more ...

drm/amdgpu: Move vram, gtt & flash defines to amdgpu_ ttm & _psp.h

As amdgpu.h is getting decomposed, move vram and gtt extern defines into
amdgpu_ttm.h & flash extern to amdgpu_psp.h

Fixes: f9acfa

drm/amdgpu: Move vram, gtt & flash defines to amdgpu_ ttm & _psp.h

As amdgpu.h is getting decomposed, move vram and gtt extern defines into
amdgpu_ttm.h & flash extern to amdgpu_psp.h

Fixes: f9acfafc3458 ("drm/amdgpu: Move externs to amdgpu.h file from amdgpu_drv.c")
Suggested-by: Christian König <[email protected]>
Cc: Mario Limonciello <[email protected]>
Cc: Christian König <[email protected]>
Cc: Alex Deucher <[email protected]>
Signed-off-by: Srinivasan Shanmugam <[email protected]>
Acked-by: Guchun Chen <[email protected]>
Reviewed-by: Mario Limonciello <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Rename DRM schedulers in amdgpu TTM

Rename mman.entity to mman.high_pr to make the distinction
clearer that this is a high priority scheduler. Similarly,
rename the recently added mman.d

drm/amdgpu: Rename DRM schedulers in amdgpu TTM

Rename mman.entity to mman.high_pr to make the distinction
clearer that this is a high priority scheduler. Similarly,
rename the recently added mman.delayed to mman.low_pr to
make it clear it is a low priority scheduler.
No functional change in this patch.

Signed-off-by: Mukul Joshi <[email protected]>
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Add a low priority scheduler for VRAM clearing

Add a low priority DRM scheduler for VRAM clearing instead of using
the exisiting high priority scheduler. Use the high priority scheduler

drm/amdgpu: Add a low priority scheduler for VRAM clearing

Add a low priority DRM scheduler for VRAM clearing instead of using
the exisiting high priority scheduler. Use the high priority scheduler
for migrations and evictions.

Signed-off-by: Mukul Joshi <[email protected]>
Acked-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: Alloc memory of GPU support memory partition

For dGPU mode VRAM allocation, create amdgpu_bo from amdgpu_vm->mem_id,
to alloc from the correct memory range.

For APU mode VRAM allocation

drm/amdkfd: Alloc memory of GPU support memory partition

For dGPU mode VRAM allocation, create amdgpu_bo from amdgpu_vm->mem_id,
to alloc from the correct memory range.

For APU mode VRAM allocation, set alloc domain to GTT, and set
bp->mem_id_plus1 from amdgpu_vm->mem_id + 1 to create amdgpu_bo, to
allocate system memory from correct NUMA node.

For GTT allocation, use mem_id -1 to allocate system memory from any
NUMA nodes.

Remove amdgpu_ttm_tt_set_mem_pool, to avoid the confusion that memory
maybe allocated from different mem_id.

Signed-off-by: Philip Yang <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: Native mode memory partition support

For native mode, after amdgpu_bo is created on CPU domain, then call
amdgpu_ttm_tt_set_mem_pool to select the TTM pool using bo->mem_id.
ttm_bo_valid

drm/amdkfd: Native mode memory partition support

For native mode, after amdgpu_bo is created on CPU domain, then call
amdgpu_ttm_tt_set_mem_pool to select the TTM pool using bo->mem_id.
ttm_bo_validate will allocate the memory to the correct memory partition
before mapping to GPUs.

Reviewed-by: Felix Kuehling <[email protected]>
Acked-and-tested-by: Mukul Joshi <[email protected]>
Signed-off-by: Philip Yang <[email protected]>
Signed-off-by: Rajneesh Bhardwaj <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Set TTM pools for memory partitions

For native mode only, create TTM pool for each memory partition to store
the NUMA node id, then the TTM pool will be selected using memory
partition i

drm/amdgpu: Set TTM pools for memory partitions

For native mode only, create TTM pool for each memory partition to store
the NUMA node id, then the TTM pool will be selected using memory
partition id to allocate memory from the correct partition.

Acked-by: Christian König <[email protected]>
(rajneesh: changed need_swiotlb and need_dma32 to false for pool init)
Reviewed-by: Felix Kuehling <[email protected]>
Acked-and-tested-by: Mukul Joshi <[email protected]>
Signed-off-by: Philip Yang <[email protected]>
Signed-off-by: Rajneesh Bhardwaj <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Use apt name for FW reserved region

Use the generic term fw_reserved_memory for FW reserve region. This
region may also hold discovery TMR in addition to other reserve
regions. This regi

drm/amdgpu: Use apt name for FW reserved region

Use the generic term fw_reserved_memory for FW reserve region. This
region may also hold discovery TMR in addition to other reserve
regions. This region size could be larger than discovery tmr size, hence
don't change the discovery tmr size based on this.

Signed-off-by: Lijo Lazar <[email protected]>
Reviewed-by: Le Ma <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: Add notifier lock for KFD userptrs

Add a per-process MMU notifier lock for processing notifiers from
userptrs. Use that lock to properly synchronize page table updates with
MMU notifiers

drm/amdgpu: Add notifier lock for KFD userptrs

Add a per-process MMU notifier lock for processing notifiers from
userptrs. Use that lock to properly synchronize page table updates with
MMU notifiers.

Signed-off-by: Felix Kuehling <[email protected]>
Reviewed-by: Xiaogang Chen<[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add drv_vram_usage_va for virt data exchange

For vram_usagebyfirmware_v2_2, fw_vram_reserve is not used. So
fw_vram_usage_va is NULL, and cannot do virt data exchange
anymore. Should add

drm/amdgpu: add drv_vram_usage_va for virt data exchange

For vram_usagebyfirmware_v2_2, fw_vram_reserve is not used. So
fw_vram_usage_va is NULL, and cannot do virt data exchange
anymore. Should add drv_vram_usage_va to do virt data exchange
in vram_usagebyfirmware_v2_2 case. And refine some code style
checks in pre add vram reservation logic patch

Signed-off-by: Tong Liu01 <[email protected]>
Acked-by: Luben Tuikov <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: fix userptr HMM range handling v2

The basic problem here is that it's not allowed to page fault while
holding the reservation lock.

So it can happen that multiple processes try to valid

drm/amdgpu: fix userptr HMM range handling v2

The basic problem here is that it's not allowed to page fault while
holding the reservation lock.

So it can happen that multiple processes try to validate an userptr
at the same time.

Work around that by putting the HMM range object into the mutex
protected bo list for now.

v2: make sure range is set to NULL in case of an error

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
CC: [email protected]
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: fix userptr HMM range handling v2

The basic problem here is that it's not allowed to page fault while
holding the reservation lock.

So it can happen that multiple processes try to valid

drm/amdgpu: fix userptr HMM range handling v2

The basic problem here is that it's not allowed to page fault while
holding the reservation lock.

So it can happen that multiple processes try to validate an userptr
at the same time.

Work around that by putting the HMM range object into the mutex
protected bo list for now.

v2: make sure range is set to NULL in case of an error

Signed-off-by: Christian König <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
CC: [email protected]
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdgpu: add vram reservation based on vram_usagebyfirmware_v2_2

Move TMR region from top of FB to 2MB for FFBM, so we need to
reserve TMR region firstly to make sure TMR can be allocated at 2MB

drm/amdgpu: add vram reservation based on vram_usagebyfirmware_v2_2

Move TMR region from top of FB to 2MB for FFBM, so we need to
reserve TMR region firstly to make sure TMR can be allocated at 2MB

Signed-off-by: Tong Liu01 <[email protected]>
Acked-by: Christian König <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

Revert "drm/amdgpu: add drm buddy support to amdgpu"

This reverts commit c9cad937c0c58618fe5b0310fd539a854dc1ae95.

This is part of a revert of the following commits:
commit 708d19d9f362 ("drm/amdgp

Revert "drm/amdgpu: add drm buddy support to amdgpu"

This reverts commit c9cad937c0c58618fe5b0310fd539a854dc1ae95.

This is part of a revert of the following commits:
commit 708d19d9f362 ("drm/amdgpu: move internal vram_mgr function into the C file")
commit 5e3f1e7729ec ("drm/amdgpu: fix start calculation in amdgpu_vram_mgr_new")
commit c9cad937c0c5 ("drm/amdgpu: add drm buddy support to amdgpu")

[WHY]
Few users reported garbaged graphics as soon as x starts,
reverting until this can be resolved.

Signed-off-by: Arunpravin Paneer Selvam <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Reviewed-by: Christian König <[email protected]>
Signed-off-by: Christian König <[email protected]>

show more ...

drm/amdgpu: add drm buddy support to amdgpu

- Switch to drm buddy allocator
- Add resource cursor support for drm buddy

v2(Matthew Auld):
- replace spinlock with mutex as we call kmem_cache_zallo

drm/amdgpu: add drm buddy support to amdgpu

- Switch to drm buddy allocator
- Add resource cursor support for drm buddy

v2(Matthew Auld):
- replace spinlock with mutex as we call kmem_cache_zalloc
(..., GFP_KERNEL) in drm_buddy_alloc() function

- lock drm_buddy_block_trim() function as it calls
mark_free/mark_split are all globally visible

v3(Matthew Auld):
- remove trim method error handling as we address the failure case
at drm_buddy_block_trim() function

v4:
- fix warnings reported by kernel test robot <[email protected]>

v5:
- fix merge conflict issue

v6:
- fix warnings reported by kernel test robot <[email protected]>

v7:
- remove DRM_BUDDY_RANGE_ALLOCATION flag usage

v8:
- keep DRM_BUDDY_RANGE_ALLOCATION flag usage
- resolve conflicts created by drm/amdgpu: remove VRAM accounting v2

v9(Christian):
- merged the below patch
- drm/amdgpu: move vram inline functions into a header
- rename label name as fallback
- move struct amdgpu_vram_mgr to amdgpu_vram_mgr.h
- remove unnecessary flags from struct amdgpu_vram_reservation
- rewrite block NULL check condition
- change else style as per coding standard
- rewrite the node max size
- add a helper function to fetch the first entry from the list

v10(Christian):
- rename amdgpu_get_node() function name as amdgpu_vram_mgr_first_block

v11:
- if size is not aligned with min_page_size, enable is_contiguous flag,
therefore, the size round up to the power of two and trimmed to the
original size.
v12:
- rename the function names having prefix as amdgpu_vram_mgr_*()
- modify the round_up() logic conforming to contiguous flag enablement
or if size is not aligned to min_block_size
- modify the trim logic
- rename node as block wherever applicable

Signed-off-by: Arunpravin Paneer Selvam <[email protected]>
Acked-by: Christian König <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Christian König <[email protected]>

show more ...

drm/amdgpu: remove VRAM accounting v2

This is provided by TTM now.

Also switch man->size to bytes instead of pages and fix the double
printing of size and usage in debugfs.

v2: fix size checking a

drm/amdgpu: remove VRAM accounting v2

This is provided by TTM now.

Also switch man->size to bytes instead of pages and fix the double
printing of size and usage in debugfs.

v2: fix size checking as well

Signed-off-by: Christian König <[email protected]>
Tested-by: Bas Nieuwenhuizen <[email protected]>
Reviewed-by: Matthew Auld <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...