drm/amdkfd: Add support for more per-process flag

Add support for more per-process flags starting with option to configure
MFMA precision for gfx 9.5

v2: Change flag name to KFD_PROC_FLAG_MFMA_HIGH

drm/amdkfd: Add support for more per-process flag

Add support for more per-process flags starting with option to configure
MFMA precision for gfx 9.5

v2: Change flag name to KFD_PROC_FLAG_MFMA_HIGH_PRECISION
Remove unused else condition
v3: Bump the KFD API version
v4: Missed SH_MEM_CONFIG__PRECISION_MODE__SHIFT define. Added it.

Signed-off-by: Harish Kasiviswanathan <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Reviewed-by: Amber Lin <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: Set per-process flags only once cik/vi

Set per-process static sh_mem config only once during process
initialization. Move all static changes from update_qpd() which is
called each time a

drm/amdkfd: Set per-process flags only once cik/vi

Set per-process static sh_mem config only once during process
initialization. Move all static changes from update_qpd() which is
called each time a queue is created to set_cache_memory_policy() which
is called once during process initialization.

set_cache_memory_policy() is currently defined only for cik and vi
family. So this commit only focuses on these two. A separate commit will
address other asics.

Signed-off-by: Harish Kasiviswanathan <[email protected]>
Reviewed-by: Amber Lin <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: drop IOMMUv2 support

Now that we use the dGPU path for all APUs, drop the
IOMMUv2 support.

v2: drop the now unused queue manager functions for gfx7/8 APUs

Reviewed-by: Felix Kuehling <

drm/amdkfd: drop IOMMUv2 support

Now that we use the dGPU path for all APUs, drop the
IOMMUv2 support.

v2: drop the now unused queue manager functions for gfx7/8 APUs

Reviewed-by: Felix Kuehling <[email protected]>
Acked-by: Christian König <[email protected]>
Tested-by: Mike Lothian <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: update SPDX license header

Update the SPDX License header for all the KFD files.

Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Rajneesh Bhardwaj <rajneesh.bhardwaj

drm/amdkfd: update SPDX license header

Update the SPDX License header for all the KFD files.

Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Rajneesh Bhardwaj <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: Introduce asic-specific mqd_manager_init function

Global function mqd_manager_init just calls asic-specific functions and it
is not necessary. Delete it and introduce a mqd_manager_init

drm/amdkfd: Introduce asic-specific mqd_manager_init function

Global function mqd_manager_init just calls asic-specific functions and it
is not necessary. Delete it and introduce a mqd_manager_init interface in
dqm for asic-specific mqd manager init. Call mqd_manager_init interface
directly to initialize mqd manager

Signed-off-by: Oak Zeng <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Signed-off-by: Felix Kuehling <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: Add dGPU support to the device queue manager

GFXv7 and v8 dGPUs use a different addressing mode for KFD compared
to APUs (GPUVM64 vs HSA64). And dGPUs don't support MTYPE_CC. They
use MT

drm/amdkfd: Add dGPU support to the device queue manager

GFXv7 and v8 dGPUs use a different addressing mode for KFD compared
to APUs (GPUVM64 vs HSA64). And dGPUs don't support MTYPE_CC. They
use MTYPE_UC instead for memory that requires coherency.

Signed-off-by: Felix Kuehling <[email protected]>
Acked-by: Oded Gabbay <[email protected]>
Signed-off-by: Oded Gabbay <[email protected]>

show more ...

drm/amdkfd: Cleanup DQM ASIC-specific ops

Remove empty initialize function.

Rename register_process to update_qpd to avoid confusion with the
non-ASIC-specific register_process.

Shorten ops_asic_s

drm/amdkfd: Cleanup DQM ASIC-specific ops

Remove empty initialize function.

Rename register_process to update_qpd to avoid confusion with the
non-ASIC-specific register_process.

Shorten ops_asic_specific to asic_ops.

Signed-off-by: Yong Zhao <[email protected]>
Signed-off-by: Felix Kuehling <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Signed-off-by: Oded Gabbay <[email protected]>

show more ...

drm/amdkfd: Adding new IOCTL for scratch memory v2

v2:
* Renamed ALLOC_MEMORY_OF_SCRATCH to SET_SCRATCH_BACKING_VA
* Removed size parameter from the ioctl, it was unused
* Removed hole in ioctl numb

drm/amdkfd: Adding new IOCTL for scratch memory v2

v2:
* Renamed ALLOC_MEMORY_OF_SCRATCH to SET_SCRATCH_BACKING_VA
* Removed size parameter from the ioctl, it was unused
* Removed hole in ioctl number space
* No more call to write_config_static_mem
* Return correct error code from ioctl

Signed-off-by: Moses Reuben <[email protected]>
Signed-off-by: Ben Goz <[email protected]>
Signed-off-by: Felix Kuehling <[email protected]>
Signed-off-by: Oded Gabbay <[email protected]>

show more ...

drm/amdkfd: Handle remaining BUG_ONs more gracefully v2

In most cases, BUG_ONs can be replaced with WARN_ON with an error
return. In some void functions just turn them into a WARN_ON and
possibly an

drm/amdkfd: Handle remaining BUG_ONs more gracefully v2

In most cases, BUG_ONs can be replaced with WARN_ON with an error
return. In some void functions just turn them into a WARN_ON and
possibly an early exit.

v2:
* Cleaned up error handling in pm_send_unmap_queue
* Removed redundant WARN_ON in kfd_process_destroy_delayed

Signed-off-by: Felix Kuehling <[email protected]>
Signed-off-by: Oded Gabbay <[email protected]>

show more ...

drm/amdkfd: Remove BUG_ONs for NULL pointer arguments

Remove BUG_ONs that check for NULL pointer arguments that are
dereferenced in the same function. Dereferencing the NULL pointer
will generate a

drm/amdkfd: Remove BUG_ONs for NULL pointer arguments

Remove BUG_ONs that check for NULL pointer arguments that are
dereferenced in the same function. Dereferencing the NULL pointer
will generate a BUG anyway, so the explicit check is redundant and
unnecessary overhead.

Signed-off-by: Felix Kuehling <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Signed-off-by: Oded Gabbay <[email protected]>

show more ...

drm/amdkfd: Consolidate and clean up log commands

Consolidate log commands so that dev_info(NULL, "Error...") uses the more
accurate pr_err, remove the module name from the log (can be seen via
dyna

drm/amdkfd: Consolidate and clean up log commands

Consolidate log commands so that dev_info(NULL, "Error...") uses the more
accurate pr_err, remove the module name from the log (can be seen via
dynamic debugging with +m), and the function name (can be seen via
dynamic debugging with +f). We also don't need debug messages saying
what function we're in. Those can be added by devs when needed

Don't print vendor and device ID in error messages. They are typically
the same for all GPUs in a multi-GPU system. So this doesn't add any
value to the message.

Lastly, remove parentheses around %d, %i and 0x%llX.
According to kernel.org:
"Printing numbers in parentheses (%d) adds no value and should be
avoided."

Signed-off-by: Kent Russell <[email protected]>
Signed-off-by: Yong Zhao <[email protected]>
Signed-off-by: Felix Kuehling <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Signed-off-by: Oded Gabbay <[email protected]>

show more ...

drm/amdkfd: allow split HQD on per-queue granularity v5

Update the KGD to KFD interface to allow sharing pipes with queue
granularity instead of pipe granularity.

This allows for more interesting p

drm/amdkfd: allow split HQD on per-queue granularity v5

Update the KGD to KFD interface to allow sharing pipes with queue
granularity instead of pipe granularity.

This allows for more interesting pipe/queue splits.

v2: fix overflow check for res.queue_mask
v3: fix shift overflow when setting res.queue_mask
v4: fix comment in is_pipeline_enabled()
v5: clamp res.queue_mask to the first MEC only

Reviewed-by: Edward O'Callaghan <[email protected]>
Reviewed-by: Felix Kuehling <[email protected]>
Acked-by: Christian König <[email protected]>
Signed-off-by: Andres Rodriguez <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: fix bug when initializing sdma vm

A logical AND operation was used during mask and shift, instead of a
bitwise AND operation. This patch fixes this bug by changing the
operation to bitwi

drm/amdkfd: fix bug when initializing sdma vm

A logical AND operation was used during mask and shift, instead of a
bitwise AND operation. This patch fixes this bug by changing the
operation to bitwise AND.

Signed-off-by: Oded Gabbay <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: Use generic defines in new amd headers

This patch makes use of the new amd headers (that are part of the new
amdgpu driver), instead of private defines.

Signed-off-by: Oded Gabbay <oded

drm/amdkfd: Use generic defines in new amd headers

This patch makes use of the new amd headers (that are part of the new
amdgpu driver), instead of private defines.

Signed-off-by: Oded Gabbay <[email protected]>

show more ...

drm/amdkfd: make the sdma vm init to be asic specific

Signed-off-by: Oded Gabbay <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>

drm/amdkfd: Use new struct for asic specific ops

This patch creates a new structure for asic specific operations, instead
of using the existing structure of operations.

This is done to make the cod

drm/amdkfd: Use new struct for asic specific ops

This patch creates a new structure for asic specific operations, instead
of using the existing structure of operations.

This is done to make the code flow more logic, readable and maintainable.

The change is done only to the device queue manager module at this point.

Signed-off-by: Oded Gabbay <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>

show more ...

drm/amdkfd: Initialize only amdkfd's assigned pipelines

This patch fixes a bug in the initialization of the pipelines. The
init_pipelines() function was called with a constant value of 0 in the
firs

drm/amdkfd: Initialize only amdkfd's assigned pipelines

This patch fixes a bug in the initialization of the pipelines. The
init_pipelines() function was called with a constant value of 0 in the
first_pipe argument. This is an error because amdkfd doesn't handle pipe 0.

The correct way is to pass the value that get_first_pipe() returns as the
argument for first_pipe.

This bug appeared in 3.19 (first version with amdkfd) and it causes around 15%
drop in CPU performance of Kaveri (A10-7850).

v2: Don't set get_first_pipe() as inline because it calls BUG_ON()

Signed-off-by: Oded Gabbay <[email protected]>
Cc: [email protected]
Tested-by: Michel Dänzer <[email protected]>

show more ...

drm/amdkfd: Add initial VI support for DQM

This patch starts to add support for the VI APU in the DQM module.

Because most (more than 90%) of the DQM code is shared among AMD's APUs, we
chose a des

drm/amdkfd: Add initial VI support for DQM

This patch starts to add support for the VI APU in the DQM module.

Because most (more than 90%) of the DQM code is shared among AMD's APUs, we
chose a design that performs most/all the code in the shared DQM file
(kfd_device_queue_manager.c). If there is H/W specific code to be executed,
than it is written in an asic-specific extension function for that H/W.

That asic-specific extension function is called from the shared function at the
appropriate time. This requires that for every asic-specific extension function
that is implemented in a specific ASIC, there will be an equivalent
implementation in ALL ASICs, even if those implementations are just stubs.

That way we achieve:

- Maintainability: by having one copy of most of the code, we only need to
fix bugs at one locations

- Readability: very clear what is the shared code and what is done per ASIC

- Extensibility: very easy to add new H/W specific files/functions

Signed-off-by: Ben Goz <[email protected]>
Signed-off-by: Oded Gabbay <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>

show more ...