drm/xe: Add SVM device memory mirroring

Add SVM device memory mirroring which enables device pages for
migration. Enabled via CONFIG_XE_DEVMEM_MIRROR Kconfig. Kconfig option
defaults to enabled. If

drm/xe: Add SVM device memory mirroring

Add SVM device memory mirroring which enables device pages for
migration. Enabled via CONFIG_XE_DEVMEM_MIRROR Kconfig. Kconfig option
defaults to enabled. If not enabled, SVM will work sans migration and
KMD memory footprint will be less.

v3:
- Add CONFIG_XE_DEVMEM_MIRROR
v4:
- Fix Kconfig (Himal)
- Use %pe to print errors (Thomas)
- Fix alignment issue (Checkpatch)
v5:
- s/xe_mem_region/xe_vram_region (Rebase)
v6:
- Only compile if CONFIG_DRM_GPUSVM selected (CI, Lucas)
- s/drm_info/drm_dbg/

Signed-off-by: Niranjana Vishwanathapura <[email protected]>
Signed-off-by: Oak Zeng <[email protected]>
Signed-off-by: Matthew Brost <[email protected]>
Reviewed-by: Thomas Hellström <[email protected]>
Reviewed-by: Himal Prasad Ghimiray <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

drm/xe: Fix xe_tile_init_noalloc() error propagation

Propagate the error to the caller so initialization properly stops if
sysfs creation fails.

Reviewed-by: Francois Dugast <francois.dugast@intel.

drm/xe: Fix xe_tile_init_noalloc() error propagation

Propagate the error to the caller so initialization properly stops if
sysfs creation fails.

Reviewed-by: Francois Dugast <[email protected]>
Reviewed-by: Himal Prasad Ghimiray <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Lucas De Marchi <[email protected]>

show more ...

drm/xe: Move VRAM manager to struct xe_vram_region

VRAM manager is related directly to struct xe_vram_region so it
should be inside this structure.
Let's move the VRAM to struct xe_vram_region.

v2:

drm/xe: Move VRAM manager to struct xe_vram_region

VRAM manager is related directly to struct xe_vram_region so it
should be inside this structure.
Let's move the VRAM to struct xe_vram_region.

v2:
- remove xe_vram_region pointer from xe_ttm_vram_mgr
- stop use dynamic alloaction for xe_ttm_vram_mgr in xe_vram_region
- rename struct xe_ttm_vram_mgr vram_mgr to ttm
v3:
- fix "'ttm' not described in 'xe_vram_region'"

Signed-off-by: Piotr Piórkowski <[email protected]>
Cc: Rodrigo Vivi <[email protected]>
Reviewed-by: Rodrigo Vivi <[email protected]>
Signed-off-by: Michal Wajdeczko <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

drm/xe: Move suballocator init to after display init

No allocations should be done before we have had a chance to preserve
the display fb.

Reviewed-by: Rodrigo Vivi <[email protected]>
Link: h

drm/xe: Move suballocator init to after display init

No allocations should be done before we have had a chance to preserve
the display fb.

Reviewed-by: Rodrigo Vivi <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Maarten Lankhorst <[email protected]>

show more ...

drm/xe: Use fault injection infrastructure to find issues at probe time

The kernel fault injection infrastructure is used to test proper error
handling during probe. The return code of the functions

drm/xe: Use fault injection infrastructure to find issues at probe time

The kernel fault injection infrastructure is used to test proper error
handling during probe. The return code of the functions using
ALLOW_ERROR_INJECTION() can be conditionnally modified at runtime by
tuning some debugfs entries. This requires CONFIG_FUNCTION_ERROR_INJECTION
(among others).

One way to use fault injection at probe time by making each of those
functions fail one at a time is:

FAILTYPE=fail_function
DEVICE="0000:00:08.0" # depends on the system
ERRNO=-12 # -ENOMEM, can depend on the function

echo N > /sys/kernel/debug/$FAILTYPE/task-filter
echo 100 > /sys/kernel/debug/$FAILTYPE/probability
echo 0 > /sys/kernel/debug/$FAILTYPE/interval
echo -1 > /sys/kernel/debug/$FAILTYPE/times
echo 0 > /sys/kernel/debug/$FAILTYPE/space
echo 1 > /sys/kernel/debug/$FAILTYPE/verbose

modprobe xe
echo $DEVICE > /sys/bus/pci/drivers/xe/unbind

grep -oP "^.* \[xe\]" /sys/kernel/debug/$FAILTYPE/injectable | \
cut -d ' ' -f 1 | while read -r FUNCTION ; do
echo "Injecting fault in $FUNCTION"
echo "" > /sys/kernel/debug/$FAILTYPE/inject
echo $FUNCTION > /sys/kernel/debug/$FAILTYPE/inject
printf %#x $ERRNO > /sys/kernel/debug/$FAILTYPE/$FUNCTION/retval
echo $DEVICE > /sys/bus/pci/drivers/xe/bind
done

rmmod xe

It will also be integrated into IGT for systematic execution by CI.

v2: Wrappers are not needed in the cases covered by this patch, so
remove them and use ALLOW_ERROR_INJECTION() directly.

v3: Document the use of fault injection at probe time in xe_pci_probe
and refer to it where ALLOW_ERROR_INJECTION() is used.

Signed-off-by: Francois Dugast <[email protected]>
Cc: Lucas De Marchi <[email protected]>
Cc: Matthew Brost <[email protected]>
Cc: Rodrigo Vivi <[email protected]>
Cc: Michal Wajdeczko <[email protected]>
Cc: Jani Nikula <[email protected]>
Reviewed-by: Rodrigo Vivi <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe/pcode: Treat pcode as per-tile rather than per-GT

There's only one instance of the pcode per tile, and for GT-related
accesses both the primary and media GT share the same register
interface.

drm/xe/pcode: Treat pcode as per-tile rather than per-GT

There's only one instance of the pcode per tile, and for GT-related
accesses both the primary and media GT share the same register
interface. Since Xe was using per-GT locking, the pcode mutex wasn't
actually protecting everything that it should since concurrent accesses
related to a tile's primary GT and media GT were possible.

Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Signed-off-by: Matt Roper <[email protected]>
Reviewed-by: Lucas De Marchi <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
(cherry picked from commit 3034cc8107b8d0c7d1b56584394e215dab57f8a3)
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe/pcode: Treat pcode as per-tile rather than per-GT

There's only one instance of the pcode per tile, and for GT-related
accesses both the primary and media GT share the same register
interface.

drm/xe/pcode: Treat pcode as per-tile rather than per-GT

There's only one instance of the pcode per tile, and for GT-related
accesses both the primary and media GT share the same register
interface. Since Xe was using per-GT locking, the pcode mutex wasn't
actually protecting everything that it should since concurrent accesses
related to a tile's primary GT and media GT were possible.

Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs")
Signed-off-by: Matt Roper <[email protected]>
Reviewed-by: Lucas De Marchi <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

drm/xe/tile: Abort driver load for sysfs creation failure

Ensure that the status of all tile associated sysfs entries creation is
relayed to xe_tile_init_noalloc, leading to a driver load abort if a

drm/xe/tile: Abort driver load for sysfs creation failure

Ensure that the status of all tile associated sysfs entries creation is
relayed to xe_tile_init_noalloc, leading to a driver load abort if any
sysfs creation failures occur.

-v2
Avoid unnecessary warn/error messages. (Lucas)

Cc: Rodrigo Vivi <[email protected]>
Cc: Lucas De Marchi <[email protected]>
Signed-off-by: Himal Prasad Ghimiray <[email protected]>
Reviewed-by: Lucas De Marchi <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Lucas De Marchi <[email protected]>

show more ...

drm/xe: Remove useless mem_access during probe

xe_pm_init is the very last thing during the xe_pci_probe(),
hence these protections are useless from the point of view
of ensuring that the device is

drm/xe: Remove useless mem_access during probe

xe_pm_init is the very last thing during the xe_pci_probe(),
hence these protections are useless from the point of view
of ensuring that the device is awake.

Let's remove it so we continue towards the goal of killing
xe_device_mem_access.

v2: Adding more cases
v3: Provide a separate fix for xe_tile_init_noalloc return (Matt)
Adding a new case where display HDCP init calls which
are also called at display probe time.

Cc: Matthew Auld <[email protected]>
Reviewed-by: Matthew Auld <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Return immediately on tile_init failure

There's no reason to proceed with applying workaround and initing
sysfs if we are going to abort the probe upon failure.

Fixes: e5a845fd8fa4 ("drm/xe

drm/xe: Return immediately on tile_init failure

There's no reason to proceed with applying workaround and initing
sysfs if we are going to abort the probe upon failure.

Fixes: e5a845fd8fa4 ("drm/xe: Add sysfs entry for tile")
Cc: Lucas De Marchi <[email protected]>
Cc: Matt Roper <[email protected]>
Cc: Matthew Auld <[email protected]>
Reviewed-by: Matt Roper <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Rodrigo Vivi <[email protected]>
(cherry picked from commit af7b93d1d7eeeef674681ddea875be6a29857a5d)
Signed-off-by: Thomas Hellström <[email protected]>

show more ...

drm/xe: Return immediately on tile_init failure

There's no reason to proceed with applying workaround and initing
sysfs if we are going to abort the probe upon failure.

Fixes: e5a845fd8fa4 ("drm/xe

drm/xe: Return immediately on tile_init failure

There's no reason to proceed with applying workaround and initing
sysfs if we are going to abort the probe upon failure.

Fixes: e5a845fd8fa4 ("drm/xe: Add sysfs entry for tile")
Cc: Lucas De Marchi <[email protected]>
Cc: Matt Roper <[email protected]>
Cc: Matthew Auld <[email protected]>
Reviewed-by: Matt Roper <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Reorder GGTT init to earlier point in probe

GuC will need to be loaded earlier during probe. Having functional GGTT
is one of the prerequisites.
Also rename xe_ggtt_init_noalloc to xe_ggtt_i

drm/xe: Reorder GGTT init to earlier point in probe

GuC will need to be loaded earlier during probe. Having functional GGTT
is one of the prerequisites.
Also rename xe_ggtt_init_noalloc to xe_ggtt_init_early to match the new
call site.

Signed-off-by: Michał Winiarski <[email protected]>
Reviewed-by: Matt Roper <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Introduce xe_tile_init_early and use at earlier point in probe

It also merges the GT (which is part of tile) initialization happening
at xe_info_init with allocating other per-tile data stru

drm/xe: Introduce xe_tile_init_early and use at earlier point in probe

It also merges the GT (which is part of tile) initialization happening
at xe_info_init with allocating other per-tile data structures into a
common helper function.

Signed-off-by: Michał Winiarski <[email protected]>
Reviewed-by: Matthew Brost <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe/wa: Apply tile workarounds at probe/resume

Although the vast majority of workarounds the driver needs to implement
are either GT-based or display-based, there are occasionally workarounds
tha

drm/xe/wa: Apply tile workarounds at probe/resume

Although the vast majority of workarounds the driver needs to implement
are either GT-based or display-based, there are occasionally workarounds
that reside outside those parts of the hardware (i.e., in they target
registers in the sgunit/soc); we can consider these to be "tile"
workarounds since there will be instance of these registers per tile.
The registers in question should only lose their values during a
function-level reset, so they only need to be applied during probe and
resume; the registers will not be affected by GT/engine resets.

Tile workarounds are rare (there's only one, 22010954014, that's
relevant to Xe at the moment) so it's probably not worth updating the
xe_rtp design to handle tile-level workarounds yet, although we may want
to consider that in the future if/when more of these show up on future
platforms.

Reviewed-by: Lucas De Marchi <[email protected]>
Acked-by: Jani Nikula <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Matt Roper <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Make usable size of VRAM readable

Current size member of vram struct does not give
complete information as what "size" contains. Does
it contain reserved portions or not. Name it usable
size

drm/xe: Make usable size of VRAM readable

Current size member of vram struct does not give
complete information as what "size" contains. Does
it contain reserved portions or not. Name it usable
size and accordingly describe other size members as
well.

Reviewed-by: Matthew Brost <[email protected]>
Reviewed-by: Lucas De Marchi <[email protected]>
Signed-off-by: Tejas Upadhyay <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Add sysfs entry for tile

We have recently introduced tile for each gpu,
so lets add sysfs entry per tile for userspace
to provide required information specific to tile.

V5:
- define ktype

drm/xe: Add sysfs entry for tile

We have recently introduced tile for each gpu,
so lets add sysfs entry per tile for userspace
to provide required information specific to tile.

V5:
- define ktype as const
V4:
- Reorder headers - Aravind
V3:
- Make API to return void and add drm_warn - Aravind
V2:
- Add logs in failure path

Reviewed-by: Aravind Iddamsetty <[email protected]>
Signed-off-by: Tejas Upadhyay <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Add kerneldoc description of multi-tile devices

v2:
- Fix doubled word. (Lucas)

Reviewed-by: Lucas De Marchi <[email protected]>
Link: https://lore.kernel.org/r/20230601215244.6786

drm/xe: Add kerneldoc description of multi-tile devices

v2:
- Fix doubled word. (Lucas)

Reviewed-by: Lucas De Marchi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Matt Roper <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Move migration from GT to tile

Migration primarily focuses on the memory associated with a tile, so it
makes more sense to track this at the tile level (especially since the
driver was alrea

drm/xe: Move migration from GT to tile

Migration primarily focuses on the memory associated with a tile, so it
makes more sense to track this at the tile level (especially since the
driver was already skipping migration operations on media GTs).

Note that the blitter engine used to perform the migration always lives
in the tile's primary GT today. In theory that could change if media
GTs ever start including blitter engines in the future, but we can
extend the design if/when that happens in the future.

v2:
- Fix kunit test build
- Kerneldoc parameter name update
v3:
- Removed leftover prototype for removed function. (Gustavo)
- Remove unrelated / unwanted error handling change. (Gustavo)

Cc: Gustavo Sousa <[email protected]>
Reviewed-by: Lucas De Marchi <[email protected]>
Acked-by: Gustavo Sousa <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Matt Roper <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Memory allocations are tile-based, not GT-based

Since memory and address spaces are a tile concept rather than a GT
concept, we need to plumb tile-based handling through lots of
memory-relat

drm/xe: Memory allocations are tile-based, not GT-based

Since memory and address spaces are a tile concept rather than a GT
concept, we need to plumb tile-based handling through lots of
memory-related code.

Note that one remaining shortcoming here that will need to be addressed
before media GT support can be re-enabled is that although the address
space is shared between a tile's GTs, each GT caches the PTEs
independently in their own TLB and thus TLB invalidation should be
handled at the GT level.

v2:
- Fix kunit test build.

Reviewed-by: Lucas De Marchi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Matt Roper <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Move VRAM from GT to tile

On platforms with VRAM, the VRAM is associated with the tile, not the
GT.

v2:
- Unsquash the GGTT handling back into its own patch.
- Fix kunit test build
v3:
-

drm/xe: Move VRAM from GT to tile

On platforms with VRAM, the VRAM is associated with the tile, not the
GT.

v2:
- Unsquash the GGTT handling back into its own patch.
- Fix kunit test build
v3:
- Tweak the "FIXME" comment to clarify that this function will be
completely gone by the end of the series. (Lucas)
v4:
- Move a few changes that were supposed to be part of the GGTT patch
back to that commit. (Gustavo)
v5:
- Kerneldoc parameter name fix.

Cc: Gustavo Sousa <[email protected]>
Reviewed-by: Lucas De Marchi <[email protected]>
Acked-by: Gustavo Sousa <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Matt Roper <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...

drm/xe: Move GGTT from GT to tile

The GGTT exists at the tile level. When a tile contains multiple GTs,
they share the same GGTT.

v2:
- Include some changes that were mis-squashed into the VRAM p

drm/xe: Move GGTT from GT to tile

The GGTT exists at the tile level. When a tile contains multiple GTs,
they share the same GGTT.

v2:
- Include some changes that were mis-squashed into the VRAM patch.
(Gustavo)

Cc: Gustavo Sousa <[email protected]>
Reviewed-by: Lucas De Marchi <[email protected]>
Acked-by: Gustavo Sousa <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Matt Roper <[email protected]>
Signed-off-by: Rodrigo Vivi <[email protected]>

show more ...