accel/ivpu: Fix the NPU's DPU frequency calculation

Fix the frequency returned to the user space by
the DRM_IVPU_PARAM_CORE_CLOCK_RATE GET_PARAM IOCTL.
The kernel driver returned CPU frequency for M

accel/ivpu: Fix the NPU's DPU frequency calculation

Fix the frequency returned to the user space by
the DRM_IVPU_PARAM_CORE_CLOCK_RATE GET_PARAM IOCTL.
The kernel driver returned CPU frequency for MTL and bare
PLL frequency for LNL - this was inconsistent and incorrect
for both platforms. With this fix the driver returns maximum
frequency of the NPU data processing unit (DPU) for all HW
generations. This is what user space always expected.

Also do not set CPU frequency in boot params - the firmware
does not use frequency passed from the driver, it was only
used by the early pre-production firmware.
With that we can remove CPU frequency calculation code.

Show NPU frequency in FREQ_CHANGE interrupt when frequency
tracking is enabled.

Fixes: 8a27ad81f7d3 ("accel/ivpu: Split IP and buttress code")
Cc: [email protected] # v6.11+
Signed-off-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Maciej Falkowski <[email protected]>
Reviewed-by: Jeff Hugo <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Link: https://lore.kernel.org/r/[email protected]

show more ...

accel/ivpu: Add API for command queue create/destroy/submit

Implement support for explicit command queue management.
To allow more flexible control over command queues add capabilities
to create, de

accel/ivpu: Add API for command queue create/destroy/submit

Implement support for explicit command queue management.
To allow more flexible control over command queues add capabilities
to create, destroy and submit jobs to specific command queues.

Signed-off-by: Karol Wachowski <[email protected]>
Signed-off-by: Maciej Falkowski <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Remove copy engine support

Copy engine was deprecated by the FW and is no longer supported.
Compute engine includes all copy engine functionality and should be used
instead.

This change

accel/ivpu: Remove copy engine support

Copy engine was deprecated by the FW and is no longer supported.
Compute engine includes all copy engine functionality and should be used
instead.

This change does not affect user space as the copy engine was never
used outside of a couple of tests.

Signed-off-by: Andrzej Kacprowski <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Stop using hardcoded DRIVER_DATE

Hardcoded driver date is useless, so use kernel version as a driver date
to make identifying .ko file easier. Also allow to pass DRIVER_DATE
on build tim

accel/ivpu: Stop using hardcoded DRIVER_DATE

Hardcoded driver date is useless, so use kernel version as a driver date
to make identifying .ko file easier. Also allow to pass DRIVER_DATE
on build time to allow versioning the driver in case it is built out
of the tree.

Reviewed-by: Karol Wachowski <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Jacek Lawrynowicz <[email protected]>

show more ...

accel/ivpu: Add NPU profiling support

Implement time based Metric Streamer profiling UAPI.

This is a generic mechanism allowing user mode tools to sample
NPU metrics. These metrics are defined by t

accel/ivpu: Add NPU profiling support

Implement time based Metric Streamer profiling UAPI.

This is a generic mechanism allowing user mode tools to sample
NPU metrics. These metrics are defined by the FW and transparent to
the driver.

The user space can check for this feature by checking
DRM_IVPU_CAP_METRIC_STREAMER driver capability.

Signed-off-by: Tomasz Rusinowicz <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add job status for jobs aborted by the driver

Add DRM_IVPU_JOB_STATUS_ABORTED to indicate that the job was aborted
by the driver due to e.g. TDR or user context MMU faults.

This will he

accel/ivpu: Add job status for jobs aborted by the driver

Add DRM_IVPU_JOB_STATUS_ABORTED to indicate that the job was aborted
by the driver due to e.g. TDR or user context MMU faults.

This will help UMD and tests distinguish if job was aborted by the FW
or the driver.

Signed-off-by: Grzegorz Trzebiatowski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Deprecate DRM_IVPU_PARAM_CONTEXT_PRIORITY param

DRM_IVPU_PARAM_CONTEXT_PRIORITY has been deprecated because it
has been replaced with DRM_IVPU_JOB_PRIORITY levels set with
submit IOCTL a

accel/ivpu: Deprecate DRM_IVPU_PARAM_CONTEXT_PRIORITY param

DRM_IVPU_PARAM_CONTEXT_PRIORITY has been deprecated because it
has been replaced with DRM_IVPU_JOB_PRIORITY levels set with
submit IOCTL and was unused anyway.

Signed-off-by: Wachowski, Karol <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Remove support for uncached buffers

Usages of DRM_IVPU_BO_UNCACHED should be replaced by DRM_IVPU_BO_WC.
There is no functional benefit from DRM_IVPU_BO_UNCACHED if these
buffers are nev

accel/ivpu: Remove support for uncached buffers

Usages of DRM_IVPU_BO_UNCACHED should be replaced by DRM_IVPU_BO_WC.
There is no functional benefit from DRM_IVPU_BO_UNCACHED if these
buffers are never mapped to host VM.

This allows to cut the buffer handling code in the kernel driver
by half.

Usage of DRM_IVPU_BO_UNCACHED buffers was removed from user-space
driver and will not be part of first UMD release.

Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Document DRM_IVPU_PARAM_CAPABILITIES

Add comments regarding new DRM_IVPU_PARAM_CAPABILITIES param.

Signed-off-by: Stanislaw Gruszka <[email protected]>
Reviewed-by: Jeff

accel/ivpu: Document DRM_IVPU_PARAM_CAPABILITIES

Add comments regarding new DRM_IVPU_PARAM_CAPABILITIES param.

Signed-off-by: Stanislaw Gruszka <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Refactor memory ranges logic

Add new dma range and change naming convention for virtual address
memory ranges managed by KMD.

New available ranges are named as follows:
* global range

accel/ivpu: Refactor memory ranges logic

Add new dma range and change naming convention for virtual address
memory ranges managed by KMD.

New available ranges are named as follows:
* global range - global context accessible by FW
* aliased range - user context accessible by FW
* dma range - user context accessible by DMA
* shave range - user context accessible by shaves
* global shave range - global context accessible by shave nn

Signed-off-by: Karol Wachowski <[email protected]>
Reviewed-by: Stanislaw Gruszka <[email protected]>
Signed-off-by: Stanislaw Gruszka <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Extend get_param ioctl to identify capabilities

Add DRM_IVPU_PARAM_CAPABILITIES parameters to get_param ioctl to query
driver capabilities. For now use it for identify metric streamer an

accel/ivpu: Extend get_param ioctl to identify capabilities

Add DRM_IVPU_PARAM_CAPABILITIES parameters to get_param ioctl to query
driver capabilities. For now use it for identify metric streamer and
new dma memory range features. Currently upstream version of intel_vpu
does not have those, they will be added it the future.

Reviewed-by: Jacek Lawrynowicz <[email protected]>
Signed-off-by: Stanislaw Gruszka <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add command buffer submission logic

Each of the user contexts has two command queues, one for compute engine
and one for the copy engine. Command queues are allocated and registered
in t

accel/ivpu: Add command buffer submission logic

Each of the user contexts has two command queues, one for compute engine
and one for the copy engine. Command queues are allocated and registered
in the device when the first job (command buffer) is submitted from
the user space to the VPU device. The userspace provides a list of
GEM buffer object handles to submit to the VPU, the driver resolves
buffer handles, pins physical memory if needed, increments ref count
for each buffer and stores pointers to buffer objects in
the ivpu_job objects that track jobs submitted to the device.
The VPU signals job completion with an asynchronous message that
contains the job id passed to firmware when the job was submitted.

Currently, the driver supports simple scheduling logic
where jobs submitted from user space are immediately pushed
to the VPU device command queues. In the future, it will be
extended to use hardware base scheduling and/or drm_sched.

Co-developed-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Implement firmware parsing and booting

Read, parse and boot VPU firmware image.

Co-developed-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Andrzej Kacprowsk

accel/ivpu: Implement firmware parsing and booting

Read, parse and boot VPU firmware image.

Co-developed-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Andrzej Kacprowski <[email protected]>
Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add GEM buffer object management

Adds four types of GEM-based BOs for the VPU:
- shmem
- internal
- prime

All types are implemented as struct ivpu_bo, based on
struct drm_gem_obje

accel/ivpu: Add GEM buffer object management

Adds four types of GEM-based BOs for the VPU:
- shmem
- internal
- prime

All types are implemented as struct ivpu_bo, based on
struct drm_gem_object. VPU address is allocated when buffer is created
except for imported prime buffers that allocate it in BO_INFO IOCTL due
to missing file_priv arg in gem_prime_import callback.
Internal buffers are pinned on creation, the rest of buffers types
can be pinned on demand (in SUBMIT IOCTL).
Buffer VPU address, allocated pages and mappings are released when the
buffer is destroyed.
Eviction mechanism is planned for future versions.

Add two new IOCTLs: BO_CREATE, BO_INFO

Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add Intel VPU MMU support

VPU Memory Management Unit is based on ARM MMU-600.
It allows the creation of multiple virtual address spaces for
the device and map noncontinuous host memory (

accel/ivpu: Add Intel VPU MMU support

VPU Memory Management Unit is based on ARM MMU-600.
It allows the creation of multiple virtual address spaces for
the device and map noncontinuous host memory (there is no dedicated
memory on the VPU).

Address space is implemented as a struct ivpu_mmu_context, it has an ID,
drm_mm allocator for VPU addresses and struct ivpu_mmu_pgtable that
holds actual 3-level, 4KB page table.
Context with ID 0 (global context) is created upon driver initialization
and it's mainly used for mapping memory required to execute
the firmware.
Contexts with non-zero IDs are user contexts allocated each time
the devices is open()-ed and they map command buffers and other
workload-related memory.
Workloads executing in a given contexts have access only
to the memory mapped in this context.

This patch is has two main files:
- ivpu_mmu_context.c handles MMU page tables and memory mapping
- ivpu_mmu.c implements a driver that programs the MMU device

Co-developed-by: Karol Wachowski <[email protected]>
Signed-off-by: Karol Wachowski <[email protected]>
Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Introduce a new DRM driver for Intel VPU

VPU stands for Versatile Processing Unit and it's a CPU-integrated
inference accelerator for Computer Vision and Deep Learning
applications.

The

accel/ivpu: Introduce a new DRM driver for Intel VPU

VPU stands for Versatile Processing Unit and it's a CPU-integrated
inference accelerator for Computer Vision and Deep Learning
applications.

The VPU device consist of following components:
- Buttress - provides CPU to VPU integration, interrupt, frequency and
power management.
- Memory Management Unit (based on ARM MMU-600) - translates VPU to
host DMA addresses, isolates user workloads.
- RISC based microcontroller - executes firmware that provides job
execution API for the kernel-mode driver
- Neural Compute Subsystem (NCS) - does the actual work, provides
Compute and Copy engines.
- Network on Chip (NoC) - network fabric connecting all the components

This driver supports VPU IP v2.7 integrated into Intel Meteor Lake
client CPUs (14th generation).

Module sources are at drivers/accel/ivpu and module name is
"intel_vpu.ko".

This patch includes only very besic functionality:
- module, PCI device and IRQ initialization
- register definitions and low level register manipulation functions
- SET/GET_PARAM ioctls
- power up without firmware

Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...