accel/ivpu: Add debug Kconfig option

Add CONFIG_DRM_ACCEL_IVPU_DEBUG option that:
- Adds -DDEBUG that enables printk regardless of the kernel config
- Enables unsafe module params (that are now di

accel/ivpu: Add debug Kconfig option

Add CONFIG_DRM_ACCEL_IVPU_DEBUG option that:
- Adds -DDEBUG that enables printk regardless of the kernel config
- Enables unsafe module params (that are now disabled by default)

Signed-off-by: Maciej Falkowski <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add tracing for IPC/PM/JOB

Add multiple trace points in couple of key places to enable tracing
with ftrace.

Signed-off-by: Jakub Pawlak <[email protected]>
Reviewed-by: Jacek Lawry

accel/ivpu: Add tracing for IPC/PM/JOB

Add multiple trace points in couple of key places to enable tracing
with ftrace.

Signed-off-by: Jakub Pawlak <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Jacek Lawrynowicz <[email protected]>

show more ...

accel/ivpu: Add coredump support

Use coredump (if available) to collect FW logs in case of a FW crash.
This makes dmesg more readable and allows to collect more log data.

Signed-off-by: Karol Wacho

accel/ivpu: Add coredump support

Use coredump (if available) to collect FW logs in case of a FW crash.
This makes dmesg more readable and allows to collect more log data.

Signed-off-by: Karol Wachowski <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]
Signed-off-by: Jacek Lawrynowicz <[email protected]>

show more ...

accel/ivpu: Split IP and buttress code

The NPU device consists of two parts: NPU buttress and NPU IP.
Buttress is a platform specific part that integrates the NPU IP with
the CPU.
NPU IP is the plat

accel/ivpu: Split IP and buttress code

The NPU device consists of two parts: NPU buttress and NPU IP.
Buttress is a platform specific part that integrates the NPU IP with
the CPU.
NPU IP is the platform agnostic part that does the inference.

This separation enables support for multiple platforms using
a single NPU IP, so for example NPU IP 37XX could be integrated into
MTL and LNL platforms.

Signed-off-by: Wachowski, Karol <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Share NPU busy time in sysfs

The driver tracks the time spent by NPU executing jobs
and shares it through sysfs `npu_busy_time_us` file.
It can be then used by user space applications to

accel/ivpu: Share NPU busy time in sysfs

The driver tracks the time spent by NPU executing jobs
and shares it through sysfs `npu_busy_time_us` file.
It can be then used by user space applications to monitor device
utilization.

NPU is considered 'busy' starting with a first job submitted
to firmware and ending when there is no more jobs pending/executing.

Signed-off-by: Tomasz Rusinowicz <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add NPU profiling support

Implement time based Metric Streamer profiling UAPI.

This is a generic mechanism allowing user mode tools to sample
NPU metrics. These metrics are defined by t

accel/ivpu: Add NPU profiling support

Implement time based Metric Streamer profiling UAPI.

This is a generic mechanism allowing user mode tools to sample
NPU metrics. These metrics are defined by the FW and transparent to
the driver.

The user space can check for this feature by checking
DRM_IVPU_CAP_METRIC_STREAMER driver capability.

Signed-off-by: Tomasz Rusinowicz <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Compile ivpu_debugfs.c conditionally

Only compile ivpu_debugfs.c file with CONFIG_DEBUG_FS.

Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Stanislaw Gruszka <stanisla

accel/ivpu: Compile ivpu_debugfs.c conditionally

Only compile ivpu_debugfs.c file with CONFIG_DEBUG_FS.

Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Stanislaw Gruszka <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add initial support for VPU 4

Add support VPU 4 - new generation of VPU IP with various
hardware design improvements. From driver point of view, it differs
in register set, initializatio

accel/ivpu: Add initial support for VPU 4

Add support VPU 4 - new generation of VPU IP with various
hardware design improvements. From driver point of view, it differs
in register set, initialization process and MMU memory ranges.

Co-developed-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Andrzej Kacprowski <[email protected]>
Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Co-developed-by: Karol Wachowski <[email protected]>
Signed-off-by: Karol Wachowski <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Signed-off-by: Stanislaw Gruszka <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Rename sources to use generation based names

Given that VPU generation can be used by multiple platforms, driver should
use VPU IP generation in names instead of a platform.

Change nami

accel/ivpu: Rename sources to use generation based names

Given that VPU generation can be used by multiple platforms, driver should
use VPU IP generation in names instead of a platform.

Change naming for sources files.

Use 37XX format, where:
3 - major VPU IP generation version
7 - minor VPU IP generation version
XX - postfix indicating this is an architecture and not marketing name

Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Stanislaw Gruszka <[email protected]>
Signed-off-by: Stanislaw Gruszka <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add firmware tracing support

Add support for firmware tracing and logging via debugfs.

Signed-off-by: Stanislaw Gruszka <[email protected]>
Reviewed-by: Jacek Lawrynowic

accel/ivpu: Add firmware tracing support

Add support for firmware tracing and logging via debugfs.

Signed-off-by: Stanislaw Gruszka <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Initial debugfs support

Add initial debugfs support. Provide below functionality:

- print buffer objects
- print latest boot mode
- trigger vpu engine reset

Signed-off-by: Stanislaw Gr

accel/ivpu: Initial debugfs support

Add initial debugfs support. Provide below functionality:

- print buffer objects
- print latest boot mode
- trigger vpu engine reset

Signed-off-by: Stanislaw Gruszka <[email protected]>
Reviewed-by: Jacek Lawrynowicz <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add PM support

- Implement cold and warm firmware boot flows
- Add hang recovery support
- Add runtime power management support

Co-developed-by: Krystian Pradzynski <krystian.prad

accel/ivpu: Add PM support

- Implement cold and warm firmware boot flows
- Add hang recovery support
- Add runtime power management support

Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add command buffer submission logic

Each of the user contexts has two command queues, one for compute engine
and one for the copy engine. Command queues are allocated and registered
in t

accel/ivpu: Add command buffer submission logic

Each of the user contexts has two command queues, one for compute engine
and one for the copy engine. Command queues are allocated and registered
in the device when the first job (command buffer) is submitted from
the user space to the VPU device. The userspace provides a list of
GEM buffer object handles to submit to the VPU, the driver resolves
buffer handles, pins physical memory if needed, increments ref count
for each buffer and stores pointers to buffer objects in
the ivpu_job objects that track jobs submitted to the device.
The VPU signals job completion with an asynchronous message that
contains the job id passed to firmware when the job was submitted.

Currently, the driver supports simple scheduling logic
where jobs submitted from user space are immediately pushed
to the VPU device command queues. In the future, it will be
extended to use hardware base scheduling and/or drm_sched.

Co-developed-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Implement firmware parsing and booting

Read, parse and boot VPU firmware image.

Co-developed-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Andrzej Kacprowsk

accel/ivpu: Implement firmware parsing and booting

Read, parse and boot VPU firmware image.

Co-developed-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Andrzej Kacprowski <[email protected]>
Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add IPC driver and JSM messages

The IPC driver is used to send and receive messages to/from firmware
running on the VPU.

The only supported IPC message format is Job Submission Model (J

accel/ivpu: Add IPC driver and JSM messages

The IPC driver is used to send and receive messages to/from firmware
running on the VPU.

The only supported IPC message format is Job Submission Model (JSM)
defined in vpu_jsm_api.h header.

Co-developed-by: Andrzej Kacprowski <[email protected]>
Signed-off-by: Andrzej Kacprowski <[email protected]>
Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add GEM buffer object management

Adds four types of GEM-based BOs for the VPU:
- shmem
- internal
- prime

All types are implemented as struct ivpu_bo, based on
struct drm_gem_obje

accel/ivpu: Add GEM buffer object management

Adds four types of GEM-based BOs for the VPU:
- shmem
- internal
- prime

All types are implemented as struct ivpu_bo, based on
struct drm_gem_object. VPU address is allocated when buffer is created
except for imported prime buffers that allocate it in BO_INFO IOCTL due
to missing file_priv arg in gem_prime_import callback.
Internal buffers are pinned on creation, the rest of buffers types
can be pinned on demand (in SUBMIT IOCTL).
Buffer VPU address, allocated pages and mappings are released when the
buffer is destroyed.
Eviction mechanism is planned for future versions.

Add two new IOCTLs: BO_CREATE, BO_INFO

Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Add Intel VPU MMU support

VPU Memory Management Unit is based on ARM MMU-600.
It allows the creation of multiple virtual address spaces for
the device and map noncontinuous host memory (

accel/ivpu: Add Intel VPU MMU support

VPU Memory Management Unit is based on ARM MMU-600.
It allows the creation of multiple virtual address spaces for
the device and map noncontinuous host memory (there is no dedicated
memory on the VPU).

Address space is implemented as a struct ivpu_mmu_context, it has an ID,
drm_mm allocator for VPU addresses and struct ivpu_mmu_pgtable that
holds actual 3-level, 4KB page table.
Context with ID 0 (global context) is created upon driver initialization
and it's mainly used for mapping memory required to execute
the firmware.
Contexts with non-zero IDs are user contexts allocated each time
the devices is open()-ed and they map command buffers and other
workload-related memory.
Workloads executing in a given contexts have access only
to the memory mapped in this context.

This patch is has two main files:
- ivpu_mmu_context.c handles MMU page tables and memory mapping
- ivpu_mmu.c implements a driver that programs the MMU device

Co-developed-by: Karol Wachowski <[email protected]>
Signed-off-by: Karol Wachowski <[email protected]>
Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...

accel/ivpu: Introduce a new DRM driver for Intel VPU

VPU stands for Versatile Processing Unit and it's a CPU-integrated
inference accelerator for Computer Vision and Deep Learning
applications.

The

accel/ivpu: Introduce a new DRM driver for Intel VPU

VPU stands for Versatile Processing Unit and it's a CPU-integrated
inference accelerator for Computer Vision and Deep Learning
applications.

The VPU device consist of following components:
- Buttress - provides CPU to VPU integration, interrupt, frequency and
power management.
- Memory Management Unit (based on ARM MMU-600) - translates VPU to
host DMA addresses, isolates user workloads.
- RISC based microcontroller - executes firmware that provides job
execution API for the kernel-mode driver
- Neural Compute Subsystem (NCS) - does the actual work, provides
Compute and Copy engines.
- Network on Chip (NoC) - network fabric connecting all the components

This driver supports VPU IP v2.7 integrated into Intel Meteor Lake
client CPUs (14th generation).

Module sources are at drivers/accel/ivpu and module name is
"intel_vpu.ko".

This patch includes only very besic functionality:
- module, PCI device and IRQ initialization
- register definitions and low level register manipulation functions
- SET/GET_PARAM ioctls
- power up without firmware

Co-developed-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Krystian Pradzynski <[email protected]>
Signed-off-by: Jacek Lawrynowicz <[email protected]>
Reviewed-by: Oded Gabbay <[email protected]>
Reviewed-by: Jeffrey Hugo <[email protected]>
Signed-off-by: Daniel Vetter <[email protected]>
Link: https://patchwork.freedesktop.org/patch/msgid/[email protected]

show more ...