dma-coherent: Warn if OF reserved memory is beyond current coherent DMA mask

When a reserved memory region described in the device tree is attached
to a device, it is expected that the device's limi

dma-coherent: Warn if OF reserved memory is beyond current coherent DMA mask

When a reserved memory region described in the device tree is attached
to a device, it is expected that the device's limitations are correctly
included in that description.

However, if the device driver failed to implement DMA address masking
or addressing beyond the default 32 bits (on arm64), then bad things
could happen because the DMA address was truncated, such as playing
back audio with no actual audio coming out, or DMA overwriting random
blocks of kernel memory.

Check against the coherent DMA mask when the memory regions are attached
to the device. Give a warning when the memory region can not be covered
by the mask.

A warning instead of a hard error was chosen, because it is possible
that existing drivers could be working fine even if they forgot to
extend the coherent DMA mask.

Signed-off-by: Chen-Yu Tsai <[email protected]>
Signed-off-by: Marek Szyprowski <[email protected]>
Link: https://lore.kernel.org/r/[email protected]

show more ...

dma-mapping: save base/size instead of pointer to shared DMA pool

On RZ/Five, which is non-coherent, and uses CONFIG_DMA_GLOBAL_POOL=y:

Oops - store (or AMO) access fault [#1]
CPU: 0 UID: 0

dma-mapping: save base/size instead of pointer to shared DMA pool

On RZ/Five, which is non-coherent, and uses CONFIG_DMA_GLOBAL_POOL=y:

Oops - store (or AMO) access fault [#1]
CPU: 0 UID: 0 PID: 1 Comm: swapper Not tainted 6.12.0-rc1-00015-g8a6e02d0c00e #201
Hardware name: Renesas SMARC EVK based on r9a07g043f01 (DT)
epc : __memset+0x60/0x100
ra : __dma_alloc_from_coherent+0x150/0x17a
epc : ffffffff8062d2bc ra : ffffffff80053a94 sp : ffffffc60000ba20
gp : ffffffff812e9938 tp : ffffffd601920000 t0 : ffffffc6000d0000
t1 : 0000000000000000 t2 : ffffffffe9600000 s0 : ffffffc60000baa0
s1 : ffffffc6000d0000 a0 : ffffffc6000d0000 a1 : 0000000000000000
a2 : 0000000000001000 a3 : ffffffc6000d1000 a4 : 0000000000000000
a5 : 0000000000000000 a6 : ffffffd601adacc0 a7 : ffffffd601a841a8
s2 : ffffffd6018573c0 s3 : 0000000000001000 s4 : ffffffd6019541e0
s5 : 0000000200000022 s6 : ffffffd6018f8410 s7 : ffffffd6018573e8
s8 : 0000000000000001 s9 : 0000000000000001 s10: 0000000000000010
s11: 0000000000000000 t3 : 0000000000000000 t4 : ffffffffdefe62d1
t5 : 000000001cd6a3a9 t6 : ffffffd601b2aad6
status: 0000000200000120 badaddr: ffffffc6000d0000 cause: 0000000000000007
[<ffffffff8062d2bc>] __memset+0x60/0x100
[<ffffffff80053e1a>] dma_alloc_from_global_coherent+0x1c/0x28
[<ffffffff80053056>] dma_direct_alloc+0x98/0x112
[<ffffffff8005238c>] dma_alloc_attrs+0x78/0x86
[<ffffffff8035fdb4>] rz_dmac_probe+0x3f6/0x50a
[<ffffffff803a0694>] platform_probe+0x4c/0x8a

If CONFIG_DMA_GLOBAL_POOL=y, the reserved_mem structure passed to
rmem_dma_setup() is saved for later use, by saving the passed pointer.
However, when dma_init_reserved_memory() is called later, the pointer
has become stale, causing a crash.

E.g. in the RZ/Five case, the referenced memory now contains the
reserved_mem structure for the "mmode_resv0@30000" node (with base
0x30000 and size 0x10000), instead of the correct "pma_resv0@58000000"
node (with base 0x58000000 and size 0x8000000).

Fix this by saving the needed reserved_mem structure's contents instead.

Fixes: 8a6e02d0c00e7b62 ("of: reserved_mem: Restructure how the reserved memory regions are processed")
Signed-off-by: Geert Uytterhoeven <[email protected]>
Reviewed-by: Oreoluwa Babatunde <[email protected]>
Tested-by: Lad Prabhakar <[email protected]>
Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: clear dev->dma_mem to NULL after freeing it

Reproduced with below sequence:
dma_declare_coherent_memory()->dma_release_coherent_memory()
->dma_declare_coherent_memory()->"return -EBUSY"

dma-mapping: clear dev->dma_mem to NULL after freeing it

Reproduced with below sequence:
dma_declare_coherent_memory()->dma_release_coherent_memory()
->dma_declare_coherent_memory()->"return -EBUSY" error

It will return -EBUSY from the dma_assign_coherent_memory()
in dma_declare_coherent_memory(), the reason is that dev->dma_mem
pointer has not been set to NULL after it's freed.

Fixes: cf65a0f6f6ff ("dma-mapping: move all DMA mapping code to kernel/dma")
Signed-off-by: Joakim Zhang <[email protected]>
Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: Add dma_release_coherent_memory to DMA API

Add dma_release_coherent_memory to DMA API to allow dma
user call it to release dev->dma_mem when the device is
removed.

Signed-off-by: Mark-

dma-mapping: Add dma_release_coherent_memory to DMA API

Add dma_release_coherent_memory to DMA API to allow dma
user call it to release dev->dma_mem when the device is
removed.

Signed-off-by: Mark-PK Tsai <[email protected]>
Acked-by: Christoph Hellwig <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Mathieu Poirier <[email protected]>

show more ...

dma-mapping: use 'bitmap_zalloc()' when applicable

'dma_mem->bitmap' is a bitmap. So use 'bitmap_zalloc()' to simplify code,
improve the semantic and avoid some open-coded arithmetic in allocator
ar

dma-mapping: use 'bitmap_zalloc()' when applicable

'dma_mem->bitmap' is a bitmap. So use 'bitmap_zalloc()' to simplify code,
improve the semantic and avoid some open-coded arithmetic in allocator
arguments.

Also change the corresponding 'kfree()' into 'bitmap_free()' to keep
consistency.

Signed-off-by: Christophe JAILLET <[email protected]>
Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: make the global coherent pool conditional

Only build the code to support the global coherent pool if support for
it is enabled.

Signed-off-by: Christoph Hellwig <[email protected]>
Tested-by:

dma-mapping: make the global coherent pool conditional

Only build the code to support the global coherent pool if support for
it is enabled.

Signed-off-by: Christoph Hellwig <[email protected]>
Tested-by: Dillon Min <[email protected]>

show more ...

dma-mapping: add a dma_init_global_coherent helper

Add a new helper to initialize the global coherent pool. This both
cleans up the existing initialization which indirects through the
reserved_mem_

dma-mapping: add a dma_init_global_coherent helper

Add a new helper to initialize the global coherent pool. This both
cleans up the existing initialization which indirects through the
reserved_mem_ops that are normally only used for struct device, and
also allows using the global pool for non-devicetree architectures.

Signed-off-by: Christoph Hellwig <[email protected]>
Tested-by: Dillon Min <[email protected]>

show more ...

dma-mapping: simplify dma_init_coherent_memory

Return the allocated dma_coherent_mem structure, set the
use_dma_pfn_offset and print the failure warning inside of
dma_init_coherent_memory instead of

dma-mapping: simplify dma_init_coherent_memory

Return the allocated dma_coherent_mem structure, set the
use_dma_pfn_offset and print the failure warning inside of
dma_init_coherent_memory instead of leaving that to the callers.

Signed-off-by: Christoph Hellwig <[email protected]>
Tested-by: Dillon Min <[email protected]>

show more ...

dma-mapping: allow using the global coherent pool for !ARM

Switch an ifdef so that the global coherent pool is initialized for
any architecture that selects the DMA_GLOBAL_POOL symbol insted of
hard

dma-mapping: allow using the global coherent pool for !ARM

Switch an ifdef so that the global coherent pool is initialized for
any architecture that selects the DMA_GLOBAL_POOL symbol insted of
hardcoding ARM.

Signed-off-by: Christoph Hellwig <[email protected]>
Tested-by: Dillon Min <[email protected]>

show more ...

dma-mapping: remove a trailing space

Signed-off-by: Zhen Lei <[email protected]>
Signed-off-by: Christoph Hellwig <[email protected]>

dma-mapping: split <linux/dma-mapping.h>

Split out all the bits that are purely for dma_map_ops implementations
and related code into a new <linux/dma-map-ops.h> header so that they
don't get pulled

dma-mapping: split <linux/dma-mapping.h>

Split out all the bits that are purely for dma_map_ops implementations
and related code into a new <linux/dma-map-ops.h> header so that they
don't get pulled into all the drivers. That also means the architecture
specific <asm/dma-mapping.h> is not pulled in by <linux/dma-mapping.h>
any more, which leads to a missing includes that were pulled in by the
x86 or arm versions in a few not overly portable drivers.

Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: introduce DMA range map, supplanting dma_pfn_offset

The new field 'dma_range_map' in struct device is used to facilitate the
use of single or multiple offsets between mapping regions of

dma-mapping: introduce DMA range map, supplanting dma_pfn_offset

The new field 'dma_range_map' in struct device is used to facilitate the
use of single or multiple offsets between mapping regions of cpu addrs and
dma addrs. It subsumes the role of "dev->dma_pfn_offset" which was only
capable of holding a single uniform offset and had no region bounds
checking.

The function of_dma_get_range() has been modified so that it takes a single
argument -- the device node -- and returns a map, NULL, or an error code.
The map is an array that holds the information regarding the DMA regions.
Each range entry contains the address offset, the cpu_start address, the
dma_start address, and the size of the region.

of_dma_configure() is the typical manner to set range offsets but there are
a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
driver code. These cases now invoke the function
dma_direct_set_offset(dev, cpu_addr, dma_addr, size).

Signed-off-by: Jim Quinlan <[email protected]>
[hch: various interface cleanups]
Signed-off-by: Christoph Hellwig <[email protected]>
Reviewed-by: Mathieu Poirier <[email protected]>
Tested-by: Mathieu Poirier <[email protected]>
Tested-by: Nathan Chancellor <[email protected]>

show more ...

dma-mapping: move the dma_declare_coherent_memory documentation

dma_declare_coherent_memory should not be in a DMA API guide aimed
at driver writers (that is consumers of the API). Move it to a com

dma-mapping: move the dma_declare_coherent_memory documentation

dma_declare_coherent_memory should not be in a DMA API guide aimed
at driver writers (that is consumers of the API). Move it to a comment
near the function instead.

Signed-off-by: Christoph Hellwig <[email protected]>
Reviewed-by: Robin Murphy <[email protected]>

show more ...

dma-coherent: fix integer overflow in the reserved-memory dma allocation

pageno is an int and the PAGE_SHIFT shift is done on an int,
overflowing if the memory is bigger than 2G

This can be reprodu

dma-coherent: fix integer overflow in the reserved-memory dma allocation

pageno is an int and the PAGE_SHIFT shift is done on an int,
overflowing if the memory is bigger than 2G

This can be reproduced using for example a reserved-memory of 4G

reserved-memory {
#address-cells = <2>;
#size-cells = <2>;
ranges;

reserved_dma: buffer@0 {
compatible = "shared-dma-pool";
no-map;
reg = <0x5 0x00000000 0x1 0x0>;
};
};

Signed-off-by: Kevin Grandemange <[email protected]>
Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: fix handling of dma-ranges for reserved memory (again)

Daniele reported that issue previously fixed in c41f9ea998f3
("drivers: dma-coherent: Account dma_pfn_offset when used with device

dma-mapping: fix handling of dma-ranges for reserved memory (again)

Daniele reported that issue previously fixed in c41f9ea998f3
("drivers: dma-coherent: Account dma_pfn_offset when used with device
tree") reappear shortly after 43fc509c3efb ("dma-coherent: introduce
interface for default DMA pool") where fix was accidentally dropped.

Lets put fix back in place and respect dma-ranges for reserved memory.

Fixes: 43fc509c3efb ("dma-coherent: introduce interface for default DMA pool")

Reported-by: Daniele Alessandrelli <[email protected]>
Tested-by: Daniele Alessandrelli <[email protected]>
Tested-by: Alexandre Torgue <[email protected]>
Signed-off-by: Vladimir Murzin <[email protected]>
Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: remove the dma_declare_coherent_memory export

dma_declare_coherent_memory is something that the platform setup code
(which pretty much means the device tree these days) need to do so th

dma-mapping: remove the dma_declare_coherent_memory export

dma_declare_coherent_memory is something that the platform setup code
(which pretty much means the device tree these days) need to do so that
drivers can use the memory as declared by the platform. Drivers
themselves have no business calling this function.

Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: remove the dma_mmap_from_dev_coherent export

dma_mmap_from_dev_coherent is only used by dma_map_ops instances,
none of which is modular.

Signed-off-by: Christoph Hellwig <[email protected]>

dma-mapping: remove dma_release_declared_memory

This function is entirely unused given that declared memory is
generally provided by platform setup code.

Signed-off-by: Christoph Hellwig <[email protected]

dma-mapping: remove dma_release_declared_memory

This function is entirely unused given that declared memory is
generally provided by platform setup code.

Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: remove the DMA_MEMORY_EXCLUSIVE flag

All users of dma_declare_coherent want their allocations to be
exclusive, so default to exclusive allocations.

Signed-off-by: Christoph Hellwig <hc

dma-mapping: remove the DMA_MEMORY_EXCLUSIVE flag

All users of dma_declare_coherent want their allocations to be
exclusive, so default to exclusive allocations.

Signed-off-by: Christoph Hellwig <[email protected]>
Reviewed-by: Greg Kroah-Hartman <[email protected]>

show more ...

dma-mapping: remove dma_mark_declared_memory_occupied

This API is not used anywhere, so remove it.

Signed-off-by: Christoph Hellwig <[email protected]>

dma-mapping: remove an incorrect __iommem annotation

memmap return a regular void pointer, not and __iomem one.

Signed-off-by: Christoph Hellwig <[email protected]>

dma-mapping: remove a few unused exports

Now that the slow path DMA API calls are implemented out of line a few
helpers only used by them don't need to be exported anymore.

Signed-off-by: Christoph

dma-mapping: remove a few unused exports

Now that the slow path DMA API calls are implemented out of line a few
helpers only used by them don't need to be exported anymore.

Signed-off-by: Christoph Hellwig <[email protected]>

show more ...

dma-mapping: move all DMA mapping code to kernel/dma

Currently the code is split over various files with dma- prefixes in the
lib/ and drives/base directories, and the number of files keeps growing.

dma-mapping: move all DMA mapping code to kernel/dma

Currently the code is split over various files with dma- prefixes in the
lib/ and drives/base directories, and the number of files keeps growing.
Move them into a single directory to keep the code together and remove
the file name prefixes. To match the irq infrastructure this directory
is placed under the kernel/ directory.

Signed-off-by: Christoph Hellwig <[email protected]>

show more ...