async_tx: fix kernel-doc notation warnings

Fix kernel-doc warnings by adding "struct" keyword or "enum" keyword.
Also fix 2 function parameter descriptions.
Change some functions and structs from ke

async_tx: fix kernel-doc notation warnings

Fix kernel-doc warnings by adding "struct" keyword or "enum" keyword.
Also fix 2 function parameter descriptions.
Change some functions and structs from kernel-doc /** notation
to regular /* comment notation.

async_pq.c:18: warning: cannot understand function prototype: 'struct page *pq_scribble_page; '
async_pq.c:18: error: Cannot parse struct or union!
async_pq.c:40: warning: No description found for return value of 'do_async_gen_syndrome'
async_pq.c:109: warning: Function parameter or member 'blocks' not described in 'do_sync_gen_syndrome'
async_pq.c:109: warning: Function parameter or member 'offsets' not described in 'do_sync_gen_syndrome'
async_pq.c:109: warning: Function parameter or member 'disks' not described in 'do_sync_gen_syndrome'
async_pq.c:109: warning: Function parameter or member 'len' not described in 'do_sync_gen_syndrome'
async_pq.c:109: warning: Function parameter or member 'submit' not described in 'do_sync_gen_syndrome'

async_tx.c:136: warning: cannot understand function prototype: 'enum submit_disposition '
async_tx.c:264: warning: Function parameter or member 'tx' not described in 'async_tx_quiesce'

Signed-off-by: Randy Dunlap <[email protected]>
Cc: Dan Williams <[email protected]>
Cc: Herbert Xu <[email protected]>
Cc: "David S. Miller" <[email protected]>
Cc: [email protected]
Reviewed-by: Dan Williams <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>

show more ...

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 335

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms and c

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 335

Based on 1 normalized pattern(s):

this program is free software you can redistribute it and or modify
it under the terms and conditions of the gnu general public license
version 2 as published by the free software foundation this program
is distributed in the hope it will be useful but without any
warranty without even the implied warranty of merchantability or
fitness for a particular purpose see the gnu general public license
for more details you should have received a copy of the gnu general
public license along with this program if not write to the free
software foundation inc 51 franklin st fifth floor boston ma 02110
1301 usa

extracted by the scancode license scanner the SPDX license identifier

GPL-2.0-only

has been chosen to replace the boilerplate/reference in 111 file(s).

Signed-off-by: Thomas Gleixner <[email protected]>
Reviewed-by: Alexios Zavras <[email protected]>
Reviewed-by: Allison Randal <[email protected]>
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Greg Kroah-Hartman <[email protected]>

show more ...

async_tx: use DMA_COMPLETE for dma completion status

Acked-by: Dan Williams <[email protected]>
Acked-by: Linus Walleij <[email protected]>
Signed-off-by: Vinod Koul <vinod.koul@intel.

async_tx: use DMA_COMPLETE for dma completion status

Acked-by: Dan Williams <[email protected]>
Acked-by: Linus Walleij <[email protected]>
Signed-off-by: Vinod Koul <[email protected]>

show more ...

async_tx: fix checking of dma_wait_for_async_tx() return value

dma_wait_for_async_tx() can also return DMA_PAUSED (which
should be considered as error).

Cc: Vinod Koul <[email protected]>
Cc: Da

async_tx: fix checking of dma_wait_for_async_tx() return value

dma_wait_for_async_tx() can also return DMA_PAUSED (which
should be considered as error).

Cc: Vinod Koul <[email protected]>
Cc: Dan Williams <[email protected]>
Cc: Tomasz Figa <[email protected]>
Signed-off-by: Bartlomiej Zolnierkiewicz <[email protected]>
Signed-off-by: Kyungmin Park <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

crypto: add module.h to those files that are explicitly using it

Part of the include cleanups means that the implicit
inclusion of module.h via device.h is going away. So
fix things up in advance.

crypto: add module.h to those files that are explicitly using it

Part of the include cleanups means that the implicit
inclusion of module.h via device.h is going away. So
fix things up in advance.

Signed-off-by: Paul Gortmaker <[email protected]>

show more ...

async_tx: trim dma_async_tx_descriptor in 'no channel switch' case

Saves 24 bytes per descriptor (64-bit) when the channel-switching
capabilities of async_tx are not required.

Signed-off-by: Dan Wi

async_tx: trim dma_async_tx_descriptor in 'no channel switch' case

Saves 24 bytes per descriptor (64-bit) when the channel-switching
capabilities of async_tx are not required.

Signed-off-by: Dan Williams <[email protected]>

show more ...

dmaengine, async_tx: add a "no channel switch" allocator

Channel switching is problematic for some dmaengine drivers as the
architecture precludes separating the ->prep from ->submit. In these
case

dmaengine, async_tx: add a "no channel switch" allocator

Channel switching is problematic for some dmaengine drivers as the
architecture precludes separating the ->prep from ->submit. In these
cases the driver can select ASYNC_TX_DISABLE_CHANNEL_SWITCH to modify
the async_tx allocator to only return channels that support all of the
required asynchronous operations.

For example MD_RAID456=y selects support for asynchronous xor, xor
validate, pq, pq validate, and memcpy. When
ASYNC_TX_DISABLE_CHANNEL_SWITCH=y any channel with all these
capabilities is marked DMA_ASYNC_TX allowing async_tx_find_channel() to
quickly locate compatible channels with the guarantee that dependency
chains will remain on one channel. When
ASYNC_TX_DISABLE_CHANNEL_SWITCH=n async_tx_find_channel() may select
channels that lead to operation chains that need to cross channel
boundaries using the async_tx channel switch capability.

Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: remove walk of tx->parent chain in dma_wait_for_async_tx

We currently walk the parent chain when waiting for a given tx to
complete however this walk may race with the driver cleanup routi

async_tx: remove walk of tx->parent chain in dma_wait_for_async_tx

We currently walk the parent chain when waiting for a given tx to
complete however this walk may race with the driver cleanup routine.
The routines in async_raid6_recov.c may fall back to the synchronous
path at any point so we need to be prepared to call async_tx_quiesce()
(which calls dma_wait_for_async_tx). To remove the ->parent walk we
guarantee that every time a dependency is attached ->issue_pending() is
invoked, then we can simply poll the initial descriptor until
completion.

This also allows for a lighter weight 'issue pending' implementation as
there is no longer a requirement to iterate through all the channels'
->issue_pending() routines as long as operations have been submitted in
an ordered chain. async_tx_issue_pending() is added for this case.

Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: kill needless module_{init|exit}

If module_init and module_exit are nops then neither need to be defined.

[ Impact: pure cleanup ]

Reviewed-by: Andre Noll <[email protected]>
Acked-by

async_tx: kill needless module_{init|exit}

If module_init and module_exit are nops then neither need to be defined.

[ Impact: pure cleanup ]

Reviewed-by: Andre Noll <[email protected]>
Acked-by: Maciej Sosnowski <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: structify submission arguments, add scribble

Prepare the api for the arrival of a new parameter, 'scribble'. This
will allow callers to identify scratchpad memory for dma address or page

async_tx: structify submission arguments, add scribble

Prepare the api for the arrival of a new parameter, 'scribble'. This
will allow callers to identify scratchpad memory for dma address or page
address conversions. As this adds yet another parameter, take this
opportunity to convert the common submission parameters (flags,
dependency, callback, and callback argument) into an object that is
passed by reference.

Also, take this opportunity to fix up the kerneldoc and add notes about
the relevant ASYNC_TX_* flags for each routine.

[ Impact: moves api pass-by-value parameters to a pass-by-reference struct ]

Signed-off-by: Andre Noll <[email protected]>
Acked-by: Maciej Sosnowski <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: kill ASYNC_TX_DEP_ACK flag

In support of inter-channel chaining async_tx utilizes an ack flag to
gate whether a dependent operation can be chained to another. While the
flag is not set th

async_tx: kill ASYNC_TX_DEP_ACK flag

In support of inter-channel chaining async_tx utilizes an ack flag to
gate whether a dependent operation can be chained to another. While the
flag is not set the chain can be considered open for appending. Setting
the ack flag closes the chain and flags the descriptor for garbage
collection. The ASYNC_TX_DEP_ACK flag essentially means "close the
chain after adding this dependency". Since each operation can only have
one child the api now implicitly sets the ack flag at dependency
submission time. This removes an unnecessary management burden from
clients of the api.

[ Impact: clean up and enforce one dependency per operation ]

Reviewed-by: Andre Noll <[email protected]>
Acked-by: Maciej Sosnowski <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

dmaengine: allow dma support for async_tx to be toggled

Provide a config option for blocking the allocation of dma channels to
the async_tx api.

Signed-off-by: Dan Williams <[email protected]

dmaengine: allow dma support for async_tx to be toggled

Provide a config option for blocking the allocation of dma channels to
the async_tx api.

Signed-off-by: Dan Williams <[email protected]>

show more ...

dmaengine: replace dma_async_client_register with dmaengine_get

Now that clients no longer need to be notified of channel arrival
dma_async_client_register can simply increment the dmaengine_ref_cou

dmaengine: replace dma_async_client_register with dmaengine_get

Now that clients no longer need to be notified of channel arrival
dma_async_client_register can simply increment the dmaengine_ref_count.

Reviewed-by: Andrew Morton <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

dmaengine: provide a common 'issue_pending_all' implementation

async_tx and net_dma each have open-coded versions of issue_pending_all,
so provide a common routine in dmaengine.

The implementation

dmaengine: provide a common 'issue_pending_all' implementation

async_tx and net_dma each have open-coded versions of issue_pending_all,
so provide a common routine in dmaengine.

The implementation needs to walk the global device list, so implement
rcu to allow dma_issue_pending_all to run lockless. Clients protect
themselves from channel removal events by holding a dmaengine reference.

Reviewed-by: Andrew Morton <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

dmaengine: centralize channel allocation, introduce dma_find_channel

Allowing multiple clients to each define their own channel allocation
scheme quickly leads to a pathological situation. For memo

dmaengine: centralize channel allocation, introduce dma_find_channel

Allowing multiple clients to each define their own channel allocation
scheme quickly leads to a pathological situation. For memory-to-memory
offload all clients can share a central allocator.

This simply moves the existing async_tx allocator to dmaengine with
minimal fixups:
* async_tx.c:get_chan_ref_by_cap --> dmaengine.c:nth_chan
* async_tx.c:async_tx_rebalance --> dmaengine.c:dma_channel_rebalance
* split out common code from async_tx.c:__async_tx_find_channel -->
dma_find_channel

Reviewed-by: Andrew Morton <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

dmaengine: up-level reference counting to the module level

Simply, if a client wants any dmaengine channel then prevent all dmaengine
modules from being removed. Once the clients are done re-enable

dmaengine: up-level reference counting to the module level

Simply, if a client wants any dmaengine channel then prevent all dmaengine
modules from being removed. Once the clients are done re-enable module
removal.

Why?, beyond reducing complication:
1/ Tracking reference counts per-transaction in an efficient manner, as
is currently done, requires a complicated scheme to avoid cache-line
bouncing effects.
2/ Per-transaction ref-counting gives the false impression that a
dma-driver can be gracefully removed ahead of its user (net, md, or
dma-slave)
3/ None of the in-tree dma-drivers talk to hot pluggable hardware, but
if such an engine were built one day we still would not need to notify
clients of remove events. The driver can simply return NULL to a
->prep() request, something that is much easier for a client to handle.

Reviewed-by: Andrew Morton <[email protected]>
Acked-by: Maciej Sosnowski <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

dmaengine: remove dependency on async_tx

async_tx.ko is a consumer of dma channels. A circular dependency arises
if modules in drivers/dma rely on common code in async_tx.ko. It
prevents either mo

dmaengine: remove dependency on async_tx

async_tx.ko is a consumer of dma channels. A circular dependency arises
if modules in drivers/dma rely on common code in async_tx.ko. It
prevents either module from being unloaded.

Move dma_wait_for_async_tx and async_tx_run_dependencies to dmaeninge.o
where they should have been from the beginning.

Reviewed-by: Andrew Morton <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: make async_tx_run_dependencies() easier to read

* Rename 'next' to 'dep'
* Move the channel switch check inside the loop to simplify
termination

Acked-by: Ilya Yanok <[email protected]>

async_tx: make async_tx_run_dependencies() easier to read

* Rename 'next' to 'dep'
* Move the channel switch check inside the loop to simplify
termination

Acked-by: Ilya Yanok <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: fix the bug in async_tx_run_dependencies

Should clear the next pointer of the TX if we are sure that the
next TX (say NXT) will be submitted to the channel too. Overwise,
we break the chai

async_tx: fix the bug in async_tx_run_dependencies

Should clear the next pointer of the TX if we are sure that the
next TX (say NXT) will be submitted to the channel too. Overwise,
we break the chain of descriptors, because we lose the information
about the next descriptor to run. So next time, when invoke
async_tx_run_dependencies() with TX, it's TX->next will be NULL, and
NXT will be never submitted.

Cc: <[email protected]> [2.6.26]
Signed-off-by: Yuri Tikhonov <[email protected]>
Signed-off-by: Ilya Yanok <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: remove depend_tx from async_tx_sync_epilog

All callers of async_tx_sync_epilog have called async_tx_quiesce on the
depend_tx, so async_tx_sync_epilog need only call the callback to
complet

async_tx: remove depend_tx from async_tx_sync_epilog

All callers of async_tx_sync_epilog have called async_tx_quiesce on the
depend_tx, so async_tx_sync_epilog need only call the callback to
complete the operation.

Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: export async_tx_quiesce

Replace open coded "wait and acknowledge" instances with async_tx_quiesce.

Signed-off-by: Dan Williams <[email protected]>

async_tx: list_for_each_entry_rcu() cleanup

In the rcu update side, don't use list_for_each_entry_rcu().

Signed-off-by: Li Zefan <[email protected]>
Signed-off-by: Dan Williams <dan.j.williams@in

async_tx: list_for_each_entry_rcu() cleanup

In the rcu update side, don't use list_for_each_entry_rcu().

Signed-off-by: Li Zefan <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

rcu: split list.h and move rcu-protected lists into rculist.h

Move rcu-protected lists from list.h into a new header file rculist.h.

This is done because list are a very used primitive structure al

rcu: split list.h and move rcu-protected lists into rculist.h

Move rcu-protected lists from list.h into a new header file rculist.h.

This is done because list are a very used primitive structure all over the
kernel and it's currently impossible to include other header files in this
list.h without creating some circular dependencies.

For example, list.h implements rcu-protected list and uses rcu_dereference()
without including rcupdate.h. It actually compiles because users of
rcu_dereference() are macros. Others RCU functions could be used too but
aren't probably because of this.

Therefore this patch creates rculist.h which includes rcupdates without to
many changes/troubles.

Signed-off-by: Franck Bui-Huu <[email protected]>
Acked-by: Paul E. McKenney <[email protected]>
Acked-by: Josh Triplett <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

show more ...

dmaengine: ack to flags: make use of the unused bits in the 'ack' field

'ack' is currently a simple integer that flags whether or not a client is done
touching fields in the given descriptor. It is

dmaengine: ack to flags: make use of the unused bits in the 'ack' field

'ack' is currently a simple integer that flags whether or not a client is done
touching fields in the given descriptor. It is effectively just a single bit
of information. Converting this to a flags parameter allows the other bits to
be put to use to control completion actions, like dma-unmap, and capture
results, like xor-zero-sum == 0.

Changes are one of:
1/ convert all open-coded ->ack manipulations to use async_tx_ack
and async_tx_test_ack.
2/ set the ack bit at prep time where possible
3/ make drivers store the flags at prep time
4/ add flags to the device_prep_dma_interrupt prototype

Acked-by: Maciej Sosnowski <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...

async_tx: fix multiple dependency submission

Shrink struct dma_async_tx_descriptor and introduce
async_tx_channel_switch to properly inject a channel switch interrupt in
the descriptor stream. This

async_tx: fix multiple dependency submission

Shrink struct dma_async_tx_descriptor and introduce
async_tx_channel_switch to properly inject a channel switch interrupt in
the descriptor stream. This simplifies the locking model as drivers no
longer need to handle dma_async_tx_descriptor.lock.

Acked-by: Shannon Nelson <[email protected]>
Signed-off-by: Dan Williams <[email protected]>

show more ...