vhost: fix field naming in guest page struct

This patch renames the host_phys_addr to host_iova in guest_page
struct. The host_phys_addr is iova, it depends on the DPDK
IOVA mode.

Fixes: e246896178

vhost: fix field naming in guest page struct

This patch renames the host_phys_addr to host_iova in guest_page
struct. The host_phys_addr is iova, it depends on the DPDK
IOVA mode.

Fixes: e246896178e6 ("vhost: get guest/host physical address mappings")
Cc: [email protected]

Signed-off-by: Xuan Ding <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>

show more ...

vhost: integrate dmadev in asynchronous data-path

Since dmadev is introduced in 21.11, to avoid the overhead of vhost DMA
abstraction layer and simplify application logics, this patch integrates
dma

vhost: integrate dmadev in asynchronous data-path

Since dmadev is introduced in 21.11, to avoid the overhead of vhost DMA
abstraction layer and simplify application logics, this patch integrates
dmadev in asynchronous data path.

Signed-off-by: Jiayu Hu <[email protected]>
Signed-off-by: Sunil Pai G <[email protected]>
Tested-by: Yvonne Yang <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>

show more ...

vhost: improve virtio-net layer logs

This patch standardizes logging done in Virtio-net, so that
the Vhost-user socket path is always prepended to the logs.
It will ease log analysis when multiple V

vhost: improve virtio-net layer logs

This patch standardizes logging done in Virtio-net, so that
the Vhost-user socket path is always prepended to the logs.
It will ease log analysis when multiple Vhost-user ports
are in use.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>
Reviewed-by: David Marchand <[email protected]>

show more ...

vhost: fix packed ring descriptor update in async enqueue

If the packet uses multiple descriptors and its descriptor indices are
wrapped, the first descriptor flag is not updated last, which may cau

vhost: fix packed ring descriptor update in async enqueue

If the packet uses multiple descriptors and its descriptor indices are
wrapped, the first descriptor flag is not updated last, which may cause
virtio read the incomplete packet. For example, given a packet uses 64
descriptors, and virtio ring size is 256, and its descriptor indices are
224~255 and 0~31, current implementation will update 224~255 descriptor
flags earlier than 0~31 descriptor flags.

This patch fixes this issue by updating descriptor flags in one loop,
so that the first descriptor flag is always updated last.

Fixes: 873e8dad6f49 ("vhost: support packed ring in async datapath")

Signed-off-by: Jiayu Hu <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>

show more ...

vhost: merge sync and async mbuf to descriptor filling

This patches merges copy_mbuf_to_desc() used by the sync
path with async_mbuf_to_desc() used by the async path.

Most of these complex function

vhost: merge sync and async mbuf to descriptor filling

This patches merges copy_mbuf_to_desc() used by the sync
path with async_mbuf_to_desc() used by the async path.

Most of these complex functions are identical, so merging
them will make the maintenance easier.

In order not to degrade performance, the patch introduces
a boolean function parameter to specify whether it is called
in async context. This boolean is statically passed to this
always-inlined function, so the compiler will optimize this
out.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: prepare sync for mbuf to descriptor refactoring

This patch extracts the descriptors buffers filling
from copy_mbuf_to_desc() into a dedicated function as a
preliminary step of merging copy_mu

vhost: prepare sync for mbuf to descriptor refactoring

This patch extracts the descriptors buffers filling
from copy_mbuf_to_desc() into a dedicated function as a
preliminary step of merging copy_mubf_to_desc() and
async_mbuf_to_desc().

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: prepare async for mbuf to descriptor refactoring

This patch extracts the IO vectors filling from
async_mbuf_to_desc() into a dedicated function as a
preliminary step of merging copy_mubf_to_d

vhost: prepare async for mbuf to descriptor refactoring

This patch extracts the IO vectors filling from
async_mbuf_to_desc() into a dedicated function as a
preliminary step of merging copy_mubf_to_desc() and
async_mbuf_to_desc().

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: simplify getting first in-flight index

This patch reworks the function getting the index
for the first packet in-flight.

When this index turns out to be zero, let's use the simple
path. Doin

vhost: simplify getting first in-flight index

This patch reworks the function getting the index
for the first packet in-flight.

When this index turns out to be zero, let's use the simple
path. Doing that avoid having to do a modulo with the
virtqueue size.

The patch also rename the function for better clarification,
and only pass the virtqueue metadata pointer, as all the
needed information are stored there.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: simplify async enqueue completion

vhost_poll_enqueue_completed() assumes some inflight
packets could have been completed in a previous call but
not returned to the application. But this is no

vhost: simplify async enqueue completion

vhost_poll_enqueue_completed() assumes some inflight
packets could have been completed in a previous call but
not returned to the application. But this is not the case,
since check_completed_copies callback is never called with
more than the current count as argument.

In other words, async->last_pkts_n is always 0. Removing it
greatly simplifies the function.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: remove notion of async descriptor

Now that IO vectors iterator have been simplified, the
rte_vhost_async_desc struct only contains a pointer on
the iterator array stored in the async metadata

vhost: remove notion of async descriptor

Now that IO vectors iterator have been simplified, the
rte_vhost_async_desc struct only contains a pointer on
the iterator array stored in the async metadata.

This patch removes it, and pass directly the iterators
array pointer to the transfer_data callback. Doing that,
we avoid declaring the descriptor array in the stack, and
also avoid the cost of filling it.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: improve IO vector logic

IO vectors and their iterators arrays were part of the
async metadata but not their indexes.

In order to makes this more consistent, the patch adds the
indexes to the

vhost: improve IO vector logic

IO vectors and their iterators arrays were part of the
async metadata but not their indexes.

In order to makes this more consistent, the patch adds the
indexes to the async metadata. Doing that, we can avoid
triggering DMA transfer within the loop as it IO vector
index overflow is now prevented in the async_mbuf_to_desc()
function.

Note that previous detection mechanism was broken
since the overflow already happened when detected, so OOB
memory access would already have happened.

With this changes done, virtio_dev_rx_async_submit_split()
and virtio_dev_rx_async_submit_packed() can be further
simplified.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: remove useless fields in async iterator struct

Offset and count fields are unused and so can be removed.
The offset field was actually in the Vhost example, but
in a way that does not make se

vhost: remove useless fields in async iterator struct

Offset and count fields are unused and so can be removed.
The offset field was actually in the Vhost example, but
in a way that does not make sense.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: introduce specific iovec structure

This patch introduces rte_vhost_iovec struct that contains
both source and destination addresses since we always have
a 1:1 mapping between source and desti

vhost: introduce specific iovec structure

This patch introduces rte_vhost_iovec struct that contains
both source and destination addresses since we always have
a 1:1 mapping between source and destination. While using
the standard iovec struct might have seemed better, having
to duplicate IO vectors and its iterators is memory
inefficient and make the implementation more complex.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: remove async batch threshold

Reaching the async batch threshold was one of the condition
to trigger the DMA transfer. However, this condition was
never met since the threshold value is 32, sa

vhost: remove async batch threshold

Reaching the async batch threshold was one of the condition
to trigger the DMA transfer. However, this condition was
never met since the threshold value is 32, same as the
MAX_PKT_BURST value.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: simplify async IO vectors iterators

This patch splits the iterator arrays in two, one for
source and one for destination. The goal is make the code
easier to understand.

Signed-off-by: Maxim

vhost: simplify async IO vectors iterators

This patch splits the iterator arrays in two, one for
source and one for destination. The goal is make the code
easier to understand.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: simplify async IO vectors

IO vectors implementation is unnecessarily complex, mixing
source and destinations vectors in the same array.

This patch declares two arrays, one for the source and

vhost: simplify async IO vectors

IO vectors implementation is unnecessarily complex, mixing
source and destinations vectors in the same array.

This patch declares two arrays, one for the source and one
for the destination. It also gets rid of seg_awaits variable
in both packed and split implementation, which is the same
as iovec_idx.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

vhost: move async data in dedicated structure

This patch moves async-related metadata from vhost_virtqueue
to a dedicated struct. It makes it clear which fields are
async related, and also saves som

vhost: move async data in dedicated structure

This patch moves async-related metadata from vhost_virtqueue
to a dedicated struct. It makes it clear which fields are
async related, and also saves some memory when async feature
is not in use.

Signed-off-by: Maxime Coquelin <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>
Reviewed-by: Jiayu Hu <[email protected]>

show more ...

mbuf: add namespace to offload flags

Fix the mbuf offload flags namespace by adding an RTE_ prefix to the
name. The old flags remain usable, but a deprecation warning is issued
at compilation.

Sign

mbuf: add namespace to offload flags

Fix the mbuf offload flags namespace by adding an RTE_ prefix to the
name. The old flags remain usable, but a deprecation warning is issued
at compilation.

Signed-off-by: Olivier Matz <[email protected]>
Acked-by: Andrew Rybchenko <[email protected]>
Acked-by: Ajit Khaparde <[email protected]>
Acked-by: Somnath Kotur <[email protected]>

show more ...

vhost: rework RARP packet injection

Caught by code review, this copy is unnecessary.

Signed-off-by: David Marchand <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]

vhost: rework RARP packet injection

Caught by code review, this copy is unnecessary.

Signed-off-by: David Marchand <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>

show more ...

vhost: remove copy threshold for async path

Copy threshold has been introduced in async vhost data
path to select the appropriate copy engine to do copies
for higher efficiency.

However, it may cau

vhost: remove copy threshold for async path

Copy threshold has been introduced in async vhost data
path to select the appropriate copy engine to do copies
for higher efficiency.

However, it may cause packets ordering issues and also
introduces performance unpredictability.

Therefore, this patch removes copy threshold support in
async vhost data path.

Signed-off-by: Jiayu Hu <[email protected]>
Signed-off-by: Cheng Jiang <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>

show more ...

vhost: add unsafe async API to clear packets

Applications need to stop DMA transfers and finish all the inflight
packets when in VM memory hot-plug case and async vhost is used. This
patch is to pro

vhost: add unsafe async API to clear packets

Applications need to stop DMA transfers and finish all the inflight
packets when in VM memory hot-plug case and async vhost is used. This
patch is to provide an unsafe API to clear inflight packets which
are submitted to DMA engine in vhost async data path. Update the
program guide and release notes for virtqueue inflight packets clear
API in vhost lib.

Signed-off-by: Cheng Jiang <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>

show more ...

vhost: fix async callbacks return type

The async vhost callback ops should return negative value when there
are something wrong in the callback, so the return type should be
changed into int32_t. Th

vhost: fix async callbacks return type

The async vhost callback ops should return negative value when there
are something wrong in the callback, so the return type should be
changed into int32_t. The issue in vhost example is also fixed.

Fixes: cd6760da1076 ("vhost: introduce async enqueue for split ring")
Fixes: 819a71685826 ("vhost: fix async callback return type")
Fixes: 6b3c81db8bb7 ("vhost: simplify async copy completion")
Fixes: abec60e7115d ("examples/vhost: support vhost async data path")
Fixes: 6e9a9d2a02ae ("examples/vhost: fix ioat dependency")
Fixes: 873e8dad6f49 ("vhost: support packed ring in async datapath")
Cc: [email protected]

Signed-off-by: Cheng Jiang <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>

show more ...

vhost: fix packed ring index wrapping

Unlike split ring, packed ring does not mandate the ring size
to be a power of 2. So we have to use a modulo operation when
wrapping ring index.

Fixes: 873e8da

vhost: fix packed ring index wrapping

Unlike split ring, packed ring does not mandate the ring size
to be a power of 2. So we have to use a modulo operation when
wrapping ring index.

Fixes: 873e8dad6f49 ("vhost: support packed ring in async datapath")
Cc: [email protected]

Signed-off-by: Maxime Coquelin <[email protected]>
Acked-by: Cheng Jiang <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>

show more ...

vhost: fix async packed ring batch datapath

We assume that in the sync path, if there is no buffer wrap in the
avail descriptors fetched in a batch, there is no buffer wrap in the
used descriptors w

vhost: fix async packed ring batch datapath

We assume that in the sync path, if there is no buffer wrap in the
avail descriptors fetched in a batch, there is no buffer wrap in the
used descriptors which need to be written back in this batch, but
this assumption is wrong in the async path since there are inflight
descriptors which are processed by the DMA device.

This patch refactors the batch copy code and adds used ring buffer
wrap check as a batch copy condition to fix this issue.

Fixes: 873e8dad6f49 ("vhost: support packed ring in async datapath")
Cc: [email protected]

Signed-off-by: Cheng Jiang <[email protected]>
Reviewed-by: Maxime Coquelin <[email protected]>

show more ...

vhost: fix index overflow for packed ring in async vhost

We introduced some new indexes in packed ring of async vhost. They
will eventually overflow and lead to errors if the ring size is not
a powe

vhost: fix index overflow for packed ring in async vhost

We introduced some new indexes in packed ring of async vhost. They
will eventually overflow and lead to errors if the ring size is not
a power of 2. This patch is to check and keep these indexes within a
reasonable range.

Fixes: 873e8dad6f49 ("vhost: support packed ring in async datapath")
Cc: [email protected]

Signed-off-by: Cheng Jiang <[email protected]>
Reviewed-by: Chenbo Xia <[email protected]>

show more ...