MFC r351747:

Implement nvme suspend / resume for pci attachment

Note: this is merged ~9 hours early due to a desire to have it in before feature
freeze in 20 minutes. Several reports of it workin

MFC r351747:

Implement nvme suspend / resume for pci attachment

Note: this is merged ~9 hours early due to a desire to have it in before feature
freeze in 20 minutes. Several reports of it working in current (and one that it
worked in -stable after copying all of -current's nvme driver) gives me
confidence that bending the rules a little here is the right trade-off.

PR: 240340
Relnotes: Yes

show more ...

MFC r343755:

Regularize the Netflix copyright

Revert r351902 ... it didn't properly exclude rack.c changes

MFC r351706:

In nvme_completion_poll, add a sanity check to make sure that we complete the
polling within a second. Panic if we don't. All the commands that use this
interface should typically

MFC r351706:

In nvme_completion_poll, add a sanity check to make sure that we complete the
polling within a second. Panic if we don't. All the commands that use this
interface should typically complete within a few tens to hundreds of
microseconds. Panic rather than return ETIMEDOUT because if the command
somehow does later complete, it will randomly corrupt memory. Also, it helps
to get a traceback from where the unexpected failure happens, rather than an
infinite loop.

show more ...

MFC r351705:

In all the places that we use the polled for completion interface, except
crash dump support code, move the while loop into an inline function. These
aren't done in the fast path,

MFC r351705:

In all the places that we use the polled for completion interface, except
crash dump support code, move the while loop into an inline function. These
aren't done in the fast path, so if the compiler choses to not inline, any
performance hit is tiny.

show more ...

MFC r351704:

Add a brief comment explaining why we can return ETIMEDOUT from the call to
the polled interface. Normally this would have the potential to corrupt stack
memory because the comple

MFC r351704:

Add a brief comment explaining why we can return ETIMEDOUT from the call to
the polled interface. Normally this would have the potential to corrupt stack
memory because the completion routines would run after we return. In this
case, however, we're doing a dump so it's safe for reasons explained in the
comment.

show more ...

MFC r351406,r351447:

r351406:
We need to define version 1 of nvme, not nvme_foo. Otherwise nvd won't load
and people who pull in nvme/nvd from modules can't load nvd.ko since it
depend

MFC r351406,r351447:

r351406:
We need to define version 1 of nvme, not nvme_foo. Otherwise nvd won't load
and people who pull in nvme/nvd from modules can't load nvd.ko since it
depends on nvme, not nvme_foo. The duplicate doesn't matter since kldxref
properly handles that case.
r351447:
It turns out the duplication is only mostly harmless.

show more ...

MFC r351411:

When we have errors resetting the device before we allocate the queues, don't
try to tear them down in the ctrlr_destroy path. Otherwise, we dereference
queue structures that are

MFC r351411:

When we have errors resetting the device before we allocate the queues, don't
try to tear them down in the ctrlr_destroy path. Otherwise, we dereference
queue structures that are NULL and we trap.

show more ...

MFC r351403:

Move releasing of resources to later

MFC r351376:

Remove stray line that was duplicated.

MFC r351356:

Create a AHCI attachment for nvme.

MFC r351355:

Separate the pci attachment from the rest of nvme

MFC r343755:

Regularize the Netflix copyright

MFC r351352: Improve NVMe hot unplug handling.

If device is unplugged from the system (CSTS register reads return
0xffffffff), it makes no sense to send any more recovery requests or
expect any resp

MFC r351352: Improve NVMe hot unplug handling.

If device is unplugged from the system (CSTS register reads return
0xffffffff), it makes no sense to send any more recovery requests or
expect any responses back. If there is a detach call in such state,
just stop all activity and free resources. If there is no detach
call (hot-plug is not supported), rely on normal timeout handling,
but when it trigger controller reset, do not wait for impossible and
quickly report failure.

Sponsored by: iXsystems, Inc.

show more ...

MFC r351320: Formalize NVMe controller consumer life cycle.

This fixes possible double call of fail_fn, for example on hot removal.
It also allows ctrlr_fn to safely return NULL cookie in case of fa

MFC r351320: Formalize NVMe controller consumer life cycle.

This fixes possible double call of fail_fn, for example on hot removal.
It also allows ctrlr_fn to safely return NULL cookie in case of failure
and not get useless ns_fn or fail_fn call with NULL cookie later.

show more ...

MFC r351028: Report NOIOB and NPWG fields as stripe size.

Namespace Optimal I/O Boundary field added in NVMe 1.3 and Namespace
Preferred Write Granularity added in 1.4 allow upper layers to align
I/

MFC r351028: Report NOIOB and NPWG fields as stripe size.

Namespace Optimal I/O Boundary field added in NVMe 1.3 and Namespace
Preferred Write Granularity added in 1.4 allow upper layers to align
I/Os for improved SSD performance and endurance.

I don't have hardware reportig those yet, but NPWG could probably be
reported by bhyve.

Sponsored by: iXsystems, Inc.

show more ...

MFC r350599, r350609: Add `nvmecontrol resv` to handle NVMe reservations.

NVMe reservations are quite alike to SCSI persistent reservations and
can be used in clustered setups with shared multiport

MFC r350599, r350609: Add `nvmecontrol resv` to handle NVMe reservations.

NVMe reservations are quite alike to SCSI persistent reservations and
can be used in clustered setups with shared multiport storage.

Relnotes: yes

show more ...

MFC r350553: Add more random bits from NVMe 1.4.

MFC r350541: Decode few more NVMe log pages.

In particular: Changed Namespace List, Commands Supported and Effects,
Reservation Notification, Sanitize Status.

Add few new arguments to `nvmecontrol

MFC r350541: Decode few more NVMe log pages.

In particular: Changed Namespace List, Commands Supported and Effects,
Reservation Notification, Sanitize Status.

Add few new arguments to `nvmecontrol log` subcommand.

show more ...

MFC r350529, r350530: Add more new fields and values from NVMe 1.4.

MFC r350523, r350524: Add IOCTL to translate nvdX into nvmeY and NSID.

While very useful by itself, it also makes `nvmecontrol` not depend on
hardcoded device names parsing, that in its turn makes s

MFC r350523, r350524: Add IOCTL to translate nvdX into nvmeY and NSID.

While very useful by itself, it also makes `nvmecontrol` not depend on
hardcoded device names parsing, that in its turn makes simple to take
nvdX (and potentially any other) device names as arguments.

Also added IOCTL bypass from nvdX to respective nvmeYnsZ makes them
interchangeable for management purposes.

show more ...

MFC r350399: Add some new fields and bits from NVMe 1.4.

MFC r350333 (by imp): Widen the type for to.

The timeout field in the CAPS register is defined to be 8 bits, so its type was
uint8_t. We recently started adding 1 to it to cope with rogue devices th

MFC r350333 (by imp): Widen the type for to.

The timeout field in the CAPS register is defined to be 8 bits, so its type was
uint8_t. We recently started adding 1 to it to cope with rogue devices that
listed 0 timeout time (which is impossible). However, in so doing, other devices
that list 0xff (for a 2 minute timeout) were broken when adding 1
overflowed. Widen the type to be uint32_t like its source register to avoid the
issue.

show more ...

MFC r350147 (by imp): Keep track of the number of commands that exhaust their retry limit.

While we print failure messages on the console, sometimes logs are lost or
overwhelmed. Keeping a count of

MFC r350147 (by imp): Keep track of the number of commands that exhaust their retry limit.

While we print failure messages on the console, sometimes logs are lost or
overwhelmed. Keeping a count of how many times we've failed retriable commands
helps get a magnitude of the problem.

show more ...

MFC r350146 (by imp): Keep track of the number of retried commands.

Retried commands can indicate a performance degredation of an nvme drive. Keep
track of the number of retries and report it out vi

MFC r350146 (by imp): Keep track of the number of retried commands.

Retried commands can indicate a performance degredation of an nvme drive. Keep
track of the number of retries and report it out via sysctl, just like number of
commands an interrupts.

show more ...