Make dnode definition uniform on !x86gcc4 requires -fms-extensions to accept anonymous union members
MFV r323530,r323533,r323534: 7431 ZFS Channel Programs, and followups7431 ZFS Channel Programsillumos/illumos-gate@dfc115332c94a2f62058ac7f2bce7631fbd20b3dhttps://github.com/illumos/illumos-gate
MFV r323530,r323533,r323534: 7431 ZFS Channel Programs, and followups7431 ZFS Channel Programsillumos/illumos-gate@dfc115332c94a2f62058ac7f2bce7631fbd20b3dhttps://github.com/illumos/illumos-gate/commit/dfc115332c94a2f62058ac7f2bce7631fbd20b3dhttps://www.illumos.org/issues/7431 ZFS channel programs (ZCP) adds support for performing compound ZFS administrative actions via Lua scripts in a sandboxed environment (with time and memory limits). This initial commit includes both base support for running ZCP scripts, and a small initial library of API calls which support getting properties and listing, destroying, and promoting datasets. Testing: in addition to the included unit tests, channel programs have been in use at Delphix for several months for batch destroying filesystems. The dsl_destroy_snaps_nvl() call has also been replaced withReviewed by: Matthew Ahrens <[email protected]>Reviewed by: George Wilson <[email protected]>Reviewed by: John Kennedy <[email protected]>Reviewed by: Dan Kimmel <[email protected]>Approved by: Garrett D'Amore <[email protected]>Author: Chris Williamson <[email protected]>8552 ZFS LUA code uses floating point mathillumos/illumos-gate@916c8d881190bd2c3ca20d9fca919aecff504435https://github.com/illumos/illumos-gate/commit/916c8d881190bd2c3ca20d9fca919aecff504435https://www.illumos.org/issues/8552 In the LUA interpreter used by "zfs program", the lua format() function accidentally includes support for '%f' and friends, which can cause compilation problems when building on platforms that don't support floating-point math in the kernel (e.g. sparc). Support for '%f' friends (%f %e %E %g %G) should be removed, since there's no way to supply a floating-point value anyway (all numbers in ZFS LUA are int64_t's).Reviewed by: Yuri Pankov <[email protected]>Reviewed by: Igor Kozhukhov <[email protected]>Approved by: Dan McDonald <[email protected]>Author: Matthew Ahrens <[email protected]>8590 memory leak in dsl_destroy_snapshots_nvl()illumos/illumos-gate@e6ab4525d156c82445c116ecf6b2b874d5e9009dhttps://github.com/illumos/illumos-gate/commit/e6ab4525d156c82445c116ecf6b2b874d5e9009dhttps://www.illumos.org/issues/8590 In dsl_destroy_snapshots_nvl(), "snaps_normalized" is not freed after it is added to "arg".Reviewed by: Pavel Zakharov <[email protected]>Reviewed by: Steve Gonczi <[email protected]>Reviewed by: George Wilson <[email protected]>Approved by: Dan McDonald <[email protected]>Author: Matthew Ahrens <[email protected]>FreeBSD notes:- zfs-program.8 manual page is taken almost as is from the vendor repository, no FreeBSD-ification done- fixed multiple instances of NULL being used where an integer is expected- replaced ETIME and ECHRNG with ETIMEDOUT and EDOM respectivelyThis commit adds a modified version of Lua 5.2.4 undersys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua, mirroring theupstream. See README.zfs in that directory for the description of Luacustomizations.See zfs-program.8 on how to use the new feature.MFC after: 5 weeksRelnotes: yesDifferential Revision: https://reviews.freebsd.org/D12528
show more ...
sys/modules: normalize .CURDIR-relative paths to SRCTOPThis simplifies make output/logicTested with: `cd sys/modules; make ALL_MODULES=` on amd64MFC after: 1 monthSponsored by: Dell EMC Isilon
zfs: clean up unused files and definitionsMFC after: 1 monthX-MFC after: r314048
Update CFLAGS for clang compatibility* Clang/llvm does not (yet) support -m(no-)spe, so make it gcc-only* Clang now supports -msoft-float, and does not appear to recognize "-disable-ppc-float-in
Update CFLAGS for clang compatibility* Clang/llvm does not (yet) support -m(no-)spe, so make it gcc-only* Clang now supports -msoft-float, and does not appear to recognize "-disable-ppc-float-in-variadic", which appears to have been a crutch until soft-float was implemented. It's now implemented for both 32- and 64-bit.* Clang/llvm use a 'medium' code model by default for powerpc64, supporting up to 4GB TOC, and does not support the '-mminimal-toc' option. Given both of these, make -mminimal-toc gcc-only.MFC after: 2 weeks
Connect the SHA-512t256 and Skein hashing algorithms to ZFSSupport for the new hashing algorithms in ZFS was introduced in r289422However it was disconnected because FreeBSD lacked implementations
Connect the SHA-512t256 and Skein hashing algorithms to ZFSSupport for the new hashing algorithms in ZFS was introduced in r289422However it was disconnected because FreeBSD lacked implementations ofSHA-512 (truncated to 256 bits), and Skein.These implementations were introduced in r300921 and r300966 respectivelyThis commit connects them to ZFS and enabled these new checksum algorithmsThis new algorithms are not supported by the boot blocks, so do not use themon your root dataset if you boot from ZFS.Relnotes: yesSponsored by: ScaleEngine Inc.
Replace sys/crypto/sha2/sha2.c with lib/libmd/sha512c.ccperciva's libmd implementation is 5-30% fasterThe same was done for SHA256 previously in r263218cperciva's implementation was lacking SHA
Replace sys/crypto/sha2/sha2.c with lib/libmd/sha512c.ccperciva's libmd implementation is 5-30% fasterThe same was done for SHA256 previously in r263218cperciva's implementation was lacking SHA-384 which I implemented, validated against OpenSSL and the NIST documentationExtend sbin/md5 to create sha384(1)Chase dependancies on sys/crypto/sha2/sha2.{c,h} and replace them with sha512{c.c,.h}Reviewed by: cperciva, des, delphijApproved by: secteam, bapt (mentor)MFC after: 2 weeksSponsored by: ScaleEngine Inc.Differential Revision: https://reviews.freebsd.org/D3929
Revert r286984 (adding opt_random.h to sys/modules/zfs/Makefile)opt_random.h is no longer needed/referenced in the kernel buildX-MFC with: r287558
fix standalone build of zfs moduleNot sure if this is a proper fix, but it does the job.
Fix `make depend` in sys/modulesReviewed by: delphijDifferential Revision: https://reviews.freebsd.org/D3291
Rename zfs nvpair files to not colidate with our nvlist.PR: 201356Approved by: pjd (mentor)
Re-support CWARNFLAGS, lost when bsd.sys.mk was removed. Also, supportCWARNFALGS.$file centrally so we don't have to have it in all theplaces. Remove a few warning flags that are no longer needed.
Re-support CWARNFLAGS, lost when bsd.sys.mk was removed. Also, supportCWARNFALGS.$file centrally so we don't have to have it in all theplaces. Remove a few warning flags that are no longer needed.Also, always use -Wno-unknown-pragma to (hopefully temporarily) workaround #pragma ident in debug.h in the opensolaris code. Remove somestale warning suppression that's no longer necessary.
Instead of requiring an edit to turn on ZFS debugging, define ZFS_DEBUG.MFC after: 1 month
Add dtrace probe support for zfs SET_ERROR(..)MFC after: 1 weekSponsored by: Multiplay
Avoid using ${.CURDIR} so that the module can be built from multipledirectories.Obtained from: Juniper Networks, Inc.
I clearly didn't test the modules... add sha256c.c to the variousmodules that included sha2.c...
opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUGDo this by forcing inclusion ofsys/cddl/compat/opensolaris/sys/debug_compat.hvia -include option into all source files from OpenSolari
opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUGDo this by forcing inclusion ofsys/cddl/compat/opensolaris/sys/debug_compat.hvia -include option into all source files from OpenSolaris.Note that this -include option must always be after -include opt_global.h.Additionally, remove forced definition of DEBUG for some modules and fixtheir build without DEBUG.Also, meaning of DEBUG was overloaded to enable WITNESS support for someOpenSolaris (primarily ZFS) locks. Now this overloading is removed andthat use of DEBUG is replaced with a new option OPENSOLARIS_WITNESS.MFC after: 17 days
Switch the vm_object mutex to be a rwlock. This will enable in thefuture further optimizations where the vm_object lock will be heldin read mode most of the time the page cache resident pool of pa
Switch the vm_object mutex to be a rwlock. This will enable in thefuture further optimizations where the vm_object lock will be heldin read mode most of the time the page cache resident pool of pagesare accessed for reading purposes.The change is mostly mechanical but few notes are reported:* The KPI changes as follow: - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK() - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK() - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK() - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED() (in order to avoid visibility of implementation details) - The read-mode operations are added: VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(), VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED()* The vm/vm_pager.h namespace pollution avoidance (forcing requiring sys/mutex.h in consumers directly to cater its inlining functions using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h consumers now must include also sys/rwlock.h.* zfs requires a quite convoluted fix to include FreeBSD rwlocks into the compat layer because the name clash between FreeBSD and solaris versions must be avoided. At this purpose zfs redefines the vm_object locking functions directly, isolating the FreeBSD components in specific compat stubs.The KPI results heavilly broken by this commit. Thirdy part ports mustbe updated accordingly (I can think off-hand of VirtualBox, for example).Sponsored by: EMC / Isilon storage divisionReviewed by: jeffReviewed by: pjd (ZFS specific review)Discussed with: alcTested by: pho
Add TRIM support.The code builds a map of regions that were freed. On every write thecode consults the map and eventually removes ranges that were freedbefore, but are now overwritten.Freed blo
Add TRIM support.The code builds a map of regions that were freed. On every write thecode consults the map and eventually removes ranges that were freedbefore, but are now overwritten.Freed blocks are not TRIMed immediately. There is a tunable that defineshow many txg we should wait with TRIMming freed blocks (64 by default).There is a low priority thread that TRIMs ranges when the time comes.During TRIM we keep in-flight ranges on a list to detect collidingwrites - we have to delay writes that collide with in-flight TRIMs incase something will be reordered and write will reached the disk beforethe TRIM. We don't have to do the same for in-flight writes, ascolliding writes just remove ranges to TRIM.Sponsored by: multiplay.co.ukThis work includes some important fixes and some improvements obtainedfrom the zfsonlinux project, including TRIMming entire vdevs on poolcreate/add/attach and on pool import for spare and cache vdevs.Obtained from: zfsonlinuxSubmitted by: Etienne Dechamps <[email protected]>
Introduce "feature flags" for ZFS pools (bump SPA version to 5000).Add first feature "com.delphix:async_destroy" (asynchronous destroyof ZFS datasets).Implement features support in ZFS boot code.
Introduce "feature flags" for ZFS pools (bump SPA version to 5000).Add first feature "com.delphix:async_destroy" (asynchronous destroyof ZFS datasets).Implement features support in ZFS boot code.Illumos revisions merged:13700:2889e2596bd613701:1949b688d5fb2619 asynchronous destruction of ZFS file systems2747 SPA versioning with zfs feature flagsReferences:https://www.illumos.org/issues/2619https://www.illumos.org/issues/2747Obtained from: illumos (issue #2619, #2747)MFC after: 1 month
Don't look for includes in userland directories. There is no need for itand it breaks building the kernel without userland sources.Reported by: andre, impMFC after: 3 days
Fix misuse of TARGET_ARCH. This should be MACHINE_ARCH.
Finally... Import the latest open-source ZFS version - (SPA) 28.Few new things available from now on:- Data deduplication.- Triple parity RAIDZ (RAIDZ3).- zfs diff.- zpool split.- Snapshot ho
Finally... Import the latest open-source ZFS version - (SPA) 28.Few new things available from now on:- Data deduplication.- Triple parity RAIDZ (RAIDZ3).- zfs diff.- zpool split.- Snapshot holds.- zpool import -F. Allows to rewind corrupted pool to earlier transaction group.- Possibility to import pool in read-only mode.MFC after: 1 month
Simplify atomic selection
Add OpenSolaris atomics for powerpc64 and connect ZFS to the build onthis platform.Reviewed by: pjd
12