release: Don't break if firmware install fails

On some platforms (e.g. powerpc) we don't have packages, so we can't
install them onto the ISOs. Proceed with building the images anyway.

Reported by

release: Don't break if firmware install fails

On some platforms (e.g. powerpc) we don't have packages, so we can't
install them onto the ISOs. Proceed with building the images anyway.

Reported by: Weekly snapshot builds
Fixes: 7e2996c1f5b4 ("release: install wireless firmware onto disc1 and dvd")
Approved by: re (cperciva)

(cherry picked from commit e8263ace39c8ecf11233c0a10d0b1839e6813046)
(cherry picked from commit 9431091ee021eca9eb4c768b566e623ba9333b41)

show more ...

release: install wireless firmware onto disc1 and dvd

Wireless driver firmware is no longer added to the src tree.
In order to have wireless support in the installer for the new drivers
we install t

release: install wireless firmware onto disc1 and dvd

Wireless driver firmware is no longer added to the src tree.
In order to have wireless support in the installer for the new drivers
we install the firmware packages onto disc1 (and memstick) and dvd
if built on FreeBSD and NOPKG is not defined (to not break cross-builds
from Linux or OSX and to allow people to opt-out).

Sponsored by: The FreeBSD Foundation
Submitted by: cperciva (the orig. commands and where to place them)
Reviewed by: jrtc27
Approved by: re (cperciva)
Differential Revision: https://reviews.freebsd.org/D47407

(cherry picked from commit 7e2996c1f5b4e684cae40c2418b68061df9997d9)
(cherry picked from commit b62012538e967b22736e8c2214b222f21146bddc)

show more ...

release: Don't copy container images into /ftp/

These are installed into /ociimages/ and stage onto the download site
at /OCI-IMAGES/; they don't belong in the same directory as the
"distribution" .

release: Don't copy container images into /ftp/

These are installed into /ociimages/ and stage onto the download site
at /OCI-IMAGES/; they don't belong in the same directory as the
"distribution" .txz files.

Approved by: re (cperciva)
Fixes: d03c82c28da8 ("release: add optional OCI images")
MFC after: 1 minute
Sponsored by: Amazon

(cherry picked from commit 976e1d36be688166e140cbaded01d433c61e6d47)
(cherry picked from commit a8bdbdbdba3e7c6e3a330026dcd82e922a4265c8)

show more ...

release: Exclude container images from globs

We use a *.txz glob to get all of the "distributions" which comprise
the FreeBSD release, but we now (optionally) also build container
images which are .

release: Exclude container images from globs

We use a *.txz glob to get all of the "distributions" which comprise
the FreeBSD release, but we now (optionally) also build container
images which are .txz files. Grep those out from the distribution
lists.

A better long-term fix would probably be to generate an explicit list
of the .txz files we want rather than using an overbroad glob and
filtering out the files we *don't* want.

Approved by: re (cperciva)
Fixes: d03c82c28da8 ("release: add optional OCI images")
MFC after: 1 minute
Sponsored by: Amazon

(cherry picked from commit 47866cdcc7278faa27250dcea16374231c676050)
(cherry picked from commit 012bb7fdafb30032f7a09212ffd6398eb4230b07)

show more ...

release: add optional OCI images

This adds three OCI archive format files to the release containing
FreeBSD base images suitable for static linked, dynamic linked and shell
workloads. The shell imag

release: add optional OCI images

This adds three OCI archive format files to the release containing
FreeBSD base images suitable for static linked, dynamic linked and shell
workloads. The shell image also contains pkg-bootstrap and can be easily
extended by installing packages (including pkgbase packages).

Approved by: re (cperciva)
Reviewed by: dch, cpersiva, jlduran, zlei
Differential Revision: https://reviews.freebsd.org/D46759
MFC after: 2 days

(cherry picked from commit d03c82c28da86e0812b98b051d24ae5980804ad7)
(cherry picked from commit 6686056ca3547c8280bd2423c59c661c2f501409)

show more ...

release: Remove useless ls

Replace "$((ls *.txz))" with "*.txz"

Approved by: re (implicit, release build fixes)
Reported by: jrtc27
Sponsored by: Amazon

(cherry picked from commit e166fea685da4ac6

release: Remove useless ls

Replace "$((ls *.txz))" with "*.txz"

Approved by: re (implicit, release build fixes)
Reported by: jrtc27
Sponsored by: Amazon

(cherry picked from commit e166fea685da4ac6a6891f10b07f98d0e9fcfce3)
(cherry picked from commit ce988aa3e80019566bf5df4cfd73f7b7195756f2)

show more ...

release: Include -dbg files in dvd image

In 2016, commit 8834318685bc reworked the code for excluding -dbg
files from install media, and in the process accidentally broke it
for dvd images ('grep -v

release: Include -dbg files in dvd image

In 2016, commit 8834318685bc reworked the code for excluding -dbg
files from install media, and in the process accidentally broke it
for dvd images ('grep -v' should have been 'grep -vE').

FreeBSD Update builds later began to depend on this, and in any case
since DVD images are intended as "include everything" images there's
no point excluding those; so remove the (broken) filtering from DVD
image builds.

Approved by: re (implicit, release build fixes)
Sponsored by: Amazon
MFC after: 2 days

(cherry picked from commit cdfd0600dc8882f0a0d0e6d9a1cdcf926edba6d6)
(cherry picked from commit ba7844e4c91505e397396ab981a0032699b0cd70)

show more ...

release: Redirect etcupdate logfile to /dev/null.

Stop shipping a log file for etcupdate. This is a source of
non-reproducability as it uses mktemp thereby guaranteeing the output is
different each

release: Redirect etcupdate logfile to /dev/null.

Stop shipping a log file for etcupdate. This is a source of
non-reproducability as it uses mktemp thereby guaranteeing the output is
different each run.

Differential Revision: https://reviews.freebsd.org/D46317

(cherry picked from commit e972e408d19a58c4f2855eface487d06ef73b799)

show more ...

release: Publish non-uploaded cloudware bits

While most "cloudware" (AWS, Azure, GCP, Vagrant) images get uploaded
to their respective clouds, we have two images -- BASIC-CI and
BASIC-CLOUDINIT -- w

release: Publish non-uploaded cloudware bits

While most "cloudware" (AWS, Azure, GCP, Vagrant) images get uploaded
to their respective clouds, we have two images -- BASIC-CI and
BASIC-CLOUDINIT -- which are published via the FTP mirrors. We have
been handling these using a manual and error-prone process of copying
them from the internal build directories into the FTP staging tree,
but they should be handled using the "make install ftp-stage"
mechanism which is used for all the other published images.

This commit 1. Adds a new 'make cloud-install' target which is run as
part of 'make install' and installs BASIC-CI and BASIC-CLOUDINIT images,
and 2. Extends 'make ftp-stage' in Makefile.mirrors to copy those bits
into the FTP staging area.

As a side benefit, this will also make BASIC-CI and BASIC-CLOUDINIT
images available for arm64; until now they have only been available
for amd64.

MFC after: 1 week

(cherry picked from commit 4771c2e9d1c7db949a82dfe4f2b9878bb358a50e)

show more ...

release: Build dvd image before staging pkgs

This missing dependency is only visible with -jN (which was otherwise
broken until recently) and only when building DVD images (which aren't
included in

release: Build dvd image before staging pkgs

This missing dependency is only visible with -jN (which was otherwise
broken until recently) and only when building DVD images (which aren't
included in weekly snapshots and thereby slipped past testing).

MFC after: 3 days

(cherry picked from commit 567cbd8ff1c4edc1f2015ec4f93608aaf5a3c546)

show more ...

release: Don't reuse disc1/bootonly directories

The disc1 and bootonly directories have files distributed into them
for use in "full" and "mini" images; the former are disc1.iso and
memstick.img, an

release: Don't reuse disc1/bootonly directories

The disc1 and bootonly directories have files distributed into them
for use in "full" and "mini" images; the former are disc1.iso and
memstick.img, and the latter is bootonly.iso and mini-memstick.img.

Unfortunately the scripts which package a directory tree into an ISO
or memory stick image also modify the directory, for example to
create an appropriate /etc/fstab file; so creating two images at the
same time breaks.

Resolve this by copying disc1 to disc1-disc1 and disc1-memstick,
and copying bootonly to bootonly-bootonly and bootonly-memstick,
before using those directories for constructing the ISO+memstick
images.

MFC after: 1 week

(cherry picked from commit a1bff97300abe4b46528357e39c83da764df1fd8)

show more ...

release: distributekernel before packagekernel

With these as a single make command, `make -j` breaks when it tries to
package up a kernel which hasn't been distributed yet.

MFC after: 1 week

(cher

release: distributekernel before packagekernel

With these as a single make command, `make -j` breaks when it tries to
package up a kernel which hasn't been distributed yet.

MFC after: 1 week

(cherry picked from commit 65c603ed65c700d6eacdf9e1e94dc42fd70fb0b7)

show more ...

release: make -j compat: cd inside subshell

Place instances of "cd foo && bar" inside subshells for compatibility
with modern make(8) which uses a single shell for the duration of a
makefile target.

release: make -j compat: cd inside subshell

Place instances of "cd foo && bar" inside subshells for compatibility
with modern make(8) which uses a single shell for the duration of a
makefile target.

MFC after: 1 week

(cherry picked from commit 93e4813b9a157396ffbcf8456ca4f5c25599807d)

show more ...

release: rework distributions list

Components like base.txz and ports.txz are called distributions in the
installer, and with the introduction of pkgbase we will start dealing
with normal pkg packag

release: rework distributions list

Components like base.txz and ports.txz are called distributions in the
installer, and with the introduction of pkgbase we will start dealing
with normal pkg packages in the installer. Rename EXTRA_PACKAGES to
DISTRIBUTIONS, and move base.txz and kernel.txz to that list.

This introduces no functional change but is a small cleanup in advance
of some pkgbase experimentation.

Reviewed by: cperciva
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D43544

(cherry picked from commit 2e9b8c8c1e3938335e17e63049887f8e8c6e11bc)

show more ...

release: update stable/14 following branching from main

Approved by: re (implicit)
Sponsored by: GoFundMe https://www.gofundme.com/f/gjbbsd
Sponsored by: PayPal https://paypal.me/gjbbsd

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

release: Fix make :@ modifier syntax

This spits out errors but seemingly isn't actually fatal, so was missed.

Fixes: 6853d893c7fe ("release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libco

release: Fix make :@ modifier syntax

This spits out errors but seemingly isn't actually fatal, so was missed.

Fixes: 6853d893c7fe ("release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libcompat}-dbg lists")

show more ...

release: Automatically generate MK_LIB${LIBCOMPAT} and lib${libcompat}-dbg lists

Reviewed by: brooks, jhb
Differential Revision: https://reviews.freebsd.org/D41184

release: Set loader_menu_multi_user_prompt to "Installer" for installer

This makes it more obvious that the media being booted is an installer
rather than an installed system, which is otherwise har

release: Set loader_menu_multi_user_prompt to "Installer" for installer

This makes it more obvious that the media being booted is an installer
rather than an installed system, which is otherwise hard to distinguish.
It also provides a more user-friendly, and more accurate, prompt.

Reviewed by: gjb
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D36419

show more ...

release: Include an etcupdate tree in -DNO_ROOT release image builds

This makes use of the new -N etcupdate flag and merges the resulting
METALOG into base.meta re-rooted to /var/db/etcupdate/curren

release: Include an etcupdate tree in -DNO_ROOT release image builds

This makes use of the new -N etcupdate flag and merges the resulting
METALOG into base.meta re-rooted to /var/db/etcupdate/current.

Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35858

show more ...

release: Forward ${MAKE} to etcupdate via the new -m flag

This is in preparation for non-FreeBSD builds where make is GNU make and
so etcupdate needs to know the name of or path to the bmake binary

release: Forward ${MAKE} to etcupdate via the new -m flag

This is in preparation for non-FreeBSD builds where make is GNU make and
so etcupdate needs to know the name of or path to the bmake binary to
use for its own builds.

Reviewed by: gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35855

show more ...

release: Use in-tree etcupdate for build

This is in preparation for non-FreeBSD and -DNO_ROOT builds. On
non-FreeBSD there is no host etcupdate to use, and -DNO_ROOT will
require additional flags th

release: Use in-tree etcupdate for build

This is in preparation for non-FreeBSD and -DNO_ROOT builds. On
non-FreeBSD there is no host etcupdate to use, and -DNO_ROOT will
require additional flags that may not be supported by the host's
etcupdate when building on FreeBSD. Moreover, there's no guarantee
anyway that the host's etcupdate is quite right for the current tree;
upgrading from source only requires that the host's is good enough for
-p which just manually copies master.passwd and group, the rest of the
upgrade is done post-installworld. For example, should a new set of
autogenerated files be added that etcupdate is taught about, the host
won't know about them and so the bootstrapped current tree will
incorrectly contain them, leading to spurious diffs on the installed
system.

Reviewed by: gjb, delphij
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35853

show more ...

Makefile.inc1 release bsd.own.mk: Introduce and use TAR_CMD

Our uses of tar rely on BSDisms, and so do not work in environments
where GNU tar is the default tar. Providing a TAR_CMD variable like
so

Makefile.inc1 release bsd.own.mk: Introduce and use TAR_CMD

Our uses of tar rely on BSDisms, and so do not work in environments
where GNU tar is the default tar. Providing a TAR_CMD variable like
some other commands allows it to be overridden to use bsdtar in such
cases.

Reviewed by: brooks, delphij, gjb
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D35813

show more ...

release: Support -DNO_ROOT image building

This requires a bunch of METALOG mangling to include the files we inject
into the tree. The mkisoimages.sh and make-memstick.sh scripts are now
called with

release: Support -DNO_ROOT image building

This requires a bunch of METALOG mangling to include the files we inject
into the tree. The mkisoimages.sh and make-memstick.sh scripts are now
called with the current directory inside the tree so that the relative
paths in the METALOG match up with the current directory. The scripts do
not require this when not using a METALOG, but for simplicity we always
do so. The Makefile mangles the real METALOG created from the install,
as those files are shared across all uses of the tree, but the shell
scripts create a temporary copy of the METALOG that they mangle as their
tree modifications are specific to that image. We also need to pass -D
to makefs to turn any duplicate METALOG entry errors into warnings, as
we have many (harmless) instances of those.

Whilst dvd1.iso should work, the !NOPKG code will need more work to
support this.

All media will also lack mergemaster and etcupdate trees, since more
work is needed to add -DNO_ROOT modes to them. Users of install media
built this way will have to manually bootstrap them.

Reviewed by: brooks, gjb
Differential Revision: https://reviews.freebsd.org/D33999

show more ...

sys/*/conf/*, docs: fix links to handbook

While here, fix all links to older en_US.ISO8859-1 documentation
in the src/ tree.

PR: 255026
Reported by: Michael Büker <freebsd@michael-bu

sys/*/conf/*, docs: fix links to handbook

While here, fix all links to older en_US.ISO8859-1 documentation
in the src/ tree.

PR: 255026
Reported by: Michael Büker <[email protected]>
Reviewed by: dbaio
Approved by: blackend (mentor), re (gjb)
MFC after: 10 days
Differential Revision: https://reviews.freebsd.org/D30265

show more ...