MFC loader+userland TSLOG support

stand/common: Add file_addbuf()
libsa: Add support for timestamp logging (tslog)
stand/common: Add support for timestamp logging (tslog)
i386/loader: Call tslog_ini

MFC loader+userland TSLOG support

stand/common: Add file_addbuf()
libsa: Add support for timestamp logging (tslog)
stand/common: Add support for timestamp logging (tslog)
i386/loader: Call tslog_init
efi/loader: Call tslog_init (+ bugfix)
stand/common command_boot: Pass tslog to kernel
kern_tslog: Include tslog data from loader
loader: Use tslog to instrument some functions
Add userland boot profiling to TSLOG (+ bugfix)

Sponsored by: https://www.patreon.com/cperciva

(cherry picked from commit 60a978bec9123fcb9c74bd925e06dd3f4faddac6)
(cherry picked from commit e193d3ba337757a27148a38494d3df106b826d1c)
(cherry picked from commit c8dfc327db2373c721a35efe857cf07262fd7dd0)
(cherry picked from commit c4b65e954f0f4754941f5b37436231f611b76d67)
(cherry picked from commit f49381ccb6bc8a5733f1aa505e716eddae299986)
(cherry picked from commit 537a44bf281559d304850b5e28b6f8b8e44fd593)
(cherry picked from commit fe51b5a76de38d20cddc05a4c6cb1103d1a0547e)
(cherry picked from commit 313724bab940c1844fda3d797cf88cd46780e62a)
(cherry picked from commit 46dd801acb2318c8464b185893bb111f09b71138)
(cherry picked from commit 52e125c2bd302982c179ecc920cc9a6e166033d6)
(cherry picked from commit 19e4f2f289fb8d2b14576a65ca9e6f931dbb6a02)

show more ...

stand: liblua: add a pager module

This is nearly a 1:1 mapping of the pager API from libsa. The only real
difference is that pager.output() will accept any number of arguments and
coerce all of the

stand: liblua: add a pager module

This is nearly a 1:1 mapping of the pager API from libsa. The only real
difference is that pager.output() will accept any number of arguments and
coerce all of them to strings for output using luaL_tolstring (i.e. the
__tostring metamethod will be used).

The only consumer planned at this time is the upcoming "show-module-options"
implementation.

MFC after: 1 week

show more ...

lualoader: drop the filename and word "LUA" from errors

The filename is nearly always wrong since it's /boot/lua/loader.lua, which
is not useful for diagnostics. The actual errmsg will include a lua

lualoader: drop the filename and word "LUA" from errors

The filename is nearly always wrong since it's /boot/lua/loader.lua, which
is not useful for diagnostics. The actual errmsg will include a lua filename
if this is relevant.

Dropping "LUA" while we're here because that's almost universally
irrelevant to whatever error follows, unless the error states that it's
actually a lua problem.

Both of these are minor nits that just detract from identifying the
pertinent information.

MFC after: 3 days

show more ...

interp_lua: fix bogus indentation, NFC

lualoader: disable autoboot on high-level interpreter errors

If we hit an error at this level, we are almost certainly not in any kind
of sane state where autoboot can do the right thing. Instead of

lualoader: disable autoboot on high-level interpreter errors

If we hit an error at this level, we are almost certainly not in any kind
of sane state where autoboot can do the right thing. Instead of letting it
try and probably failing, disable autoboot so they immediately get kicked
into a loader prompt for manual remediation/diagnosis.

Reviewed by: tsoome
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D23611

show more ...

stand: consolidate knowledge of lua path

Multiple places coordinate to 'know' where lua scripts are installed. Knock
this down to being formally defined (and overridable) in exactly one spot,
defs.m

stand: consolidate knowledge of lua path

Multiple places coordinate to 'know' where lua scripts are installed. Knock
this down to being formally defined (and overridable) in exactly one spot,
defs.mk, and spread the knowledge to loaders and liblua alike. A future
commit will expose this to lua as loader.lua_path, so it can build absolute
paths to lua scripts as needed.

MFC after: 1 week

show more ...

userboot: handle guest interpreter mismatches more intelligently

The switch to lualoader creates a problem with userboot: the host is
inclined to build userboot with Lua, but the host userboot's int

userboot: handle guest interpreter mismatches more intelligently

The switch to lualoader creates a problem with userboot: the host is
inclined to build userboot with Lua, but the host userboot's interpreter
must match what's available on the guest. For almost all FreeBSD guests in
the wild, Lua is not yet available and a Lua-based userboot will fail.

This revision updates userboot protocol to version 5, which adds a
swap_interpreter callback to request a different interpreter, and tries to
determine the proper interpreter to be used based on how the guest
/boot/loader is compiled. This is still a bit of a guess, but it's likely
the best possible guess we can make in order to get it right. The
interpreter is now embedded in the resulting executable, so we can open
/boot/loader on the guest and hunt that down to derive the interpreter it
was built with.

Using -l with bhyveload will not allow an intepreter swap, even if the
loader specified happens to be a userboot with the wrong interpreter. We'll
simply complain about the mismatch and bail out.

For legacy guests without the interpreter marker, we assume they're 4th.
For new guests with the interpreter marker, we'll read it and swap over
to the proper interpreter if it doesn't match what the userboot we're using
was compiled with.

Both flavors of userboot are installed by default, userboot_4th.so and
userboot_lua.so. This fixes the build WITHOUT_FORTH as a coincidence, which
was broken by userboot being forced to 4th.

Reviewed by: imp, jhb, araujo (earlier version)
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D16945

show more ...

lualoader: Print error messages from command failures at the prompt

Previously lualoader would remain silent, rather than printing
command_errmsg or noting that a command had failed or was not found

lualoader: Print error messages from command failures at the prompt

Previously lualoader would remain silent, rather than printing
command_errmsg or noting that a command had failed or was not found.

Approved by: re (gjb)

show more ...

stand/interp_lua: correct errorr => error

loader.lua: Expose errno table to lua

Reviewed by: kevans
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14454

lualoader: Add ability to intercept cli commands

If we failed to execute the input line as pure lua, run the command through
parse for consistent argument parsing. Pass the parsed arguments through

lualoader: Add ability to intercept cli commands

If we failed to execute the input line as pure lua, run the command through
parse for consistent argument parsing. Pass the parsed arguments through to
a global "cli_execute" written in Lua, which is expected to either handle it
or pass it back through to interp_builtin_cmd (via loader.command).

lua-handled cli commands will then exist as globals in whatever module they
most belong in, and invocations at the loader prompt will magically dispatch
to them if they exist.

Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D14450

show more ...

Lua loader: Add barebones "lfs" module

Add a Lua FileSystem module, an emulation of a subset of the permissively
licensed (MIT) Lua library of the same name[0], to our loader's Lua
environment.

[0]

Lua loader: Add barebones "lfs" module

Add a Lua FileSystem module, an emulation of a subset of the permissively
licensed (MIT) Lua library of the same name[0], to our loader's Lua
environment.

[0]: https://github.com/keplerproject/luafilesystem/

Reviewed by: kevans
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14418

show more ...

interp_lua: Register io/loader with regular Lua module system

Reviewed by: kevans
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14421

Add Lua as a scripting langauge to /boot/loader

liblua glues the lua run time into the boot loader. It implements all
the runtime routines that lua expects. In addition, it has a few
standard 'C' he

Add Lua as a scripting langauge to /boot/loader

liblua glues the lua run time into the boot loader. It implements all
the runtime routines that lua expects. In addition, it has a few
standard 'C' headers that nueter various aspects of the LUA build that
are too specific to lua to be in libsa. Many refinements from the
original code to improve implementation and the number of included lua
libraries. Use int64_t for lua_Number. Have "/boot/lua" be the default
module path. Numerous cleanups from the original GSoC project,
including hacking libsa to allow lua to be built with only one change
outside luaconf.h.

Add the final bit of lua glue to bring in liblua and plug into the
multiple interpreter framework, previously committed.

Add LOADER_LUA option, currently off by default.

Presently, this is an experimental option. One must opt-in to using
this by defining WITH_LOADER_LUA and WITHOUT_FORTH. It's been
lightly tested, so keep a backup copy of your old loader handy.
The menu code, coming in the next commit, hasn't been exhaustively
tested. A LUA boot loader is 60k larger than a FORTH one, which is
80k larger than a no-interpreter one. Subtle changes in size
may tip things past some subtle limit (the binary is ~430k now
when built with LUA). A future version may offer coexistance.

Bump FreeBSD version to 1200058 to mark the milestone.

Pedro Souza's 2014 Summer of Code project. Rui Paulo, Pedro Arthur,
Zakary Nafziger and Wojciech A. Koszek also contributed. Warner Losh
reworked it extensively into its current form.

Obtained from: https://wiki.freebsd.org/SummerOfCode2014/LuaLoader
Sponsored by: Google Summer of Code
Relnotes: Yes
MFC After: 1 month
Differential Review: https://reviews.freebsd.org/D14295

show more ...