libfetch: don't rely on ca_root_nss for certificate validation

Before certctl(8), there was no system trust store, and libfetch
relied on the CA certificate bundle from the ca_root_nss port to
verif

libfetch: don't rely on ca_root_nss for certificate validation

Before certctl(8), there was no system trust store, and libfetch
relied on the CA certificate bundle from the ca_root_nss port to
verify peers.

We now have a system trust store and a reliable mechanism for
manipulating it (to explicitly add, remove, or revoke certificates),
but if ca_root_nss is installed, libfetch will still prefer that to
the system trust store.

With this change, unless explicitly overridden, libfetch will rely on
OpenSSL to pick up the default system trust store.

PR: 256902
MFC after: 3 days
Reviewed by: kevans
Differential Revision: https://reviews.freebsd.org/D42059

(cherry picked from commit 09f5c1e118bb4eca77b83a0d08f559b20f60aa59)

show more ...

Remove $FreeBSD$: one-line .c pattern

Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/

libfetch: remove all old OpenSSL support

This change removes pre-OpenSSL 1.1 supporting code and removes/adjusted
preprocessor conditionals which were tautilogically true as FreeBSD main
has shipped

libfetch: remove all old OpenSSL support

This change removes pre-OpenSSL 1.1 supporting code and removes/adjusted
preprocessor conditionals which were tautilogically true as FreeBSD main
has shipped with OpenSSL 1.1+ for some time.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D40711

show more ...

libfetch: do not call deprecated OpenSSL functions

As of OpenSSL 1.1 SSL_library_init() and SSL_load_error_strings() are
deprecated. There are replacement initialization functions but they do
not n

libfetch: do not call deprecated OpenSSL functions

As of OpenSSL 1.1 SSL_library_init() and SSL_load_error_strings() are
deprecated. There are replacement initialization functions but they do
not need to be called: "As of version 1.1.0 OpenSSL will automatically
allocate all resources that it needs so no explicit initialisation is
required."

Wrap both calls in an OPENSSL_VERSION_NUMBER block.

PR: 271615
Reviewed by: Pierre Pronchery <[email protected]>
Event: Kitchener-Waterloo Hackathon 202305
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40265

show more ...

fetch: support EAI_ADDRFAMILY error, correct two error messages

With the change to return EAI_ADDRFAMILY from getaddrinfo(), fetch
would print "Unknown resolver error" for that error. Add that erro

fetch: support EAI_ADDRFAMILY error, correct two error messages

With the change to return EAI_ADDRFAMILY from getaddrinfo(), fetch
would print "Unknown resolver error" for that error. Add that error
and its string to libfetch's table, using an #ifdef just in case.
Correct error strings for EAI_NODATA (although it is currently unused)
and EAI_NONAME. Should maybe rework the code to use gai_strerror(3),
but that doesn't map directly, and the current strings are shortened.

Reviewed in https://reviews.freebsd.org/D37139 with related changes.

Reviewed by: bz
MFC after: 1 month

show more ...

libfetch: Use memcpy in place of an odd strncpy.

The length passed to strncpy is the length of the source string, not
the destination buffer. This triggers a non-fatal warning in GCC 12.
Hoewver, t

libfetch: Use memcpy in place of an odd strncpy.

The length passed to strncpy is the length of the source string, not
the destination buffer. This triggers a non-fatal warning in GCC 12.
Hoewver, the code is also odd. It is really just a memcpy of the
string without its nul terminator. For that use case, memcpy is
clearer.

Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D36824

show more ...

Remove support for SSLv3 from fetch(3).

Support for SSLv3 was already removed from OpenSSL (r361392).

Differential Revision: https://reviews.freebsd.org/D24947

Replace literal uses of /usr/local in C sources with _PATH_LOCALBASE

Literal references to /usr/local exist in a large number of files in
the FreeBSD base system. Many are in contributed software, i

Replace literal uses of /usr/local in C sources with _PATH_LOCALBASE

Literal references to /usr/local exist in a large number of files in
the FreeBSD base system. Many are in contributed software, in configuration
files, or in the documentation, but 19 uses have been identified in C
source files or headers outside the contrib and sys/contrib directories.

This commit makes it possible to set _PATH_LOCALBASE in paths.h to use
a different prefix for locally installed software.

In order to avoid changes to openssh source files, LOCALBASE is passed to
the build via Makefiles under src/secure. While _PATH_LOCALBASE could have
been used here, there is precedent in the construction of the path used to
a xauth program which depends on the LOCALBASE value passed on the compiler
command line to select a non-default directory.

This could be changed in a later commit to make the openssh build
consistently use _PATH_LOCALBASE. It is considered out-of-scope for this
commit.

Reviewed by: imp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D26942

show more ...

fetch(3): plug some leaks

In the successful case, sockshost is not freed prior to return.

The failure case can now be hit after fetch_reopen(), which was not true
before. Thus, we need to make sure

fetch(3): plug some leaks

In the successful case, sockshost is not freed prior to return.

The failure case can now be hit after fetch_reopen(), which was not true
before. Thus, we need to make sure to clean up all of the conn resources
which will also close sd. For all of the points prior to fetch_reopen(), we
continue to just close sd.

CID: 1419598, 1419616

show more ...

fetch(3): don't leak sockshost on failure

fetch_socks5_getenv will allocate memory for the host (or set it to NULL) in
all cases through the function; the caller is responsible for freeing it if
we

fetch(3): don't leak sockshost on failure

fetch_socks5_getenv will allocate memory for the host (or set it to NULL) in
all cases through the function; the caller is responsible for freeing it if
we end up allocating.

While I'm here, I've eliminated a label that just jumps to the next line...

show more ...

fetch(3): fix regression in IPv6:port spec from r357977

In case the port was specified, we never actually populated *host. Do so
now.

Pointy hat: kevans

fetch(3): move bits of fetch_socks5_getenv around

This commit separates out port parsing and validation from grabbing the host
from the env var. The only related bit really is that we need to be mor

fetch(3): move bits of fetch_socks5_getenv around

This commit separates out port parsing and validation from grabbing the host
from the env var. The only related bit really is that we need to be more
specific with the delimiter in the IPv6 case.

show more ...

fetch(3): Add SOCKS5 support

This change adds SOCKS5 support to the library fetch(3) and updates the man
page.

Details: Within the fetch_connect() function, fetch(3) checks if the
SOCKS5_PROXY envi

fetch(3): Add SOCKS5 support

This change adds SOCKS5 support to the library fetch(3) and updates the man
page.

Details: Within the fetch_connect() function, fetch(3) checks if the
SOCKS5_PROXY environment variable is set. If so, it connects to this host
rather than the end-host. It then initializes the SOCKS5 connection in
accordance with RFC 1928 and returns the resulting conn_t (file descriptor)
for usage by the regular FTP/HTTP handlers.

Design Decision: This change defaults all DNS resolutions through the proxy
by sending all IPs as hostnames. Going forward, another feature might be to
create another environmental variable to toggle resolutions through the
proxy or not..

One may set the SOCKS5_PROXY environment variable in any of the formats:

SOCKS5_PROXY=proxy.example.com
SOCKS5_PROXY=proxy.example.com:1080
SOCKS5_PROXY=192.0.2.0
SOCKS5_PROXY=198.51.100.0:1080
SOCKS5_PROXY=[2001:db8::1]
SOCKS5_PROXY=[2001:db8::2]:1080

Then perform a request with fetch(1).

(note by kevans)
I've since been informed that Void Linux/xbps has a fork of libfetch that
also implements SOCKS5. I may compare/contrast the two in the mid-to-near
future.

Submitted by: Farhan Khan <farhan farhan codes>
Differential Revision: https://reviews.freebsd.org/D18908

show more ...

[libfetch] Fix compilation with WITHOUT_CRYPT.

Improve URL parsing. In particular, convert scheme and host to lowercase.

MFC after: 1 week

Make libfetch buildable.

Fix an inverted conditional in the netrc code, which would ignore the
value of $HOME and always use the home directory from the passwd
database, unless $HOME was unset, in which case it would use (nu

Fix an inverted conditional in the netrc code, which would ignore the
value of $HOME and always use the home directory from the passwd
database, unless $HOME was unset, in which case it would use (null).

While there, clean up handling of netrcfd and add debugging aids.

MFC after: 3 weeks

show more ...

Use __VA_ARGS__ to simplify the DEBUG macro.

MFC after: 3 weeks

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
pr

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

show more ...

In fetch_resolve(), if the port number or service name is included in
the host argument (e.g. "www.freebsd.org:443"), the service pointer,
which is supposed to point to the port or service part, inst

In fetch_resolve(), if the port number or service name is included in
the host argument (e.g. "www.freebsd.org:443"), the service pointer,
which is supposed to point to the port or service part, instead points
to the separator, causing getaddrinfo() to fail.

Note that I have not been able to trigger this bug with fetch(1), nor
do I believe it is possible, as libfetch always parses the host:port
specification itself. I discovered it when I copied fetch_resolve()
into an unrelated project.

MFC after: 3 days

show more ...

r308996 broke IP literals by assuming that a colon could only occur as
a separator between host and port, and using strchr() to search for it.
Rewrite fetch_resolve() so it handles bracketed literals

r308996 broke IP literals by assuming that a colon could only occur as
a separator between host and port, and using strchr() to search for it.
Rewrite fetch_resolve() so it handles bracketed literals correctly, and
remove similar code elsewhere to avoid passing unbracketed literals to
fetch_resolve(). Remove #ifdef INET6 so we still parse IP literals
correctly even if we do not have the ability to connect to them.

While there, fix an off-by-one error which caused HTTP 400 errors to be
misinterpreted as redirects.

PR: 217723
MFC after: 1 week
Reported by: bapt, bz, cem, ngie

show more ...

libfetch: extra bounds checking through reallocarray(3).

Reviewed by: des
MFC after: 1 week

Properly indent a default: label and avoid crashing when running
under -v but cannot connect due to trying to print an int as %s [1].

Reported by: andrew [1]
MFC after: 3 days

Add a file descriptor in struct url for netrc

When using libfetch in an application that drops privileges when fetching
like pkg(8) then user complain because the application does not read anymore
$

Add a file descriptor in struct url for netrc

When using libfetch in an application that drops privileges when fetching
like pkg(8) then user complain because the application does not read anymore
${HOME}/.netrc. Now a caller can prepare a fd to the said file and manually
assign it to the structure.

It is also a first step to allow to capsicumize libfetch applications

Reviewed by: allanjude, des
Approved by: des
Differential Revision: https://reviews.freebsd.org/D9678

show more ...

More debugging code I missed in r309051.

Reported by: jbeich, jkim