Remove support for SSLv3 from fetch(3).

Support for SSLv3 was already removed from OpenSSL (r361392).

Differential Revision: https://reviews.freebsd.org/D24947

Replace literal uses of /usr/local in C sources with _PATH_LOCALBASE

Literal references to /usr/local exist in a large number of files in
the FreeBSD base system. Many are in contributed software, i

Replace literal uses of /usr/local in C sources with _PATH_LOCALBASE

Literal references to /usr/local exist in a large number of files in
the FreeBSD base system. Many are in contributed software, in configuration
files, or in the documentation, but 19 uses have been identified in C
source files or headers outside the contrib and sys/contrib directories.

This commit makes it possible to set _PATH_LOCALBASE in paths.h to use
a different prefix for locally installed software.

In order to avoid changes to openssh source files, LOCALBASE is passed to
the build via Makefiles under src/secure. While _PATH_LOCALBASE could have
been used here, there is precedent in the construction of the path used to
a xauth program which depends on the LOCALBASE value passed on the compiler
command line to select a non-default directory.

This could be changed in a later commit to make the openssh build
consistently use _PATH_LOCALBASE. It is considered out-of-scope for this
commit.

Reviewed by: imp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D26942

show more ...

fetch(3): plug some leaks

In the successful case, sockshost is not freed prior to return.

The failure case can now be hit after fetch_reopen(), which was not true
before. Thus, we need to make sure

fetch(3): plug some leaks

In the successful case, sockshost is not freed prior to return.

The failure case can now be hit after fetch_reopen(), which was not true
before. Thus, we need to make sure to clean up all of the conn resources
which will also close sd. For all of the points prior to fetch_reopen(), we
continue to just close sd.

CID: 1419598, 1419616

show more ...

fetch(3): don't leak sockshost on failure

fetch_socks5_getenv will allocate memory for the host (or set it to NULL) in
all cases through the function; the caller is responsible for freeing it if
we

fetch(3): don't leak sockshost on failure

fetch_socks5_getenv will allocate memory for the host (or set it to NULL) in
all cases through the function; the caller is responsible for freeing it if
we end up allocating.

While I'm here, I've eliminated a label that just jumps to the next line...

show more ...

fetch(3): fix regression in IPv6:port spec from r357977

In case the port was specified, we never actually populated *host. Do so
now.

Pointy hat: kevans

fetch(3): move bits of fetch_socks5_getenv around

This commit separates out port parsing and validation from grabbing the host
from the env var. The only related bit really is that we need to be mor

fetch(3): move bits of fetch_socks5_getenv around

This commit separates out port parsing and validation from grabbing the host
from the env var. The only related bit really is that we need to be more
specific with the delimiter in the IPv6 case.

show more ...

fetch(3): Add SOCKS5 support

This change adds SOCKS5 support to the library fetch(3) and updates the man
page.

Details: Within the fetch_connect() function, fetch(3) checks if the
SOCKS5_PROXY envi

fetch(3): Add SOCKS5 support

This change adds SOCKS5 support to the library fetch(3) and updates the man
page.

Details: Within the fetch_connect() function, fetch(3) checks if the
SOCKS5_PROXY environment variable is set. If so, it connects to this host
rather than the end-host. It then initializes the SOCKS5 connection in
accordance with RFC 1928 and returns the resulting conn_t (file descriptor)
for usage by the regular FTP/HTTP handlers.

Design Decision: This change defaults all DNS resolutions through the proxy
by sending all IPs as hostnames. Going forward, another feature might be to
create another environmental variable to toggle resolutions through the
proxy or not..

One may set the SOCKS5_PROXY environment variable in any of the formats:

SOCKS5_PROXY=proxy.example.com
SOCKS5_PROXY=proxy.example.com:1080
SOCKS5_PROXY=192.0.2.0
SOCKS5_PROXY=198.51.100.0:1080
SOCKS5_PROXY=[2001:db8::1]
SOCKS5_PROXY=[2001:db8::2]:1080

Then perform a request with fetch(1).

(note by kevans)
I've since been informed that Void Linux/xbps has a fork of libfetch that
also implements SOCKS5. I may compare/contrast the two in the mid-to-near
future.

Submitted by: Farhan Khan <farhan farhan codes>
Differential Revision: https://reviews.freebsd.org/D18908

show more ...

[libfetch] Fix compilation with WITHOUT_CRYPT.

Improve URL parsing. In particular, convert scheme and host to lowercase.

MFC after: 1 week

Make libfetch buildable.

Fix an inverted conditional in the netrc code, which would ignore the
value of $HOME and always use the home directory from the passwd
database, unless $HOME was unset, in which case it would use (nu

Fix an inverted conditional in the netrc code, which would ignore the
value of $HOME and always use the home directory from the passwd
database, unless $HOME was unset, in which case it would use (null).

While there, clean up handling of netrcfd and add debugging aids.

MFC after: 3 weeks

show more ...

Use __VA_ARGS__ to simplify the DEBUG macro.

MFC after: 3 weeks

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
pr

lib: further adoption of SPDX licensing ID tags.

Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

show more ...

In fetch_resolve(), if the port number or service name is included in
the host argument (e.g. "www.freebsd.org:443"), the service pointer,
which is supposed to point to the port or service part, inst

In fetch_resolve(), if the port number or service name is included in
the host argument (e.g. "www.freebsd.org:443"), the service pointer,
which is supposed to point to the port or service part, instead points
to the separator, causing getaddrinfo() to fail.

Note that I have not been able to trigger this bug with fetch(1), nor
do I believe it is possible, as libfetch always parses the host:port
specification itself. I discovered it when I copied fetch_resolve()
into an unrelated project.

MFC after: 3 days

show more ...

r308996 broke IP literals by assuming that a colon could only occur as
a separator between host and port, and using strchr() to search for it.
Rewrite fetch_resolve() so it handles bracketed literals

r308996 broke IP literals by assuming that a colon could only occur as
a separator between host and port, and using strchr() to search for it.
Rewrite fetch_resolve() so it handles bracketed literals correctly, and
remove similar code elsewhere to avoid passing unbracketed literals to
fetch_resolve(). Remove #ifdef INET6 so we still parse IP literals
correctly even if we do not have the ability to connect to them.

While there, fix an off-by-one error which caused HTTP 400 errors to be
misinterpreted as redirects.

PR: 217723
MFC after: 1 week
Reported by: bapt, bz, cem, ngie

show more ...

libfetch: extra bounds checking through reallocarray(3).

Reviewed by: des
MFC after: 1 week

Properly indent a default: label and avoid crashing when running
under -v but cannot connect due to trying to print an int as %s [1].

Reported by: andrew [1]
MFC after: 3 days

Add a file descriptor in struct url for netrc

When using libfetch in an application that drops privileges when fetching
like pkg(8) then user complain because the application does not read anymore
$

Add a file descriptor in struct url for netrc

When using libfetch in an application that drops privileges when fetching
like pkg(8) then user complain because the application does not read anymore
${HOME}/.netrc. Now a caller can prepare a fd to the said file and manually
assign it to the structure.

It is also a first step to allow to capsicumize libfetch applications

Reviewed by: allanjude, des
Approved by: des
Differential Revision: https://reviews.freebsd.org/D9678

show more ...

More debugging code I missed in r309051.

Reported by: jbeich, jkim

Remove debugging code.

Refactor fetch_connect() and fetch_bind() to improve readability and avoid
repeating the same DNS lookups.

MFC after: 3 weeks

Don't leak addrinfo in fetch_bind()

Reported by: Coverity
CID: 1225038
MFC after: 1 week

lib: minor spelling fixes in comments.

No functional change.

Test for /etc/ssl/cert.pem existence to avoid masking SSL_CA_CERT_PATH

Prior to this patch, unless SSL_CA_CERT_FILE is set in the environment,
libfetch will set the CA file to "/usr/local/etc/cert.p

Test for /etc/ssl/cert.pem existence to avoid masking SSL_CA_CERT_PATH

Prior to this patch, unless SSL_CA_CERT_FILE is set in the environment,
libfetch will set the CA file to "/usr/local/etc/cert.pem" if it exists,
and to "/etc/ssl/cert.pem" otherwise. This has the consequence of
masking SSL_CA_CERT_PATH, because OpenSSL will ignore the CA path if a CA
file is set but fails to load (see X509_STORE_load_locations()).

While here, fall back to OpenSSL defaults if neither SSL_CA_CERT_FILE nor
SSL_CA_CERT_PATH are set in the environment, and if neither of the
libfetch default CA files exists.

PR: 193871
Submitted by: John W. O'Brien <[email protected]>
Approved by: des
MFC after: 1 week

show more ...

Fix non-POSIX-compliant use of getaddrinfo in libfetch

Submitted by: Boris Kolpackov <[email protected]>
Reviewed by: bapt
Approved by: bapt
MFC after: 1 week
Differential Revision: https://re

Fix non-POSIX-compliant use of getaddrinfo in libfetch

Submitted by: Boris Kolpackov <[email protected]>
Reviewed by: bapt
Approved by: bapt
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D3724

show more ...