libc: regoranize malloc build

Create a stdlib/malloc to hold the definition of the malloc interface
(e.g., the Symbol.map file) and make jemalloc a subdirectory. This will
make it easier to integra

libc: regoranize malloc build

Create a stdlib/malloc to hold the definition of the malloc interface
(e.g., the Symbol.map file) and make jemalloc a subdirectory. This will
make it easier to integrate alternative allocators such as snmalloc
while making it clear that the current jemalloc symbols are the FreeBSD
API/ABI (for better or worse).

Suggested by: jrtc27
Reviewed by: jrtc27, emaste
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D41457

show more ...

libc: include malloc via stdlib/Makefile.inc

There's a hierarchy here and we should use it.

Improves: cbeacb7c46f3a3650e5dbefa9a1a18bc9943a8cc

Reviewed by: jrtc27, jhb, emaste
Sponsored by: DARPA

libc: include malloc via stdlib/Makefile.inc

There's a hierarchy here and we should use it.

Improves: cbeacb7c46f3a3650e5dbefa9a1a18bc9943a8cc

Reviewed by: jrtc27, jhb, emaste
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D41456

show more ...

Remove $FreeBSD$: one-line sh pattern

Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/

Revert "libc: Implement bsort(3) a bitonic type of sorting algorithm."

Some points for the future:
- libc is not the right place for sorting algorithms.
Probably libutil is better suited for thi

Revert "libc: Implement bsort(3) a bitonic type of sorting algorithm."

Some points for the future:
- libc is not the right place for sorting algorithms.
Probably libutil is better suited for this purpose or
a dedicated libsort. Should move all sorting algorithms
away from libc eventually.
- CheriBSD uses capabilities for memory access, and could
benefit from a standard memswap() function.
- Do something about qsort() in FreeBSD's libc like:
- Mark it deprecated on FreeBSD, as a first step,
due to missing limits on CPU time.
- Audit the use of qsort() in the FreeBSD base system
and consider swapping to other existing sorting
algorithms.

Discussed with: brooks@

Differential Revision: https://reviews.freebsd.org/D36493

This reverts commit a7469c9c0a504a5e6e9b89e148cd78df5e67ff7f.
This reverts commit 7d65a450cdcc7cc743f2ecd114ba3428a21c0033.
This reverts commit 8dcf3a82c54cb216df3213a013047907636a01da.

show more ...

libc: Implement bsort(3) a bitonic type of sorting algorithm.

The bsort(3) algorithm works by swapping objects, similarly to qsort(3),
and does not require any significant amount of additional memor

libc: Implement bsort(3) a bitonic type of sorting algorithm.

The bsort(3) algorithm works by swapping objects, similarly to qsort(3),
and does not require any significant amount of additional memory.

The bsort(3) algorithm doesn't suffer from the processing time issues
known the plague the qsort(3) family of algorithms, and is bounded by
a complexity of O(log2(N) * log2(N) * N), where N is the number of
elements in the sorting array. The additional complexity compared to
mergesort(3) is a fair tradeoff in situations where no memory may
be allocated.

The bsort(3) APIs are identical to those of qsort(3), allowing for
easy drop-in and testing.

The design of the bsort(3) algorithm allows for future parallell CPU
execution when sorting arrays. The current version of the bsort(3)
algorithm is single threaded. This is possible because fixed areas
of the sorting data is compared at a time, and can easily be divided
among different CPU's to sort large arrays faster.

Reviewed by: gbe@, delphij@, pauamma_gundo.com (manpages)
Sponsored by: NVIDIA Networking
Differential Revision: https://reviews.freebsd.org/D36493

show more ...

Add GNU glibc compatible secure_getenv

Add mostly glibc and msl compatible secure_getenv. Return NULL if
issetugid() indicates the process is tainted, otherwise getenv(x). The
rational behind this

Add GNU glibc compatible secure_getenv

Add mostly glibc and msl compatible secure_getenv. Return NULL if
issetugid() indicates the process is tainted, otherwise getenv(x). The
rational behind this is the fact that many Linux applications use this
function instead of getenv() as it's widely consider a, "best
practice".

Reviewed by: imp, mjg (feedback)
Pull Request: https://github.com/freebsd/freebsd-src/pull/686
Signed-off-by: Lucy Marsh <[email protected]>

show more ...

Alter the prototype of qsort_r(3) to match POSIX, which adopted the
glibc-based interface.

Unfortunately, the glibc maintainers, despite knowing the existence
of the FreeBSD qsort_r(3) interface in

Alter the prototype of qsort_r(3) to match POSIX, which adopted the
glibc-based interface.

Unfortunately, the glibc maintainers, despite knowing the existence
of the FreeBSD qsort_r(3) interface in 2004 and refused to add the
same interface to glibc based on grounds of the lack of standardization
and portability concerns, has decided it was a good idea to introduce
their own qsort_r(3) interface in 2007 as a GNU extension with a
slightly different and incompatible interface.

With the adoption of their interface as POSIX standard, let's switch
to the same prototype, there is no need to remain incompatible.

C++ and C applications written for the historical FreeBSD interface
get source level compatibility when building in C++ mode, or when
building with a C compiler with C11 generics support, provided that
the caller passes a fifth parameter of qsort_r() that exactly matches
the historical FreeBSD comparator function pointer type and does not
redefine the historical qsort_r(3) prototype in their source code.

Symbol versioning is used to keep old binaries working.

MFC: never
Relnotes: yes
Reviewed by: cem, imp, hps, pauamma
Differential revision: https://reviews.freebsd.org/D17083

show more ...

libc: add clearenv function

The clearenv(3) function allows us to clear all environment
variable in one shot. This may be useful for security programs that
want to control the environment or what va

libc: add clearenv function

The clearenv(3) function allows us to clear all environment
variable in one shot. This may be useful for security programs that
want to control the environment or what variables are passed to new
spawned programs.

Reviewed by: scf, markj (secteam), 0mp (manpages)
Differential Revision: https://reviews.freebsd.org/D28223

show more ...

libc qsort(3): Eliminate ambiguous sign comparison

The left side of the MIN() expression is the (signed) result of pointer
subtraction (ptrdiff_t). The right hand side is the also the (signed)
resu

libc qsort(3): Eliminate ambiguous sign comparison

The left side of the MIN() expression is the (signed) result of pointer
subtraction (ptrdiff_t). The right hand side is the also the (signed)
result of pointer subtraction, additionally subtracting the element size
('es'), which is unsigned size_t. This coerces the right-hand
expression into an unsigned value. MIN(signed, unsigned) triggers
-Wsign-compare.

Sorting elements of size greater than SSIZE_MAX is nonsensical, so we
can instead treat the element size as ssize_t, leaving the right-hand
result the same signedness as the left.

Reviewed by: arichardson, kib
Differential Revision: https://reviews.freebsd.org/D31292

show more ...

Implement ptsname_r.

MFC after: 2 weeks
PR: 250062
Reviewed by: jilles, 0mp, Ray <i maskray me>
Differential Revision: https://reviews.freebsd.org/D26647

Add qsort_s(3). Apart from the constraints, it also makes it easier
to port software written for Linux variant of qsort_r(3).

Reviewed by: kib, arichardson
MFC after: 2 weeks
Relnotes: yes
Sponsore

Add qsort_s(3). Apart from the constraints, it also makes it easier
to port software written for Linux variant of qsort_r(3).

Reviewed by: kib, arichardson
MFC after: 2 weeks
Relnotes: yes
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D23174

show more ...

Deprecate sranddev(3) API

It serves no useful purpose and wasn't as popular as its equally meritless
cousin, srandomdev(3).

Setting aside the problems with rand(3) in general, the problem with this

Deprecate sranddev(3) API

It serves no useful purpose and wasn't as popular as its equally meritless
cousin, srandomdev(3).

Setting aside the problems with rand(3) in general, the problem with this
interface is that the seed isn't shared with the caller (other than by
attacking the output of the generator, which is trivial, but not a hallmark of
pleasant API design). The (arguable) utility of rand(3) or random(3) is as a
semi-fast simulation generator which produces consistent results from a given
seed. These are mutually at odd. Furthermore, sometimes people got the
mistaken impression that a high quality random seed meant a weak generator like
rand(3) or random(3) could be used for things like cryptographic key
generation. This is absolutely not so.

The API was never part of a standard and was not widely used in tree. Existing
in-tree uses have all been removed.

Possible replacement in out of tree codebases:

char buf[3];
time_t t;

time(t);
strftime(buf, sizeof(buf), "%S", gmtime(&t));
srand(atoi(buf));

Relnotes: yes

show more ...

Provide set_constraint_handler_s(3) man page.

Mention abort_handler_s(3) and ignore_handler_s(3), provide
cross-reference from memset(3).

Submitted by: Yuri Pankov <[email protected]>
MFC after: 3

Provide set_constraint_handler_s(3) man page.

Mention abort_handler_s(3) and ignore_handler_s(3), provide
cross-reference from memset(3).

Submitted by: Yuri Pankov <[email protected]>
MFC after: 3 days
Differential revision: https://reviews.freebsd.org/D16797

show more ...

Use tab for indent.

Submitted by: Yuri Pankov <[email protected]>
MFC after: 3 days

Move strtold wrapper from strtol.c to its own strtold.c. This code
was written by theraven@ (David Chisnall) entirely, there's no
original Berkeley code left here so just copy his copyright over.

Implement the memset_s(3) function as specified by the C11 ISO/IEC
9899:2011 Appendix K 3.7.4.1.

Other needed supporting types, defines and constraint_handler
infrastructure is added as specified in

Implement the memset_s(3) function as specified by the C11 ISO/IEC
9899:2011 Appendix K 3.7.4.1.

Other needed supporting types, defines and constraint_handler
infrastructure is added as specified in the C11 spec.

Submitted by: Tom Rix <[email protected]>
Sponsored by: Juniper Networks
Discussed with: ed
MFC after: 3 weeks
Differential revision: https://reviews.freebsd.org/D9903
Differential revision: https://reviews.freebsd.org/D10161

show more ...

Export __cxa_thread_atexit_impl as an alias for __cxa_thread_atexit.

libstdc++ before gcc r244057 expected that libc provided
__cxa_thread_atexit_impl, and libstdc++ implemented
__cxa_thread_atexit,

Export __cxa_thread_atexit_impl as an alias for __cxa_thread_atexit.

libstdc++ before gcc r244057 expected that libc provided
__cxa_thread_atexit_impl, and libstdc++ implemented
__cxa_thread_atexit, by forwarding the calls to _impl. Mentioned gcc
revision checks for __cxa_thread_atexit in libc and does not provide
the symbol from libstdc++ if found.

This change helps older gcc, in particular, all released versions
which implement thread_local, by consolidating the implementation into
libc. For that versions, if configured with the current libc, the
__cxa_thread_atexit is exported from libstdc++ as a trivial wrapper
around libc::__cxa_thread_atexit_impl.

The __cxa_thread_atexit implementation is put into separate source
file to allow for static linking with older libstdc++.a.

gcc bugzilla: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78968
Reported by: Hannes Hauswedell <[email protected]>
PR: 215709
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks

show more ...

When MAKEOBJDIRPREFIX points to a case-insensitive file system, the
build can break when different source files create the same object
files (case-insensitivity speaking). This is the case for _Exit

When MAKEOBJDIRPREFIX points to a case-insensitive file system, the
build can break when different source files create the same object
files (case-insensitivity speaking). This is the case for _Exit.c
and _exit.s. Compile _Exit.c as C99_Exit.c

Reviewed by: sjg@
MFC after: completion
Sponsored by: Bracket Computing
Differential Revision: https://reviews.freebsd.org/D7893

show more ...

Add __cxa_thread_atexit(3) API implementation.

This is the backing feature to implement C++11 thread storage duration
specified by the thread_local keyword. A destructor for given
thread-local obje

Add __cxa_thread_atexit(3) API implementation.

This is the backing feature to implement C++11 thread storage duration
specified by the thread_local keyword. A destructor for given
thread-local object is registered to be executed at the thread
termination time using __cxa_thread_atexit(). Libc calls the
__cxa_thread_calls_dtors() during exit(3), before finalizers and
atexit functions, and libthr calls the function at the thread
termination time, after the stack unwinding and thread-specific key
destruction.

There are several uncertainties in the API which lacks a formal
specification. Among them:
- is it allowed to register destructors during destructing;
we allow, but limiting the nesting level. If too many iterations
detected, a diagnostic is issued to stderr and thread forcibly
terminates for now.
- how to handle destructors which belong to an unloading dso;
for now, we ignore destructor calls for such entries, and
issue a diagnostic. Linux does prevent dso unload until all
threads with destructors from the dso terminated.
It is supposed that the diagnostics allow to detect real-world
applications relying on the above details and possibly adjust
our implementation. Right now the choices were to provide the slim
API (but that rarely stands the practice test).

Tests are added to check generic functionality and to specify some of
the above implementation choices.

Submitted by: Mahdi Mokhtari <mokhi64_gmail.com>
Reviewed by: theraven
Discussed with: dim (detection of -std=c++11 supoort for tests)
Sponsored by: The FreeBSD Foundation (my involvement)
MFC after: 2 weeks
Differential revisions: https://reviews.freebsd.org/D7224,
https://reviews.freebsd.org/D7427

show more ...

Replace implementation of hsearch() by one that scales.

Traditionally the hcreate() function creates a hash table that uses
chaining, using a fixed user-provided size. The problem with this
approach

Replace implementation of hsearch() by one that scales.

Traditionally the hcreate() function creates a hash table that uses
chaining, using a fixed user-provided size. The problem with this
approach is that this often either wastes memory (table too big) or
yields bad performance (table too small). For applications it may not
always be easy to estimate the right hash table size. A fixed number
only increases performance compared to a linked list by a constant
factor.

This problem can be solved easily by dynamically resizing the hash
table. If the size of the hash table is at least doubled, this has no
negative on the running time complexity. If a dynamically sized hash
table is used, we can also switch to using open addressing instead of
chaining, which has the advantage of just using a single allocation for
the entire table, instead of allocating many small objects.

Finally, a problem with the existing implementation is that its
deterministic algorithm for hashing makes it possible to come up with
fixed patterns to trigger an excessive number of collisions. We can
easily solve this by using FNV-1a as a hashing algorithm in combination
with a randomly generated offset basis.

Measurements have shown that this implementation is about 20-25% faster
than the existing implementation (even if the existing implementation is
given an excessive number of buckets). Though it allocates more memory
through malloc() than the old implementation (between 4-8 pointers per
used entry instead of 3), process memory use is similar to the old
implementation as if the estimated size was underestimated by a factor
10. This is due to the fact that malloc() needs to perform less
bookkeeping.

Reviewed by: jilles, pfg
Obtained from: https://github.com/NuxiNL/cloudlibc
Differential Revision: https://reviews.freebsd.org/D4644

show more ...

Import reallocarray(3) from OpenBSD

Add a manpage for it, assign the copyright to the OpenBSD project on it since it
is mostly copy/paste from OpenBSD manpage.
style(9) fixes

Differential Revision:

Import reallocarray(3) from OpenBSD

Add a manpage for it, assign the copyright to the OpenBSD project on it since it
is mostly copy/paste from OpenBSD manpage.
style(9) fixes

Differential Revision: https://reviews.freebsd.org/D2420
Reviewed by: kib

show more ...

Add re-entrant versions of the hash functions based on the GNU api.

While testing this I found a conformance issue in hdestroy()
that will be fixed in a subsequent commit.

Obtained from: NetBSD (hc

Add re-entrant versions of the hash functions based on the GNU api.

While testing this I found a conformance issue in hdestroy()
that will be fixed in a subsequent commit.

Obtained from: NetBSD (hcreate.c, CVS Rev. 1.7)

show more ...

Add support for some block functions that come from OS X. These are
intended to build with any C compiler.

Reviewed by: pfg
MFC after: 3 weeks

Replace use of ${.CURDIR} by ${LIBC_SRCTOP} and define ${LIBC_SRCTOP}
if not already defined. This allows building libc from outside of
lib/libc using a reach-over makefile.

A typical use-case is to

Replace use of ${.CURDIR} by ${LIBC_SRCTOP} and define ${LIBC_SRCTOP}
if not already defined. This allows building libc from outside of
lib/libc using a reach-over makefile.

A typical use-case is to build a standard ILP32 version and a COMPAT32
version in a single iteration by building the COMPAT32 version using a
reach-over makefile.

Obtained from: Juniper Networks, Inc.

show more ...

Add more locale-specific functions to the relevant man pages and
Makefiles:
- libc/stdtime/strftime.3
- libc/stdtime/strptime.3
- libc/stdlib/strfmon.3

Reviewed by: theraven
Approved by: gabor (ment

Add more locale-specific functions to the relevant man pages and
Makefiles:
- libc/stdtime/strftime.3
- libc/stdtime/strptime.3
- libc/stdlib/strfmon.3

Reviewed by: theraven
Approved by: gabor (mentor)

show more ...