[libc++][format] Exposes basic-format-string

This paper was accepted during the last plenary and is intended to be
backported to LLVM 15. When backporting the release notes in the branch
should be u

[libc++][format] Exposes basic-format-string

This paper was accepted during the last plenary and is intended to be
backported to LLVM 15. When backporting the release notes in the branch
should be updated too.

Note the feature-test macro isn't updated since this will change; three
papers have updated the same macro in the same plenary.

Implements:
- P2508R1 Exposing std::basic-format-string

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D130643

(cherry picked from commit f712775dafdb221fdf73f38819fa9e618aec6b67)

show more ...

[libc++] Implements Unicode grapheme clustering

This implements the Grapheme clustering as required by
P1868R2 width: clarifying units of width and precision in std::format

This was omitted in the

[libc++] Implements Unicode grapheme clustering

This implements the Grapheme clustering as required by
P1868R2 width: clarifying units of width and precision in std::format

This was omitted in the initial patch, but the paper was marked as completed. This really completes the paper.

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D126971

show more ...

[libc++] Treat incomplete features just like other experimental features

In particular remove the ability to expel incomplete features from the
library at configure-time, since this can now be done

[libc++] Treat incomplete features just like other experimental features

In particular remove the ability to expel incomplete features from the
library at configure-time, since this can now be done through the
_LIBCPP_ENABLE_EXPERIMENTAL macro.

Also, never provide symbols related to incomplete features inside the
dylib, instead provide them in c++experimental.a (this changes the
symbols list, but not for any configuration that should have shipped).

Differential Revision: https://reviews.llvm.org/D128928

show more ...

[libc++][format] Improves parsing speed.

A format string like "{}" is quite common. In this case avoid parsing
the format-spec when it's not present. Before the parsing was always
called, therefore

[libc++][format] Improves parsing speed.

A format string like "{}" is quite common. In this case avoid parsing
the format-spec when it's not present. Before the parsing was always
called, therefore some refactoring is done to make sure the formatters
work properly when their parse member isn't called.

From the wording it's not entirely clear whether this optimization is
allowed

[tab:formatter]
```
and the range [pc.begin(), pc.end()) from the last call to f.parse(pc).
```
Implies there's always a call to `f.parse` even when the format-spec
isn't present. Therefore this optimization isn't done for handle
classes; it's unclear whether that would break user defined formatters.

The improvements give a small reduciton is code size:
719408 12472 488 732368 b2cd0 before
718824 12472 488 731784 b2a88 after

The performance benefits when not using a format-spec are:

```
Comparing ./formatter_int.libcxx.out-baseline to ./formatter_int.libcxx.out
Benchmark Time CPU Time Old Time New CPU Old CPU New
----------------------------------------------------------------------------------------------------------------------------------------------------
BM_Basic<uint32_t> -0.0688 -0.0687 67 62 67 62
BM_Basic<int32_t> -0.1105 -0.1107 73 65 73 65
BM_Basic<uint64_t> -0.1053 -0.1049 95 85 95 85
BM_Basic<int64_t> -0.0889 -0.0888 93 85 93 85
BM_BasicLow<__uint128_t> -0.0655 -0.0655 96 90 96 90
BM_BasicLow<__int128_t> -0.0693 -0.0694 97 90 97 90
BM_Basic<__uint128_t> -0.0359 -0.0359 256 247 256 247
BM_Basic<__int128_t> -0.0414 -0.0414 239 229 239 229
```

For the cases where a format-spec is used the results remain similar,
some are faster some are slower, differing per run.

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D129426

show more ...

[libc++][format] Use forwarding references.

This implements a not accepted LWG issue. Not doing so would require
integral types to use the handle class instead of being directly stored
in the basic_

[libc++][format] Use forwarding references.

This implements a not accepted LWG issue. Not doing so would require
integral types to use the handle class instead of being directly stored
in the basic_format_arg.

The previous code used `std::forward` in places where it wasn't required
by the Standard. These are now removed.

Implements:
- P2418R2 Add support for std::generator-like types to std::format
- LWG 3631 basic_format_arg(T&&) should use remove_cvref_t<T> throughout

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D127570

show more ...

[libc++][format] Improve string formatters

This changes the implementation of the formatter. Instead of inheriting
from a specialized parser all formatters will use the same generic
parser. This red

[libc++][format] Improve string formatters

This changes the implementation of the formatter. Instead of inheriting
from a specialized parser all formatters will use the same generic
parser. This reduces the binary size.

The new parser contains some additional fields only used in the chrono
formatting. Since this doesn't change the size of the parser the fields
are in the generic parser. The parser is designed to fit in 128-bit,
making it cheap to pass by value.

The new format function is a const member function. This isn't required
by the Standard yet, but it will be after LWG-3636 is accepted.
Additionally P2286 adds a formattable concept which requires the member
function to be const qualified in C++23. This paper is likely to be
accepted in the 2022 July plenary.

Depends on D121530

NOTE parts of the code now contains duplicates for the current and new parser.
The intention is to remove the duplication in followup patches. A general
overview of the final code is available in D124620. That review however lacks a
bit of polish.

Most of the new code is based on the same algorithms used in the current code.

The final version of this code reduces the binary size by 17 KB for this example
code
```

int main() {
{
std::string_view sv{"hello world"};
std::format("{}{}|{}{}{}{}{}{}|{}{}{}{}{}{}|{}{}{}|{}{}|{}", true, '*',
(signed char)(42), (short)(42), (int)(42), (long)(42), (long long)(42), (__int128_t)(42),
(unsigned char)(42), (unsigned short)(42), (unsigned int)(42), (unsigned long)(42),
(unsigned long long)(42), (__uint128_t)(42),
(float)(42), (double)(42), (long double)(42),
"hello world", sv,
nullptr);
}
{
std::wstring_view sv{L"hello world"};
std::format(L"{}{}|{}{}{}{}{}{}|{}{}{}{}{}{}|{}{}{}|{}{}|{}", true, L'*',
(signed char)(42), (short)(42), (int)(42), (long)(42), (long long)(42), (__int128_t)(42),
(unsigned char)(42), (unsigned short)(42), (unsigned int)(42), (unsigned long)(42),
(unsigned long long)(42), (__uint128_t)(42),
(float)(42), (double)(42), (long double)(42),
L"hello world", sv,
nullptr);
}
}

```

Reviewed By: #libc, ldionne

Differential Revision: https://reviews.llvm.org/D125606

show more ...

[libc++] Removes unneeded <iterator> includes.

Reviewed By: #libc, philnik

Differential Revision: https://reviews.llvm.org/D127675

[libc++][format] Implement format-string.

Implements the compile-time checking of the formatting arguments.

Completes:
- P2216 std::format improvements

Reviewed By: #libc, ldionne

Differential Re

[libc++][format] Implement format-string.

Implements the compile-time checking of the formatting arguments.

Completes:
- P2216 std::format improvements

Reviewed By: #libc, ldionne

Differential Revision: https://reviews.llvm.org/D121530

show more ...

[libc++] Granularize <iterator> includes

Reviewed By: ldionne, #libc

Spies: libcxx-commits, wenlei

Differential Revision: https://reviews.llvm.org/D127445

[libc++][format][5/6] Improve format_to_n.

Use a specialized buffer wrapper to limit the number of insertions in the
buffer. After the limit has been reached the buffer only needs to count
the numbe

[libc++][format][5/6] Improve format_to_n.

Use a specialized buffer wrapper to limit the number of insertions in the
buffer. After the limit has been reached the buffer only needs to count
the number of insertions to return the buffer size required to store the
entire output.

Depends on D110498

Reviewed By: #libc, Mordante

Differential Revision: https://reviews.llvm.org/D110499

show more ...

[libc++][format] Improve format-arg-store.

This optimizes the __format_arg_store type to allow a more efficient
storage of the basic_format_args.

It stores the data in two arrays:
- A struct with t

[libc++][format] Improve format-arg-store.

This optimizes the __format_arg_store type to allow a more efficient
storage of the basic_format_args.

It stores the data in two arrays:
- A struct with the tag of the exposition only variant's type and the
offset of the element in the data array. Since this array only depends
on the type information it's calculated at compile time and can be
shared by different instances of this class.
- The arguments converted to the types used in the exposition only
variant of basic_format_arg. This means the packed data can be
directly copied to an element of this variant.

The new code uses rvalue reference arguments in preparation for P2418.
The handle class also has some changes to prepare for P2418. The real
changed for P2418 will be done separately, but these parts make it
easier to implement that paper.

Some parts of existing test code are removed since they were no longer
valid after the changes, but new tests have been added.

Implements parts of:
- P2418 Add support for std::generator-like types to std::format

Completes:
- LWG3473 Normative encouragement in non-normative note

Depends on D121138

Reviewed By: #libc, vitaut, Mordante

Differential Revision: https://reviews.llvm.org/D121514

show more ...

[libc++][format] Adds a formattable concept.

The concept is based on P2286R2 Formatting Ranges. It will be used to
optimise the storage of __format_arg_store as required by LWG-3473.

Depends on D12

[libc++][format] Adds a formattable concept.

The concept is based on P2286R2 Formatting Ranges. It will be used to
optimise the storage of __format_arg_store as required by LWG-3473.

Depends on D120916

Reviewed By: #libc, Mordante

Differential Revision: https://reviews.llvm.org/D120921

show more ...

[libc++][format][4/6] Improve formatted_size.

Use a specialized "buffer" to count the number of insertions instead of
using a `string` as storage type.

Depends on D110497.

Reviewed By: ldionne, #l

[libc++][format][4/6] Improve formatted_size.

Use a specialized "buffer" to count the number of insertions instead of
using a `string` as storage type.

Depends on D110497.

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D110498

show more ...

[libc++][format][3/6] Adds a __container_buffer.

Instead of writing every character directly into the container by using
a `back_insert_iterator` the data is buffered in an `array`. This buffer
is t

[libc++][format][3/6] Adds a __container_buffer.

Instead of writing every character directly into the container by using
a `back_insert_iterator` the data is buffered in an `array`. This buffer
is then inserted to the container by calling its `insert` member function.

Since there's no guarantee every container's `insert` behaves properly
containers need to opt-in to this behaviour. The appropriate standard
containers opt-in to this behaviour.

This change improves the performance of the format functions that use a
`back_insert_iterator`.

Depends on D110495

Reviewed By: ldionne, vitaut, #libc

Differential Revision: https://reviews.llvm.org/D110497

show more ...

[libc++] Ensure that all public C++ headers include <__assert>

This patch changes the requirement for getting the declaration of the
assertion handler from including <__assert> to including any publ

[libc++] Ensure that all public C++ headers include <__assert>

This patch changes the requirement for getting the declaration of the
assertion handler from including <__assert> to including any public
C++ header of the library. Note that C compatibility headers are
excluded because we don't implement all the C headers ourselves --
some of them are taken straight from the C library, like assert.h.

It also adds a generated test to check it. Furthermore, this new
generated test is designed in a way that will make it possible to
replace almost all the existing test-generation scripts with this
system in upcoming patches.

Differential Revision: https://reviews.llvm.org/D122506

show more ...

[libc++][format][2/6] Adds a __output_iterator.

Instead of using a temporary `string` in `__vformat_to_wrapped` use a new
generic iterator. This aids to reduce the number of template instantions
and

[libc++][format][2/6] Adds a __output_iterator.

Instead of using a temporary `string` in `__vformat_to_wrapped` use a new
generic iterator. This aids to reduce the number of template instantions
and avoids using a `string` to buffer the entire formatted output.

This changes the type of `format_context` and `wformat_context`, this can
still be done since the code isn't ABI stable yet.

Several approaches have been evaluated:
- Using a __output_buffer base class with:
- a put function to store the buffer in its internal buffer
- a virtual flush function to copy the internal buffer to the output
- Using a `function` to forward the output operation to the output buffer,
much like the next method.
- Using a type erased function point to store the data in the buffer.
The last version resulted in the best performance. For some cases there's
still a loss of speed over the original method. This loss many becomes
apparent when large strings are copied to a pointer like iterator, before
the compiler optimized this using `memcpy`.

Reviewed By: ldionne, vitaut, #libc

Differential Revision: https://reviews.llvm.org/D110495

show more ...

[libc++] Replace _LIBCPP_HAS_NO_CONCEPTS with _LIBCPP_STD_VER > 17. NFCI.

All supported compilers that support C++20 now support concepts. So, remove
`_LIB_LIBCPP_HAS_NO_CONCEPTS` in favor of `_LIBC

[libc++] Replace _LIBCPP_HAS_NO_CONCEPTS with _LIBCPP_STD_VER > 17. NFCI.

All supported compilers that support C++20 now support concepts. So, remove
`_LIB_LIBCPP_HAS_NO_CONCEPTS` in favor of `_LIBCPP_STD_VER > 17`. Similarly in
the tests, remove `// UNSUPPORTED: libcpp-no-concepts`.

Differential Revision: https://reviews.llvm.org/D121528

show more ...

[libc++] Granularize algorithm includes

Reviewed By: Mordante, ldionne, Quuxplusone, #libc, #libc_abi

Spies: #libc_vendors, libcxx-commits, miyuki

Differential Revision: https://reviews.llvm.org/D

[libc++] Granularize algorithm includes

Reviewed By: Mordante, ldionne, Quuxplusone, #libc, #libc_abi

Spies: #libc_vendors, libcxx-commits, miyuki

Differential Revision: https://reviews.llvm.org/D119667

show more ...

[libc++][format][nfc] Header cleanup.

Remove the unneeded macro protection, forward declarations, and
includes.

Reviewed By: #libc, Quuxplusone, ldionne, philnik

Differential Revision: https://rev

[libc++][format][nfc] Header cleanup.

Remove the unneeded macro protection, forward declarations, and
includes.

Reviewed By: #libc, Quuxplusone, ldionne, philnik

Differential Revision: https://reviews.llvm.org/D118925

show more ...

[libc++] Normalize all our '#pragma GCC system_header', and regression-test.

Now we'll notice if a header forgets to include this magic phrase.

Differential Revision: https://reviews.llvm.org/D1188

[libc++] Normalize all our '#pragma GCC system_header', and regression-test.

Now we'll notice if a header forgets to include this magic phrase.

Differential Revision: https://reviews.llvm.org/D118800

show more ...

[libc++][format] Finish P0645 Text Formatting.

This adjust the version macro and sets it as completed. All parts of the paper
have been implemented, except for the parts replaced by later papers and

[libc++][format] Finish P0645 Text Formatting.

This adjust the version macro and sets it as completed. All parts of the paper
have been implemented, except for the parts replaced by later papers and
LWG-issues.

Adjusted the synopsis to match the synopsis in the Standard. Not yet
implemented parts of P2216 and P2418 still use the P0645 wording.

Completes:
- P0645 Text Formatting

Depends on D115991

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D115999

show more ...

[libc++][format] Adds formatter handle.

This implements the handler according to P0645. P2418 changes the wording
in the Standard. That isn't implemented and requires changes in more
places. LWG3631

[libc++][format] Adds formatter handle.

This implements the handler according to P0645. P2418 changes the wording
in the Standard. That isn't implemented and requires changes in more
places. LWG3631 applies modifications to P2418, but is currently
unresolved.

Implements parts of:
* P0645 Text Formatting

Depends on D115989

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D115991

show more ...

[libc++][format] Adds formatter pointer.

This implements the last required formatter specialization.

Completes:
- LWG 3251 Are std::format alignment specifiers applied to string arguments?
- LWG 33

[libc++][format] Adds formatter pointer.

This implements the last required formatter specialization.

Completes:
- LWG 3251 Are std::format alignment specifiers applied to string arguments?
- LWG 3340 Formatting functions should throw on argument/format string mismatch in §[format.functions]
- LWG 3540 §[format.arg] There should be no const in basic_format_arg(const T* p)

Implements parts of:
- P0645 Text Formatting

Depends on D114001

Reviewed By: ldionne, vitaut, #libc

Differential Revision: https://reviews.llvm.org/D115988

show more ...

[libc++][format] Adds formatter floating-point.

This properly implements the formatter for floating-point types.

Completes:
- P1652R1 Printf corner cases in std::format
- LWG 3250 std::format: # (a

[libc++][format] Adds formatter floating-point.

This properly implements the formatter for floating-point types.

Completes:
- P1652R1 Printf corner cases in std::format
- LWG 3250 std::format: # (alternate form) for NaN and inf
- LWG 3243 std::format and negative zeroes

Implements parts of:
- P0645 Text Formatting

Reviewed By: #libc, ldionne, vitaut

Differential Revision: https://reviews.llvm.org/D114001

show more ...

[libc++][format][1/6] Reduce binary size.

This removes the `format_args_t` from `<format>` and adjusts the type of
the `format_args` for the `vformat_to` overloads.

The `format_context` uses a `bac

[libc++][format][1/6] Reduce binary size.

This removes the `format_args_t` from `<format>` and adjusts the type of
the `format_args` for the `vformat_to` overloads.

The `format_context` uses a `back_insert_iterator<string>` therefore the
new `output_iterator` function uses a `string` as its temporary storage
buffer. This isn't ideal. The next patches in this series will improve
this. These improvements make it easy to also improve `format_to_n` and
`formatted_size`.

This addresses P2216 `6. Binary size`.
P2216 `5. Compile-time checks` are not part of this change.

Implements parts of:
- P2216 std::format improvements

Depends on D103670

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D110494

show more ...