[llvm][json] Fix UINT64 json parsing

https://reviews.llvm.org/D109347 added support for UINT64 json numeric
types. However, it seems that it didn't properly test uint64_t numbers
larger than the int

[llvm][json] Fix UINT64 json parsing

https://reviews.llvm.org/D109347 added support for UINT64 json numeric
types. However, it seems that it didn't properly test uint64_t numbers
larger than the int64_t because the number parsing logic doesn't
have any special handling for these large numbers.

This diffs adds a handler for large numbers, and besides that, fixes the
parsing of signed types by checking for errno ERANGE, which is the
recommended way to check if parsing fails because of out of bounds
errors. Before this diff, strtoll was always returning a number within
the bounds of an int64_t and the bounds check it was doing was completely
superfluous.

As an interesting fact about the old implementation, when calling strtoll
with "18446744073709551615", the largest uint64_t, End was S.end(), even
though it didn't use all digits. Which means that this check can only be
used to identify if the numeric string is malformed or not.

This patch also adds additional tests for extreme cases.

Differential Revision: https://reviews.llvm.org/D125322

show more ...

[NFCI][Support] Avoid ASSERT_/EXPECT_TRUE(A <op> B)

The error messages in tests are far better when a test fails if the test
is written using ASSERT_/EXPECT_<operator>(A, B) rather than
ASSERT_/EXPE

[NFCI][Support] Avoid ASSERT_/EXPECT_TRUE(A <op> B)

The error messages in tests are far better when a test fails if the test
is written using ASSERT_/EXPECT_<operator>(A, B) rather than
ASSERT_/EXPECT_TRUE(A <operator> B).

This commit updates all of llvm/unittests/Support to use these macros
where possible.

This change has not been possible in:
- llvm/unittests/Support/FSUniqueIDTest.cpp - due to not overloading
operators beyond ==, != and <.
- llvm/unittests/Support/BranchProbabilityTest.cpp - where the unchanged
tests are of the operator overloads themselves.

There are other possibilities of this conversion not being valid, which
have not applied in these tests, as they do not use NULL (they use
nullptr), and they do not use const char* (they use std::string or
StringRef).

Reviewed By: mubashar_

Differential Revision: https://reviews.llvm.org/D117319

show more ...

[JSON] Handle uint64_t type

There was no handling of uint64_t in the LLVM JSON library.
This patch adds support for that. The motivation is
the https://reviews.llvm.org/D109217.

Differential Revisi

[JSON] Handle uint64_t type

There was no handling of uint64_t in the LLVM JSON library.
This patch adds support for that. The motivation is
the https://reviews.llvm.org/D109217.

Differential Revision: https://reviews.llvm.org/D109347

show more ...

[JSON] Add ObjectMapper::mapOptional to validate optional data.

Currently the idiom for mapping optional fields is:
ObjectMapper O(Val, P);
if (!O.map("required1", Out.R1) || !O.map("required2",

[JSON] Add ObjectMapper::mapOptional to validate optional data.

Currently the idiom for mapping optional fields is:
ObjectMapper O(Val, P);
if (!O.map("required1", Out.R1) || !O.map("required2", Out.R2))
return false;
O.map("optional1", Out.O1); // ignore result
return true;

If `optional1` is present but malformed, then we won't detect/report
that error. We may even leave `Out` in an incomplete state while returning true.
Instead, we'd often prefer to ignore `optional1` if it is absent, but otherwise
behave just like map().

Differential Revision: https://reviews.llvm.org/D89128

show more ...

[json] Provide a means to delegate writing a value to another API

(Based on D87170 by dsanders)

I recently had need to call out to an external API to emit a JSON object as part
of one an LLVM tool

[json] Provide a means to delegate writing a value to another API

(Based on D87170 by dsanders)

I recently had need to call out to an external API to emit a JSON object as part
of one an LLVM tool was emitting. However, our JSON support didn't provide a way
to delegate part of the JSON output to that API.

Add rawValueBegin() and rawValueEnd() to maintain and check the internal state
while something else is writing to the stream. It's the users responsibility to
ensure that the resulting JSON output is still valid.

Differential Revision: https://reviews.llvm.org/D88902

show more ...

[JSON] Add error reporting to fromJSON and ObjectMapper

Translating between JSON objects and C++ strutctures is common.
From experience in clangd, fromJSON/ObjectMapper work well and save a lot of
c

[JSON] Add error reporting to fromJSON and ObjectMapper

Translating between JSON objects and C++ strutctures is common.
From experience in clangd, fromJSON/ObjectMapper work well and save a lot of
code, but aren't adopted elsewhere at least partly due to total lack of error
reporting beyond "ok"/"bad".

The recently-added error model should be rich enough for most applications.
It requires tracking the path within the root object and reporting local
errors at appropriate places.
To do this, we exploit the fact that the call graph of recursive
parse functions mirror the structure of the JSON itself.
The current path is represented as a linked list of segments, each of which is
on the stack as a parameter. Concretely, fromJSON now looks like:
bool fromJSON(const Value&, T&, Path);

Beyond the signature change, this is reasonably unobtrusive: building
the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON.
However the root caller of fromJSON must now create a Root object to
store the errors, which is a little clunky.

I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not
general enough to be the primary interface I think (at least, not usable in
clangd).

All existing users (mostly just clangd) are updated in this patch,
making this change backwards-compatible is a bit hairy.

Differential Revision: https://reviews.llvm.org/D88103

show more ...

[JSON] Display errors associated with Paths in context

When an error occurs processing a JSON object, seeing the actual
surrounding data helps. Dumping just the node where the problem
was identified

[JSON] Display errors associated with Paths in context

When an error occurs processing a JSON object, seeing the actual
surrounding data helps. Dumping just the node where the problem
was identified can be too much or too little information.

printErrorContext() shows the error message in its context, as a comment.
JSON values along the path to the broken place are shown in some detail,
the rest of the document is elided. For example:

```
{
"credentials": [
{
"username": /* error: expected string */ 42,
"password": "secret"
},
{ ... }
]
"backups": { ... }
}
```

Differential Revision: https://reviews.llvm.org/D88103

show more ...

[JSON] Facility to track position within an object and report errors.

This error model should be rich enough for most applications. It comprises:

- a name for the root object, so the user knows wha

[JSON] Facility to track position within an object and report errors.

This error model should be rich enough for most applications. It comprises:

- a name for the root object, so the user knows what we're parsing
- a path from the root object to the JSON node most associated with the error
- a local error message

This can be presented as an llvm::Error e.g.
"expected string at ConfigFile.credentials[0].username"

It's designed to be cheap: Paths are a linked list of lightweight
objects on the stack. No heap allocations unless errors are encountered.

A subsequent commit will make use of this in the JSON-to-object
translation facilities: fromJSON and ObjectMapper.
However it's independent of these and can be used for e.g. validation alone.

Another subsequent commit will support showing the error in its context
within the parsed value.

Differential Revision: https://reviews.llvm.org/D88103

show more ...

[JSON] Allow emitting comments in json::OStream

This isn't standard JSON, but is a popular extension.
It will be used to show errors in context, rendering pseudo-json for humans.

Differential Revis

[JSON] Allow emitting comments in json::OStream

This isn't standard JSON, but is a popular extension.
It will be used to show errors in context, rendering pseudo-json for humans.

Differential Revision: https://reviews.llvm.org/D88103

show more ...

Revert "Disable a JSONTest.Integers test with all MSVCs, PR46470"

This reverts commit 63bcf89125fdfe8a41b0125454b006b49abb0184.
Turns out the more targeted disablement in the previous change
was goo

Revert "Disable a JSONTest.Integers test with all MSVCs, PR46470"

This reverts commit 63bcf89125fdfe8a41b0125454b006b49abb0184.
Turns out the more targeted disablement in the previous change
was good enough.

show more ...

Disable a JSONTest.Integers test with all MSVCs, PR46470

Disable a JSONTest.Integers test with newer MSVCs, PR46470

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly m

Make llvm::StringRef to std::string conversions explicit.

This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.

show more ...

[Support] Add erase() to json::Object

[Support] Add JSON streaming output API, faster where the heavy value types aren't needed.

Summary:
There's still a little bit of constant factor that could be trimmed (e.g.
more overloads to avoid

[Support] Add JSON streaming output API, faster where the heavy value types aren't needed.

Summary:
There's still a little bit of constant factor that could be trimmed (e.g.
more overloads to avoid round-tripping primitives through json::Value).
But this solves the memory scaling problem, and greatly improves the performance
constant factor, and the API should leave room for optimization if needed.

Adapt TimeProfiler to use it, eliminating almost all the performance regression
from r358476.

Performance test on my machine:
perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp

Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms
json::Value (HEAD): 2757ms (+11%)
After this patch: 2520 ms (+1.6%)

Reviewers: anton-afanasyev, lebedev.ri

Subscribers: kristina, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D60804

llvm-svn: 359186

show more ...

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the ne

Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636

show more ...

[Support] json::Value construction from std::vector<T> and std::map<string,T>.

Summary: Previously this required a conversion to json::Array/json::Object first.

Reviewers: ioeric

Subscribers: kris

[Support] json::Value construction from std::vector<T> and std::map<string,T>.

Summary: Previously this required a conversion to json::Array/json::Object first.

Reviewers: ioeric

Subscribers: kristina, llvm-commits

Differential Revision: https://reviews.llvm.org/D53385

llvm-svn: 344732

show more ...

[Support] Harded JSON against invalid UTF-8.

Parsing invalid UTF-8 input is now a parse error.
Creating JSON values from invalid UTF-8 now triggers an assertion, and
(in no-assert builds) substitute

[Support] Harded JSON against invalid UTF-8.

Parsing invalid UTF-8 input is now a parse error.
Creating JSON values from invalid UTF-8 now triggers an assertion, and
(in no-assert builds) substitutes the unicode replacement character.
Strings retrieved from json::Value are always valid UTF-8.

llvm-svn: 336657

show more ...

[Support] Allow JSON serialization of Optional<T> for supported T.

This is ported from r333881 to JSON's new home.

llvm-svn: 336542

[Support] Make JSON handle doubles and int64s losslessly

Summary:
This patch adds a new "integer" ValueType, and renames Number -> Double.
This allows us to preserve the full precision of int64_t wh

[Support] Make JSON handle doubles and int64s losslessly

Summary:
This patch adds a new "integer" ValueType, and renames Number -> Double.
This allows us to preserve the full precision of int64_t when parsing integers
from the wire, or constructing from an integer.
The API is unchanged, other than giving asInteger() a clearer contract.

In addition, always output doubles with enough precision that parsing will
reconstruct the same double.

Reviewers: simon_tatham

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46209

llvm-svn: 336541

show more ...

Lift JSON library from clang-tools-extra/clangd to llvm/Support.

Summary:
This consists of four main parts:
- an type json::Expr representing JSON values of dynamic kind, which can be
composed,

Lift JSON library from clang-tools-extra/clangd to llvm/Support.

Summary:
This consists of four main parts:
- an type json::Expr representing JSON values of dynamic kind, which can be
composed, inspected, and modified
- a JSON parser from string -> json::Expr
- a JSON printer from json::Expr -> string, with optional pretty-printing
- a convention for mapping json::Expr <=> native types (fromJSON/toJSON)
Mapping functions are provided for primitives (e.g. int, vector) and the
ObjectMapper helper helps implement fromJSON for struct/object types.

Based on clangd's usage, a couple of places I'd appreciate review attention:
- fromJSON returns only bool. A richer error-signaling mechanism may be useful
to provide useful messages, or let recursive fromJSONs (containers/structs)
do careful error recovery.
- should json::obj be always explicitly written (like json::ary)
- there's no streaming parse API. I suspect there are some simple wins like
a callback API where the document is a long array, and each element is small.
But this can probably be bolted on easily when we see the need.

Reviewers: bkramer, labath

Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits

Differential Revision: https://reviews.llvm.org/D45753

llvm-svn: 336534

show more ...