[pseudo] Eliminate LRTable::Action. NFC

The last remaining uses are in tests/test builders.
Replace with a builder struct.

Differential Revision: https://reviews.llvm.org/D129093

[pseudo] Store shift and goto actions in a compact structure with faster lookup.

The actions table is very compact but the binary search to find the
correct action is relatively expensive.
A hashtab

[pseudo] Store shift and goto actions in a compact structure with faster lookup.

The actions table is very compact but the binary search to find the
correct action is relatively expensive.
A hashtable is faster but pretty large (64 bits per value, plus empty
slots, and lookup is constant time but not trivial due to collisions).

The structure in this patch uses 1.25 bits per entry (whether present or absent)
plus the size of the values, and lookup is trivial.

The Shift table is 119KB = 27KB values + 92KB keys.
The Goto table is 86KB = 30KB values + 57KB keys.
(Goto has a smaller keyspace as #nonterminals < #terminals, and more entries).

This patch improves glrParse speed by 28%: 4.69 => 5.99 MB/s
Overall the table grows by 60%: 142 => 228KB.

By comparison, DenseMap<unsigned, StateID> is "only" 16% faster (5.43 MB/s),
and results in a 285% larger table (547 KB) vs the baseline.

Differential Revision: https://reviews.llvm.org/D128485

show more ...

[pseudo] Check follow-sets instead of tying reduce actions to lookahead tokens.

Previously, the action table stores a reduce action for each lookahead
token it should allow. These tokens are the fol

[pseudo] Check follow-sets instead of tying reduce actions to lookahead tokens.

Previously, the action table stores a reduce action for each lookahead
token it should allow. These tokens are the followSet(action.rule.target).

In practice, the follow sets are large, so we spend a bunch of time binary
searching around all these essentially-duplicates to check whether our lookahead
token is there.
However the number of reduces for a given state is very small, so we're
much better off linear scanning over them and performing a fast check for each.

D128318 was an attempt at this, storing a bitmap for each reduce.
However it's even more compact just to use the follow sets directly, as
there are fewer nonterminals than (state, rule) pairs. It's also faster.

This specialized approach means unbundling Reduce from other actions in
LRTable, so it's no longer useful to support it in Action. I suspect
Action will soon go away, as we store each kind of action separately.

This improves glrParse speed by 42% (3.30 -> 4.69 MB/s).
It also reduces LR table size by 59% (343 -> 142kB).

Differential Revision: https://reviews.llvm.org/D128472

show more ...

Reland "[pseudo] Track heads as GSS nodes, rather than as "pending actions"."

This reverts commit 2c80b5319870b57fbdbb6c9cef9c86c26c65371d.

Fixes LRTable::buildForTest to create states that are ref

Reland "[pseudo] Track heads as GSS nodes, rather than as "pending actions"."

This reverts commit 2c80b5319870b57fbdbb6c9cef9c86c26c65371d.

Fixes LRTable::buildForTest to create states that are referenced but
have no actions.

show more ...

Revert "[pseudo] Track heads as GSS nodes, rather than as "pending actions"."

This reverts commit e3ec054dfdf48f19cb6726cb3f4965b9ab320ed9.

Tests fail in asserts mode: https://lab.llvm.org/buildbot

Revert "[pseudo] Track heads as GSS nodes, rather than as "pending actions"."

This reverts commit e3ec054dfdf48f19cb6726cb3f4965b9ab320ed9.

Tests fail in asserts mode: https://lab.llvm.org/buildbot/#/builders/109/builds/41217

show more ...

[pseudo] Track heads as GSS nodes, rather than as "pending actions".

IMO this model is simpler to understand (borrowed from the LR0 patch D127357).
It also makes error recovery easier to implement,

[pseudo] Track heads as GSS nodes, rather than as "pending actions".

IMO this model is simpler to understand (borrowed from the LR0 patch D127357).
It also makes error recovery easier to implement, as we have a simple list of
head nodes lying around to recover from when needed.
(It's not quite as nice as LR0 in this respect though).

It's slightly slower (2.24 -> 2.12 MB/S on my machine = 5%) but nothing close
to as bad as LR0.
However
- I think we'd have to eat a litle performance loss otherwise to implement
error recovery.
- this frees up some complexity budget for optimizations like fastpath push/pop
(this + fastpath is already faster than head)
- I haven't changed the data structure here and it's now pretty dumb, we can
make it faster

Differential Revision: https://reviews.llvm.org/D128297

show more ...

[pseudo] Move grammar-related headers to a separate dir, NFC.

We did that for .cpp, but forgot the headers.

Differential Revision: https://reviews.llvm.org/D127388

[pseudo] Remove the explicit Accept actions.

As pointed out in the previous review section, having a dedicated accept
action doesn't seem to be necessary. This patch implements the the same behavior

[pseudo] Remove the explicit Accept actions.

As pointed out in the previous review section, having a dedicated accept
action doesn't seem to be necessary. This patch implements the the same behavior
without accept acction, which will save some code complexity.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D125677

show more ...

[pseudo] Fix a sign-compare warning in debug build, NFC.

[pseudo] Restore accidentally removed debug print

[pseudo] Invert rows/columns of LRTable storage for speedup. NFC

There are more states than symbols.
This means first partioning the action list by state leaves us with a smaller
range to binary sea

[pseudo] Invert rows/columns of LRTable storage for speedup. NFC

There are more states than symbols.
This means first partioning the action list by state leaves us with a smaller
range to binary search over. This improves find() a lot and glrParse() by 7%.
The tradeoff is storing more smaller ranges increases the size of the offsets
array, overall grammar memory is +1% (337->340KB).

Before:
glrParse 188795975 ns 188778003 ns 77 bytes_per_second=1.98068M/s
After:
glrParse 175936203 ns 175916873 ns 81 bytes_per_second=2.12548M/s

Differential Revision: https://reviews.llvm.org/D127006

show more ...

[pseudo] A basic implementation of compiling cxx grammar at build time.

The main idea is to compile the cxx grammar at build time, and construct
the core pieces (Grammar, LRTable) of the pseudoparse

[pseudo] A basic implementation of compiling cxx grammar at build time.

The main idea is to compile the cxx grammar at build time, and construct
the core pieces (Grammar, LRTable) of the pseudoparse based on the compiled
data sources.

This is a tiny implementation, which is good for start:

- defines how the public API should look like;
- integrates the cxx grammar compilation workflow with the cmake system.
- onlynonterminal symbols of the C++ grammar are compiled, anything
else are still doing the real compilation work at runtime, we can opt-in more
bits in the future;
- splits the monolithic clangPsuedo library for better layering;

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D125667

show more ...