en Copyright 2015 Java http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#1662cfa4 [CSSPGO][CSProfileConverter] Remove call target samples when including callee samples into caller.When a flat CS profile is converted to a nested profile, the call target samples for inlined callee contexts are left over in the callsite target map. This could cause indirect call promotion to function improperly. One issue is that the inlined callsites are treated with double amount of samples. The other is the inlined callsites are reconsidered for subsequent PGO ICP.I'm fixing this by excluding call targets from the callsite for inlined targets. While fixing this I found that callsite target sum and the number of body samples for that callsite could be mismatched. {D122609} has an explanation and a fix for that on llvm-profgen side. For now I'm tolerating it in this change.Reviewed By: wenleiDifferential Revision: https://reviews.llvm.org/D125266 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Mon, 09 May 2022 20:23:59 +0000 Hongtao Yu <hoy@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#23191a4f [CSSPGO][llvm-profgen] Do not duplicate context profiles into base profile when converting CS flat profile to nested.Recent experiments with our two large internal services showed that duplicating context profiles into base profile caused code size inflation and didn't deliver good performance compared to no such duplication. It was a trick we made to catch up with the CS flat profile and I'm now turning it off by default.The code size inflation mainly comes from the enriched based profiles. A base profile for a function represents the uninlined (or outlined) portion of the whole function running time. Such portion could be very small if a function is inlined into most of its hot callsites. Duplicating context profiles of the function into its base profiles could cause the outlined body to be hot enough and in turn get many of its callees inlined, thus increases the code size. The size inflation could further cause perf regression.Reviewed By: wenleiDifferential Revision: https://reviews.llvm.org/D124796 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Thu, 12 May 2022 16:28:25 +0000 Hongtao Yu <hoy@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#e36786d1 [CSSPGO] Rename ProfileIsCSNested and ProfileIsCSFlatTo be more clear and definitive, I'm renaming `ProfileIsCSFlat` back to `ProfileIsCS` which stands for full context-sensitive flat profiles. `ProfileIsCSNested` is now renamed to `ProfileIsPreInlined` and is extended to be applicable for CS flat profiles too. More specifically, `ProfileIsPreInlined` is for any kind of profiles (flat or nested) that contain 'ShouldBeInlined' contexts. The flag is encoded in the profile summary section for extbinary profiles and is computed on-the-fly for text profiles.Reviewed By: wenleiDifferential Revision: https://reviews.llvm.org/D122602 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Thu, 28 Apr 2022 18:31:02 +0000 Hongtao Yu <hoy@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#bc380c09 [llvm-profgen] Turn on CS nested profile generation by default for CSSPGO.CS nested profile has a benefit over the CS flat profile that is to speed up the build while achieve an on-par performance. I'm turning it on by default for CSSPGO.Reviewed By: wenleiDifferential Revision: https://reviews.llvm.org/D121142 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Mon, 07 Mar 2022 19:41:11 +0000 Hongtao Yu <hoy@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#f0f70ae6 [CSSPGO] Do not recount callee samples when computing profile summary for nested CS profile.When generating nested CS profile with all calling contexts of a function duplicated into a base profile under `--generate-merged-base-profiles`, do not recount callee samples when computing profile summary. This fixes the profile summary mismatch between flat cs profile and nested cs profile, for both extbinary and text format.Reviewed By: wenleiDifferential Revision: https://reviews.llvm.org/D119494 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Fri, 11 Feb 2022 05:55:19 +0000 Hongtao Yu <hoy@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#f6f0409f [llvm-profgen] Turn on preinliner by defaultpreinliner has been tuned on large server workloads and it's not ready to be turned on by default. this change also updates the thresholds based on tuning.Differential Revision: https://reviews.llvm.org/D115770 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Wed, 15 Dec 2021 01:10:47 +0000 Wenlei He <wenlei@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#5740bb80 [CSSPGO] Use nested context-sensitive profile.CSSPGO currently employs a flat profile format for context-sensitive profiles. Such a flat profile allows for precisely manipulating contexts that is either inlined or not inlined. This is a benefit over the nested profile format used by non-CS AutoFDO. A downside of this is the longer build time due to parsing the indexing the full CS contexts.For a CS flat profile, though only the context profiles relevant to a module are loaded when that module is compiled, the cost to figure out what profiles are relevant is noticeably high when there're many contexts, since the sample reader will need to scan all context strings anyway. On the contrary, a nested function profile has its related inline subcontexts isolated from other unrelated contexts. Therefore when compiling a set of functions, unrelated contexts will never need to be scanned.In this change we are exploring using nested profile format for CSSPGO. This is expected to work based on an assumption that with a preinliner-computed profile all contexts are precomputed and expected to be inlined by the compiler. Contexts not expected to be inlined will be cut off and returned to corresponding base profiles (for top-level outlined functions). This naturally forms a nested profile where all nested contexts are expected to be inlined. The compiler will less likely optimize on derived contexts that are not precomputed.A CS-nested profile will look exactly the same with regular nested profile except that each nested profile can come with an attributes. With pseudo probes, a nested profile shown as below can also have a CFG checksum.```main:1968679:12 2: 24 3: 28 _Z5funcAi:18 3.1: 28 _Z5funcBi:30 3: _Z5funcAi:1467398 0: 10 1: 10 _Z8funcLeafi:11 3: 24 1: _Z8funcLeafi:1467299 0: 6 1: 6 3: 287884 4: 287864 _Z3fibi:315608 15: 23 !CFGChecksum: 138828622701 !Attributes: 2 !CFGChecksum: 281479271677951 !Attributes: 2```Specific work included in this change:- A recursive profile converter to convert CS flat profile to nested profile.- Extend function checksum and attribute metadata to be stored in nested way for text profile and extbinary profile.- Unifiy sample loader inliner path for CS and preinlined nested profile. - Changes in the sample loader to support probe-based nested profile.I've seen promising results regarding build time. A nested profile can result in a 20% shorter build time than a CS flat profile while keep an on-par performance. This is with -duplicate-contexts-into-base=1.Test Plan:Reviewed By: wenleiDifferential Revision: https://reviews.llvm.org/D115205 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Tue, 14 Dec 2021 18:03:05 +0000 Hongtao Yu <hoy@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#484a569e [llvm-profgen] Fix total samples related issuesSince total sample and body sample are used to compute hotness threshold in compiler, we found in some services changing the total samples computation will cause noticeable regression. Hence, here we will revert the changes and just keep all total samples number identical to the old tool.Three changes in this diff:1. Revert previous diff(https://reviews.llvm.org/D112672: [llvm-profgen] Update total samples by accumulating all its body samples) and put it under a switch.2. Keep the negative line number. Although compiler doesn't consume the count but it will be used to compute hot threshold.3. Change to accumulate total samples per byte instead of per instruction.Reviewed By: hoy, wenleiDifferential Revision: https://reviews.llvm.org/D115013 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Fri, 03 Dec 2021 00:51:42 +0000 wlei <wlei@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#27cb3707 [llvm-profgen] Trim cold function profiles for non-CS AutoFDOThis change allows to trim the profile if it's considered to be cold for baseline AutoFDO. We reuse the cold threshold from `ProfileSummaryBuilder::getColdCountThreshold(..)` which can be set by percent(--profile-summary-cutoff-cold) or by value(--profile-summary-cold-count).Reviewed By: hoy, wenleiDifferential Revision: https://reviews.llvm.org/D113785 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Mon, 29 Nov 2021 07:43:11 +0000 wlei <wlei@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#f5537643 [llvm-profgen] Update total samples by accumulating all its body samplesLike probe-based profile, the total samples is the sum of all its body samples. This patch fix it by a post-processing update for the line-number based profile. Tested it on our internal services, results showed no performance change.Reviewed By: hoy, wenleiDifferential Revision: https://reviews.llvm.org/D112672 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Wed, 27 Oct 2021 07:25:50 +0000 wlei <wlei@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#259e4c56 [CSSPGO] Trim cold base profiles for the CS preinliner.Adding support to the CS preinliner to trim cold base profiles. This makes trimming consistent with the inline decision made by the preinliner. Also disable the existing profile merger when preinliner is on unless explicitly specified.Reviewed By: wenlei, wleiDifferential Revision: https://reviews.llvm.org/D112489 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Wed, 27 Oct 2021 23:56:06 +0000 Hongtao Yu <hoy@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#31a5cb32 [llvm-profgen] Filter out invalid debug lineDifferential Revision: https://reviews.llvm.org/D110081 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Sat, 02 Oct 2021 00:20:56 +0000 wlei <wlei@fb.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#aaa826fa [CSSPGO][llvm-profgen] Make extended binary the default output formatMake extended binary the default output format for CSSPGO. This avoids having to pass flag every time when generating profile. It also matches llvm-profdata where binary profile is the default (should we switch to extbinary as default for llvm-profdata?).We plan to compress name table for context profile, which depends on the built-in compression of extbinary.Differential Revision: https://reviews.llvm.org/D103650 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Thu, 03 Jun 2021 20:42:24 +0000 Wenlei He <aktoon@gmail.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#dff83158 [CSSPGO][llvm-profdata] Support trimming cold context when merging profilesThe change adds support for triming and merging cold context when mergine CSSPGO profiles using llvm-profdata. This is similar to the context profile trimming in llvm-profgen, however the flexibility to trim cold context after profile is generated can be useful.Differential Revision: https://reviews.llvm.org/D100528 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Thu, 15 Apr 2021 05:53:40 +0000 Wenlei He <aktoon@gmail.com> http://172.16.0.5:8080/history/llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test#30b02323 [CSSPGO][llvm-profgen] Context-sensitive global pre-inlinerThis change sets up a framework in llvm-profgen to estimate inline decision and adjust context-sensitive profile based on that. We call it a global pre-inliner in llvm-profgen.It will serve two purposes: 1) Since context profile for not inlined context will be merged into base profile, if we estimate a context will not be inlined, we can merge the context profile in the output to save profile size. 2) For thinLTO, when a context involving functions from different modules is not inined, we can't merge functions profiles across modules, leading to suboptimal post-inline count quality. By estimating some inline decisions, we would be able to adjust/merge context profiles beforehand as a mitigation.Compiler inline heuristic uses inline cost which is not available in llvm-profgen. But since inline cost is closely related to size, we could get an estimate through function size from debug info. Because the size we have in llvm-profgen is the final size, it could also be more accurate than the inline cost estimation in the compiler.This change only has the framework, with a few TODOs left for follow up patches for a complete implementation: 1) We need to retrieve size for funciton//inlinee from debug info for inlining estimation. Currently we use number of samples in a profile as place holder for size estimation. 2) Currently the thresholds are using the values used by sample loader inliner. But they need to be tuned since the size here is fully optimized machine code size, instead of inline cost based on not yet fully optimized IR.Differential Revision: https://reviews.llvm.org/D99146 List of files: /llvm-project-15.0.7/llvm/test/tools/llvm-profgen/cs-preinline.test Fri, 05 Mar 2021 15:50:36 +0000 Wenlei He <wenlei@fb.com>