Changes in procfs-smaps_rollup

30934843 - mm/smaps: add Pss_Dirty

Mon, 20 Jun 2022 08:12:50 +0000

mm/smaps: add Pss_DirtyPss is the sum of the sizes of clean and dirty private pages, and theproportional sizes of clean and dirty shared pages: Private = Private_Dirty + Private_Clean Shared_Proportional = Shared_Dirty_Proportional + Shared_Clean_Proportional Pss = Private + Shared_ProportionalThe Shared*Proportional fields are not present in smaps, so it is notalways possible to determine how much of the Pss is from dirty pages andhow much is from clean pages. This information can be useful formeasuring memory usage for the purpose of optimisation, since clean pagescan usually be discarded by the kernel immediately while dirty pagescannot.The smaps routines in the kernel already have access to this data, so adda Pss_Dirty to show it to userspace. Pss_Clean is not added since it canbe calculated from Pss and Pss_Dirty.Link: https://lkml.kernel.org/r/20220620081251.2928103-1-vincent.whitchurch@axis.comSigned-off-by: Vincent Whitchurch Cc: Jonathan Corbet Cc: Alexey Dobriyan Signed-off-by: Andrew Morton List of files: /linux-6.15/Documentation/ABI/testing/procfs-smaps_rollup

54a19b4d - docs: ABI: cleanup several ABI documents

Fri, 30 Oct 2020 07:40:50 +0000

docs: ABI: cleanup several ABI documentsThere are some ABI documents that, while they don't generateany warnings, they have issues when parsed by get_abi.pl scripton its output result.Address them, in order to provide a clean output.Reviewed-by: Tom Rix # for fpga-managerReviewed-By: Kajol Jain # for sysfs-bus-event_source-devices-hv_gpci and sysfs-bus-event_source-devices-hv_24x7Acked-by: Jonathan Cameron #for IIOAcked-by: Oded Gabbay # for HabanalabsAcked-by: Vaibhav Jain # for sysfs-bus-papr-pmemAcked-by: Cezary Rojewski # for catptAcked-by: Suzuki K Poulose Acked-by: Ilya Dryomov # for rbdAcked-by: Jonathan Corbet Signed-off-by: Mauro Carvalho Chehab Link: https://lore.kernel.org/r/5bc78e5b68ed1e9e39135173857cb2e753be868f.1604042072.git.mchehab+huawei@kernel.orgSigned-off-by: Greg Kroah-Hartman List of files: /linux-6.15/Documentation/ABI/testing/procfs-smaps_rollup

0c1bc6b8 - docs: filesystems: fix renamed references

Tue, 14 Apr 2020 16:48:37 +0000

docs: filesystems: fix renamed referencesSome filesystem references got broken by a previous patchseries I submitted. Address those.Signed-off-by: Mauro Carvalho Chehab Acked-by: David Sterba # fs/affs/KconfigLink: https://lore.kernel.org/r/57318c53008dbda7f6f4a5a9e5787f4d37e8565a.1586881715.git.mchehab+huawei@kernel.orgSigned-off-by: Jonathan Corbet List of files: /linux-6.15/Documentation/ABI/testing/procfs-smaps_rollup

ee2ad71b - mm: smaps: split PSS into components

Fri, 12 Jul 2019 04:00:10 +0000

mm: smaps: split PSS into componentsReport separate components (anon, file, and shmem) for PSS insmaps_rollup.This helps understand and tune the memory manager behavior in consumerdevices, particularly mobile devices. Many of them (e.g. chromebooks andAndroid-based devices) use zram for anon memory, and perform disk readsfor discarded file pages. The difference in latency is large (e.g.reading a single page from SSD is 30 times slower than decompressing azram page on one popular device), thus it is useful to know how much ofthe PSS is anon vs. file.All the information is already present in /proc/pid/smaps, but much moreexpensive to obtain because of the large size of that procfs entry.This patch also removes a small code duplication in smaps_account, whichwould have gotten worse otherwise.Also updated Documentation/filesystems/proc.txt (the smaps section was abit stale, and I added a smaps_rollup section) andDocumentation/ABI/testing/procfs-smaps_rollup.[semenzato@chromium.org: v5] Link: http://lkml.kernel.org/r/20190626234333.44608-1-semenzato@chromium.orgLink: http://lkml.kernel.org/r/20190626180429.174569-1-semenzato@chromium.orgSigned-off-by: Luigi Semenzato Acked-by: Yu Zhao Cc: Sonny Rao Cc: Yu Zhao Cc: Brian Geffon Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/Documentation/ABI/testing/procfs-smaps_rollup

493b0e9d - mm: add /proc/pid/smaps_rollup

Wed, 06 Sep 2017 23:25:08 +0000

mm: add /proc/pid/smaps_rollup/proc/pid/smaps_rollup is a new proc file that improves the performanceof user programs that determine aggregate memory statistics (e.g., totalPSS) of a process.Android regularly "samples" the memory usage of various processes inorder to balance its memory pool sizes. This sampling process involvesopening /proc/pid/smaps and summing certain fields. For very largeprocesses, sampling memory use this way can take several hundredmilliseconds, due mostly to the overhead of the seq_printf calls intask_mmu.c.smaps_rollup improves the situation. It contains most of the fields of/proc/pid/smaps, but instead of a set of fields for each VMA,smaps_rollup instead contains one synthetic smaps-format entryrepresenting the whole process. In the single smaps_rollup syntheticentry, each field is the summation of the corresponding field in all ofthe real-smaps VMAs. Using a common format for smaps_rollup and smapsallows userspace parsers to repurpose parsers meant for use withnon-rollup smaps for smaps_rollup, and it allows userspace to switchbetween smaps_rollup and smaps at runtime (say, based on theavailability of smaps_rollup in a given kernel) with minimal fuss.By using smaps_rollup instead of smaps, a caller can avoid thesignificant overhead of formatting, reading, and parsing each of a largeprocess's potentially very numerous memory mappings. For samplingsystem_server's PSS in Android, we measured a 12x speedup, representinga savings of several hundred milliseconds.One alternative to a new per-process proc file would have been includingPSS information in /proc/pid/status. We considered this option butthought that PSS would be too expensive (by a few orders of magnitude)to collect relative to what's already emitted as part of/proc/pid/status, and slowing every user of /proc/pid/status for thesake of readers that happen to want PSS feels wrong.The code itself works by reusing the existing VMA-walking framework weuse for regular smaps generation and keeping the mem_size_statsstructure around between VMA walks instead of using a fresh one for eachVMA. In this way, summation happens automatically. We let seq_filewalk over the VMAs just as it does for regular smaps and just emitnothing to the seq_file until we hit the last VMA.Benchmarks: using smaps: iterations:1000 pid:1163 pss:220023808 0m29.46s real 0m08.28s user 0m20.98s system using smaps_rollup: iterations:1000 pid:1163 pss:220702720 0m04.39s real 0m00.03s user 0m04.31s systemWe're using the PSS samples we collect asynchronously forsystem-management tasks like fine-tuning oom_adj_score, memory usetracking for debugging, application-level memory-use attribution, anddeciding whether we want to kill large processes during system idlemaintenance windows. Android has been using PSS for these purposes fora long time; as the average process VMA count has increased and anddevices become more efficiency-conscious, PSS-collection inefficiencyhas started to matter more. IMHO, it'd be a lot safer to optimize theexisting PSS-collection model, which has been fine-tuned over the years,instead of changing the memory tracking approach entirely to work aroundsmaps-generation inefficiency.Tim said:: There are two main reasons why Android gathers PSS information::: 1. Android devices can show the user the amount of memory used per: application via the settings app. This is a less important use case.:: 2. We log PSS to help identify leaks in applications. We have found: an enormous number of bugs (in the Android platform, in Google's own: apps, and in third-party applications) using this data.:: To do this, system_server (the main process in Android userspace) will: sample the PSS of a process three seconds after it changes state (for: example, app is launched and becomes the foreground application) and about: every ten minutes after that. The net result is that PSS collection is: regularly running on at least one process in the system (usually a few: times a minute while the screen is on, less when screen is off due to: suspend). PSS of a process is an incredibly useful stat to track, and we: aren't going to get rid of it. We've looked at some very hacky approaches: using RSS ("take the RSS of the target process, subtract the RSS of the: zygote process that is the parent of all Android apps") to reduce the: accounting time, but it regularly overestimated the memory used by 20+: percent. Accordingly, I don't think that there's a good alternative to: using PSS.:: We started looking into PSS collection performance after we noticed random: frequency spikes while a phone's screen was off; occasionally, one of the: CPU clusters would ramp to a high frequency because there was 200-300ms of: constant CPU work from a single thread in the main Android userspace: process. The work causing the spike (which is reasonable governor: behavior given the amount of CPU time needed) was always PSS collection.: As a result, Android is burning more power than we should be on PSS: collection.:: The other issue (and why I'm less sure about improving smaps as a: long-term solution) is that the number of VMAs per process has increased: significantly from release to release. After trying to figure out why we: were seeing these 200-300ms PSS collection times on Android O but had not: noticed it in previous versions, we found that the number of VMAs in the: main system process increased by 50% from Android N to Android O (from: ~1800 to ~2700) and varying increases in every userspace process. Android: M to N also had an increase in the number of VMAs, although not as much.: I'm not sure why this is increasing so much over time, but thinking about: ASLR and ways to make ASLR better, I expect that this will continue to: increase going forward. I would not be surprised if we hit 5000 VMAs on: the main Android process (system_server) by 2020.:: If we assume that the number of VMAs is going to increase over time, then: doing anything we can do to reduce the overhead of each VMA during PSS: collection seems like the right way to go, and that means outputting an: aggregate statistic (to avoid whatever overhead there is per line in: writing smaps and in reading each line from userspace).Link: http://lkml.kernel.org/r/20170812022148.178293-1-dancol@google.comSigned-off-by: Daniel Colascione Cc: Tim Murray Cc: Joel Fernandes Cc: Al Viro Cc: Randy Dunlap Cc: Minchan Kim Cc: Michal Hocko Cc: Sonny Rao Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/Documentation/ABI/testing/procfs-smaps_rollup