Changes in Makefile

5ec4333b - mm/damon: remove DAMON debugfs interface

Mon, 06 Jan 2025 19:19:41 +0000

mm/damon: remove DAMON debugfs interfaceIt's time to remove DAMON debugfs interface, which has deprecated longbefore in February 2023. Read the cover letter of this patch series formore details.All documents and related tests are also removed. Finally remove theinterface.Link: https://lkml.kernel.org/r/20250106191941.107070-9-sj@kernel.orgSigned-off-by: SeongJae Park Cc: Alex Shi Cc: Brendan Higgins Cc: David Gow Cc: Hu Haowen <2023002089@link.tyut.edu.cn>Cc: Jonathan Corbet Cc: Rae Moar Cc: Shuah Khan Cc: Yanteng Si Signed-off-by: Andrew Morton List of files: /linux-6.15/mm/damon/Makefile

7ae2c17f - mm/damon/modules: deduplicate init steps for DAMON context setup

Wed, 26 Oct 2022 22:59:42 +0000

mm/damon/modules: deduplicate init steps for DAMON context setupDAMON_RECLAIM and DAMON_LRU_SORT has duplicated code for DAMON context andtarget initializations. Deduplicate the part by implementing a functionfor the initialization in 'modules-common.c' and using it.Link: https://lkml.kernel.org/r/20221026225943.100429-12-sj@kernel.orgSigned-off-by: SeongJae Park Signed-off-by: Andrew Morton List of files: /linux-6.15/mm/damon/Makefile

c8e7b4d0 - mm/damon/sysfs: split out schemes directory implementation to separate file

Wed, 26 Oct 2022 22:59:41 +0000

mm/damon/sysfs: split out schemes directory implementation to separate fileDAMON sysfs interface for 'schemes' directory is implemented using aboutone thousand lines of code. It has no strong dependency with otherparts of its file, so split it out to another file for better codemanagement.Link: https://lkml.kernel.org/r/20221026225943.100429-11-sj@kernel.orgSigned-off-by: SeongJae Park Signed-off-by: Andrew Morton List of files: /linux-6.15/mm/damon/Makefile

39240595 - mm/damon/sysfs: move sysfs_lock to common module

Wed, 26 Oct 2022 22:59:38 +0000

mm/damon/sysfs: move sysfs_lock to common moduleDAMON sysfs interface is implemented in a single file, sysfs.c, which hasabout 2,800 lines of code. As the interface is hierarchical and some ofthe code can be reused by different hierarchies, it would make more senseto split out the implementation into common parts and different parts inmultiple files. As the beginning of the work, create files for commoncode and move the global mutex for directories modifications protectioninto the new file.Link: https://lkml.kernel.org/r/20221026225943.100429-8-sj@kernel.orgSigned-off-by: SeongJae Park Signed-off-by: Andrew Morton List of files: /linux-6.15/mm/damon/Makefile

40e983cc - mm/damon: introduce DAMON-based LRU-lists Sorting

Mon, 13 Jun 2022 19:23:00 +0000

mm/damon: introduce DAMON-based LRU-lists SortingUsers can do data access-aware LRU-lists sorting using 'LRU_PRIO' and'LRU_DEPRIO' DAMOS actions. However, finding best parameters includingthe hotness/coldness thresholds, CPU quota, and watermarks could bechallenging for some users. To make the scheme easy to be used withoutcomplex tuning for common situations, this commit implements a statickernel module called 'DAMON_LRU_SORT' using the 'LRU_PRIO' and'LRU_DEPRIO' DAMOS actions.It proactively sorts LRU-lists using DAMON with conservatively chosendefault values of the parameters. That is, the module under its defaultparameters will make no harm for common situations but provide some levelof efficiency improvements for systems having clear hot/cold accesspattern under a level of memory pressure while consuming only a limitedsmall portion of CPU time.Link: https://lkml.kernel.org/r/20220613192301.8817-9-sj@kernel.orgSigned-off-by: SeongJae Park Cc: Jonathan Corbet Signed-off-by: Andrew Morton List of files: /linux-6.15/mm/damon/Makefile

c951cd3b - mm/damon: implement a minimal stub for sysfs-based DAMON interface

Tue, 22 Mar 2022 21:49:27 +0000

mm/damon: implement a minimal stub for sysfs-based DAMON interfaceDAMON's debugfs-based user interface served very well, so far. However,it unnecessarily depends on debugfs, while DAMON is not aimed to be usedfor only debugging. Also, the interface receives multiple values via onefile. For example, schemes file receives 18 values separated by whitespaces. As a result, it is ineffient, hard to be used, and difficult tobe extended. Especially, keeping backward compatibility of user spacetools is getting only challenging. It would be better to implementanother reliable and flexible interface and deprecate the debugfsinterface in long term.To this end, this commit implements a stub of a part of the new userinterface of DAMON using sysfs. Specifically, this commit implements thesysfs control parts for virtual address space monitoring.More specifically, the idea of the new interface is, using directoryhierarchies and making one file for one value. The hierarchy that thiscommit is introducing is as below. In the below figure, parents-childrenrelations are represented with indentations, each directory is having``/`` suffix, and files in each directory are separated by comma (","). /sys/kernel/mm/damon/admin │ kdamonds/nr_kdamonds │ │ 0/state,pid │ │ │ contexts/nr_contexts │ │ │ │ 0/operations │ │ │ │ │ monitoring_attrs/ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ nr_regions/min,max │ │ │ │ │ targets/nr_targets │ │ │ │ │ │ 0/pid_target │ │ │ │ │ │ ... │ │ │ │ ... │ │ ...Writing a number to each 'nr' file makes directories of name <0> to in the directory of the 'nr' file. That's all this commit does.Writing proper values to relevant files will construct the DAMON contexts,and writing a special keyword, 'on', to 'state' files for each kdamondwill ask DAMON to start the constructed contexts.For a short example, using below commands for monitoring virtual addressspaces of a given workload is imaginable: # cd /sys/kernel/mm/damon/admin/ # echo 1 > kdamonds/nr_kdamonds # echo 1 > kdamonds/0/contexts/nr_contexts # echo vaddr > kdamonds/0/contexts/0/operations # echo 1 > kdamonds/0/contexts/0/targets/nr_targets # echo $(pidof ) > kdamonds/0/contexts/0/targets/0/pid_target # echo on > kdamonds/0/statePlease note that this commit is implementing only the sysfs part stub asabovely mentioned. This commit doesn't implement the special keywords for'state' files. Following commits will do that.[jiapeng.chong@linux.alibaba.com: fix missing error code in damon_sysfs_attrs_add_dirs()] Link: https://lkml.kernel.org/r/20220302111120.24984-1-jiapeng.chong@linux.alibaba.comLink: https://lkml.kernel.org/r/20220228081314.5770-4-sj@kernel.orgSigned-off-by: SeongJae Park Signed-off-by: Jiapeng Chong Cc: David Rientjes Cc: Greg Kroah-Hartman Cc: Jonathan Corbet Cc: Shuah Khan Cc: Xin Hao Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile

3213a3c1 - mm/damon: remove unnecessary CONFIG_DAMON option

Tue, 22 Mar 2022 21:49:09 +0000

mm/damon: remove unnecessary CONFIG_DAMON optionIn mm/Makefile has: obj-$(CONFIG_DAMON) += damon/So that we don't need 'obj-$(CONFIG_DAMON) :=' in mm/damon/Makefile,delete it from mm/damon/Makefile.Link: https://lkml.kernel.org/r/20220221065255.19991-1-tangmeng@uniontech.comSigned-off-by: tangmeng Cc: SeongJae Park Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile

f7d911c3 - mm/damon: rename damon_primitives to damon_operations

Tue, 22 Mar 2022 21:48:46 +0000

mm/damon: rename damon_primitives to damon_operationsPatch series "Allow DAMON user code independent of monitoring primitives".In-kernel DAMON user code is required to configure the monitoring context(struct damon_ctx) with proper monitoring primitives (structdamon_primitive). This makes the user code dependent to all supportingmonitoring primitives. For example, DAMON debugfs interface depends onboth DAMON_VADDR and DAMON_PADDR, though some users have interest in onlyone use case. As more monitoring primitives are introduced, the problemwill be bigger.To minimize such unnecessary dependency, this patchset makes monitoringprimitives can be registered by the implemnting code and later dynamicallysearched and selected by the user code.In addition to that, this patchset renames monitoring primitives tomonitoring operations, which is more easy to intuitively understand whatit means and how it would be structed.This patch (of 8):DAMON has a set of callback functions called monitoring primitives and letit can be configured with various implementations for easy extension fordifferent address spaces and usages. However, the word 'primitive' is notso explicit. Meanwhile, many other structs resembles similar purposecalls themselves 'operations'. To make the code easier to be understood,this commit renames 'damon_primitives' to 'damon_operations' before it istoo late to rename.Link: https://lkml.kernel.org/r/20220215184603.1479-1-sj@kernel.orgLink: https://lkml.kernel.org/r/20220215184603.1479-2-sj@kernel.orgSigned-off-by: SeongJae Park Cc: Xin Hao Cc: David Rientjes Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile

43b0536c - mm/damon: introduce DAMON-based Reclamation (DAMON_RECLAIM)

Fri, 05 Nov 2021 20:47:57 +0000

mm/damon: introduce DAMON-based Reclamation (DAMON_RECLAIM)This implements a new kernel subsystem that finds cold memory regionsusing DAMON and reclaims those immediately. It is intended to be usedas proactive lightweigh reclamation logic for light memory pressure.For heavy memory pressure, it could be inactivated and fall back to thetraditional page-scanning based reclamation.It's implemented on top of DAMON framework to use the DAMON-basedOperation Schemes (DAMOS) feature. It utilizes all the DAMOS featuresincluding speed limit, prioritization, and watermarks.It could be enabled and tuned in boot time via the kernel bootparameter, and in run time via its module parameters('/sys/module/damon_reclaim/parameters/') interface.[yangyingliang@huawei.com: fix error return code in damon_reclaim_turn()] Link: https://lkml.kernel.org/r/20211025124500.2758060-1-yangyingliang@huawei.comLink: https://lkml.kernel.org/r/20211019150731.16699-15-sj@kernel.orgSigned-off-by: SeongJae Park Signed-off-by: Yang Yingliang Cc: Amit Shah Cc: Benjamin Herrenschmidt Cc: David Hildenbrand Cc: David Rientjes Cc: David Woodhouse Cc: Greg Thelen Cc: Jonathan Cameron Cc: Jonathan Corbet Cc: Leonard Foerster Cc: Marco Elver Cc: Markus Boehme Cc: Shakeel Butt Cc: Shuah Khan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile

a28397be - mm/damon: implement primitives for physical address space monitoring

Fri, 05 Nov 2021 20:46:56 +0000

mm/damon: implement primitives for physical address space monitoringThis implements the monitoring primitives for the physical memoryaddress space. Internally, it uses the PTE Accessed bit, similar tothat of the virtual address spaces monitoring primitives. It supportsonly user memory pages, as idle pages tracking does. If the monitoringtarget physical memory address range contains non-user memory pages,access check of the pages will do nothing but simply treat the pages asnot accessed.Link: https://lkml.kernel.org/r/20211012205711.29216-6-sj@kernel.orgSigned-off-by: SeongJae Park Cc: Amit Shah Cc: Benjamin Herrenschmidt Cc: Brendan Higgins Cc: David Hildenbrand Cc: David Rienjes Cc: David Woodhouse Cc: Greg Thelen Cc: Jonathan Cameron Cc: Jonathan Corbet Cc: Leonard Foerster Cc: Marco Elver Cc: Markus Boehme Cc: Shakeel Butt Cc: Shuah Khan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile

46c3a0ac - mm/damon/vaddr: separate commonly usable functions

Fri, 05 Nov 2021 20:46:53 +0000

mm/damon/vaddr: separate commonly usable functionsThis moves functions in the default virtual address spaces monitoringprimitives that commonly usable from other address spaces like physicaladdress space into a header file. Those will be reused by the physicaladdress space monitoring primitives which will be implemented by thefollowing commit.[sj@kernel.org: include 'highmem.h' to fix a build failure] Link: https://lkml.kernel.org/r/20211014110848.5204-1-sj@kernel.orgLink: https://lkml.kernel.org/r/20211012205711.29216-5-sj@kernel.orgSigned-off-by: SeongJae Park Cc: Amit Shah Cc: Benjamin Herrenschmidt Cc: Brendan Higgins Cc: David Hildenbrand Cc: David Rienjes Cc: David Woodhouse Cc: Greg Thelen Cc: Jonathan Cameron Cc: Jonathan Corbet Cc: Leonard Foerster Cc: Marco Elver Cc: Markus Boehme Cc: Shakeel Butt Cc: Shuah Khan Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile

4bc05954 - mm/damon: implement a debugfs-based user space interface

Wed, 08 Sep 2021 02:56:53 +0000

mm/damon: implement a debugfs-based user space interfaceDAMON is designed to be used by kernel space code such as the memorymanagement subsystems, and therefore it provides only kernel space API.That said, letting the user space control DAMON could provide somebenefits to them. For example, it will allow user space to analyze theirspecific workloads and make their own special optimizations.For such cases, this commit implements a simple DAMON application kernelmodule, namely 'damon-dbgfs', which merely wraps the DAMON api and exportsthose to the user space via the debugfs.'damon-dbgfs' exports three files, ``attrs``, ``target_ids``, and``monitor_on`` under its debugfs directory, ``/damon/``.Attributes----------Users can read and write the ``sampling interval``, ``aggregationinterval``, ``regions update interval``, and min/max number of monitoringtarget regions by reading from and writing to the ``attrs`` file. Forexample, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10,1000 and check it again:: # cd /damon # echo 5000 100000 1000000 10 1000 > attrs # cat attrs 5000 100000 1000000 10 1000Target IDs----------Some types of address spaces supports multiple monitoring target. Forexample, the virtual memory address spaces monitoring can have multipleprocesses as the monitoring targets. Users can set the targets by writingrelevant id values of the targets to, and get the ids of the currenttargets by reading from the ``target_ids`` file. In case of the virtualaddress spaces monitoring, the values should be pids of the monitoringtarget processes. For example, below commands set processes having pids42 and 4242 as the monitoring targets and check it again:: # cd /damon # echo 42 4242 > target_ids # cat target_ids 42 4242Note that setting the target ids doesn't start the monitoring.Turning On/Off--------------Setting the files as described above doesn't incur effect unless youexplicitly start the monitoring. You can start, stop, and check thecurrent status of the monitoring by writing to and reading from the``monitor_on`` file. Writing ``on`` to the file starts the monitoring ofthe targets with the attributes. Writing ``off`` to the file stops those.DAMON also stops if every targets are invalidated (in case of the virtualmemory monitoring, target processes are invalidated when terminated).Below example commands turn on, off, and check the status of DAMON:: # cd /damon # echo on > monitor_on # echo off > monitor_on # cat monitor_on offPlease note that you cannot write to the above-mentioned debugfs fileswhile the monitoring is turned on. If you write to the files while DAMONis running, an error code such as ``-EBUSY`` will be returned.[akpm@linux-foundation.org: remove unneeded "alloc failed" printks][akpm@linux-foundation.org: replace macro with static inline]Link: https://lkml.kernel.org/r/20210716081449.22187-8-sj38.park@gmail.comSigned-off-by: SeongJae Park Reviewed-by: Leonard Foerster Reviewed-by: Fernand Sieber Cc: Alexander Shishkin Cc: Amit Shah Cc: Benjamin Herrenschmidt Cc: Brendan Higgins Cc: David Hildenbrand Cc: David Rientjes Cc: David Woodhouse Cc: Fan Du Cc: Greg Kroah-Hartman Cc: Greg Thelen Cc: Ingo Molnar Cc: Joe Perches Cc: Jonathan Cameron Cc: Jonathan Corbet Cc: Marco Elver Cc: Markus Boehme Cc: Maximilian Heyne Cc: Mel Gorman Cc: Minchan Kim Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Rik van Riel Cc: Shakeel Butt Cc: Shuah Khan Cc: Steven Rostedt (VMware) Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile

3f49584b - mm/damon: implement primitives for the virtual memory address spaces

Wed, 08 Sep 2021 02:56:44 +0000

mm/damon: implement primitives for the virtual memory address spacesThis commit introduces a reference implementation of the address spacespecific low level primitives for the virtual address space, so that usersof DAMON can easily monitor the data accesses on virtual address spaces ofspecific processes by simply configuring the implementation to be used byDAMON.The low level primitives for the fundamental access monitoring are definedin two parts:1. Identification of the monitoring target address range for the address space.2. Access check of specific address range in the target space.The reference implementation for the virtual address space does the worksas below.PTE Accessed-bit Based Access Check-----------------------------------The implementation uses PTE Accessed-bit for basic access checks. Thatis, it clears the bit for the next sampling target page and checks whetherit is set again after one sampling period. This could disturb the reclaimlogic. DAMON uses ``PG_idle`` and ``PG_young`` page flags to solve theconflict, as Idle page tracking does.VMA-based Target Address Range Construction-------------------------------------------Only small parts in the super-huge virtual address space of the processesare mapped to physical memory and accessed. Thus, tracking the unmappedaddress regions is just wasteful. However, because DAMON can deal withsome level of noise using the adaptive regions adjustment mechanism,tracking every mapping is not strictly required but could even incur ahigh overhead in some cases. That said, too huge unmapped areas insidethe monitoring target should be removed to not take the time for theadaptive mechanism.For the reason, this implementation converts the complex mappings to threedistinct regions that cover every mapped area of the address space. Also,the two gaps between the three regions are the two biggest unmapped areasin the given address space. The two biggest unmapped areas would be thegap between the heap and the uppermost mmap()-ed region, and the gapbetween the lowermost mmap()-ed region and the stack in most of the cases.Because these gaps are exceptionally huge in usual address spaces,excluding these will be sufficient to make a reasonable trade-off. Belowshows this in detail:: (small mmap()-ed regions and munmap()-ed regions) [akpm@linux-foundation.org: mm/damon/vaddr.c needs highmem.h for kunmap_atomic()][sjpark@amazon.de: remove unnecessary PAGE_EXTENSION setup] Link: https://lkml.kernel.org/r/20210806095153.6444-2-sj38.park@gmail.com[sjpark@amazon.de: safely walk page table] Link: https://lkml.kernel.org/r/20210831161800.29419-1-sj38.park@gmail.comLink: https://lkml.kernel.org/r/20210716081449.22187-6-sj38.park@gmail.comSigned-off-by: SeongJae Park Reviewed-by: Leonard Foerster Reviewed-by: Fernand Sieber Acked-by: Shakeel Butt Cc: Alexander Shishkin Cc: Amit Shah Cc: Benjamin Herrenschmidt Cc: Brendan Higgins Cc: David Hildenbrand Cc: David Rientjes Cc: David Woodhouse Cc: Fan Du Cc: Greg Kroah-Hartman Cc: Greg Thelen Cc: Ingo Molnar Cc: Joe Perches Cc: Jonathan Cameron Cc: Jonathan Corbet Cc: Marco Elver Cc: Markus Boehme Cc: Maximilian Heyne Cc: Mel Gorman Cc: Minchan Kim Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Rik van Riel Cc: Shuah Khan Cc: Steven Rostedt (VMware) Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile

2224d848 - mm: introduce Data Access MONitor (DAMON)

Wed, 08 Sep 2021 02:56:28 +0000

mm: introduce Data Access MONitor (DAMON)Patch series "Introduce Data Access MONitor (DAMON)", v34.Introduction============DAMON is a data access monitoring framework for the Linux kernel. Thecore mechanisms of DAMON called 'region based sampling' and 'adaptiveregions adjustment' (refer to 'mechanisms.rst' in the 11th patch of thispatchset for the detail) make it- accurate (The monitored information is useful for DRAM level memory management. It might not appropriate for Cache-level accuracy, though.),- light-weight (The monitoring overhead is low enough to be applied online while making no impact on the performance of the target workloads.), and- scalable (the upper-bound of the instrumentation overhead is controllable regardless of the size of target workloads.).Using this framework, therefore, several memory management mechanisms suchas reclamation and THP can be optimized to aware real data accesspatterns. Experimental access pattern aware memory managementoptimization works that incurring high instrumentation overhead will beable to have another try.Though DAMON is for kernel subsystems, it can be easily exposed to theuser space by writing a DAMON-wrapper kernel subsystem. Then, user spaceusers who have some special workloads will be able to write personalizedtools or applications for deeper understanding and specializedoptimizations of their systems.DAMON is also merged in two public Amazon Linux kernel trees that based onv5.4.y[1] and v5.10.y[2].[1] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon[2] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damonThe userspace tool[1] is available, released under GPLv2, and activelybeing maintained. I am also planning to implement another basic userinterface in perf[2]. Also, the basic test suite for DAMON is availableunder GPLv2[3].[1] https://github.com/awslabs/damo[2] https://lore.kernel.org/linux-mm/20210107120729.22328-1-sjpark@amazon.com/[3] https://github.com/awslabs/damon-testsLong-term Plan--------------DAMON is a part of a project called Data Access-aware Operating System(DAOS). As the name implies, I want to improve the performance andefficiency of systems using fine-grained data access patterns. Theoptimizations are for both kernel and user spaces. I will thereforemodify or create kernel subsystems, export some of those to user space andimplement user space library / tools. Below shows the layers andcomponents for the project. --------------------------------------------------------------------------- Primitives: PTE Accessed bit, PG_idle, rmap, (Intel CMT), ... Framework: DAMON Features: DAMOS, virtual addr, physical addr, ... Applications: DAMON-debugfs, (DARC), ... ^^^^^^^^^^^^^^^^^^^^^^^ KERNEL SPACE ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Raw Interface: debugfs, (sysfs), (damonfs), tracepoints, (sys_damon), ... vvvvvvvvvvvvvvvvvvvvvvv USER SPACE vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv Library: (libdamon), ... Tools: DAMO, (perf), ... ---------------------------------------------------------------------------The components in parentheses or marked as '...' are not implemented yetbut in the future plan. IOW, those are the TODO tasks of DAOS project.For more detail, please refer to the plans:https://lore.kernel.org/linux-mm/20201202082731.24828-1-sjpark@amazon.com/Evaluations===========We evaluated DAMON's overhead, monitoring quality and usefulness using 24realistic workloads on my QEMU/KVM based virtual machine running a kernelthat v24 DAMON patchset is applied.DAMON is lightweight. It increases system memory usage by 0.39% and slowstarget workloads down by 1.16%.DAMON is accurate and useful for memory management optimizations. Anexperimental DAMON-based operation scheme for THP, namely 'ethp', removes76.15% of THP memory overheads while preserving 51.25% of THP speedup.Another experimental DAMON-based 'proactive reclamation' implementation,'prcl', reduces 93.38% of residential sets and 23.63% of system memoryfootprint while incurring only 1.22% runtime overhead in the best case(parsec3/freqmine).NOTE that the experimental THP optimization and proactive reclamation arenot for production but only for proof of concepts.Please refer to the official document[1] or "Documentation/admin-guide/mm:Add a document for DAMON" patch in this patchset for detailed evaluationsetup and results.[1] https://damonitor.github.io/doc/html/latest-damon/admin-guide/mm/damon/eval.htmlReal-world User Story=====================In summary, DAMON has used on production systems and proved its usefulness.DAMON as a profiler-------------------We analyzed characteristics of a large scale production systems of ourcustomers using DAMON. The systems utilize 70GB DRAM and 36 CPUs. Fromthis, we were able to find interesting things below.There were obviously different access pattern under idle workload andactive workload. Under the idle workload, it accessed large memoryregions with low frequency, while the active workload accessed smallmemory regions with high freuqnecy.DAMON found a 7GB memory region that showing obviously high accessfrequency under the active workload. We believe this is theperformance-effective working set and need to be protected.There was a 4KB memory region that showing highest access frequency undernot only active but also idle workloads. We think this must be a hottestcode section like thing that should never be paged out.For this analysis, DAMON used only 0.3-1% of single CPU time. Because weused recording-based analysis, it consumed about 3-12 MB of disk space per20 minutes. This is only small amount of disk space, but we can furtherreduce the disk usage by using non-recording-based DAMON features. I'dlike to argue that only DAMON can do such detailed analysis (finding 4KBhighest region in 70GB memory) with the light overhead.DAMON as a system optimization tool-----------------------------------We also found below potential performance problems on the systems and madeDAMON-based solutions.The system doesn't want to make the workload suffer from the pagereclamation and thus it utilizes enough DRAM but no swap device. However,we found the system is actively reclaiming file-backed pages, because thesystem has intensive file IO. The file IO turned out to be notperformance critical for the workload, but the customer wanted to ensureperformance critical file-backed pages like code section to not mistakenlybe evicted.Using direct IO should or `mlock()` would be a straightforward solution,but modifying the user space code is not easy for the customer.Alternatively, we could use DAMON-based operation scheme[1]. By using it,we can ask DAMON to track access frequency of each region and make'process_madvise(MADV_WILLNEED)[2]' call for regions having specific sizeand access frequency for a time interval.We also found the system is having high number of TLB misses. We tried'always' THP enabled policy and it greatly reduced TLB misses, but thepage reclamation also been more frequent due to the THP internalfragmentation caused memory bloat. We could try another DAMON-basedoperation scheme that applies 'MADV_HUGEPAGE' to memory regions having>=2MB size and high access frequency, while applying 'MADV_NOHUGEPAGE' toregions having <2MB size and low access frequency.We do not own the systems so we only reported the analysis results andpossible optimization solutions to the customers. The customers satisfiedabout the analysis results and promised to try the optimization guides.[1] https://lore.kernel.org/linux-mm/20201006123931.5847-1-sjpark@amazon.com/[2] https://lore.kernel.org/linux-api/20200622192900.22757-4-minchan@kernel.org/Comparison with Idle Page Tracking==================================Idle Page Tracking allows users to set and read idleness of pages using abitmap file which represents each page with each bit of the file. Onerecommended usage of it is working set size detection. Users can do thatby 1. find PFN of each page for workloads in interest, 2. set all the pages as idle by doing writes to the bitmap file, 3. wait until the workload accesses its working set, and 4. read the idleness of the pages again and count pages became not idle.NOTE: While Idle Page Tracking is for user space users, DAMON is primarilydesigned for kernel subsystems though it can easily exposed to the userspace. Hence, this section only assumes such user space use of DAMON.For what use cases Idle Page Tracking would be better?------------------------------------------------------1. Flexible usecases other than hotness monitoring.Because Idle Page Tracking allows users to control the primitive (Pageidleness) by themselves, Idle Page Tracking users can do anything theywant. Meanwhile, DAMON is primarily designed to monitor the hotness ofeach memory region. For this, DAMON asks users to provide samplinginterval and aggregation interval. For the reason, there could be someuse case that using Idle Page Tracking is simpler.2. Physical memory monitoring.Idle Page Tracking receives PFN range as input, so natively supportsphysical memory monitoring.DAMON is designed to be extensible for multiple address spaces and usecases by implementing and using primitives for the given use case.Therefore, by theory, DAMON has no limitation in the type of targetaddress space as long as primitives for the given address space exists.However, the default primitives introduced by this patchset supports onlyvirtual address spaces.Therefore, for physical memory monitoring, you should implement your ownprimitives and use it, or simply use Idle Page Tracking.Nonetheless, RFC patchsets[1] for the physical memory address spaceprimitives is already available. It also supports user memory same toIdle Page Tracking.[1] https://lore.kernel.org/linux-mm/20200831104730.28970-1-sjpark@amazon.com/For what use cases DAMON is better?-----------------------------------1. Hotness Monitoring.Idle Page Tracking let users know only if a page frame is accessed or not.For hotness check, the user should write more code and use more memory.DAMON do that by itself.2. Low Monitoring OverheadDAMON receives user's monitoring request with one step and then providethe results. So, roughly speaking, DAMON require only O(1) user/kernelcontext switches.In case of Idle Page Tracking, however, because the interface receivescontiguous page frames, the number of user/kernel context switchesincreases as the monitoring target becomes complex and huge. As a result,the context switch overhead could be not negligible.Moreover, DAMON is born to handle with the monitoring overhead. Becausethe core mechanism is pure logical, Idle Page Tracking users might be ableto implement the mechanism on their own, but it would be time consumingand the user/kernel context switching will still more frequent than thatof DAMON. Also, the kernel subsystems cannot use the logic in this case.3. Page granularity working set size detection.Until v22 of this patchset, this was categorized as the thing Idle PageTracking could do better, because DAMON basically maintains additionalmetadata for each of the monitoring target regions. So, in the pagegranularity working set size detection use case, DAMON would incur (numberof monitoring target pages * size of metadata) memory overhead. Size ofthe single metadata item is about 54 bytes, so assuming 4KB pages, about1.3% of monitoring target pages will be additionally used.All essential metadata for Idle Page Tracking are embedded in 'structpage' and page table entries. Therefore, in this use case, only onecounter variable for working set size accounting is required if Idle PageTracking is used.There are more details to consider, but roughly speaking, this is true inmost cases.However, the situation changed from v23. Now DAMON supports arbitrarytypes of monitoring targets, which don't use the metadata. Using that,DAMON can do the working set size detection with no additional spaceoverhead but less user-kernel context switch. A first draft for theimplementation of monitoring primitives for this usage is available in aDAMON development tree[1]. An RFC patchset for it based on this patchsetwill also be available soon.Since v24, the arbitrary type support is dropped from this patchsetbecause this patchset doesn't introduce real use of the type. You canstill get it from the DAMON development tree[2], though.[1] https://github.com/sjp38/linux/tree/damon/pgidle_hack[2] https://github.com/sjp38/linux/tree/damon/master4. More future usecasesWhile Idle Page Tracking has tight coupling with base primitives (PG_Idleand page table Accessed bits), DAMON is designed to be extensible for manyuse cases and address spaces. If you need some special address type orwant to use special h/w access check primitives, you can write your ownprimitives for that and configure DAMON to use those. Therefore, if youruse case could be changed a lot in future, using DAMON could be better.Can I use both Idle Page Tracking and DAMON?--------------------------------------------Yes, though using them concurrently for overlapping memory regions couldresult in interference to each other. Nevertheless, such use case wouldbe rare or makes no sense at all. Even in the case, the noise would botbe really significant. So, you can choose whatever you want depending onthe characteristics of your use cases.More Information================We prepared a showcase web site[1] that you can get more information.There are- the official documentations[2],- the heatmap format dynamic access pattern of various realistic workloads for heap area[3], mmap()-ed area[4], and stack[5] area,- the dynamic working set size distribution[6] and chronological working set size changes[7], and- the latest performance test results[8].[1] https://damonitor.github.io/_index[2] https://damonitor.github.io/doc/html/latest-damon[3] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.0.png.html[4] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html[5] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.2.png.html[6] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html[7] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html[8] https://damonitor.github.io/test/result/perf/latest/html/index.htmlBaseline and Complete Git Trees===============================The patches are based on the latest -mm tree, specificallyv5.14-rc1-mmots-2021-07-15-18-47 of https://github.com/hnaz/linux-mm. You canalso clone the complete git tree: $ git clone git://github.com/sjp38/linux -b damon/patches/v34The web is also available:https://github.com/sjp38/linux/releases/tag/damon/patches/v34Development Trees-----------------There are a couple of trees for entire DAMON patchset series and featuresfor future release.- For latest release: https://github.com/sjp38/linux/tree/damon/master- For next release: https://github.com/sjp38/linux/tree/damon/nextLong-term Support Trees-----------------------For people who want to test DAMON but using LTS kernels, there are anothercouple of trees based on two latest LTS kernels respectively andcontaining the 'damon/master' backports.- For v5.4.y: https://github.com/sjp38/linux/tree/damon/for-v5.4.y- For v5.10.y: https://github.com/sjp38/linux/tree/damon/for-v5.10.yAmazon Linux Kernel Trees-------------------------DAMON is also merged in two public Amazon Linux kernel trees that based onv5.4.y[1] and v5.10.y[2].[1] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon[2] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damonGit Tree for Diff of Patches============================For easy review of diff between different versions of each patch, Iprepared a git tree containing all versions of the DAMON patchset series:https://github.com/sjp38/damon-patchesYou can clone it and use 'diff' for easy review of changes betweendifferent versions of the patchset. For example: $ git clone https://github.com/sjp38/damon-patches && cd damon-patches $ diff -u damon/v33 damon/v34Sequence Of Patches===================First three patches implement the core logics of DAMON. The 1st patchintroduces basic sampling based hotness monitoring for arbitrary types oftargets. Following two patches implement the core mechanisms for controlof overhead and accuracy, namely regions based sampling (patch 2) andadaptive regions adjustment (patch 3).Now the essential parts of DAMON is complete, but it cannot work unlesssomeone provides monitoring primitives for a specific use case. Thefollowing two patches make it just work for virtual address spacesmonitoring. The 4th patch makes 'PG_idle' can be used by DAMON and the5th patch implements the virtual memory address space specific monitoringprimitives using page table Accessed bits and the 'PG_idle' page flag.Now DAMON just works for virtual address space monitoring via the kernelspace api. To let the user space users can use DAMON, following fourpatches add interfaces for them. The 6th patch adds a tracepoint formonitoring results. The 7th patch implements a DAMON application kernelmodule, namely damon-dbgfs, that simply wraps DAMON and exposes DAMONinterface to the user space via the debugfs interface. The 8th patchfurther exports pid of monitoring thread (kdamond) to user space foreasier cpu usage accounting, and the 9th patch makes the debugfs interfaceto support multiple contexts.Three patches for maintainability follows. The 10th patch addsdocumentations for both the user space and the kernel space. The 11thpatch provides unit tests (based on the kunit) while the 12th patch addsuser space tests (based on the kselftest).Finally, the last patch (13th) updates the MAINTAINERS file.This patch (of 13):DAMON is a data access monitoring framework for the Linux kernel. Thecore mechanisms of DAMON make it - accurate (the monitoring output is useful enough for DRAM level performance-centric memory management; It might be inappropriate for CPU cache levels, though), - light-weight (the monitoring overhead is normally low enough to be applied online), and - scalable (the upper-bound of the overhead is in constant range regardless of the size of target workloads).Using this framework, hence, we can easily write efficient kernel spacedata access monitoring applications. For example, the kernel's memorymanagement mechanisms can make advanced decisions using this.Experimental data access aware optimization works that incurring highaccess monitoring overhead could again be implemented on top of this.Due to its simple and flexible interface, providing user space interfacewould be also easy. Then, user space users who have some specialworkloads can write personalized applications for better understanding andoptimizations of their workloads and systems.===Nevertheless, this commit is defining and implementing only basic accesscheck part without the overhead-accuracy handling core logic. The basicaccess check is as below.The output of DAMON says what memory regions are how frequently accessedfor a given duration. The resolution of the access frequency iscontrolled by setting ``sampling interval`` and ``aggregation interval``.In detail, DAMON checks access to each page per ``sampling interval`` andaggregates the results. In other words, counts the number of the accessesto each region. After each ``aggregation interval`` passes, DAMON callscallback functions that previously registered by users so that users canread the aggregated results and then clears the results. This can bedescribed in below simple pseudo-code:: init() while monitoring_on: for page in monitoring_target: if accessed(page): nr_accesses[page] += 1 if time() % aggregation_interval == 0: for callback in user_registered_callbacks: callback(monitoring_target, nr_accesses) for page in monitoring_target: nr_accesses[page] = 0 if time() % update_interval == 0: update() sleep(sampling interval)The target regions constructed at the beginning of the monitoring andupdated after each ``regions_update_interval``, because the target regionscould be dynamically changed (e.g., mmap() or memory hotplug). Themonitoring overhead of this mechanism will arbitrarily increase as thesize of the target workload grows.The basic monitoring primitives for actual access check and dynamic targetregions construction aren't in the core part of DAMON. Instead, it allowsusers to implement their own primitives that are optimized for their usecase and configure DAMON to use those. In other words, users cannot usecurrent version of DAMON without some additional works.Following commits will implement the core mechanisms for theoverhead-accuracy control and default primitives implementations.Link: https://lkml.kernel.org/r/20210716081449.22187-1-sj38.park@gmail.comLink: https://lkml.kernel.org/r/20210716081449.22187-2-sj38.park@gmail.comSigned-off-by: SeongJae Park Reviewed-by: Leonard Foerster Reviewed-by: Fernand Sieber Acked-by: Shakeel Butt Cc: Jonathan Cameron Cc: Alexander Shishkin Cc: Amit Shah Cc: Benjamin Herrenschmidt Cc: Jonathan Corbet Cc: David Hildenbrand Cc: David Woodhouse Cc: Marco Elver Cc: Fan Du Cc: Greg Kroah-Hartman Cc: Greg Thelen Cc: Joe Perches Cc: Mel Gorman Cc: Maximilian Heyne Cc: Minchan Kim Cc: Ingo Molnar Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Rik van Riel Cc: David Rientjes Cc: Steven Rostedt (VMware) Cc: Shuah Khan Cc: Vlastimil Babka Cc: Vladimir Davydov Cc: Brendan Higgins Cc: Markus Boehme Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds List of files: /linux-6.15/mm/damon/Makefile