Page Replacement in Linux - AKA Kernel大会

Page Replacement in LinuxWu FengguangOctober 22, 2009

Outline. .1 page types and organization. .2 page reclaim logicsWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 2 / 32

page reclaimWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 3 / 32

memory zonesWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 4 / 32

zoned page allocationDMA 24-bit addressableDMA32 32-bit addressableNORMAL direct addressableHIGHMEM addressable via (temp) mapping.__get_free_page(gfp_mask)..zone modifier | zone fallback order-----------------+---------------------------------------__GFP_DMA | DMA__GFP_DMA32 | DMA32 => (DMA)(unspecified) | NORMAL => DMA32 => (DMA)__GFP_HIGHMEM | HIGHMEM => NORMAL => DMA32 => (DMA).....Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 5 / 32

zone pagesWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 6 / 32

zone pages (cont.)free pagesLRU cache (in 5 lists)file backed pages (active + inactive)swap backed pages (active + inactive)unevictable pagesslab cache (reclaimable)icachedcacheother pagesWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 7 / 32

file backed pagesManaged in both radix tree and file LRU lists.Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 8 / 32

page referenced bitsread() sets PageReferenced or PageActive (activate it)mmap read sets pte_young, to be examined at reclaim timeWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 9 / 32

general cache rulesIn order to stay in memory:unmapped have to be referenced 2 times when in active+inactive listmapped have to be referenced 1 times when in inactive listWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 10 / 32

activate rulesactivate on the 2nd read()/write() accessactivate accessed mapped pages; some unfreeable pagesre-activate accessed program textWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 11 / 32

evict rulesde-activate all unmapped pages; mapped data pagespageout all unmapped pages; not accessed mapped pagesrecycle pages put to writeback; some unreclaimable pages for nowWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 12 / 32

alanced aging: whyOn sequential read, need a small (24kb) I/O to fill the hole.Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 13 / 32

alanced aging: howfor ratio in 14096 , 12048 , 11024 , . . . , 12 , 1for each zonescan (zone_nr_lru_pages * ratio) pages;if enough pages reclaimedbreak;Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 14 / 32

alanced aging: overscan of high zonesproblemic scenario:the 16MB DMA zone is cycled once per second=>the 500MB NORMAL zone is cycled once per second=>all pages not referenced within 1 second are evicted=>huge nr_free_pagesWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 15 / 32

alanced aging: how 2for ratio in 14096 , 12048 , 11024 , . . . , 12 , 1for each zone from target zone to ZONE_DMAscan (zone_nr_lru_pages * ratio) pages;if enough pages reclaimedbreak;Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 16 / 32

alanced aging: overscan of lower zonesWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 17 / 32

alanced aging: low mem reserveWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 18 / 32

alanced aging: over reclaimfor ratio in 14096 , 12048 , 11024 , . . . , 12 , 1for each zone from target zone to lowest zonescan (zone_nr_lru_pages * ratio) pagesif enough pages reclaimedbreak;Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 19 / 32

alanced aging: early breakSo we reach the current 2.6.32 logic. Problems of the loops:risks overscanning lower zonesrisks overscanning small zonesfor ratio in 14096 , 12048 , 11024 , . . . , 12 , 1for each zone from target zone to lowest zonescan (zone_nr_lru_pages * ratio) pages,until enough pages reclaimed && ...if enough pages reclaimedbreak;Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 20 / 32

alanced aging: NUMA nodesPrimary goals:avoid off node allocation, orspread NUMA interlink traffic evenlyno explicit efforts for balanced scan of NUMA nodesaging rates are heavily impacted by page allocation policyNUMA memory policies: interleave, bind, preferred, defaultWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 21 / 32

zone reclaim in a NUMA systemmemory is exhausted locally => off node allocation of anonymous pagessource: Local and Remote Memory: Memory in a Linux/NUMA System, Christoph Lameterzone reclaim:start reclaiming (unmapped) pages if a zone’s free memory is lowWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 22 / 32

direct reclaim in a busy systemkswapd reclaim kswapd frees memory in the background, so that lightpage allocations can complete instantly;direct reclaim when kswapd cannot keep up with the allocation rate,applications try to free memory for themselves.helps throttle heavy allocation appsNUMA: focus reclaim on allocation heavy nodesmay isolate too many LRU pages on fork bombsWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 23 / 32

eclaim scalability: unevictable listto avoiding scanning the unreclaimable pages:pages pinned by mlock() or SHM_LOCKmemory backed pages (ramfs, ramdisk)Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 24 / 32

eclaim scalability: mapped pageswe used to avoid scanning and evicting mapped pagesmapped pages are more costly to scan/reclaim/refaultmapped pages are more valuable to cache in typical desktopnow tend to de-activate mapped pages unconditionallylong round trip time (eg. 1TB mem)every page have pte_youngVM_EXEC mapped file pages are still protected as usual(for better desktop responsiveness)Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 25 / 32

eclaim efficiency: lumpy reclaimrational excessive reclaims for high-order page allocationsolution lumpy reclaim for high-order page:.1 the first page is taken from LRU list (as before),. .2 try to reclaim the (physically) surrounding pages(to form a high-order block)problem distort LRU; randomly punching holes in cached filesWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 26 / 32

split file/anon LRU listsfile LRU file backed pagesanon LRU mem/swap backed pagesanonymous pagestmpfs pagesscan ratio ∝ recent_scanned : recent_referenced(more referenced, less scanned)protects anonymous pagesimproves reclaim efficiencyWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 27 / 32

anon LRU active:inactive ratiobalanced scan in normalshrink active list iif inactive list is lowinactive list serves as the only grace window for active referencesnote that when inactive list => 0, LRU reduce to FIFOactive:inactive => sqrt(10 * GB)Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 28 / 32

file LRU active:inactive ratiobalanced scan in normalshrink active list only when inactive list is lowactive list: working setinactive list: use-once dataprotect working set from being flushed by use-once dataactive:inactive => 1:1Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 29 / 32

advanced page replacementcurrent logic: balanced but not optimalobvious optimization: streaming IOdrop behinduse frequency instead of recencyavailable solutionsCLOCK-Pro, CART, CAR, ARC, LIRS etc.problemscomplexitypossible regressionsWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 30 / 32

eadings and resourcesLinuxMMhttp://linux-mm.org/linux-mm mailing listhttp://marc.info/?l=linux-mmlinux kernel source code/usr/src/linux $ git log mmbook Understanding The Linux Virtual Memory Managerhttp://www.skynet.ie/~mel/projects/vm/guide/pdf/understand.pdfWu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 31 / 32

Thank you!Wu Fengguang (Intel OTC) Page Replacement in Linux 2009 AKA kernel conference 32 / 32