malloc, arena, brk, mmap | Linux System Programming

QUESTION

Why does process RSS remain unchanged after free()?

malloc/free are libc allocator APIs; the kernel does not know individual allocation sizes or object types. The allocator manages small objects by dividing the heap grown with brk and arenas/chunks obtained with mmap.

free only returns an object to the allocator. It does not guarantee an immediate munmap or return of the pages to the kernel. Live allocations in the same arena, bin caches, and fragmentation can leave both virtual ranges and resident pages in place.

STRUCTURE

Structure diagram

Figure 1. Four nested layers from malloc objects to physical pages

malloc APIrequested size · ownership · free

Thread cache / bin

free chunk
size class
fast reuse

Arena

allocated chunk
free hole
top chunk

Virtual mappings

brk heap
large mmap
allocator metadata

Resident pages

faulted page
Private_Dirty
return through trim/madvise

free returns a user object to the allocator. If the same page contains a live chunk, its VMA and RSS can remain unchanged.

CALL PATH

Call path

Figure 2. From userspace code to observable results

malloc request size/alignment

allocator bin find free chunk

brk/mmap if arena expansion is needed

user object caller ownership

free/trim return to cache or kernel

The malloc object graph, allocator chunks, VMAs, and physical pages have four different lifetimes. Do not use one tool for both leak analysis and RSS analysis.

Figure 3. Major points along the kernel-internal path

brk syscall adjust end of heap VMA

mmap new anonymous VMA

page fault obtain actual pages

madvise/munmap return page/range

RSS resident accounting

This diagram is not for memorizing function names. Follow which return value, file descriptor, memory mapping, or wait queue is passed to the next stage.

SOURCE COORDINATES

Linux 6.18.37 LTS source locations

Go beyond the glibc function to the files where the syscall implementation meets kernel objects. Each link points to the original file at the same tag.

File	Function / structure	What to inspect
mm/mmap.c	do_brk_flags(), do_mmap()	Change the virtual range requested by the allocator
mm/memory.c	do_anonymous_page()	Demand allocation of heap/mmap pages
mm/madvise.c	madvise_dontneed_single_vma()	A path that may return resident pages when the allocator trims

COMPLETE PROGRAM

Complete runnable example

The code below is not pseudocode with explanatory lines omitted. It is a minimal example that can be built and run as a file.

Buildcc -std=c17 -Wall -Wextra -O2 allocator_rss.c -o allocator_rss

01#define _GNU_SOURCE
02#include <malloc.h>
03#include <stdio.h>
04#include <stdlib.h>
05#include <string.h>
06#include <unistd.h>
07
08int main(void)
09{
10    size_t length = 128UL * 1024 * 1024;
11    unsigned char *buffer = malloc(length);
12    if (buffer == NULL)
13        return 1;
14    memset(buffer, 0xa5, length);
15    printf("allocated and touched; pid=%ld\n", (long)getpid());
16    getchar();
17
18    free(buffer);
19    puts("freed to allocator");
20    getchar();
21
22    int released = malloc_trim(0);
23    printf("malloc_trim=%d\n", released);
24    getchar();
25    return 0;
26}

CODE NOTES

Code notes

Source line 11malloc(length)

The request allocates a virtual object. Depending on size and settings, glibc chooses the arena top chunk or a separate mmap.

Source line 14memset(buffer

Faults in each page with a write, distinguishing an overcommit reservation from an actual increase in RSS.

Source line 18free(buffer)

Pointer ownership returns to the allocator, but whether pages immediately return to the kernel depends on chunk placement and allocator policy.

Source line 22malloc_trim(0)

A nonstandard extension that asks glibc to return unused heap pages to the kernel. A return value of 1 does not mean every free page disappeared.

Source line 16getchar();

Pauses at each stage so an external shell can compare /proc/PID/smaps_rollup with pmap.

DETAILS

Detailed behavior

Large- and small-allocation paths differ

A large allocation can use a separate mmap and be easy to munmap on free. Small allocations are mixed in arena pages, making an entire page hard to return while even one object remains live. The threshold depends on implementation, tuning, and execution history.

Do not treat a fixed behavior at one size as an API contract.

Multithreaded arenas trade RSS for contention

An allocator may maintain multiple arenas and thread caches to reduce lock contention. If empty chunks are scattered across arenas after a peak, process-wide RSS can remain high.

Apply a tunable such as MALLOC_ARENA_MAX only after measuring workload latency and fragmentation.

Distinguish leaks from retention

A leak is an application failing to free an unreachable object; retention is the allocator keeping freed pages for reuse. Both produce high RSS, but the remedies differ.

Use ASan/LSan, a heap profiler, malloc_info, and smaps together.

OBJECTS

Objects and lifetimes

Object	Creation and release	Values to inspect
`user allocation`	Passed to the caller by malloc and returned to allocator ownership by free	requested/usable size, owner
`allocator chunk/arena`	Reused throughout the process lifetime, with some ranges returned by trim/unmap	bin, top chunk, fragmentation
`anonymous VMA/page`	Created by brk/mmap and faults, and reduced by madvise/munmap/reclaim	Rss, Private_Dirty

FAILURE PATH

Failure conditions and common misconceptions

Observed symptom	Likely causes	How to verify
RSS remains high after free	arena retention/fragmentation	Compare malloc_info with smaps
malloc NULL/abort	ENOMEM or heap corruption	errno, sanitizer, core dump
latency spike	arena lock, page fault, mmap/munmap	Inspect perf lock/fault and an allocator profiler

LAB

Verify it yourself

At each of the three getchar stages, record Rss/Private_Dirty from smaps_rollup and brk/mmap/munmap from strace.
Instead of one 128 MiB allocation, intermix many 4 KiB objects and partially free them to observe the fragmentation difference.
Vary MALLOC_ARENA_MAX and measure both peak RSS and throughput in a multithreaded allocation benchmark.

Run./allocator_rss

Tracestrace -e trace=brk,mmap,madvise,munmap ./allocator_rss

PRIMARY REFERENCES

Primary sources

malloc(3)malloc_trim(3)glibc Memory Allocation