요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
1
========================================
2
Short users guide for the slab allocator
3
========================================
5
The slab allocator includes full debugging support (when built with
6
CONFIG_SLUB_DEBUG=y) but it is off by default (unless built with
7
CONFIG_SLUB_DEBUG_ON=y). You can enable debugging only for selected
8
slabs in order to avoid an impact on overall system performance which
9
may make a bug more difficult to find.
11
In order to switch debugging on one can add an option ``slab_debug``
12
to the kernel command line. That will enable full debugging for
13
all slabs.
15
Typically one would then use the ``slabinfo`` command to get statistical
16
data and perform operation on the slabs. By default ``slabinfo`` only lists
17
slabs that have data in them. See "slabinfo -h" for more options when
18
running the command. ``slabinfo`` can be compiled with
19
::
21
gcc -o slabinfo tools/mm/slabinfo.c
23
Some of the modes of operation of ``slabinfo`` require that slub debugging
24
be enabled on the command line. F.e. no tracking information will be
25
available without debugging on and validation can only partially
26
be performed if debugging was not switched on.
28
Some more sophisticated uses of slab_debug:
29
-------------------------------------------
31
Parameters may be given to ``slab_debug``. If none is specified then full
32
debugging is enabled. Format:
34
slab_debug=<Debug-Options>
35
Enable options for all slabs
37
slab_debug=<Debug-Options>,<slab name1>,<slab name2>,...
38
Enable options only for select slabs (no spaces
39
after a comma)
41
Multiple blocks of options for all slabs or selected slabs can be given, with
42
blocks of options delimited by ';'. The last of "all slabs" blocks is applied
43
to all slabs except those that match one of the "select slabs" block. Options
44
of the first "select slabs" blocks that matches the slab's name are applied.
46
Possible debug options are::
48
F Sanity checks on (enables SLAB_DEBUG_CONSISTENCY_CHECKS
49
Sorry SLAB legacy issues)
50
Z Red zoning
51
P Poisoning (object and padding)
52
U User tracking (free and alloc)
53
T Trace (please only use on single slabs)
54
A Enable failslab filter mark for the cache
55
O Switch debugging off for caches that would have
56
caused higher minimum slab orders
57
- Switch all debugging off (useful if the kernel is
58
configured with CONFIG_SLUB_DEBUG_ON)
60
F.e. in order to boot just with sanity checks and red zoning one would specify::
62
slab_debug=FZ
64
Trying to find an issue in the dentry cache? Try::
66
slab_debug=,dentry
68
to only enable debugging on the dentry cache. You may use an asterisk at the
69
end of the slab name, in order to cover all slabs with the same prefix. For
70
example, here's how you can poison the dentry cache as well as all kmalloc
71
slabs::
73
slab_debug=P,kmalloc-*,dentry
75
Red zoning and tracking may realign the slab. We can just apply sanity checks
76
to the dentry cache with::
78
slab_debug=F,dentry
80
Debugging options may require the minimum possible slab order to increase as
81
a result of storing the metadata (for example, caches with PAGE_SIZE object
82
sizes). This has a higher likelihood of resulting in slab allocation errors
83
in low memory situations or if there's high fragmentation of memory. To
84
switch off debugging for such caches by default, use::
86
slab_debug=O
88
You can apply different options to different list of slab names, using blocks
89
of options. This will enable red zoning for dentry and user tracking for
90
kmalloc. All other slabs will not get any debugging enabled::
92
slab_debug=Z,dentry;U,kmalloc-*
94
You can also enable options (e.g. sanity checks and poisoning) for all caches
95
except some that are deemed too performance critical and don't need to be
96
debugged by specifying global debug options followed by a list of slab names
97
with "-" as options::
99
slab_debug=FZ;-,zs_handle,zspage
101
The state of each debug option for a slab can be found in the respective files
102
under::
104
/sys/kernel/slab/<slab name>/
106
If the file contains 1, the option is enabled, 0 means disabled. The debug
107
options from the ``slab_debug`` parameter translate to the following files::
109
F sanity_checks
110
Z red_zone
111
P poison
112
U store_user
113
T trace
114
A failslab
116
failslab file is writable, so writing 1 or 0 will enable or disable
117
the option at runtime. Write returns -EINVAL if cache is an alias.
118
Careful with tracing: It may spew out lots of information and never stop if
119
used on the wrong slab.
121
Slab merging
122
============
124
If no debug options are specified then SLUB may merge similar slabs together
125
in order to reduce overhead and increase cache hotness of objects.
126
``slabinfo -a`` displays which slabs were merged together.
128
Slab validation
129
===============
131
SLUB can validate all object if the kernel was booted with slab_debug. In
132
order to do so you must have the ``slabinfo`` tool. Then you can do
133
::
135
slabinfo -v
137
which will test all objects. Output will be generated to the syslog.
139
This also works in a more limited way if boot was without slab debug.
140
In that case ``slabinfo -v`` simply tests all reachable objects. Usually
141
these are in the cpu slabs and the partial slabs. Full slabs are not
142
tracked by SLUB in a non debug situation.
144
Getting more performance
145
========================
147
To some degree SLUB's performance is limited by the need to take the
148
list_lock once in a while to deal with partial slabs. That overhead is
149
governed by the order of the allocation for each slab. The allocations
150
can be influenced by kernel parameters:
152
.. slab_min_objects=x (default: automatically scaled by number of cpus)
153
.. slab_min_order=x (default 0)
154
.. slab_max_order=x (default 3 (PAGE_ALLOC_COSTLY_ORDER))
156
``slab_min_objects``
157
allows to specify how many objects must at least fit into one
158
slab in order for the allocation order to be acceptable. In
159
general slub will be able to perform this number of
160
allocations on a slab without consulting centralized resources
161
(list_lock) where contention may occur.
163
``slab_min_order``
164
specifies a minimum order of slabs. A similar effect like
165
``slab_min_objects``.
167
``slab_max_order``
168
specified the order at which ``slab_min_objects`` should no
169
longer be checked. This is useful to avoid SLUB trying to
170
generate super large order pages to fit ``slab_min_objects``
171
of a slab cache with large object sizes into one high order
172
page. Setting command line parameter
173
``debug_guardpage_minorder=N`` (N > 0), forces setting
174
``slab_max_order`` to 0, what cause minimum possible order of
175
slabs allocation.
177
``slab_strict_numa``
178
Enables the application of memory policies on each
179
allocation. This results in more accurate placement of
180
objects which may result in the reduction of accesses
181
to remote nodes. The default is to only apply memory
182
policies at the folio level when a new folio is acquired
183
or a folio is retrieved from the lists. Enabling this
184
option reduces the fastpath performance of the slab allocator.
186
SLUB Debug output
187
=================
189
Here is a sample of slub debug output::
191
====================================================================
192
BUG kmalloc-8: Right Redzone overwritten
193
--------------------------------------------------------------------
195
INFO: 0xc90f6d28-0xc90f6d2b. First byte 0x00 instead of 0xcc
196
INFO: Slab 0xc528c530 flags=0x400000c3 inuse=61 fp=0xc90f6d58
197
INFO: Object 0xc90f6d20 @offset=3360 fp=0xc90f6d58
198
INFO: Allocated in get_modalias+0x61/0xf5 age=53 cpu=1 pid=554
200
Bytes b4 (0xc90f6d10): 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ
201
Object (0xc90f6d20): 31 30 31 39 2e 30 30 35 1019.005
202
Redzone (0xc90f6d28): 00 cc cc cc .
203
Padding (0xc90f6d50): 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
205
[<c010523d>] dump_trace+0x63/0x1eb
206
[<c01053df>] show_trace_log_lvl+0x1a/0x2f
207
[<c010601d>] show_trace+0x12/0x14
208
[<c0106035>] dump_stack+0x16/0x18
209
[<c017e0fa>] object_err+0x143/0x14b
210
[<c017e2cc>] check_object+0x66/0x234
211
[<c017eb43>] __slab_free+0x239/0x384
212
[<c017f446>] kfree+0xa6/0xc6
213
[<c02e2335>] get_modalias+0xb9/0xf5
214
[<c02e23b7>] dmi_dev_uevent+0x27/0x3c
215
[<c027866a>] dev_uevent+0x1ad/0x1da
216
[<c0205024>] kobject_uevent_env+0x20a/0x45b
217
[<c020527f>] kobject_uevent+0xa/0xf
218
[<c02779f1>] store_uevent+0x4f/0x58
219
[<c027758e>] dev_attr_store+0x29/0x2f
220
[<c01bec4f>] sysfs_write_file+0x16e/0x19c
221
[<c0183ba7>] vfs_write+0xd1/0x15a
222
[<c01841d7>] sys_write+0x3d/0x72
223
[<c0104112>] sysenter_past_esp+0x5f/0x99
224
[<b7f7b410>] 0xb7f7b410
225
=======================
227
FIX kmalloc-8: Restoring Redzone 0xc90f6d28-0xc90f6d2b=0xcc
229
If SLUB encounters a corrupted object (full detection requires the kernel
230
to be booted with slab_debug) then the following output will be dumped
231
into the syslog:
233
1. Description of the problem encountered
235
This will be a message in the system log starting with::
237
===============================================
238
BUG <slab cache affected>: <What went wrong>
239
-----------------------------------------------
241
INFO: <corruption start>-<corruption_end> <more info>
242
INFO: Slab <address> <slab information>
243
INFO: Object <address> <object information>
244
INFO: Allocated in <kernel function> age=<jiffies since alloc> cpu=<allocated by
245
cpu> pid=<pid of the process>
246
INFO: Freed in <kernel function> age=<jiffies since free> cpu=<freed by cpu>
247
pid=<pid of the process>
249
(Object allocation / free information is only available if SLAB_STORE_USER is
250
set for the slab. slab_debug sets that option)
252
2. The object contents if an object was involved.
254
Various types of lines can follow the BUG SLUB line:
256
Bytes b4 <address> : <bytes>
257
Shows a few bytes before the object where the problem was detected.
258
Can be useful if the corruption does not stop with the start of the
259
object.
261
Object <address> : <bytes>
262
The bytes of the object. If the object is inactive then the bytes
263
typically contain poison values. Any non-poison value shows a
264
corruption by a write after free.
266
Redzone <address> : <bytes>
267
The Redzone following the object. The Redzone is used to detect
268
writes after the object. All bytes should always have the same
269
value. If there is any deviation then it is due to a write after
270
the object boundary.
272
(Redzone information is only available if SLAB_RED_ZONE is set.
273
slab_debug sets that option)
275
Padding <address> : <bytes>
276
Unused data to fill up the space in order to get the next object
277
properly aligned. In the debug case we make sure that there are
278
at least 4 bytes of padding. This allows the detection of writes
279
before the object.
281
3. A stackdump
283
The stackdump describes the location where the error was detected. The cause
284
of the corruption is may be more likely found by looking at the function that
285
allocated or freed the object.
287
4. Report on how the problem was dealt with in order to ensure the continued
288
operation of the system.
290
These are messages in the system log beginning with::
292
FIX <slab cache affected>: <corrective action taken>
294
In the above sample SLUB found that the Redzone of an active object has
295
been overwritten. Here a string of 8 characters was written into a slab that
296
has the length of 8 characters. However, a 8 character string needs a
297
terminating 0. That zero has overwritten the first byte of the Redzone field.
298
After reporting the details of the issue encountered the FIX SLUB message
299
tells us that SLUB has restored the Redzone to its proper value and then
300
system operations continue.
302
Emergency operations
303
====================
305
Minimal debugging (sanity checks alone) can be enabled by booting with::
307
slab_debug=F
309
This will be generally be enough to enable the resiliency features of slub
310
which will keep the system running even if a bad kernel component will
311
keep corrupting objects. This may be important for production systems.
312
Performance will be impacted by the sanity checks and there will be a
313
continual stream of error messages to the syslog but no additional memory
314
will be used (unlike full debugging).
316
No guarantees. The kernel component still needs to be fixed. Performance
317
may be optimized further by locating the slab that experiences corruption
318
and enabling debugging only for that cache
320
I.e.::
322
slab_debug=F,dentry
324
If the corruption occurs by writing after the end of the object then it
325
may be advisable to enable a Redzone to avoid corrupting the beginning
326
of other objects::
328
slab_debug=FZ,dentry
330
Extended slabinfo mode and plotting
331
===================================
333
The ``slabinfo`` tool has a special 'extended' ('-X') mode that includes:
334
- Slabcache Totals
335
- Slabs sorted by size (up to -N <num> slabs, default 1)
336
- Slabs sorted by loss (up to -N <num> slabs, default 1)
338
Additionally, in this mode ``slabinfo`` does not dynamically scale
339
sizes (G/M/K) and reports everything in bytes (this functionality is
340
also available to other slabinfo modes via '-B' option) which makes
341
reporting more precise and accurate. Moreover, in some sense the `-X'
342
mode also simplifies the analysis of slabs' behaviour, because its
343
output can be plotted using the ``slabinfo-gnuplot.sh`` script. So it
344
pushes the analysis from looking through the numbers (tons of numbers)
345
to something easier -- visual analysis.
347
To generate plots:
349
a) collect slabinfo extended records, for example::
351
while [ 1 ]; do slabinfo -X >> FOO_STATS; sleep 1; done
353
b) pass stats file(-s) to ``slabinfo-gnuplot.sh`` script::
355
slabinfo-gnuplot.sh FOO_STATS [FOO_STATS2 .. FOO_STATSN]
357
The ``slabinfo-gnuplot.sh`` script will pre-processes the collected records
358
and generates 3 png files (and 3 pre-processing cache files) per STATS
359
file:
360
- Slabcache Totals: FOO_STATS-totals.png
361
- Slabs sorted by size: FOO_STATS-slabs-by-size.png
362
- Slabs sorted by loss: FOO_STATS-slabs-by-loss.png
364
Another use case, when ``slabinfo-gnuplot.sh`` can be useful, is when you
365
need to compare slabs' behaviour "prior to" and "after" some code
366
modification. To help you out there, ``slabinfo-gnuplot.sh`` script
367
can 'merge' the `Slabcache Totals` sections from different
368
measurements. To visually compare N plots:
370
a) Collect as many STATS1, STATS2, .. STATSN files as you need::
372
while [ 1 ]; do slabinfo -X >> STATS<X>; sleep 1; done
374
b) Pre-process those STATS files::
376
slabinfo-gnuplot.sh STATS1 STATS2 .. STATSN
378
c) Execute ``slabinfo-gnuplot.sh`` in '-t' mode, passing all of the
379
generated pre-processed \*-totals::
381
slabinfo-gnuplot.sh -t STATS1-totals STATS2-totals .. STATSN-totals
383
This will produce a single plot (png file).
385
Plots, expectedly, can be large so some fluctuations or small spikes
386
can go unnoticed. To deal with that, ``slabinfo-gnuplot.sh`` has two
387
options to 'zoom-in'/'zoom-out':
389
a) ``-s %d,%d`` -- overwrites the default image width and height
390
b) ``-r %d,%d`` -- specifies a range of samples to use (for example,
391
in ``slabinfo -X >> FOO_STATS; sleep 1;`` case, using a ``-r
392
40,60`` range will plot only samples collected between 40th and
393
60th seconds).
396
DebugFS files for SLUB
397
======================
399
For more information about current state of SLUB caches with the user tracking
400
debug option enabled, debugfs files are available, typically under
401
/sys/kernel/debug/slab/<cache>/ (created only for caches with enabled user
402
tracking). There are 2 types of these files with the following debug
403
information:
405
1. alloc_traces::
407
Prints information about unique allocation traces of the currently
408
allocated objects. The output is sorted by frequency of each trace.
410
Information in the output:
411
Number of objects, allocating function, possible memory wastage of
412
kmalloc objects(total/per-object), minimal/average/maximal jiffies
413
since alloc, pid range of the allocating processes, cpu mask of
414
allocating cpus, numa node mask of origins of memory, and stack trace.
416
Example:::
418
338 pci_alloc_dev+0x2c/0xa0 waste=521872/1544 age=290837/291891/293509 pid=1 cpus=106 nodes=0-1
419
__kmem_cache_alloc_node+0x11f/0x4e0
420
kmalloc_trace+0x26/0xa0
421
pci_alloc_dev+0x2c/0xa0
422
pci_scan_single_device+0xd2/0x150
423
pci_scan_slot+0xf7/0x2d0
424
pci_scan_child_bus_extend+0x4e/0x360
425
acpi_pci_root_create+0x32e/0x3b0
426
pci_acpi_scan_root+0x2b9/0x2d0
427
acpi_pci_root_add.cold.11+0x110/0xb0a
428
acpi_bus_attach+0x262/0x3f0
429
device_for_each_child+0xb7/0x110
430
acpi_dev_for_each_child+0x77/0xa0
431
acpi_bus_attach+0x108/0x3f0
432
device_for_each_child+0xb7/0x110
433
acpi_dev_for_each_child+0x77/0xa0
434
acpi_bus_attach+0x108/0x3f0
436
2. free_traces::
438
Prints information about unique freeing traces of the currently allocated
439
objects. The freeing traces thus come from the previous life-cycle of the
440
objects and are reported as not available for objects allocated for the first
441
time. The output is sorted by frequency of each trace.
443
Information in the output:
444
Number of objects, freeing function, minimal/average/maximal jiffies since free,
445
pid range of the freeing processes, cpu mask of freeing cpus, and stack trace.
447
Example:::
449
1980 <not-available> age=4294912290 pid=0 cpus=0
450
51 acpi_ut_update_ref_count+0x6a6/0x782 age=236886/237027/237772 pid=1 cpus=1
451
kfree+0x2db/0x420
452
acpi_ut_update_ref_count+0x6a6/0x782
453
acpi_ut_update_object_reference+0x1ad/0x234
454
acpi_ut_remove_reference+0x7d/0x84
455
acpi_rs_get_prt_method_data+0x97/0xd6
456
acpi_get_irq_routing_table+0x82/0xc4
457
acpi_pci_irq_find_prt_entry+0x8e/0x2e0
458
acpi_pci_irq_lookup+0x3a/0x1e0
459
acpi_pci_irq_enable+0x77/0x240
460
pcibios_enable_device+0x39/0x40
461
do_pci_enable_device.part.0+0x5d/0xe0
462
pci_enable_device_flags+0xfc/0x120
463
pci_enable_device+0x13/0x20
464
virtio_pci_probe+0x9e/0x170
465
local_pci_probe+0x48/0x80
466
pci_device_probe+0x105/0x1c0
468
Christoph Lameter, May 30, 2007
469
Sergey Senozhatsky, October 23, 2015
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
SLUB debug와 slabinfo
1-27Slab allocator는 `CONFIG_SLUB_DEBUG=y`로 build하면 완전한 debugging을 지원하지만 기본적으로 꺼져 있습니다. `CONFIG_SLUB_DEBUG_ON=y`이면 기본 활성화됩니다. 전체 system 성능 저하가 bug 재현을 어렵게 만들 수 있으므로 선택한 slab에만 debug를 켤 수 있습니다.
Kernel command line에 `slab_debug`를 추가하면 모든 slab의 full debugging을 켭니다. 통계 조회와 slab operation에는 보통 `slabinfo`를 사용하며 기본적으로 data가 있는 slab만 나열합니다. 추가 option은 `slabinfo -h`에서 확인합니다.
gcc -o slabinfo tools/mm/slabinfo.c`slabinfo`의 일부 mode는 boot command line에서 SLUB debugging을 활성화해야 합니다. Debugging이 없으면 tracking information이 없고 validation도 일부 object에만 가능합니다.
slab_debug option 문법
28-59`slab_debug`에 parameter가 없으면 full debugging을 활성화합니다. `slab_debug=<Debug-Options>`는 모든 slab에 적용하고, 뒤에 공백 없이 comma-separated slab name을 붙이면 선택한 slab에만 적용합니다.
Semicolon으로 여러 global 또는 selected block을 구분할 수 있습니다. Global block 중 마지막 것이 selected block에 match하지 않는 모든 slab에 적용되고, selected block은 slab name에 처음 match한 block의 option이 적용됩니다.
F Sanity checks on (enables SLAB_DEBUG_CONSISTENCY_CHECKS
Sorry SLAB legacy issues)
Z Red zoning
P Poisoning (object and padding)
U User tracking (free and alloc)
T Trace (please only use on single slabs)
A Enable failslab filter mark for the cache
O Switch debugging off for caches that would have
caused higher minimum slab orders
- Switch all debugging off (useful if the kernel is
configured with CONFIG_SLUB_DEBUG_ON)| Option | 기능 | 주의 |
|---|---|---|
| F | Sanity checks | SLAB_DEBUG_CONSISTENCY_CHECKS |
| Z | Red zoning | object 경계 밖 write 감지 |
| P | Poisoning | object와 padding poison |
| U | User tracking | alloc/free call site 추적 |
| T | Trace | 단일 slab에서만 사용 권장 |
| A | Failslab mark | cache를 failslab filter 대상으로 표시 |
| O | Order 보호 | minimum slab order가 커지는 cache의 debug 해제 |
| - | 모든 debug 해제 | CONFIG_SLUB_DEBUG_ON override에 유용 |
선택적 debug 예제
60-100Sanity check와 red zoning만 모든 slab에 켭니다.
slab_debug=FZDentry cache에만 full debugging을 켭니다.
slab_debug=,dentrySlab name 끝에 asterisk를 사용하면 같은 prefix의 모든 slab을 선택할 수 있습니다. 다음은 dentry와 모든 kmalloc slab을 poison합니다.
slab_debug=P,kmalloc-*,dentryRealignment를 피하고 dentry에 sanity check만 적용합니다.
slab_debug=F,dentryRed zoning과 tracking metadata는 minimum slab order를 높일 수 있고, low-memory 또는 fragmentation 상황에서 slab allocation error 가능성을 키웁니다. 그러한 cache의 debugging을 기본 해제하려면 `O`를 사용합니다.
slab_debug=OBlock별로 다른 option을 적용할 수 있습니다. 다음은 dentry에 red zoning, kmalloc prefix에 user tracking을 켜고 나머지는 끕니다.
slab_debug=Z,dentry;U,kmalloc-*Global option 뒤에 `-` selected block을 두면 성능에 민감해 debug가 필요 없는 cache만 제외할 수 있습니다.
slab_debug=FZ;-,zs_handle,zspageCache별 debug 상태
101-120각 slab의 debug option 상태는 다음 sysfs directory에서 확인합니다.
/sys/kernel/slab/<slab name>/File 값이 1이면 enabled, 0이면 disabled입니다. `slab_debug` option은 다음 file로 대응됩니다.
F sanity_checks
Z red_zone
P poison
U store_user
T trace
A failslab| Option | sysfs file |
|---|---|
| F | sanity_checks |
| Z | red_zone |
| P | poison |
| U | store_user |
| T | trace |
| A | failslab |
`failslab`은 writable이므로 1 또는 0을 써 runtime에 option을 켜거나 끌 수 있습니다. Cache가 alias이면 write는 `-EINVAL`을 반환합니다.
잘못된 slab에서 tracing을 켜면 매우 많은 정보가 끝없이 출력될 수 있으므로 단일 cache에 제한해 사용합니다.
Slab merging과 validation
121-143Debug option이 없으면 SLUB는 overhead를 줄이고 object cache hotness를 높이기 위해 유사한 slab을 merge할 수 있습니다. `slabinfo -a`는 merge된 slab 관계를 보여 줍니다.
`slab_debug`로 boot했다면 `slabinfo`로 모든 object를 validate할 수 있습니다.
slabinfo -vValidation 결과는 syslog에 출력됩니다. Slab debug 없이 boot해도 제한적으로 동작하지만 CPU slab과 partial slab처럼 reachable object만 검사합니다. Non-debug 상황에서는 full slab을 SLUB가 추적하지 않습니다.
SLUB 성능 parameter
144-185SLUB는 partial slab을 처리하기 위해 때때로 `list_lock`을 잡아야 하며, 그 overhead는 slab별 allocation order의 영향을 받습니다.
| Parameter | 기본값 | 효과 |
|---|---|---|
| slab_min_objects=x | CPU 수에 따라 자동 scale | acceptable order의 slab 하나에 들어가야 하는 최소 object 수 |
| slab_min_order=x | 0 | slab allocation의 최소 order |
| slab_max_order=x | 3 (PAGE_ALLOC_COSTLY_ORDER) | slab_min_objects 검사를 중단할 order |
| slab_strict_numa | disabled | 각 allocation마다 memory policy 적용; placement 정확도와 fastpath 성능 교환 |
`slab_min_objects`는 acceptable allocation order에서 slab 하나에 최소 몇 object가 들어가야 하는지 정합니다. 일반적으로 SLUB는 centralized resource인 `list_lock`을 다시 확인하기 전에 그 수만큼 allocation할 수 있습니다. `slab_min_order`는 유사한 방식으로 minimum slab order를 직접 지정합니다.
`slab_max_order`는 `slab_min_objects` 검사를 중단할 order입니다. Large object cache가 요구 수를 맞추려고 지나치게 큰 high-order page를 만들지 않게 합니다. `debug_guardpage_minorder=N`에서 N이 0보다 크면 `slab_max_order`가 0으로 강제되어 가능한 최소 order를 사용합니다.
`slab_strict_numa`는 allocation마다 memory policy를 적용해 object placement를 더 정확하게 하고 remote-node access를 줄일 수 있습니다. 기본 동작은 새 folio를 얻거나 list에서 folio를 가져올 때만 folio level에서 policy를 적용합니다. 이 option은 slab allocator fastpath 성능을 낮춥니다.
SLUB corruption report 예제
186-228다음은 `kmalloc-8` object 뒤 Right Redzone이 overwrite된 SLUB debug output 원문입니다.
====================================================================
BUG kmalloc-8: Right Redzone overwritten
--------------------------------------------------------------------
INFO: 0xc90f6d28-0xc90f6d2b. First byte 0x00 instead of 0xcc
INFO: Slab 0xc528c530 flags=0x400000c3 inuse=61 fp=0xc90f6d58
INFO: Object 0xc90f6d20 @offset=3360 fp=0xc90f6d58
INFO: Allocated in get_modalias+0x61/0xf5 age=53 cpu=1 pid=554
Bytes b4 (0xc90f6d10): 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ
Object (0xc90f6d20): 31 30 31 39 2e 30 30 35 1019.005
Redzone (0xc90f6d28): 00 cc cc cc .
Padding (0xc90f6d50): 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
[<c010523d>] dump_trace+0x63/0x1eb
[<c01053df>] show_trace_log_lvl+0x1a/0x2f
[<c010601d>] show_trace+0x12/0x14
[<c0106035>] dump_stack+0x16/0x18
[<c017e0fa>] object_err+0x143/0x14b
[<c017e2cc>] check_object+0x66/0x234
[<c017eb43>] __slab_free+0x239/0x384
[<c017f446>] kfree+0xa6/0xc6
[<c02e2335>] get_modalias+0xb9/0xf5
[<c02e23b7>] dmi_dev_uevent+0x27/0x3c
[<c027866a>] dev_uevent+0x1ad/0x1da
[<c0205024>] kobject_uevent_env+0x20a/0x45b
[<c020527f>] kobject_uevent+0xa/0xf
[<c02779f1>] store_uevent+0x4f/0x58
[<c027758e>] dev_attr_store+0x29/0x2f
[<c01bec4f>] sysfs_write_file+0x16e/0x19c
[<c0183ba7>] vfs_write+0xd1/0x15a
[<c01841d7>] sys_write+0x3d/0x72
[<c0104112>] sysenter_past_esp+0x5f/0x99
[<b7f7b410>] 0xb7f7b410
=======================
FIX kmalloc-8: Restoring Redzone 0xc90f6d28-0xc90f6d2b=0xccReport는 corruption byte range, slab·object metadata, allocation call site와 age·CPU·PID, object 주변 byte, stack trace를 기록합니다. 마지막 `FIX` line은 SLUB가 Redzone 값을 복구했음을 알립니다.
SLUB report 해석
229-300SLUB가 corrupted object를 만나면 full detection 기준으로 `slab_debug`가 필요하며 report를 syslog에 기록합니다. Report는 네 부분으로 읽습니다.
| 단계 | 구성 | 핵심 |
|---|---|---|
| 1 | Problem description | BUG·INFO line과 alloc/free context |
| 2 | Object contents | Bytes b4, Object, Redzone, Padding |
| 3 | Stackdump | error detection 위치와 원인 후보 call site |
| 4 | Corrective action | `FIX <cache>: <action>` |
순서질문대표 marker
1무엇이 손상됐는가?BUG / INFO
2어느 byte가 달라졌는가?Bytes b4 / Object / Redzone / Padding
3어디서 발견·할당·해제됐는가?stackdump / Allocated in / Freed in
4계속 동작하기 위해 무엇을 고쳤는가?FIX
오류 식별부터 byte 증거, call stack, 자동 복구 순서로 분석합니다.
===============================================
BUG <slab cache affected>: <What went wrong>
-----------------------------------------------
INFO: <corruption start>-<corruption_end> <more info>
INFO: Slab <address> <slab information>
INFO: Object <address> <object information>
INFO: Allocated in <kernel function> age=<jiffies since alloc> cpu=<allocated by
cpu> pid=<pid of the process>
INFO: Freed in <kernel function> age=<jiffies since free> cpu=<freed by cpu>
pid=<pid of the process>Object allocation/free 정보는 해당 slab에 `SLAB_STORE_USER`가 설정됐을 때만 있으며 `slab_debug`가 이 option을 설정합니다.
| Line | 진단 의미 |
|---|---|
| Bytes b4 | object 앞 byte; corruption이 object 시작 전까지 이어지는지 확인 |
| Object | inactive object의 poison 값과 write-after-free 확인 |
| Redzone | object 뒤 경계; 값 차이는 object boundary 밖 write |
| Padding | 다음 object alignment 공간; object 앞 write 감지 |
`Redzone`은 `SLAB_RED_ZONE`이 있을 때만 제공됩니다. Debug mode는 alignment padding을 최소 4 byte 확보해 object 앞 write도 감지합니다. Stackdump는 error detection 위치를 보여 주지만 실제 원인은 object를 allocate하거나 free한 function에 있을 가능성이 큽니다.
FIX <slab cache affected>: <corrective action taken>예제에서는 길이 8 slab에 8-character string과 terminating zero를 쓰면서 zero가 Redzone 첫 byte를 overwrite했습니다. SLUB는 상세 내용을 보고한 뒤 Redzone을 정상 값으로 복구하고 system operation을 계속합니다.
Emergency debugging
301-329Production system에서 최소 debug, 즉 sanity check만 켜 SLUB resiliency 기능을 사용할 수 있습니다.
slab_debug=FSanity check는 성능을 낮추고 지속적인 syslog error를 만들지만 full debugging과 달리 추가 memory를 사용하지 않습니다. 손상시키는 kernel component가 계속 있어도 system을 유지하는 데 도움을 줄 수 있지만 문제 component는 반드시 수정해야 하며 보장은 없습니다.
문제가 발생하는 cache만 찾았다면 범위를 dentry로 제한합니다.
slab_debug=F,dentryObject 끝 뒤 write가 corruption 원인이면 다음처럼 Redzone도 켜 다른 object 시작 부분을 보호합니다.
slab_debug=FZ,dentryExtended slabinfo mode
330-345`slabinfo -X` extended mode는 세 종류의 section을 포함합니다.
| Section | 내용 |
|---|---|
| Slabcache Totals | cache 전체 합계 |
| Slabs sorted by size | size 순 상위 `-N <num>`, 기본 1 |
| Slabs sorted by loss | loss 순 상위 `-N <num>`, 기본 1 |
Extended mode는 G/M/K 단위로 dynamic scale하지 않고 byte로 보고합니다. 이 기능은 다른 slabinfo mode에서도 `-B`로 사용할 수 있습니다. 정확한 byte output은 `slabinfo-gnuplot.sh`로 plot할 수 있어 많은 숫자를 직접 읽는 대신 시각적으로 slab behavior를 분석하게 합니다.
Extended record plot 생성
346-363먼저 일정 간격으로 extended record를 수집합니다.
while [ 1 ]; do slabinfo -X >> FOO_STATS; sleep 1; done수집한 stats file을 plotting script에 전달합니다.
slabinfo-gnuplot.sh FOO_STATS [FOO_STATS2 .. FOO_STATSN]| Output PNG | 내용 |
|---|---|
| FOO_STATS-totals.png | Slabcache Totals |
| FOO_STATS-slabs-by-size.png | size 순 slab |
| FOO_STATS-slabs-by-loss.png | loss 순 slab |
`slabinfo-gnuplot.sh`는 각 STATS file을 전처리해 PNG 3개와 전처리 cache file 3개를 만듭니다.
변경 전후 plot 비교와 확대
364-394Code 변경 전후 slab behavior를 비교하려면 필요한 수만큼 STATS file을 수집합니다.
while [ 1 ]; do slabinfo -X >> STATS<X>; sleep 1; done각 STATS file을 전처리합니다.
slabinfo-gnuplot.sh STATS1 STATS2 .. STATSN생성된 모든 `*-totals`를 `-t` mode로 전달하면 하나의 비교 PNG를 만듭니다.
slabinfo-gnuplot.sh -t STATS1-totals STATS2-totals .. STATSN-totals| Option | 효과 |
|---|---|
| -s %d,%d | 기본 image width와 height override |
| -r %d,%d | 사용할 sample range 지정; `-r 40,60`은 40~60초 sample |
Plot이 커 작은 fluctuation이나 spike가 보이지 않으면 `-s`로 image 크기를 바꾸거나 `-r`로 sample 구간을 제한합니다.
alloc_traces debugfs
395-435User tracking debug option이 켜진 cache에는 보통 `/sys/kernel/debug/slab/<cache>/` 아래 debugfs file이 만들어집니다. `alloc_traces`는 현재 allocated object의 unique allocation trace를 frequency 순으로 출력합니다.
| Field | 의미 |
|---|---|
| objects | 현재 allocated object 수 |
| function | allocating function |
| waste | kmalloc object의 total/per-object 낭비 |
| age | alloc 뒤 최소/평균/최대 jiffies |
| pid | allocating process pid range |
| cpus | allocating CPU mask |
| nodes | memory origin NUMA node mask |
| stack | allocation stack trace |
338 pci_alloc_dev+0x2c/0xa0 waste=521872/1544 age=290837/291891/293509 pid=1 cpus=106 nodes=0-1
__kmem_cache_alloc_node+0x11f/0x4e0
kmalloc_trace+0x26/0xa0
pci_alloc_dev+0x2c/0xa0
pci_scan_single_device+0xd2/0x150
pci_scan_slot+0xf7/0x2d0
pci_scan_child_bus_extend+0x4e/0x360
acpi_pci_root_create+0x32e/0x3b0
pci_acpi_scan_root+0x2b9/0x2d0
acpi_pci_root_add.cold.11+0x110/0xb0a
acpi_bus_attach+0x262/0x3f0
device_for_each_child+0xb7/0x110
acpi_dev_for_each_child+0x77/0xa0
acpi_bus_attach+0x108/0x3f0
device_for_each_child+0xb7/0x110
acpi_dev_for_each_child+0x77/0xa0
acpi_bus_attach+0x108/0x3f0free_traces debugfs
436-467`free_traces`는 현재 allocated object가 이전 life-cycle에서 free된 unique trace를 frequency 순으로 출력합니다. 처음 allocate된 object에는 이전 free trace가 없어 `<not-available>`로 보고됩니다.
| Field | 의미 |
|---|---|
| objects | 현재 allocated object 수 |
| function | 이전 life-cycle의 freeing function |
| age | free 뒤 최소/평균/최대 jiffies |
| pid | freeing process pid range |
| cpus | freeing CPU mask |
| stack | free stack trace |
1980 <not-available> age=4294912290 pid=0 cpus=0
51 acpi_ut_update_ref_count+0x6a6/0x782 age=236886/237027/237772 pid=1 cpus=1
kfree+0x2db/0x420
acpi_ut_update_ref_count+0x6a6/0x782
acpi_ut_update_object_reference+0x1ad/0x234
acpi_ut_remove_reference+0x7d/0x84
acpi_rs_get_prt_method_data+0x97/0xd6
acpi_get_irq_routing_table+0x82/0xc4
acpi_pci_irq_find_prt_entry+0x8e/0x2e0
acpi_pci_irq_lookup+0x3a/0x1e0
acpi_pci_irq_enable+0x77/0x240
pcibios_enable_device+0x39/0x40
do_pci_enable_device.part.0+0x5d/0xe0
pci_enable_device_flags+0xfc/0x120
pci_enable_device+0x13/0x20
virtio_pci_probe+0x9e/0x170
local_pci_probe+0x48/0x80
pci_device_probe+0x105/0x1c0
문서 이력
468-469초기 문서는 Christoph Lameter가 2007년 5월 30일 작성했고 Sergey Senozhatsky가 2015년 10월 23일 갱신했습니다.
운영 핵심
slab.rst:1-469SLUB debug는 corruption을 잡는 강력한 기능이지만 metadata·allocation order·fastpath 비용을 늘릴 수 있습니다. Cache pattern으로 범위를 좁히고 sanity, redzone, poisoning, user tracking을 목적에 맞게 조합한 뒤 slabinfo·syslog·debugfs trace를 함께 분석하는 것이 핵심입니다.