요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
1
======================
2
Firmware-Assisted Dump
3
======================
5
July 2011
7
The goal of firmware-assisted dump is to enable the dump of
8
a crashed system, and to do so from a fully-reset system, and
9
to minimize the total elapsed time until the system is back
10
in production use.
12
- Firmware-Assisted Dump (FADump) infrastructure is intended to replace
13
the existing phyp assisted dump.
14
- Fadump uses the same firmware interfaces and memory reservation model
15
as phyp assisted dump.
16
- Unlike phyp dump, FADump exports the memory dump through /proc/vmcore
17
in the ELF format in the same way as kdump. This helps us reuse the
18
kdump infrastructure for dump capture and filtering.
19
- Unlike phyp dump, userspace tool does not need to refer any sysfs
20
interface while reading /proc/vmcore.
21
- Unlike phyp dump, FADump allows user to release all the memory reserved
22
for dump, with a single operation of echo 1 > /sys/kernel/fadump_release_mem.
23
- Once enabled through kernel boot parameter, FADump can be
24
started/stopped through /sys/kernel/fadump_registered interface (see
25
sysfs files section below) and can be easily integrated with kdump
26
service start/stop init scripts.
28
Comparing with kdump or other strategies, firmware-assisted
29
dump offers several strong, practical advantages:
31
- Unlike kdump, the system has been reset, and loaded
32
with a fresh copy of the kernel. In particular,
33
PCI and I/O devices have been reinitialized and are
34
in a clean, consistent state.
35
- Once the dump is copied out, the memory that held the dump
36
is immediately available to the running kernel. And therefore,
37
unlike kdump, FADump doesn't need a 2nd reboot to get back
38
the system to the production configuration.
40
The above can only be accomplished by coordination with,
41
and assistance from the Power firmware. The procedure is
42
as follows:
44
- The first kernel registers the sections of memory with the
45
Power firmware for dump preservation during OS initialization.
46
These registered sections of memory are reserved by the first
47
kernel during early boot.
49
- When system crashes, the Power firmware will copy the registered
50
low memory regions (boot memory) from source to destination area.
51
It will also save hardware PTE's.
53
NOTE:
54
The term 'boot memory' means size of the low memory chunk
55
that is required for a kernel to boot successfully when
56
booted with restricted memory. By default, the boot memory
57
size will be the larger of 5% of system RAM or 256MB.
58
Alternatively, user can also specify boot memory size
59
through boot parameter 'crashkernel=' which will override
60
the default calculated size. Use this option if default
61
boot memory size is not sufficient for second kernel to
62
boot successfully. For syntax of crashkernel= parameter,
63
refer to Documentation/admin-guide/kdump/kdump.rst. If any
64
offset is provided in crashkernel= parameter, it will be
65
ignored as FADump uses a predefined offset to reserve memory
66
for boot memory dump preservation in case of a crash.
68
- After the low memory (boot memory) area has been saved, the
69
firmware will reset PCI and other hardware state. It will
70
*not* clear the RAM. It will then launch the bootloader, as
71
normal.
73
- The freshly booted kernel will notice that there is a new node
74
(rtas/ibm,kernel-dump on pSeries or ibm,opal/dump/mpipl-boot
75
on OPAL platform) in the device tree, indicating that
76
there is crash data available from a previous boot. During
77
the early boot OS will reserve rest of the memory above
78
boot memory size effectively booting with restricted memory
79
size. This will make sure that this kernel (also, referred
80
to as second kernel or capture kernel) will not touch any
81
of the dump memory area.
83
- User-space tools will read /proc/vmcore to obtain the contents
84
of memory, which holds the previous crashed kernel dump in ELF
85
format. The userspace tools may copy this info to disk, or
86
network, nas, san, iscsi, etc. as desired.
88
- Once the userspace tool is done saving dump, it will echo
89
'1' to /sys/kernel/fadump_release_mem to release the reserved
90
memory back to general use, except the memory required for
91
next firmware-assisted dump registration.
93
e.g.::
95
# echo 1 > /sys/kernel/fadump_release_mem
97
Please note that the firmware-assisted dump feature
98
is only available on POWER6 and above systems on pSeries
99
(PowerVM) platform and POWER9 and above systems with OP940
100
or later firmware versions on PowerNV (OPAL) platform.
101
Note that, OPAL firmware exports ibm,opal/dump node when
102
FADump is supported on PowerNV platform.
104
On OPAL based machines, system first boots into an intermittent
105
kernel (referred to as petitboot kernel) before booting into the
106
capture kernel. This kernel would have minimal kernel and/or
107
userspace support to process crash data. Such kernel needs to
108
preserve previously crash'ed kernel's memory for the subsequent
109
capture kernel boot to process this crash data. Kernel config
110
option CONFIG_PRESERVE_FA_DUMP has to be enabled on such kernel
111
to ensure that crash data is preserved to process later.
113
-- On OPAL based machines (PowerNV), if the kernel is build with
114
CONFIG_OPAL_CORE=y, OPAL memory at the time of crash is also
115
exported as /sys/firmware/opal/mpipl/core file. This procfs file is
116
helpful in debugging OPAL crashes with GDB. The kernel memory
117
used for exporting this procfs file can be released by echo'ing
118
'1' to /sys/firmware/opal/mpipl/release_core node.
120
e.g.
121
# echo 1 > /sys/firmware/opal/mpipl/release_core
123
-- Support for Additional Kernel Arguments in Fadump
124
Fadump has a feature that allows passing additional kernel arguments
125
to the fadump kernel. This feature was primarily designed to disable
126
kernel functionalities that are not required for the fadump kernel
127
and to reduce its memory footprint while collecting the dump.
129
Command to Add Additional Kernel Parameters to Fadump:
130
e.g.
131
# echo "nr_cpus=16" > /sys/kernel/fadump/bootargs_append
133
The above command is sufficient to add additional arguments to fadump.
134
An explicit service restart is not required.
136
Command to Retrieve the Additional Fadump Arguments:
137
e.g.
138
# cat /sys/kernel/fadump/bootargs_append
140
Note: Additional kernel arguments for fadump with HASH MMU is only
141
supported if the RMA size is greater than 768 MB. If the RMA
142
size is less than 768 MB, the kernel does not export the
143
/sys/kernel/fadump/bootargs_append sysfs node.
145
Implementation details:
146
-----------------------
148
During boot, a check is made to see if firmware supports
149
this feature on that particular machine. If it does, then
150
we check to see if an active dump is waiting for us. If yes
151
then everything but boot memory size of RAM is reserved during
152
early boot (See Fig. 2). This area is released once we finish
153
collecting the dump from user land scripts (e.g. kdump scripts)
154
that are run. If there is dump data, then the
155
/sys/kernel/fadump_release_mem file is created, and the reserved
156
memory is held.
158
If there is no waiting dump data, then only the memory required to
159
hold CPU state, HPTE region, boot memory dump, and FADump header is
160
usually reserved at an offset greater than boot memory size (see Fig. 1).
161
This area is *not* released: this region will be kept permanently
162
reserved, so that it can act as a receptacle for a copy of the boot
163
memory content in addition to CPU state and HPTE region, in the case
164
a crash does occur.
166
Since this reserved memory area is used only after the system crash,
167
there is no point in blocking this significant chunk of memory from
168
production kernel. Hence, the implementation uses the Linux kernel's
169
Contiguous Memory Allocator (CMA) for memory reservation if CMA is
170
configured for kernel. With CMA reservation this memory will be
171
available for applications to use it, while kernel is prevented from
172
using it. With this FADump will still be able to capture all of the
173
kernel memory and most of the user space memory except the user pages
174
that were present in CMA region::
176
o Memory Reservation during first kernel
178
Low memory Top of memory
179
0 boot memory size |<------ Reserved dump area ----->| |
180
| | | Permanent Reservation | |
181
V V | | V
182
+-----------+-----/ /---+---+----+-----------+-------+----+-----+
183
| | |///|////| DUMP | HDR |////| |
184
+-----------+-----/ /---+---+----+-----------+-------+----+-----+
185
| ^ ^ ^ ^ ^
186
| | | | | |
187
\ CPU HPTE / | |
188
-------------------------------- | |
189
Boot memory content gets transferred | |
190
to reserved area by firmware at the | |
191
time of crash. | |
192
FADump Header |
193
(meta area) |
194
|
195
|
196
Metadata: This area holds a metadata structure whose
197
address is registered with f/w and retrieved in the
198
second kernel after crash, on platforms that support
199
tags (OPAL). Having such structure with info needed
200
to process the crashdump eases dump capture process.
202
Fig. 1
205
o Memory Reservation during second kernel after crash
207
Low memory Top of memory
208
0 boot memory size |
209
| |<------------ Crash preserved area ------------>|
210
V V |<--- Reserved dump area --->| |
211
+----+---+--+-----/ /---+---+----+-------+-----+-----+-------+
212
| |ELF| | |///|////| DUMP | HDR |/////| |
213
+----+---+--+-----/ /---+---+----+-------+-----+-----+-------+
214
| | | | | |
215
----- ------------------------------ ---------------
216
\ | |
217
\ | |
218
\ | |
219
\ | ----------------------------
220
\ | /
221
\ | /
222
\ | /
223
/proc/vmcore
226
+---+
227
|///| -> Regions (CPU, HPTE & Metadata) marked like this in the above
228
+---+ figures are not always present. For example, OPAL platform
229
does not have CPU & HPTE regions while Metadata region is
230
not supported on pSeries currently.
232
+---+
233
|ELF| -> elfcorehdr, it is created in second kernel after crash.
234
+---+
236
Note: Memory from 0 to the boot memory size is used by second kernel
238
Fig. 2
241
Currently the dump will be copied from /proc/vmcore to a new file upon
242
user intervention. The dump data available through /proc/vmcore will be
243
in ELF format. Hence the existing kdump infrastructure (kdump scripts)
244
to save the dump works fine with minor modifications. KDump scripts on
245
major Distro releases have already been modified to work seamlessly (no
246
user intervention in saving the dump) when FADump is used, instead of
247
KDump, as dump mechanism.
249
The tools to examine the dump will be same as the ones
250
used for kdump.
252
How to enable firmware-assisted dump (FADump):
253
----------------------------------------------
255
1. Set config option CONFIG_FA_DUMP=y and build kernel.
256
2. Boot into linux kernel with 'fadump=on' kernel cmdline option.
257
By default, FADump reserved memory will be initialized as CMA area.
258
Alternatively, user can boot linux kernel with 'fadump=nocma' to
259
prevent FADump to use CMA.
260
3. Optionally, user can also set 'crashkernel=' kernel cmdline
261
to specify size of the memory to reserve for boot memory dump
262
preservation.
264
NOTE:
265
1. 'fadump_reserve_mem=' parameter has been deprecated. Instead
266
use 'crashkernel=' to specify size of the memory to reserve
267
for boot memory dump preservation.
268
2. If firmware-assisted dump fails to reserve memory then it
269
will fallback to existing kdump mechanism if 'crashkernel='
270
option is set at kernel cmdline.
271
3. if user wants to capture all of user space memory and ok with
272
reserved memory not available to production system, then
273
'fadump=nocma' kernel parameter can be used to fallback to
274
old behaviour.
276
Sysfs/debugfs files:
277
--------------------
279
Firmware-assisted dump feature uses sysfs file system to hold
280
the control files and debugfs file to display memory reserved region.
282
Here is the list of files under kernel sysfs:
284
/sys/kernel/fadump_enabled
285
This is used to display the FADump status.
287
- 0 = FADump is disabled
288
- 1 = FADump is enabled
290
This interface can be used by kdump init scripts to identify if
291
FADump is enabled in the kernel and act accordingly.
293
/sys/kernel/fadump_registered
294
This is used to display the FADump registration status as well
295
as to control (start/stop) the FADump registration.
297
- 0 = FADump is not registered.
298
- 1 = FADump is registered and ready to handle system crash.
300
To register FADump echo 1 > /sys/kernel/fadump_registered and
301
echo 0 > /sys/kernel/fadump_registered for un-register and stop the
302
FADump. Once the FADump is un-registered, the system crash will not
303
be handled and vmcore will not be captured. This interface can be
304
easily integrated with kdump service start/stop.
306
/sys/kernel/fadump/mem_reserved
308
This is used to display the memory reserved by FADump for saving the
309
crash dump.
311
/sys/kernel/fadump_release_mem
312
This file is available only when FADump is active during
313
second kernel. This is used to release the reserved memory
314
region that are held for saving crash dump. To release the
315
reserved memory echo 1 to it::
317
echo 1 > /sys/kernel/fadump_release_mem
319
After echo 1, the content of the /sys/kernel/debug/powerpc/fadump_region
320
file will change to reflect the new memory reservations.
322
The existing userspace tools (kdump infrastructure) can be easily
323
enhanced to use this interface to release the memory reserved for
324
dump and continue without 2nd reboot.
326
Note: /sys/kernel/fadump_release_opalcore sysfs has moved to
327
/sys/firmware/opal/mpipl/release_core
329
/sys/firmware/opal/mpipl/release_core
331
This file is available only on OPAL based machines when FADump is
332
active during capture kernel. This is used to release the memory
333
used by the kernel to export /sys/firmware/opal/mpipl/core file. To
334
release this memory, echo '1' to it:
336
echo 1 > /sys/firmware/opal/mpipl/release_core
338
Note: The following FADump sysfs files are deprecated.
340
+----------------------------------+--------------------------------+
341
| Deprecated | Alternative |
342
+----------------------------------+--------------------------------+
343
| /sys/kernel/fadump_enabled | /sys/kernel/fadump/enabled |
344
+----------------------------------+--------------------------------+
345
| /sys/kernel/fadump_registered | /sys/kernel/fadump/registered |
346
+----------------------------------+--------------------------------+
347
| /sys/kernel/fadump_release_mem | /sys/kernel/fadump/release_mem |
348
+----------------------------------+--------------------------------+
350
Here is the list of files under powerpc debugfs:
351
(Assuming debugfs is mounted on /sys/kernel/debug directory.)
353
/sys/kernel/debug/powerpc/fadump_region
354
This file shows the reserved memory regions if FADump is
355
enabled otherwise this file is empty. The output format
356
is::
358
<region>: [<start>-<end>] <reserved-size> bytes, Dumped: <dump-size>
360
and for kernel DUMP region is:
362
DUMP: Src: <src-addr>, Dest: <dest-addr>, Size: <size>, Dumped: # bytes
364
e.g.
365
Contents when FADump is registered during first kernel::
367
# cat /sys/kernel/debug/powerpc/fadump_region
368
CPU : [0x0000006ffb0000-0x0000006fff001f] 0x40020 bytes, Dumped: 0x0
369
HPTE: [0x0000006fff0020-0x0000006fff101f] 0x1000 bytes, Dumped: 0x0
370
DUMP: [0x0000006fff1020-0x0000007fff101f] 0x10000000 bytes, Dumped: 0x0
372
Contents when FADump is active during second kernel::
374
# cat /sys/kernel/debug/powerpc/fadump_region
375
CPU : [0x0000006ffb0000-0x0000006fff001f] 0x40020 bytes, Dumped: 0x40020
376
HPTE: [0x0000006fff0020-0x0000006fff101f] 0x1000 bytes, Dumped: 0x1000
377
DUMP: [0x0000006fff1020-0x0000007fff101f] 0x10000000 bytes, Dumped: 0x10000000
378
: [0x00000010000000-0x0000006ffaffff] 0x5ffb0000 bytes, Dumped: 0x5ffb0000
381
NOTE:
382
Please refer to Documentation/filesystems/debugfs.rst on
383
how to mount the debugfs filesystem.
386
TODO:
387
-----
388
- Need to come up with the better approach to find out more
389
accurate boot memory size that is required for a kernel to
390
boot successfully when booted with restricted memory.
392
Author: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>
394
This document is based on the original documentation written for phyp
396
assisted dump by Linas Vepstas and Manish Ahuja.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
Firmware-Assisted Dump의 목표
1-27문서 날짜는 2011년 7월입니다. Firmware-Assisted Dump (`FADump`)의 목표는 crash system의 dump를 fully-reset system에서 수집하고 production 복귀까지 걸리는 전체 시간을 줄이는 것입니다.
- FADump infrastructure는 기존 phyp assisted dump를 대체하려는 기능입니다.
- Phyp-assisted dump와 같은 firmware interface 및 memory-reservation model을 사용합니다.
- Phyp dump와 달리 kdump처럼 ELF-format memory dump를 `/proc/vmcore`로 export하므로 dump capture/filtering에 기존 kdump infrastructure를 재사용합니다.
- Userspace tool이 `/proc/vmcore`를 읽을 때 별도 sysfs interface를 참조할 필요가 없습니다.
- `echo 1 > /sys/kernel/fadump_release_mem` 한 번으로 dump용 reserved memory 전체를 release할 수 있습니다.
- Boot parameter로 enable한 뒤 `/sys/kernel/fadump_registered`에서 registration을 start/stop할 수 있어 kdump service init script와 쉽게 통합됩니다.
Kdump 대비 장점
28-39Kdump와 달리 system은 reset된 뒤 fresh kernel copy로 boot하므로 PCI와 I/O device가 reinitialize되어 clean하고 consistent한 상태입니다.
Dump copy가 끝나면 dump를 담았던 memory를 running kernel이 즉시 사용할 수 있습니다. 따라서 production configuration으로 돌아가기 위한 두 번째 reboot가 필요 없습니다.
Firmware 협력과 crash-capture 절차
40-103- 첫 kernel은 OS initialization 중 crash 때 보존할 memory section을 Power firmware에 등록하고 early boot에서 reserve합니다.
- System crash가 발생하면 firmware가 registered low-memory region, 즉 boot memory를 source에서 destination area로 copy하고 hardware PTE도 저장합니다.
- Boot memory 저장 뒤 firmware가 PCI와 다른 hardware state를 reset하되 RAM은 clear하지 않고 정상적으로 bootloader를 시작합니다.
- 새 kernel은 pSeries의 `rtas/ibm,kernel-dump` 또는 OPAL의 `ibm,opal/dump/mpipl-boot` device-tree node로 이전 boot의 crash data를 감지합니다. Capture kernel은 boot-memory size 위의 나머지 memory를 reserve해 dump area를 건드리지 않습니다.
- Userspace tool은 `/proc/vmcore`에서 이전 crashed kernel memory를 ELF format으로 읽어 disk, network, NAS, SAN, iSCSI 등에 저장합니다.
- 저장이 끝나면 `/sys/kernel/fadump_release_mem`에 `1`을 써 다음 FADump registration에 필요한 영역을 제외한 reserved memory를 general use로 돌립니다.
`boot memory`는 restricted-memory boot에서 kernel이 성공적으로 시작하는 데 필요한 low-memory chunk입니다. 기본값은 system RAM의 5%와 256MB 중 큰 값입니다. 부족하면 `crashkernel=` boot parameter로 override하며 syntax는 `Documentation/admin-guide/kdump/kdump.rst`를 따릅니다. FADump는 predefined offset을 사용하므로 `crashkernel=`의 offset은 무시합니다.
# echo 1 > /sys/kernel/fadump_release_memPSeries PowerVM에서는 POWER6 이상, PowerNV OPAL에서는 POWER9 이상과 OP940 이상 firmware에서 FADump를 사용할 수 있습니다. PowerNV 지원 시 OPAL firmware가 `ibm,opal/dump` node를 export합니다.
첫 kernel memory 등록→Crash→Firmware가 boot memory/PTE 저장→Hardware reset, RAM 보존→Capture kernel→`/proc/vmcore`→Dump 저장→Reserved memory release
Firmware가 RAM을 보존한 채 hardware를 reset하고 capture kernel이 vmcore를 export합니다.
OPAL petitboot와 additional boot argument
104-144OPAL machine은 capture kernel 전에 petitboot kernel이라는 중간 kernel로 먼저 boot합니다. 이 최소 kernel/userspace는 crash data를 처리하고 다음 capture-kernel boot까지 이전 crashed-kernel memory를 보존해야 하므로 `CONFIG_PRESERVE_FA_DUMP`를 enable해야 합니다.
PowerNV kernel이 `CONFIG_OPAL_CORE=y`로 build되면 crash 당시 OPAL memory를 `/sys/firmware/opal/mpipl/core`로 export해 GDB로 OPAL crash를 debug할 수 있습니다. 이 file을 export하는 데 쓴 memory는 다음 command로 release합니다.
# echo 1 > /sys/firmware/opal/mpipl/release_coreFADump kernel에 additional kernel argument를 전달하면 dump 수집에 불필요한 기능을 disable해 memory footprint를 줄일 수 있습니다. 다음 write는 service restart 없이 즉시 argument를 추가합니다.
# echo "nr_cpus=16" > /sys/kernel/fadump/bootargs_append현재 추가된 argument는 다음 command로 읽습니다.
# cat /sys/kernel/fadump/bootargs_appendHASH MMU에서 additional FADump argument는 RMA size가 768 MB보다 클 때만 지원합니다. 768 MB 미만이면 `/sys/kernel/fadump/bootargs_append` node를 export하지 않습니다.
Boot-time reservation과 CMA
145-175Boot 중 firmware support를 확인하고 active dump가 기다리는지 검사합니다. Dump가 있으면 early boot에서 boot-memory size를 제외한 RAM 전체를 reserve하며, userspace kdump script가 수집을 마치면 release합니다. 이때 `/sys/kernel/fadump_release_mem` file을 만들고 그때까지 memory를 유지합니다.
대기 dump가 없으면 CPU state, HPTE region, boot-memory dump, FADump header를 담는 데 필요한 영역만 boot-memory size보다 큰 offset에 reserve합니다. 이 permanent region은 future crash 때 boot-memory copy를 받을 receptacle이므로 release하지 않습니다.
이 영역은 crash 뒤에만 필요하므로 production kernel에서 큰 memory chunk를 막지 않기 위해 kernel에 CMA가 configure되어 있으면 Contiguous Memory Allocator로 reserve합니다. Application은 CMA memory를 사용할 수 있지만 kernel은 사용하지 않습니다.
CMA reservation을 쓰면 FADump가 kernel memory 전체와 userspace memory 대부분을 capture할 수 있지만 crash 때 CMA region에 있던 user page는 제외됩니다.
Active dump 있음→Boot memory 제외 RAM reserve→Userspace capture→Release
Active dump 없음→CPU/HPTE/DUMP/HDR permanent area→CMA 사용 가능→Future crash receptacle
대기 dump 유무와 CMA 설정에 따라 reserved area의 용도가 달라집니다.
첫 kernel의 memory reservation (Fig. 1)
176-204영역상태용도
`0 .. boot memory size`Production 사용Boot memory content
CPU regionPermanent/CMA reserveCPU state
HPTE regionPermanent/CMA reserveHardware page-table state
DUMP regionPermanent/CMA reserveBoot memory copy destination
HDRPermanent reserveFADump header/meta area
MetadataPlatform-dependentFirmware 등록 address와 capture 정보
Crash copyFirmware가 boot memory content를 DUMP region으로 전송합니다.
OPAL tagsMetadata structure가 있으면 second kernel이 capture 정보를 쉽게 복구합니다.
Boot memory 위의 permanent dump area가 crash 시 firmware copy의 destination이 됩니다.
Metadata area는 firmware에 등록한 address에서 second kernel이 crash 뒤 다시 찾는 structure를 담습니다. OPAL처럼 tag를 지원하는 platform에서 dump-processing 정보를 보관해 capture 과정을 단순화합니다.
Boot memory content→Firmware crash handler→DUMP destination
CPU state→CPU region
Hardware PTE→HPTE region
Capture metadata→HDR/Metadata
Firmware가 low boot memory와 machine state를 permanent dump area에 보존합니다.
Crash 뒤 second kernel memory layout (Fig. 2)
205-240Second kernel은 address 0부터 boot-memory size까지를 자기 실행에 사용하고 나머지 crash-preserved area를 reserve합니다. Crash 뒤 생성한 `elfcorehdr`와 preserved DUMP/HDR 영역을 조합해 `/proc/vmcore`를 제공합니다.
그림에서 hatch 처리된 CPU, HPTE, Metadata region은 항상 존재하지 않습니다. OPAL에는 CPU/HPTE region이 없고 현재 pSeries는 Metadata region을 지원하지 않습니다. `ELF` 영역은 crash 뒤 second kernel이 만든 `elfcorehdr`입니다.
영역Second-kernel 처리`/proc/vmcore` 기여
`0 .. boot memory size`Capture kernel 실행Boot-memory view
ELF`elfcorehdr` 생성Vmcore header
CPU/HPTEPlatform-dependent reserveMachine state
DUMP/HDRCrash-preserved reserveKernel memory와 metadata
나머지 preserved RAM접근 금지Previous userspace/kernel pages
Capture kernel 실행 영역과 crash-preserved 영역을 분리합니다.
`elfcorehdr`→CPU/HPTE state→DUMP/HDR→Preserved RAM→`/proc/vmcore`
Second kernel이 header와 preserved memory segment를 하나의 ELF vmcore로 노출합니다.
Dump 저장과 분석 tool
241-251현재 dump는 `/proc/vmcore`에서 새 file로 copy합니다. Data가 ELF format이므로 기존 kdump script가 작은 수정만으로 저장할 수 있고 major distribution의 KDump script는 FADump를 선택해도 user 개입 없이 동작하도록 이미 변경되었습니다.
Dump 분석 tool도 kdump에서 사용하는 것과 같습니다.
FADump enable 절차
252-275- `CONFIG_FA_DUMP=y`로 kernel을 build합니다.
- `fadump=on` kernel command-line option으로 boot합니다. 기본 FADump reserved memory는 CMA area로 initialize되며 `fadump=nocma`로 CMA 사용을 막을 수 있습니다.
- 필요하면 `crashkernel=`로 boot-memory dump preservation용 reserve size를 지정합니다.
`fadump_reserve_mem=`은 deprecated되었으므로 `crashkernel=`을 사용합니다. FADump가 memory reserve에 실패하고 command line에 `crashkernel=`이 있으면 기존 kdump로 fallback합니다. Userspace memory 전체를 capture하기 위해 reserved memory를 production system에서 포기할 수 있다면 `fadump=nocma`로 과거 behavior를 선택합니다.
Sysfs와 OPAL control file
276-337FADump는 sysfs에 control file을, debugfs에 reserved-memory region 표시 file을 둡니다.
| Path | 기능 |
|---|---|
| `/sys/kernel/fadump_enabled` | `0`은 disabled, `1`은 enabled. Kdump init script가 FADump 상태를 판별 |
| `/sys/kernel/fadump_registered` | `0`은 미등록, `1`은 등록되어 crash 처리 준비 완료. Write로 registration start/stop |
| `/sys/kernel/fadump/mem_reserved` | Crash dump 저장을 위해 FADump가 reserve한 memory 표시 |
| `/sys/kernel/fadump_release_mem` | Second kernel에서 FADump active일 때만 존재하며 dump용 reserved region release |
| `/sys/kernel/debug/powerpc/fadump_region` | Release 뒤 변경된 memory reservation을 반영 |
| `/sys/firmware/opal/mpipl/core` | OPAL crash 당시 memory를 core file로 export |
| `/sys/firmware/opal/mpipl/release_core` | Capture kernel에서 OPAL core export용 memory release |
FADump registration은 `/sys/kernel/fadump_registered`에 `1`을 써 시작하고 `0`을 써 해제합니다. Unregister 뒤 crash는 처리되지 않고 vmcore도 capture되지 않습니다.
Dump reserved memory release command는 다음과 같습니다.
echo 1 > /sys/kernel/fadump_release_memUserspace kdump infrastructure는 이 interface로 두 번째 reboot 없이 dump memory를 release할 수 있습니다. 과거 `/sys/kernel/fadump_release_opalcore`는 `/sys/firmware/opal/mpipl/release_core`로 이동했습니다.
OPAL core export memory는 다음 command로 release합니다.
echo 1 > /sys/firmware/opal/mpipl/release_coreDeprecated sysfs path
338-349다음 기존 sysfs 경로는 폐기 예정이며 새 대체 경로를 사용해야 합니다.
| Deprecated | Alternative |
|---|---|
| `/sys/kernel/fadump_enabled` | `/sys/kernel/fadump/enabled` |
| `/sys/kernel/fadump_registered` | `/sys/kernel/fadump/registered` |
| `/sys/kernel/fadump_release_mem` | `/sys/kernel/fadump/release_mem` |
Debugfs reserved-region 출력
350-383Debugfs가 `/sys/kernel/debug`에 mount되어 있다고 가정할 때 `/sys/kernel/debug/powerpc/fadump_region`은 FADump가 enable된 경우 reserved-memory region을 표시하고 아니면 비어 있습니다.
일반 region output format은 다음과 같습니다.
<region>: [<start>-<end>] <reserved-size> bytes, Dumped: <dump-size>Kernel DUMP region format은 다음과 같습니다.
DUMP: Src: <src-addr>, Dest: <dest-addr>, Size: <size>, Dumped: # bytes첫 kernel에서 FADump가 registered된 출력 예입니다.
# cat /sys/kernel/debug/powerpc/fadump_region
CPU : [0x0000006ffb0000-0x0000006fff001f] 0x40020 bytes, Dumped: 0x0
HPTE: [0x0000006fff0020-0x0000006fff101f] 0x1000 bytes, Dumped: 0x0
DUMP: [0x0000006fff1020-0x0000007fff101f] 0x10000000 bytes, Dumped: 0x0Second kernel에서 FADump가 active인 출력 예입니다.
# cat /sys/kernel/debug/powerpc/fadump_region
CPU : [0x0000006ffb0000-0x0000006fff001f] 0x40020 bytes, Dumped: 0x40020
HPTE: [0x0000006fff0020-0x0000006fff101f] 0x1000 bytes, Dumped: 0x1000
DUMP: [0x0000006fff1020-0x0000007fff101f] 0x10000000 bytes, Dumped: 0x10000000
: [0x00000010000000-0x0000006ffaffff] 0x5ffb0000 bytes, Dumped: 0x5ffb0000Debugfs mount 방법은 `Documentation/filesystems/debugfs.rst`를 참조합니다.
TODO와 저작 정보
384-396Restricted-memory boot에서 kernel이 성공적으로 시작하는 데 필요한 boot-memory size를 더 정확히 구하는 방법이 필요합니다.
저자는 Mahesh Salgaonkar이며, 이 문서는 Linas Vepstas와 Manish Ahuja가 작성한 phyp assisted dump 원문을 바탕으로 합니다.
요약과 해설
firmware-assisted-dump.rst:1-396FADump는 RAM을 지우지 않고 hardware를 reset한 뒤 fresh capture kernel에서 ELF vmcore를 제공합니다. Permanent/CMA reservation, platform별 metadata와 release interface를 정확히 운영해야 두 번째 reboot 없이 production memory를 회수할 수 있습니다.