요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
====================
APEI Error INJection
====================
EINJ provides a hardware error injection mechanism. It is very useful
for debugging and testing APEI and RAS features in general.
You need to check whether your BIOS supports EINJ first. For that, look
for early boot messages similar to this one::
ACPI: EINJ 0x000000007370A000 000150 (v01 INTEL 00000001 INTL 00000001)
which shows that the BIOS is exposing an EINJ table - it is the
mechanism through which the injection is done.
Alternatively, look in /sys/firmware/acpi/tables for an "EINJ" file,
which is a different representation of the same thing.
It doesn't necessarily mean that EINJ is not supported if those above
don't exist: before you give up, go into BIOS setup to see if the BIOS
has an option to enable error injection. Look for something called WHEA
or similar. Often, you need to enable an ACPI5 support option prior, in
order to see the APEI,EINJ,... functionality supported and exposed by
the BIOS menu.
To use EINJ, make sure the following are options enabled in your kernel
configuration::
CONFIG_DEBUG_FS
CONFIG_ACPI_APEI
CONFIG_ACPI_APEI_EINJ
...and to (optionally) enable CXL protocol error injection set::
CONFIG_ACPI_APEI_EINJ_CXL
The EINJ user interface is in <debugfs mount point>/apei/einj.
The following files belong to it:
- available_error_type
This file shows which error types are supported:
================ ===================================
Error Type Value Error Description
================ ===================================
0x00000001 Processor Correctable
0x00000002 Processor Uncorrectable non-fatal
0x00000004 Processor Uncorrectable fatal
0x00000008 Memory Correctable
0x00000010 Memory Uncorrectable non-fatal
0x00000020 Memory Uncorrectable fatal
0x00000040 PCI Express Correctable
0x00000080 PCI Express Uncorrectable non-fatal
0x00000100 PCI Express Uncorrectable fatal
0x00000200 Platform Correctable
0x00000400 Platform Uncorrectable non-fatal
0x00000800 Platform Uncorrectable fatal
V2_0x00000001 EINJV2 Processor Error
V2_0x00000002 EINJV2 Memory Error
V2_0x00000004 EINJV2 PCI Express Error
================ ===================================
The format of the file contents are as above, except present are only
the available error types.
- error_type
Set the value of the error type being injected. Possible error types
are defined in the file available_error_type above.
- error_inject
Write any integer to this file to trigger the error injection. Make
sure you have specified all necessary error parameters, i.e. this
write should be the last step when injecting errors.
- flags
Present for kernel versions 3.13 and above. Used to specify which
of param{1..4} are valid and should be used by the firmware during
injection. Value is a bitmask as specified in ACPI5.0 spec for the
SET_ERROR_TYPE_WITH_ADDRESS data structure:
Bit 0
Processor APIC field valid (see param3 below).
Bit 1
Memory address and mask valid (param1 and param2).
Bit 2
PCIe (seg,bus,dev,fn) valid (see param4 below).
Bit 3
EINJv2 extension structure is valid
If set to zero, legacy behavior is mimicked where the type of
injection specifies just one bit set, and param1 is multiplexed.
- param1
This file is used to set the first error parameter value. Its effect
depends on the error type specified in error_type. For example, if
error type is memory related type, the param1 should be a valid
physical memory address. [Unless "flag" is set - see above]
- param2
Same use as param1 above. For example, if error type is of memory
related type, then param2 should be a physical memory address mask.
Linux requires page or narrower granularity, say, 0xfffffffffffff000.
- param3
Used when the 0x1 bit is set in "flags" to specify the APIC id
- param4
Used when the 0x4 bit is set in "flags" to specify target PCIe device
- notrigger
The error injection mechanism is a two-step process. First inject the
error, then perform some actions to trigger it. Setting "notrigger"
to 1 skips the trigger phase, which *may* allow the user to cause the
error in some other context by a simple access to the CPU, memory
location, or device that is the target of the error injection. Whether
this actually works depends on what operations the BIOS actually
includes in the trigger phase.
- component_id0 .. component_idN, component_syndrome0 .. component_syndromeN
These files are used to set the "Component Array" field
of the EINJv2 Extension Structure. Each holds a 128-bit
hex value. Writing just a newline to any of these files
sets an invalid (all-ones) value.
CXL error types are supported from ACPI 6.5 onwards (given a CXL port
is present). The EINJ user interface for CXL error types is at
<debugfs mount point>/cxl. The following files belong to it:
- einj_types:
Provides the same functionality as available_error_types above, but
for CXL error types
- $dport_dev/einj_inject:
Injects a CXL error type into the CXL port represented by $dport_dev,
where $dport_dev is the name of the CXL port (usually a PCIe device name).
Error injections targeting a CXL 2.0+ port can use the legacy interface
under <debugfs mount point>/apei/einj, while CXL 1.1/1.0 port injections
must use this file.
BIOS versions based on the ACPI 4.0 specification have limited options
in controlling where the errors are injected. Your BIOS may support an
extension (enabled with the param_extension=1 module parameter, or boot
command line einj.param_extension=1). This allows the address and mask
for memory injections to be specified by the param1 and param2 files in
apei/einj.
BIOS versions based on the ACPI 5.0 specification have more control over
the target of the injection. For processor-related errors (type 0x1, 0x2
and 0x4), you can set flags to 0x3 (param3 for bit 0, and param1 and
param2 for bit 1) so that you have more information added to the error
signature being injected. The actual data passed is this::
memory_address = param1;
memory_address_range = param2;
apicid = param3;
pcie_sbdf = param4;
For memory errors (type 0x8, 0x10 and 0x20) the address is set using
param1 with a mask in param2 (0x0 is equivalent to all ones). For PCI
express errors (type 0x40, 0x80 and 0x100) the segment, bus, device and
function are specified using param1::
31 24 23 16 15 11 10 8 7 0
+-------------------------------------------------+
| segment | bus | device | function | reserved |
+-------------------------------------------------+
Anyway, you get the idea, if there's doubt just take a look at the code
in drivers/acpi/apei/einj.c.
An ACPI 5.0 BIOS may also allow vendor-specific errors to be injected.
In this case a file named vendor will contain identifying information
from the BIOS that hopefully will allow an application wishing to use
the vendor-specific extension to tell that they are running on a BIOS
that supports it. All vendor extensions have the 0x80000000 bit set in
error_type. A file vendor_flags controls the interpretation of param1
and param2 (1 = PROCESSOR, 2 = MEMORY, 4 = PCI). See your BIOS vendor
documentation for details (and expect changes to this API if vendors
creativity in using this feature expands beyond our expectations).
An error injection example::
# cd /sys/kernel/debug/apei/einj
# cat available_error_type # See which errors can be injected
0x00000002 Processor Uncorrectable non-fatal
0x00000008 Memory Correctable
0x00000010 Memory Uncorrectable non-fatal
# echo 0x12345000 > param1 # Set memory address for injection
# echo 0xfffffffffffff000 > param2 # Mask - anywhere in this page
# echo 0x8 > error_type # Choose correctable memory error
# echo 1 > error_inject # Inject now
An EINJv2 error injection example::
# cd /sys/kernel/debug/apei/einj
# cat available_error_type # See which errors can be injected
0x00000002 Processor Uncorrectable non-fatal
0x00000008 Memory Correctable
0x00000010 Memory Uncorrectable non-fatal
V2_0x00000001 EINJV2 Processor Error
V2_0x00000002 EINJV2 Memory Error
# echo 0x12345000 > param1 # Set memory address for injection
# echo 0xfffffffffffff000 > param2 # Range - anywhere in this page
# echo 0x1 > component_id0 # First device ID
# echo 0x4 > component_syndrome0 # First error syndrome
# echo 0x2 > component_id1 # Second device ID
# echo 0x4 > component_syndrome1 # Second error syndrome
# echo '' > component_id2 # Mark id2 invalid to terminate list
# echo V2_0x2 > error_type # Choose EINJv2 memory error
# echo 0xa > flags # set flags to indicate EINJv2
# echo 1 > error_inject # Inject now
You should see something like this in dmesg::
[22715.830801] EDAC sbridge MC3: HANDLING MCE MEMORY ERROR
[22715.834759] EDAC sbridge MC3: CPU 0: Machine Check Event: 0 Bank 7: 8c00004000010090
[22715.834759] EDAC sbridge MC3: TSC 0
[22715.834759] EDAC sbridge MC3: ADDR 12345000 EDAC sbridge MC3: MISC 144780c86
[22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0
[22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0)
A CXL error injection example with $dport_dev=0000:e0:01.1::
# cd /sys/kernel/debug/cxl/
# ls
0000:e0:01.1 0000:0c:00.0
# cat einj_types # See which errors can be injected
0x00008000 CXL.mem Protocol Correctable
0x00010000 CXL.mem Protocol Uncorrectable non-fatal
0x00020000 CXL.mem Protocol Uncorrectable fatal
# cd 0000:e0:01.1 # Navigate to dport to inject into
# echo 0x8000 > einj_inject # Inject error
Special notes for injection into SGX enclaves:
There may be a separate BIOS setup option to enable SGX injection.
The injection process consists of setting some special memory controller
trigger that will inject the error on the next write to the target
address. But the h/w prevents any software outside of an SGX enclave
from accessing enclave pages (even BIOS SMM mode).
The following sequence can be used:
1) Determine physical address of enclave page
2) Use "notrigger=1" mode to inject (this will setup
the injection address, but will not actually inject)
3) Enter the enclave
4) Store data to the virtual address matching physical address from step 1
5) Execute CLFLUSH for that virtual address
6) Spin delay for 250ms
7) Read from the virtual address. This will trigger the error
For more information about EINJ, please refer to ACPI specification
version 4.0, section 17.5 and ACPI 5.0, section 18.6.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
BIOS 지원 확인과 kernel 설정
1-39EINJ(Error INJection)는 hardware error를 의도적으로 주입하여 APEI(ACPI Platform Error Interface)와 RAS(Reliability, Availability, Serviceability) 기능을 debug·test하는 mechanism이다. 실제 machine check, memory·PCIe·platform error를 발생시킬 수 있으므로 production system이나 보존해야 할 data가 있는 환경이 아니라 격리된 test system에서만 사용해야 한다.
먼저 BIOS가 EINJ를 지원하는지 확인한다. Early boot log의 `ACPI: EINJ ...` message는 BIOS가 injection mechanism을 기술하는 EINJ ACPI table을 노출했다는 뜻이다. 같은 table의 다른 표현은 `/sys/firmware/acpi/tables/EINJ`에서 확인할 수 있다.
둘 다 보이지 않아도 곧바로 미지원이라고 단정하지 않는다. BIOS setup에서 error injection, WHEA 또는 유사한 option을 찾고, APEI·EINJ menu를 노출하기 전에 필요한 ACPI5 support option을 먼저 켜야 하는지도 확인한다.
Kernel에서는 `CONFIG_DEBUG_FS`, `CONFIG_ACPI_APEI`, `CONFIG_ACPI_APEI_EINJ`가 필요하다. CXL protocol error injection까지 사용하려면 `CONFIG_ACPI_APEI_EINJ_CXL`을 추가한다. 기본 interface는 `<debugfs mount point>/apei/einj`에 생성된다.
CONFIG_DEBUG_FS
CONFIG_ACPI_APEI
CONFIG_ACPI_APEI_EINJ
CONFIG_ACPI_APEI_EINJ_CXL # optional CXL injectionFirmware capability부터 debugfs interface까지 단계별로 검증한다.
기본 APEI injection과 선택 CXL injection에 필요한 option이다.
.. SPDX-License-Identifier: GPL-2.0
====================
APEI Error INJection
====================
EINJ provides a hardware error injection mechanism. It is very useful
for debugging and testing APEI and RAS features in general.
You need to check whether your BIOS supports EINJ first. For that, look
for early boot messages similar to this one::
ACPI: EINJ 0x000000007370A000 000150 (v01 INTEL 00000001 INTL 00000001)
which shows that the BIOS is exposing an EINJ table - it is the
mechanism through which the injection is done.
Alternatively, look in /sys/firmware/acpi/tables for an "EINJ" file,
which is a different representation of the same thing.
It doesn't necessarily mean that EINJ is not supported if those above
don't exist: before you give up, go into BIOS setup to see if the BIOS
has an option to enable error injection. Look for something called WHEA
or similar. Often, you need to enable an ACPI5 support option prior, in
order to see the APEI,EINJ,... functionality supported and exposed by
the BIOS menu.
To use EINJ, make sure the following are options enabled in your kernel
configuration::
CONFIG_DEBUG_FS
CONFIG_ACPI_APEI
CONFIG_ACPI_APEI_EINJ
...and to (optionally) enable CXL protocol error injection set::
CONFIG_ACPI_APEI_EINJ_CXL
The EINJ user interface is in <debugfs mount point>/apei/einj.지원 error type과 최종 trigger
40-80`available_error_type`은 현재 BIOS와 platform이 지원하는 error type만 표 형식으로 보여 준다. Processor, memory, PCI Express, platform 범주는 각각 correctable, uncorrectable non-fatal, uncorrectable fatal bit를 가지며 EINJv2는 processor·memory·PCI Express extension type을 별도로 표시한다.
원문의 error type bit와 설명을 구조화한 표다.
`error_type`에 `available_error_type`에 실제로 표시된 값 중 주입할 type을 쓴다. 문서의 전체 표에 값이 있어도 현재 platform file에 없으면 지원한다고 가정하면 안 된다.
`error_inject`에 임의의 integer를 쓰면 injection이 실행된다. 이 write는 되돌릴 수 없는 최종 trigger이므로 address, mask, APIC ID, PCIe target, component array, flags와 `notrigger`를 모두 설정하고 검토한 뒤 마지막에 수행해야 한다.
`error_inject` write 전 모든 parameter를 확정하는 안전한 순서다.
The following files belong to it:
- available_error_type
This file shows which error types are supported:
================ ===================================
Error Type Value Error Description
================ ===================================
0x00000001 Processor Correctable
0x00000002 Processor Uncorrectable non-fatal
0x00000004 Processor Uncorrectable fatal
0x00000008 Memory Correctable
0x00000010 Memory Uncorrectable non-fatal
0x00000020 Memory Uncorrectable fatal
0x00000040 PCI Express Correctable
0x00000080 PCI Express Uncorrectable non-fatal
0x00000100 PCI Express Uncorrectable fatal
0x00000200 Platform Correctable
0x00000400 Platform Uncorrectable non-fatal
0x00000800 Platform Uncorrectable fatal
V2_0x00000001 EINJV2 Processor Error
V2_0x00000002 EINJV2 Memory Error
V2_0x00000004 EINJV2 PCI Express Error
================ ===================================
The format of the file contents are as above, except present are only
the available error types.
- error_type
Set the value of the error type being injected. Possible error types
are defined in the file available_error_type above.
- error_inject
Write any integer to this file to trigger the error injection. Make
sure you have specified all necessary error parameters, i.e. this
write should be the last step when injecting errors.
flags와 param1..4 의미
81-119Kernel 3.13 이상에서 `flags`는 firmware가 `param1`부터 `param4` 중 어떤 field를 유효하게 사용할지 나타내는 bitmask다. ACPI 5.0 `SET_ERROR_TYPE_WITH_ADDRESS` data structure의 validity bit를 그대로 따른다.
각 bit가 활성화하는 target parameter field다.
`flags=0`은 legacy behavior를 흉내 낸다. Injection type에는 bit 하나만 설정하고 `param1`을 type에 따라 multiplex한다. 명시적 target field를 쓰는 새 방식과 legacy 해석을 섞지 않도록 BIOS specification을 확인해야 한다.
`param1`은 error type에 따른 첫 parameter다. Memory error이면 valid physical address가 된다. `param2`는 같은 범주의 두 번째 parameter이며 memory error에서는 physical address mask다. Linux는 page 또는 그보다 좁은 granularity를 요구하며 예시는 `0xfffffffffffff000`이다.
`flags` bit 0, 즉 `0x1`이 설정되면 `param3`은 APIC ID를 지정한다. Bit 2, 즉 `0x4`가 설정되면 `param4`는 target PCIe device를 지정한다. Parameter 값을 써 두는 것만으로는 부족하고 대응 validity bit도 일치해야 firmware가 그 값을 사용한다.
Error category와 flags에 따라 같은 file의 해석이 달라진다.
- flags
Present for kernel versions 3.13 and above. Used to specify which
of param{1..4} are valid and should be used by the firmware during
injection. Value is a bitmask as specified in ACPI5.0 spec for the
SET_ERROR_TYPE_WITH_ADDRESS data structure:
Bit 0
Processor APIC field valid (see param3 below).
Bit 1
Memory address and mask valid (param1 and param2).
Bit 2
PCIe (seg,bus,dev,fn) valid (see param4 below).
Bit 3
EINJv2 extension structure is valid
If set to zero, legacy behavior is mimicked where the type of
injection specifies just one bit set, and param1 is multiplexed.
- param1
This file is used to set the first error parameter value. Its effect
depends on the error type specified in error_type. For example, if
error type is memory related type, the param1 should be a valid
physical memory address. [Unless "flag" is set - see above]
- param2
Same use as param1 above. For example, if error type is of memory
related type, then param2 should be a physical memory address mask.
Linux requires page or narrower granularity, say, 0xfffffffffffff000.
- param3
Used when the 0x1 bit is set in "flags" to specify the APIC id
- param4
Used when the 0x4 bit is set in "flags" to specify target PCIe device
notrigger·EINJv2 component·CXL interface
120-152Error injection은 보통 error를 장치에 설정하는 단계와 실제 access 등으로 error를 trigger하는 두 단계로 이뤄진다. `notrigger=1`은 BIOS-defined trigger phase를 건너뛰어 사용자가 이후 CPU, memory location 또는 device access라는 다른 context에서 error를 발생시킬 가능성을 준다.
실제로 이 방식이 동작하는지는 BIOS trigger phase에 어떤 operation이 포함됐는지에 달려 있다. `notrigger`는 error 설정을 취소하는 option이 아니며, target에 대한 다음 access가 치명적 error를 일으킬 수 있으므로 injection state와 후속 operation을 통제해야 한다.
`component_id0..N`과 `component_syndrome0..N`은 EINJv2 Extension Structure의 Component Array를 설정한다. 각 file은 128-bit hexadecimal value 하나를 가지며, newline만 쓰면 all-ones invalid value가 되어 array의 유효 entry를 끝내는 데 사용할 수 있다.
CXL error type은 CXL port가 있을 때 ACPI 6.5부터 지원된다. `<debugfs mount point>/cxl/einj_types`는 CXL 지원 type을 보여 주고 `$dport_dev/einj_inject`는 해당 CXL port에 type을 주입한다. CXL 2.0 이상 port는 legacy `/apei/einj` interface도 쓸 수 있지만 CXL 1.1·1.0 port는 dport별 file을 사용해야 한다.
Firmware trigger를 생략하고 target access로 error를 일으키는 경로다.
CXL port generation에 따라 사용할 debugfs path가 다르다.
- notrigger
The error injection mechanism is a two-step process. First inject the
error, then perform some actions to trigger it. Setting "notrigger"
to 1 skips the trigger phase, which *may* allow the user to cause the
error in some other context by a simple access to the CPU, memory
location, or device that is the target of the error injection. Whether
this actually works depends on what operations the BIOS actually
includes in the trigger phase.
- component_id0 .. component_idN, component_syndrome0 .. component_syndromeN
These files are used to set the "Component Array" field
of the EINJv2 Extension Structure. Each holds a 128-bit
hex value. Writing just a newline to any of these files
sets an invalid (all-ones) value.
CXL error types are supported from ACPI 6.5 onwards (given a CXL port
is present). The EINJ user interface for CXL error types is at
<debugfs mount point>/cxl. The following files belong to it:
- einj_types:
Provides the same functionality as available_error_types above, but
for CXL error types
- $dport_dev/einj_inject:
Injects a CXL error type into the CXL port represented by $dport_dev,
where $dport_dev is the name of the CXL port (usually a PCIe device name).
Error injections targeting a CXL 2.0+ port can use the legacy interface
under <debugfs mount point>/apei/einj, while CXL 1.1/1.0 port injections
must use this file.ACPI 4.0·5.0 target 지정과 PCIe SBDF
153-184ACPI 4.0 기반 BIOS는 injection target 제어가 제한적일 수 있다. BIOS extension을 지원하면 module parameter `param_extension=1` 또는 boot command line `einj.param_extension=1`로 켜고, `apei/einj`의 `param1`과 `param2`로 memory address와 mask를 지정한다.
ACPI 5.0 BIOS는 더 세밀한 target을 허용한다. Processor error type `0x1`, `0x2`, `0x4`에서 `flags=0x3`을 쓰면 bit 0의 `param3` APIC ID와 bit 1의 `param1` address·`param2` range를 error signature에 함께 넣는다.
Debugfs parameter가 firmware data structure field로 대응되는 관계다.
Memory error type `0x8`, `0x10`, `0x20`은 `param1`에 address, `param2`에 mask를 쓴다. Mask `0x0`은 all-ones와 동등하다. PCI Express error type `0x40`, `0x80`, `0x100`은 segment, bus, device, function을 packed value로 지정한다.
원문의 32-bit diagram을 bit range 표로 정리한다.
BIOS 구현과 specification 해석이 의심스러우면 kernel source `drivers/acpi/apei/einj.c`를 확인한다. User interface 값, flags와 firmware method 호출의 실제 대응은 이 driver code가 최종 기준이다.
Error category에 맞는 parameter와 validity bit를 조합한다.
BIOS versions based on the ACPI 4.0 specification have limited options
in controlling where the errors are injected. Your BIOS may support an
extension (enabled with the param_extension=1 module parameter, or boot
command line einj.param_extension=1). This allows the address and mask
for memory injections to be specified by the param1 and param2 files in
apei/einj.
BIOS versions based on the ACPI 5.0 specification have more control over
the target of the injection. For processor-related errors (type 0x1, 0x2
and 0x4), you can set flags to 0x3 (param3 for bit 0, and param1 and
param2 for bit 1) so that you have more information added to the error
signature being injected. The actual data passed is this::
memory_address = param1;
memory_address_range = param2;
apicid = param3;
pcie_sbdf = param4;
For memory errors (type 0x8, 0x10 and 0x20) the address is set using
param1 with a mask in param2 (0x0 is equivalent to all ones). For PCI
express errors (type 0x40, 0x80 and 0x100) the segment, bus, device and
function are specified using param1::
31 24 23 16 15 11 10 8 7 0
+-------------------------------------------------+
| segment | bus | device | function | reserved |
+-------------------------------------------------+
Anyway, you get the idea, if there's doubt just take a look at the code
in drivers/acpi/apei/einj.c.Vendor-specific error extension
185-196ACPI 5.0 BIOS는 vendor-specific error injection을 제공할 수 있다. 이때 `vendor` file에는 application이 현재 BIOS가 해당 extension을 지원하는지 식별할 수 있도록 BIOS가 제공한 정보가 들어간다.
모든 vendor extension은 `error_type`의 `0x80000000` bit를 설정한다. `vendor_flags`는 `param1`과 `param2` 해석 범주를 정하며 값 1은 PROCESSOR, 2는 MEMORY, 4는 PCI를 뜻한다.
구체적인 parameter layout과 side effect는 BIOS vendor documentation을 따라야 한다. 원문도 vendor가 기능을 확장하는 방식에 따라 API가 바뀔 수 있다고 경고하므로 generic EINJ value를 vendor extension에 그대로 적용해서는 안 된다.
Generic interface에서 vendor-specific payload를 구분하는 field다.
An ACPI 5.0 BIOS may also allow vendor-specific errors to be injected.
In this case a file named vendor will contain identifying information
from the BIOS that hopefully will allow an application wishing to use
the vendor-specific extension to tell that they are running on a BIOS
that supports it. All vendor extensions have the 0x80000000 bit set in
error_type. A file vendor_flags controls the interpretation of param1
and param2 (1 = PROCESSOR, 2 = MEMORY, 4 = PCI). See your BIOS vendor
documentation for details (and expect changes to this API if vendors
creativity in using this feature expands beyond our expectations).
Correctable memory error 예제
197-208기본 예제는 debugfs EINJ directory로 이동해 `available_error_type`을 확인한 뒤 physical address `0x12345000`과 page mask `0xfffffffffffff000`을 설정한다. `error_type=0x8`은 correctable memory error다.
마지막 `echo 1 > error_inject`가 실제 injection을 실행한다. 예제 address가 현재 system에서 안전하거나 test memory에 속한다는 보장은 없으므로 그대로 실행하지 말고 physical allocation, NUMA·DIMM mapping과 workload 격리를 먼저 확인해야 한다.
# cd /sys/kernel/debug/apei/einj
# echo 0x12345000 > param1
# echo 0xfffffffffffff000 > param2
# echo 0x8 > error_type
# echo 1 > error_inject지원 type 조회부터 final trigger까지의 command 의미다.
각 command가 선택하는 memory target과 error다.
An error injection example::
# cd /sys/kernel/debug/apei/einj
# cat available_error_type # See which errors can be injected
0x00000002 Processor Uncorrectable non-fatal
0x00000008 Memory Correctable
0x00000010 Memory Uncorrectable non-fatal
# echo 0x12345000 > param1 # Set memory address for injection
# echo 0xfffffffffffff000 > param2 # Mask - anywhere in this page
# echo 0x8 > error_type # Choose correctable memory error
# echo 1 > error_inject # Inject now
EINJv2 component array와 dmesg 결과
209-238EINJv2 예제는 address·range에 더해 두 component의 ID와 syndrome을 설정한다. `component_id0=0x1`, `component_syndrome0=0x4`, `component_id1=0x2`, `component_syndrome1=0x4`이며 `component_id2`에는 newline만 써 invalid marker로 component list를 끝낸다.
`error_type=V2_0x2`는 EINJv2 memory error를 선택한다. `flags=0xa`는 bit 1의 memory address·mask validity와 bit 3의 EINJv2 extension validity를 함께 설정한다. 모든 field 뒤에 `error_inject=1`로 실행한다.
Address와 component array를 함께 firmware에 전달한다.
성공한 correctable memory injection은 dmesg에서 EDAC·Machine Check Event로 관찰할 수 있다. 예제 log는 `ADDR 12345000`, CPU·bank·socket·APIC, channel·DIMM·page·offset·syndrome을 포함해 지정 target과 실제 RAS report를 대조하게 한다.
Injection parameter가 RAS log에 반영됐는지 확인하는 field다.
An EINJv2 error injection example::
# cd /sys/kernel/debug/apei/einj
# cat available_error_type # See which errors can be injected
0x00000002 Processor Uncorrectable non-fatal
0x00000008 Memory Correctable
0x00000010 Memory Uncorrectable non-fatal
V2_0x00000001 EINJV2 Processor Error
V2_0x00000002 EINJV2 Memory Error
# echo 0x12345000 > param1 # Set memory address for injection
# echo 0xfffffffffffff000 > param2 # Range - anywhere in this page
# echo 0x1 > component_id0 # First device ID
# echo 0x4 > component_syndrome0 # First error syndrome
# echo 0x2 > component_id1 # Second device ID
# echo 0x4 > component_syndrome1 # Second error syndrome
# echo '' > component_id2 # Mark id2 invalid to terminate list
# echo V2_0x2 > error_type # Choose EINJv2 memory error
# echo 0xa > flags # set flags to indicate EINJv2
# echo 1 > error_inject # Inject now
You should see something like this in dmesg::
[22715.830801] EDAC sbridge MC3: HANDLING MCE MEMORY ERROR
[22715.834759] EDAC sbridge MC3: CPU 0: Machine Check Event: 0 Bank 7: 8c00004000010090
[22715.834759] EDAC sbridge MC3: TSC 0
[22715.834759] EDAC sbridge MC3: ADDR 12345000 EDAC sbridge MC3: MISC 144780c86
[22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0
[22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0)
CXL dport error injection 예제
239-250CXL 예제의 target dport는 PCIe device name `0000:e0:01.1`이다. `/sys/kernel/debug/cxl/`에서 port directory를 나열하고 `einj_types`를 읽어 CXL.mem protocol correctable, uncorrectable non-fatal, fatal type의 지원 여부를 확인한다.
Target directory `0000:e0:01.1`로 이동해 `einj_inject`에 `0x8000`을 쓰면 CXL.mem Protocol Correctable error를 해당 port에 주입한다. Dport 이름과 type value는 system마다 다르므로 예제 값을 다른 machine에 그대로 사용하지 않는다.
# cd /sys/kernel/debug/cxl/
# cat einj_types
# cd 0000:e0:01.1
# echo 0x8000 > einj_inject공통 type 조회 뒤 특정 downstream port file에서 error를 실행한다.
원문 예제에서 보고된 protocol error value다.
A CXL error injection example with $dport_dev=0000:e0:01.1::
# cd /sys/kernel/debug/cxl/
# ls
0000:e0:01.1 0000:0c:00.0
# cat einj_types # See which errors can be injected
0x00008000 CXL.mem Protocol Correctable
0x00010000 CXL.mem Protocol Uncorrectable non-fatal
0x00020000 CXL.mem Protocol Uncorrectable fatal
# cd 0000:e0:01.1 # Navigate to dport to inject into
# echo 0x8000 > einj_inject # Inject error
SGX enclave 대상 injection
251-271SGX injection은 별도의 BIOS setup option을 요구할 수 있다. Memory controller의 special trigger는 target address에 대한 다음 write에서 error를 주입하지만, hardware는 BIOS SMM을 포함해 enclave 밖의 software가 enclave page에 접근하지 못하게 한다.
따라서 enclave 외부에서 target만 arm하고 실제 memory access는 enclave 안에서 수행한다. 먼저 enclave page의 physical address를 구한 뒤 `notrigger=1`로 injection address를 설정하되 즉시 trigger하지 않는다.
원문의 7단계를 enclave 경계와 cache operation 중심으로 구조화했다.
Store와 `CLFLUSH`, 250ms delay 뒤 같은 virtual address를 read하면 error가 trigger된다. 이 절차는 enclave memory와 machine-check handling을 실제로 교란하므로 disposable test enclave와 system recovery path를 준비해야 한다.
더 자세한 protocol은 ACPI specification 4.0 section 17.5와 ACPI 5.0 section 18.6을 참고한다. Platform firmware version에 맞는 specification과 vendor guide를 함께 대조해야 한다.
어떤 단계가 enclave 밖과 안에서 수행되는지 구분한다.
Special notes for injection into SGX enclaves:
There may be a separate BIOS setup option to enable SGX injection.
The injection process consists of setting some special memory controller
trigger that will inject the error on the next write to the target
address. But the h/w prevents any software outside of an SGX enclave
from accessing enclave pages (even BIOS SMM mode).
The following sequence can be used:
1) Determine physical address of enclave page
2) Use "notrigger=1" mode to inject (this will setup
the injection address, but will not actually inject)
3) Enter the enclave
4) Store data to the virtual address matching physical address from step 1
5) Execute CLFLUSH for that virtual address
6) Spin delay for 250ms
7) Read from the virtual address. This will trigger the error
For more information about EINJ, please refer to ACPI specification
version 4.0, section 17.5 and ACPI 5.0, section 18.6.
요약·해설
einj.rst:1-271EINJ는 BIOS ACPI table과 debugfs interface를 통해 processor·memory·PCIe·platform·CXL hardware error를 주입한다. 실제 crash와 data loss를 일으킬 수 있으므로 격리된 RAS test system에서만 사용하고 `error_inject` write는 모든 target parameter 검토 후 마지막에 수행한다.
`flags`는 param1..4와 EINJv2 extension의 validity를 지정한다. `notrigger`는 firmware trigger를 생략하며, vendor·CXL·SGX 경로는 generic memory example과 다른 interface·실행 context를 요구한다.
Injection 후에는 dmesg의 EDAC·MCE address, topology, syndrome을 설정값과 대조해야 한다. 지원 type과 target encoding은 BIOS ACPI version, vendor documentation과 `drivers/acpi/apei/einj.c`를 함께 기준으로 삼는다.
Capability 확인부터 RAS log 검증과 복구까지의 전체 흐름이다.