요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
==========================
Coresight CPU Debug Module
==========================
:Author: Leo Yan <leo.yan@linaro.org>
:Date: April 5th, 2017
Introduction
------------
Coresight CPU debug module is defined in ARMv8-a architecture reference manual
(ARM DDI 0487A.k) Chapter 'Part H: External debug', the CPU can integrate
debug module and it is mainly used for two modes: self-hosted debug and
external debug. Usually the external debug mode is well known as the external
debugger connects with SoC from JTAG port; on the other hand the program can
explore debugging method which rely on self-hosted debug mode, this document
is to focus on this part.
The debug module provides sample-based profiling extension, which can be used
to sample CPU program counter, secure state and exception level, etc; usually
every CPU has one dedicated debug module to be connected. Based on self-hosted
debug mechanism, Linux kernel can access these related registers from mmio
region when the kernel panic happens. The callback notifier for kernel panic
will dump related registers for every CPU; finally this is good for assistant
analysis for panic.
Implementation
--------------
- During driver registration, it uses EDDEVID and EDDEVID1 - two device ID
registers to decide if sample-based profiling is implemented or not. On some
platforms this hardware feature is fully or partially implemented; and if
this feature is not supported then registration will fail.
- At the time this documentation was written, the debug driver mainly relies on
information gathered by the kernel panic callback notifier from three
sampling registers: EDPCSR, EDVIDSR and EDCIDSR: from EDPCSR we can get
program counter; EDVIDSR has information for secure state, exception level,
bit width, etc; EDCIDSR is context ID value which contains the sampled value
of CONTEXTIDR_EL1.
- The driver supports a CPU running in either AArch64 or AArch32 mode. The
registers naming convention is a bit different between them, AArch64 uses
'ED' for register prefix (ARM DDI 0487A.k, chapter H9.1) and AArch32 uses
'DBG' as prefix (ARM DDI 0487A.k, chapter G5.1). The driver is unified to
use AArch64 naming convention.
- ARMv8-a (ARM DDI 0487A.k) and ARMv7-a (ARM DDI 0406C.b) have different
register bits definition. So the driver consolidates two difference:
If PCSROffset=0b0000, on ARMv8-a the feature of EDPCSR is not implemented;
but ARMv7-a defines "PCSR samples are offset by a value that depends on the
instruction set state". For ARMv7-a, the driver checks furthermore if CPU
runs with ARM or thumb instruction set and calibrate PCSR value, the
detailed description for offset is in ARMv7-a ARM (ARM DDI 0406C.b) chapter
C11.11.34 "DBGPCSR, Program Counter Sampling Register".
If PCSROffset=0b0010, ARMv8-a defines "EDPCSR implemented, and samples have
no offset applied and do not sample the instruction set state in AArch32
state". So on ARMv8 if EDDEVID1.PCSROffset is 0b0010 and the CPU operates
in AArch32 state, EDPCSR is not sampled; when the CPU operates in AArch64
state EDPCSR is sampled and no offset are applied.
Clock and power domain
----------------------
Before accessing debug registers, we should ensure the clock and power domain
have been enabled properly. In ARMv8-a ARM (ARM DDI 0487A.k) chapter 'H9.1
Debug registers', the debug registers are spread into two domains: the debug
domain and the CPU domain.
::
+---------------+
| |
| |
+----------+--+ |
dbg_clock -->| |**| |<-- cpu_clock
| Debug |**| CPU |
dbg_power_domain -->| |**| |<-- cpu_power_domain
+----------+--+ |
| |
| |
+---------------+
For debug domain, the user uses DT binding "clocks" and "power-domains" to
specify the corresponding clock source and power supply for the debug logic.
The driver calls the pm_runtime_{put|get} operations as needed to handle the
debug power domain.
For CPU domain, the different SoC designs have different power management
schemes and finally this heavily impacts external debug module. So we can
divide into below cases:
- On systems with a sane power controller which can behave correctly with
respect to CPU power domain, the CPU power domain can be controlled by
register EDPRCR in driver. The driver firstly writes bit EDPRCR.COREPURQ
to power up the CPU, and then writes bit EDPRCR.CORENPDRQ for emulation
of CPU power down. As result, this can ensure the CPU power domain is
powered on properly during the period when access debug related registers;
- Some designs will power down an entire cluster if all CPUs on the cluster
are powered down - including the parts of the debug registers that should
remain powered in the debug power domain. The bits in EDPRCR are not
respected in these cases, so these designs do not support debug over
power down in the way that the CoreSight / Debug designers anticipated.
This means that even checking EDPRSR has the potential to cause a bus hang
if the target register is unpowered.
In this case, accessing to the debug registers while they are not powered
is a recipe for disaster; so we need preventing CPU low power states at boot
time or when user enable module at the run time. Please see chapter
"How to use the module" for detailed usage info for this.
Device Tree Bindings
--------------------
See Documentation/devicetree/bindings/arm/arm,coresight-cpu-debug.yaml for
details.
How to use the module
---------------------
If you want to enable debugging functionality at boot time, you can add
"coresight_cpu_debug.enable=1" to the kernel command line parameter.
The driver also can work as module, so can enable the debugging when insmod
module::
# insmod coresight_cpu_debug.ko debug=1
When boot time or insmod module you have not enabled the debugging, the driver
uses the debugfs file system to provide a knob to dynamically enable or disable
debugging:
To enable it, write a '1' into /sys/kernel/debug/coresight_cpu_debug/enable::
# echo 1 > /sys/kernel/debug/coresight_cpu_debug/enable
To disable it, write a '0' into /sys/kernel/debug/coresight_cpu_debug/enable::
# echo 0 > /sys/kernel/debug/coresight_cpu_debug/enable
As explained in chapter "Clock and power domain", if you are working on one
platform which has idle states to power off debug logic and the power
controller cannot work well for the request from EDPRCR, then you should
firstly constraint CPU idle states before enable CPU debugging feature; so can
ensure the accessing to debug logic.
If you want to limit idle states at boot time, you can use "nohlt" or
"cpuidle.off=1" in the kernel command line.
At the runtime you can disable idle states with below methods:
It is possible to disable CPU idle states by way of the PM QoS
subsystem, more specifically by using the "/dev/cpu_dma_latency"
interface (see Documentation/power/pm_qos_interface.rst for more
details). As specified in the PM QoS documentation the requested
parameter will stay in effect until the file descriptor is released.
For example::
# exec 3<> /dev/cpu_dma_latency; echo 0 >&3
...
Do some work...
...
# exec 3<>-
The same can also be done from an application program.
Disable specific CPU's specific idle state from cpuidle sysfs (see
Documentation/admin-guide/pm/cpuidle.rst)::
# echo 1 > /sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable
Output format
-------------
Here is an example of the debugging output format::
ARM external debug module:
coresight-cpu-debug 850000.debug: CPU[0]:
coresight-cpu-debug 850000.debug: EDPRSR: 00000001 (Power:On DLK:Unlock)
coresight-cpu-debug 850000.debug: EDPCSR: handle_IPI+0x174/0x1d8
coresight-cpu-debug 850000.debug: EDCIDSR: 00000000
coresight-cpu-debug 850000.debug: EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
coresight-cpu-debug 852000.debug: CPU[1]:
coresight-cpu-debug 852000.debug: EDPRSR: 00000001 (Power:On DLK:Unlock)
coresight-cpu-debug 852000.debug: EDPCSR: debug_notifier_call+0x23c/0x358
coresight-cpu-debug 852000.debug: EDCIDSR: 00000000
coresight-cpu-debug 852000.debug: EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
Self-hosted CPU debug와 panic 분석
1-27이 문서는 Leo Yan이 2017년 4월 5일 작성했다. CoreSight CPU debug module은 ARMv8-A Architecture Reference Manual(ARM DDI 0487A.k)의 `Part H: External debug`에 정의된다.
CPU가 통합할 수 있는 debug module의 주된 두 사용 방식은 self-hosted debug와 external debug다. external debug는 JTAG port로 외부 debugger를 SoC에 연결하는 익숙한 방식이고, 이 문서는 프로그램과 Linux kernel이 자체적으로 활용하는 self-hosted debug에 초점을 둔다.
debug module의 sample-based profiling extension은 CPU program counter, secure state, exception level 등을 표본화한다. 일반적으로 CPU마다 전용 debug module 하나가 연결된다. kernel panic이 발생하면 Linux kernel이 MMIO 영역의 관련 register에 접근하고 panic callback notifier가 모든 CPU의 register를 dump하여 사후 원인 분석을 돕는다.
CPU별 debug module 표본을 panic notifier가 수집한다.
이 문서가 다루는 범위를 구분한다.
==========================
Coresight CPU Debug Module
==========================
:Author: Leo Yan <leo.yan@linaro.org>
:Date: April 5th, 2017
Introduction
------------
Coresight CPU debug module is defined in ARMv8-a architecture reference manual
(ARM DDI 0487A.k) Chapter 'Part H: External debug', the CPU can integrate
debug module and it is mainly used for two modes: self-hosted debug and
external debug. Usually the external debug mode is well known as the external
debugger connects with SoC from JTAG port; on the other hand the program can
explore debugging method which rely on self-hosted debug mode, this document
is to focus on this part.
The debug module provides sample-based profiling extension, which can be used
to sample CPU program counter, secure state and exception level, etc; usually
every CPU has one dedicated debug module to be connected. Based on self-hosted
debug mechanism, Linux kernel can access these related registers from mmio
region when the kernel panic happens. The callback notifier for kernel panic
will dump related registers for every CPU; finally this is good for assistant
analysis for panic.
Profiling register와 ARMv7/ARMv8 PC 보정
28-65driver 등록 중 `EDDEVID`와 `EDDEVID1` device ID register로 sample-based profiling 구현 여부를 판정한다. platform에 따라 완전 또는 부분 구현될 수 있으며 지원하지 않으면 등록에 실패한다.
문서 작성 당시 panic callback notifier는 주로 세 sampling register를 수집했다. `EDPCSR`은 program counter, `EDVIDSR`은 secure state·exception level·bit width 등, `EDCIDSR`은 `CONTEXTIDR_EL1`에서 표본화한 context ID 값을 제공한다.
driver는 AArch64와 AArch32 CPU mode를 모두 지원한다. AArch64 register 이름은 `ED` 접두어를, AArch32는 `DBG` 접두어를 사용하지만 driver 내부 표기는 AArch64의 `ED` 규칙으로 통일한다.
ARMv8-A와 ARMv7-A는 register bit 정의가 다르다. `PCSROffset=0b0000`이면 ARMv8-A에서는 `EDPCSR`가 구현되지 않았다는 뜻이다. ARMv7-A에서는 instruction set state에 따른 값만큼 PCSR sample이 offset되므로 driver가 ARM 또는 Thumb 실행 여부를 더 확인해 PCSR 값을 보정한다. 상세 offset은 ARMv7-A ARM의 C11.11.34 `DBGPCSR, Program Counter Sampling Register`에 있다.
`PCSROffset=0b0010`이면 ARMv8-A에서 `EDPCSR`가 구현되며 sample에 offset을 적용하지 않고 AArch32 state의 instruction set state는 표본화하지 않는다. 따라서 CPU가 AArch32로 동작하면 EDPCSR를 표본화하지 않고, AArch64로 동작하면 offset 없이 표본화한다.
callback notifier가 CPU별 실행 상태를 복원할 때 쓰는 값이다.
아키텍처와 실행 state에 따른 EDPCSR 처리 차이다.
서로 다른 architecture naming을 driver의 한 규칙으로 정규화한다.
Implementation
--------------
- During driver registration, it uses EDDEVID and EDDEVID1 - two device ID
registers to decide if sample-based profiling is implemented or not. On some
platforms this hardware feature is fully or partially implemented; and if
this feature is not supported then registration will fail.
- At the time this documentation was written, the debug driver mainly relies on
information gathered by the kernel panic callback notifier from three
sampling registers: EDPCSR, EDVIDSR and EDCIDSR: from EDPCSR we can get
program counter; EDVIDSR has information for secure state, exception level,
bit width, etc; EDCIDSR is context ID value which contains the sampled value
of CONTEXTIDR_EL1.
- The driver supports a CPU running in either AArch64 or AArch32 mode. The
registers naming convention is a bit different between them, AArch64 uses
'ED' for register prefix (ARM DDI 0487A.k, chapter H9.1) and AArch32 uses
'DBG' as prefix (ARM DDI 0487A.k, chapter G5.1). The driver is unified to
use AArch64 naming convention.
- ARMv8-a (ARM DDI 0487A.k) and ARMv7-a (ARM DDI 0406C.b) have different
register bits definition. So the driver consolidates two difference:
If PCSROffset=0b0000, on ARMv8-a the feature of EDPCSR is not implemented;
but ARMv7-a defines "PCSR samples are offset by a value that depends on the
instruction set state". For ARMv7-a, the driver checks furthermore if CPU
runs with ARM or thumb instruction set and calibrate PCSR value, the
detailed description for offset is in ARMv7-a ARM (ARM DDI 0406C.b) chapter
C11.11.34 "DBGPCSR, Program Counter Sampling Register".
If PCSROffset=0b0010, ARMv8-a defines "EDPCSR implemented, and samples have
no offset applied and do not sample the instruction set state in AArch32
state". So on ARMv8 if EDDEVID1.PCSROffset is 0b0010 and the CPU operates
in AArch32 state, EDPCSR is not sampled; when the CPU operates in AArch64
state EDPCSR is sampled and no offset are applied.
Debug domain과 CPU domain 전원 제약
66-116debug register에 접근하기 전에 clock과 power domain이 올바르게 활성화되어야 한다. ARMv8-A ARM H9.1에 따르면 register는 debug domain과 CPU domain 두 영역에 걸쳐 있다.
구조화하면 `dbg_clock`과 `dbg_power_domain`은 Debug logic에, `cpu_clock`과 `cpu_power_domain`은 CPU에 공급된다. 두 영역 경계에 걸친 debug register는 양쪽 전원 상태의 영향을 받는다.
debug domain은 DT binding의 `clocks`와 `power-domains`로 debug logic의 clock source와 power supply를 지정한다. driver는 필요할 때 `pm_runtime_get`과 `pm_runtime_put` 계열 연산으로 debug power domain을 관리한다.
CPU power domain 처리 방식은 SoC 설계에 따라 다르다. 정상적인 power controller에서는 driver가 `EDPRCR.COREPURQ`를 써 CPU를 power-up한 뒤 `EDPRCR.CORENPDRQ`로 CPU power-down을 모의한다. 그 결과 debug register에 접근하는 동안 CPU domain이 켜진 상태를 보장한다.
일부 설계는 cluster의 모든 CPU가 꺼질 때 debug domain에 남아 있어야 할 register 일부까지 cluster 전체와 함께 끈다. 이런 경우 `EDPRCR` bit가 존중되지 않아 설계자가 의도한 debug-over-power-down을 지원하지 않는다. 전원이 꺼진 target의 `EDPRSR`를 확인하는 것만으로도 bus hang이 날 수 있다.
따라서 이런 platform에서는 전원이 꺼진 debug register 접근을 막기 위해 부팅 시 또는 runtime module 활성화 전에 CPU low-power state를 제한해야 한다. 구체적인 명령은 다음 사용법 절에서 설명한다.
원문의 ASCII 그림을 clock·power 공급과 공유 경계로 재구성한다.
EDPRCR 동작 여부에 따라 안전한 접근 방법이 갈린다.
debug register 접근 구간을 CPU power-up 상태로 감싼다.
Clock and power domain
----------------------
Before accessing debug registers, we should ensure the clock and power domain
have been enabled properly. In ARMv8-a ARM (ARM DDI 0487A.k) chapter 'H9.1
Debug registers', the debug registers are spread into two domains: the debug
domain and the CPU domain.
::
+---------------+
| |
| |
+----------+--+ |
dbg_clock -->| |**| |<-- cpu_clock
| Debug |**| CPU |
dbg_power_domain -->| |**| |<-- cpu_power_domain
+----------+--+ |
| |
| |
+---------------+
For debug domain, the user uses DT binding "clocks" and "power-domains" to
specify the corresponding clock source and power supply for the debug logic.
The driver calls the pm_runtime_{put|get} operations as needed to handle the
debug power domain.
For CPU domain, the different SoC designs have different power management
schemes and finally this heavily impacts external debug module. So we can
divide into below cases:
- On systems with a sane power controller which can behave correctly with
respect to CPU power domain, the CPU power domain can be controlled by
register EDPRCR in driver. The driver firstly writes bit EDPRCR.COREPURQ
to power up the CPU, and then writes bit EDPRCR.CORENPDRQ for emulation
of CPU power down. As result, this can ensure the CPU power domain is
powered on properly during the period when access debug related registers;
- Some designs will power down an entire cluster if all CPUs on the cluster
are powered down - including the parts of the debug registers that should
remain powered in the debug power domain. The bits in EDPRCR are not
respected in these cases, so these designs do not support debug over
power down in the way that the CoreSight / Debug designers anticipated.
This means that even checking EDPRSR has the potential to cause a bus hang
if the target register is unpowered.
In this case, accessing to the debug registers while they are not powered
is a recipe for disaster; so we need preventing CPU low power states at boot
time or when user enable module at the run time. Please see chapter
"How to use the module" for detailed usage info for this.
Device Tree binding
117-123Device Tree 속성과 binding 세부 사항은 `Documentation/devicetree/bindings/arm/arm,coresight-cpu-debug.yaml`을 참조한다.
debug logic의 clock·power와 장치 기술은 YAML binding이 기준이다.
Device Tree Bindings
--------------------
See Documentation/devicetree/bindings/arm/arm,coresight-cpu-debug.yaml for
details.
부팅·module·debugfs 활성화
124-146부팅 시 debug 기능을 켜려면 kernel command line에 `coresight_cpu_debug.enable=1`을 추가한다.
driver를 module로 사용할 때는 `insmod coresight_cpu_debug.ko debug=1`로 로드와 동시에 활성화할 수 있다. 부팅이나 insmod 시 켜지 않았다면 debugfs의 `/sys/kernel/debug/coresight_cpu_debug/enable` knob로 동적으로 제어한다.
해당 파일에 1을 쓰면 활성화하고 0을 쓰면 비활성화한다. 원문 명령과 경로는 각각 `echo 1 > .../enable`, `echo 0 > .../enable`이다.
시점에 따라 kernel parameter, module parameter, debugfs를 사용한다.
어느 시점에든 최종적으로 driver의 debug 기능 상태를 설정한다.
How to use the module
---------------------
If you want to enable debugging functionality at boot time, you can add
"coresight_cpu_debug.enable=1" to the kernel command line parameter.
The driver also can work as module, so can enable the debugging when insmod
module::
# insmod coresight_cpu_debug.ko debug=1
When boot time or insmod module you have not enabled the debugging, the driver
uses the debugfs file system to provide a knob to dynamically enable or disable
debugging:
To enable it, write a '1' into /sys/kernel/debug/coresight_cpu_debug/enable::
# echo 1 > /sys/kernel/debug/coresight_cpu_debug/enable
To disable it, write a '0' into /sys/kernel/debug/coresight_cpu_debug/enable::
# echo 0 > /sys/kernel/debug/coresight_cpu_debug/enable
Idle state 제한과 PM QoS
147-177idle state가 debug logic의 전원을 끄고 power controller가 `EDPRCR` 요청을 제대로 처리하지 못하는 platform에서는 CPU debug를 켜기 전에 CPU idle state를 제한해야 한다. 그래야 debug logic 접근 중 전원이 유지된다.
부팅 시 모든 idle state를 제한하려면 kernel command line에 `nohlt` 또는 `cpuidle.off=1`을 사용한다.
runtime에는 PM QoS subsystem의 `/dev/cpu_dma_latency` interface로 CPU idle state를 제한할 수 있다. 요청은 file descriptor가 닫힐 때까지 유지된다. 예제는 fd 3으로 파일을 열고 0을 쓴 뒤 작업을 수행하고 fd 3을 닫는다. 같은 동작을 application program에서도 구현할 수 있다.
특정 CPU의 특정 idle state만 끄려면 cpuidle sysfs의 `/sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable`에 1을 쓴다. 자세한 내용은 `Documentation/power/pm_qos_interface.rst`와 `Documentation/admin-guide/pm/cpuidle.rst`를 참조한다.
범위와 유지 기간에 따라 선택한다.
fd가 열린 구간에서 latency 요청과 idle 제한이 유효하다.
As explained in chapter "Clock and power domain", if you are working on one
platform which has idle states to power off debug logic and the power
controller cannot work well for the request from EDPRCR, then you should
firstly constraint CPU idle states before enable CPU debugging feature; so can
ensure the accessing to debug logic.
If you want to limit idle states at boot time, you can use "nohlt" or
"cpuidle.off=1" in the kernel command line.
At the runtime you can disable idle states with below methods:
It is possible to disable CPU idle states by way of the PM QoS
subsystem, more specifically by using the "/dev/cpu_dma_latency"
interface (see Documentation/power/pm_qos_interface.rst for more
details). As specified in the PM QoS documentation the requested
parameter will stay in effect until the file descriptor is released.
For example::
# exec 3<> /dev/cpu_dma_latency; echo 0 >&3
...
Do some work...
...
# exec 3<>-
The same can also be done from an application program.
Disable specific CPU's specific idle state from cpuidle sysfs (see
Documentation/admin-guide/pm/cpuidle.rst)::
# echo 1 > /sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable
CPU별 debug register 출력
178-193출력은 `ARM external debug module` 제목 아래 장치와 CPU를 나열하고 CPU마다 `EDPRSR`, `EDPCSR`, `EDCIDSR`, `EDVIDSR`를 표시한다.
예제의 CPU 0과 CPU 1 모두 `EDPRSR=00000001`로 Power On, DLK Unlock 상태다. CPU 0의 PC는 `handle_IPI+0x174/0x1d8`, CPU 1은 `debug_notifier_call+0x23c/0x358`이다. 두 CPU의 `EDCIDSR`는 0이고 `EDVIDSR=90000000`은 Non-secure, EL1/0, 64-bit, VMID 0으로 해석된다.
panic dump에서 각 줄이 제공하는 진단 정보다.
두 CPU가 panic 시점에 실행하던 위치다.
전원 상태를 먼저 확인한 뒤 PC와 실행 context를 결합한다.
Output format
-------------
Here is an example of the debugging output format::
ARM external debug module:
coresight-cpu-debug 850000.debug: CPU[0]:
coresight-cpu-debug 850000.debug: EDPRSR: 00000001 (Power:On DLK:Unlock)
coresight-cpu-debug 850000.debug: EDPCSR: handle_IPI+0x174/0x1d8
coresight-cpu-debug 850000.debug: EDCIDSR: 00000000
coresight-cpu-debug 850000.debug: EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
coresight-cpu-debug 852000.debug: CPU[1]:
coresight-cpu-debug 852000.debug: EDPRSR: 00000001 (Power:On DLK:Unlock)
coresight-cpu-debug 852000.debug: EDPCSR: debug_notifier_call+0x23c/0x358
coresight-cpu-debug 852000.debug: EDCIDSR: 00000000
coresight-cpu-debug 852000.debug: EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
요약·해설
coresight-cpu-debug.rst:1-193CoreSight CPU debug module의 self-hosted panic sampling, EDPCSR·EDVIDSR·EDCIDSR 해석, ARMv7/ARMv8 PCSROffset 차이, debug·CPU 전원 도메인과 안전한 idle 제한 절차를 설명합니다.