← Documents Documentation/ABI/testing/debugfs-intel-iommu GitHub 원문 ↗

Linux 6.18.37 · ABI / testing

Intel IOMMU debugfs ABI

Intel IOMMU의 register, interrupt remapping, DMA translation, invalidation queue, latency sampling, device·PASID page table을 관찰하는 debugfs ABI를 Kabylake 출력과 함께 설명합니다.

Source pathDocumentation/ABI/testing/debugfs-intel-iommu
Source versionLinux v6.18.37
TranslationDUJINLABS 전문 번역 + 해설

요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.

1. 요약·해설

원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.

IOMMU register set

debugfs-intel-iommu:1-44

iommu_regset은 각 IOMMU device의 register base와 VER·GCMD·GSTS·FSTS·FECTL을 비롯한 register 내용을 dump합니다.

Interrupt remapping과 posting table

debugfs-intel-iommu:46-78

ir_translation_struct는 remapped interrupt의 SrcID·DstID·vector·IRTE와 posted interrupt의 PDA·vector·IRTE table entry를 출력합니다.

DMA remapping structures

debugfs-intel-iommu:79-115

dmar_translation_struct는 root·context table과 PASID directory·table entry를 dump하며, PASID가 없는 legacy mode에서는 PASID를 -1로 표시합니다.

Invalidation queue internals

debugfs-intel-iommu:117-151

invalidation_queue는 IOMMU별 queue base·head·tail과 descriptor quadword·status를 출력합니다.

DMAR invalidation latency sampling

debugfs-intel-iommu:153-220

dmar_perf_latency는 0..3 control 값으로 sampling을 끄거나 IOTLB·devTLB·interrupt entry cache invalidation latency를 선택하고 histogram snapshot을 제공합니다.

Legacy·scalable domain page table

debugfs-intel-iommu:222-276

BDF별 domain_translation_struct는 legacy device page table을, BDF/PASID 경로는 scalable mode의 지정 PASID page table을 dump합니다.

2. 영어 원문 전체

번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.

원문 전체 펼치기
1 What: /sys/kernel/debug/iommu/intel/iommu_regset
2 Date: December 2023
3 Contact: Jingqi Liu <Jingqi.liu@intel.com>
4 Description:
5 This file dumps all the register contents for each IOMMU device.
6
7 Example in Kabylake:
8
9 ::
10
11 $ sudo cat /sys/kernel/debug/iommu/intel/iommu_regset
12
13 IOMMU: dmar0 Register Base Address: 26be37000
14
15 Name Offset Contents
16 VER 0x00 0x0000000000000010
17 GCMD 0x18 0x0000000000000000
18 GSTS 0x1c 0x00000000c7000000
19 FSTS 0x34 0x0000000000000000
20 FECTL 0x38 0x0000000000000000
21
22 [...]
23
24 IOMMU: dmar1 Register Base Address: fed90000
25
26 Name Offset Contents
27 VER 0x00 0x0000000000000010
28 GCMD 0x18 0x0000000000000000
29 GSTS 0x1c 0x00000000c7000000
30 FSTS 0x34 0x0000000000000000
31 FECTL 0x38 0x0000000000000000
32
33 [...]
34
35 IOMMU: dmar2 Register Base Address: fed91000
36
37 Name Offset Contents
38 VER 0x00 0x0000000000000010
39 GCMD 0x18 0x0000000000000000
40 GSTS 0x1c 0x00000000c7000000
41 FSTS 0x34 0x0000000000000000
42 FECTL 0x38 0x0000000000000000
43
44 [...]
45
46 What: /sys/kernel/debug/iommu/intel/ir_translation_struct
47 Date: December 2023
48 Contact: Jingqi Liu <Jingqi.liu@intel.com>
49 Description:
50 This file dumps the table entries for Interrupt
51 remapping and Interrupt posting.
52
53 Example in Kabylake:
54
55 ::
56
57 $ sudo cat /sys/kernel/debug/iommu/intel/ir_translation_struct
58
59 Remapped Interrupt supported on IOMMU: dmar0
60 IR table address:100900000
61
62 Entry SrcID DstID Vct IRTE_high IRTE_low
63 0 00:0a.0 00000080 24 0000000000040050 000000800024000d
64 1 00:0a.0 00000001 ef 0000000000040050 0000000100ef000d
65
66 Remapped Interrupt supported on IOMMU: dmar1
67 IR table address:100300000
68 Entry SrcID DstID Vct IRTE_high IRTE_low
69 0 00:02.0 00000002 26 0000000000040010 000000020026000d
70
71 [...]
72
73 ****
74
75 Posted Interrupt supported on IOMMU: dmar0
76 IR table address:100900000
77 Entry SrcID PDA_high PDA_low Vct IRTE_high IRTE_low
78
79 What: /sys/kernel/debug/iommu/intel/dmar_translation_struct
80 Date: December 2023
81 Contact: Jingqi Liu <Jingqi.liu@intel.com>
82 Description:
83 This file dumps Intel IOMMU DMA remapping tables, such
84 as root table, context table, PASID directory and PASID
85 table entries in debugfs. For legacy mode, it doesn't
86 support PASID, and hence PASID field is defaulted to
87 '-1' and other PASID related fields are invalid.
88
89 Example in Kabylake:
90
91 ::
92
93 $ sudo cat /sys/kernel/debug/iommu/intel/dmar_translation_struct
94
95 IOMMU dmar1: Root Table Address: 0x103027000
96 B.D.F Root_entry
97 00:02.0 0x0000000000000000:0x000000010303e001
98
99 Context_entry
100 0x0000000000000102:0x000000010303f005
101
102 PASID PASID_table_entry
103 -1 0x0000000000000000:0x0000000000000000:0x0000000000000000
104
105 IOMMU dmar0: Root Table Address: 0x103028000
106 B.D.F Root_entry
107 00:0a.0 0x0000000000000000:0x00000001038a7001
108
109 Context_entry
110 0x0000000000000000:0x0000000103220e7d
111
112 PASID PASID_table_entry
113 0 0x0000000000000000:0x0000000000800002:0x00000001038a5089
114
115 [...]
116
117 What: /sys/kernel/debug/iommu/intel/invalidation_queue
118 Date: December 2023
119 Contact: Jingqi Liu <Jingqi.liu@intel.com>
120 Description:
121 This file exports invalidation queue internals of each
122 IOMMU device.
123
124 Example in Kabylake:
125
126 ::
127
128 $ sudo cat /sys/kernel/debug/iommu/intel/invalidation_queue
129
130 Invalidation queue on IOMMU: dmar0
131 Base: 0x10022e000 Head: 20 Tail: 20
132 Index qw0 qw1 qw2
133 0 0000000000000014 0000000000000000 0000000000000000
134 1 0000000200000025 0000000100059c04 0000000000000000
135 2 0000000000000014 0000000000000000 0000000000000000
136
137 qw3 status
138 0000000000000000 0000000000000000
139 0000000000000000 0000000000000000
140 0000000000000000 0000000000000000
141
142 [...]
143
144 Invalidation queue on IOMMU: dmar1
145 Base: 0x10026e000 Head: 32 Tail: 32
146 Index qw0 qw1 status
147 0 0000000000000004 0000000000000000 0000000000000000
148 1 0000000200000025 0000000100059804 0000000000000000
149 2 0000000000000011 0000000000000000 0000000000000000
150
151 [...]
152
153 What: /sys/kernel/debug/iommu/intel/dmar_perf_latency
154 Date: December 2023
155 Contact: Jingqi Liu <Jingqi.liu@intel.com>
156 Description:
157 This file is used to control and show counts of
158 execution time ranges for various types per DMAR.
159
160 Firstly, write a value to
161 /sys/kernel/debug/iommu/intel/dmar_perf_latency
162 to enable sampling.
163
164 The possible values are as follows:
165
166 * 0 - disable sampling all latency data
167
168 * 1 - enable sampling IOTLB invalidation latency data
169
170 * 2 - enable sampling devTLB invalidation latency data
171
172 * 3 - enable sampling intr entry cache invalidation latency data
173
174 Next, read /sys/kernel/debug/iommu/intel/dmar_perf_latency gives
175 a snapshot of sampling result of all enabled monitors.
176
177 Examples in Kabylake:
178
179 ::
180
181 1) Disable sampling all latency data:
182
183 $ sudo echo 0 > /sys/kernel/debug/iommu/intel/dmar_perf_latency
184
185 2) Enable sampling IOTLB invalidation latency data
186
187 $ sudo echo 1 > /sys/kernel/debug/iommu/intel/dmar_perf_latency
188
189 $ sudo cat /sys/kernel/debug/iommu/intel/dmar_perf_latency
190
191 IOMMU: dmar0 Register Base Address: 26be37000
192 <0.1us 0.1us-1us 1us-10us 10us-100us 100us-1ms
193 inv_iotlb 0 0 0 0 0
194
195 1ms-10ms >=10ms min(us) max(us) average(us)
196 inv_iotlb 0 0 0 0 0
197
198 [...]
199
200 IOMMU: dmar2 Register Base Address: fed91000
201 <0.1us 0.1us-1us 1us-10us 10us-100us 100us-1ms
202 inv_iotlb 0 0 18 0 0
203
204 1ms-10ms >=10ms min(us) max(us) average(us)
205 inv_iotlb 0 0 2 2 2
206
207 3) Enable sampling devTLB invalidation latency data
208
209 $ sudo echo 2 > /sys/kernel/debug/iommu/intel/dmar_perf_latency
210
211 $ sudo cat /sys/kernel/debug/iommu/intel/dmar_perf_latency
212
213 IOMMU: dmar0 Register Base Address: 26be37000
214 <0.1us 0.1us-1us 1us-10us 10us-100us 100us-1ms
215 inv_devtlb 0 0 0 0 0
216
217 >=10ms min(us) max(us) average(us)
218 inv_devtlb 0 0 0 0
219
220 [...]
221
222 What: /sys/kernel/debug/iommu/intel/<bdf>/domain_translation_struct
223 Date: December 2023
224 Contact: Jingqi Liu <Jingqi.liu@intel.com>
225 Description:
226 This file dumps a specified page table of Intel IOMMU
227 in legacy mode or scalable mode.
228
229 For a device that only supports legacy mode, dump its
230 page table by the debugfs file in the debugfs device
231 directory. e.g.
232 /sys/kernel/debug/iommu/intel/0000:00:02.0/domain_translation_struct.
233
234 For a device that supports scalable mode, dump the
235 page table of specified pasid by the debugfs file in
236 the debugfs pasid directory. e.g.
237 /sys/kernel/debug/iommu/intel/0000:00:02.0/1/domain_translation_struct.
238
239 Examples in Kabylake:
240
241 ::
242
243 1) Dump the page table of device "0000:00:02.0" that only supports legacy mode.
244
245 $ sudo cat /sys/kernel/debug/iommu/intel/0000:00:02.0/domain_translation_struct
246
247 Device 0000:00:02.0 @0x1017f8000
248 IOVA_PFN PML5E PML4E
249 0x000000008d800 | 0x0000000000000000 0x00000001017f9003
250 0x000000008d801 | 0x0000000000000000 0x00000001017f9003
251 0x000000008d802 | 0x0000000000000000 0x00000001017f9003
252
253 PDPE PDE PTE
254 0x00000001017fa003 0x00000001017fb003 0x000000008d800003
255 0x00000001017fa003 0x00000001017fb003 0x000000008d801003
256 0x00000001017fa003 0x00000001017fb003 0x000000008d802003
257
258 [...]
259
260 2) Dump the page table of device "0000:00:0a.0" with PASID "1" that
261 supports scalable mode.
262
263 $ sudo cat /sys/kernel/debug/iommu/intel/0000:00:0a.0/1/domain_translation_struct
264
265 Device 0000:00:0a.0 with pasid 1 @0x10c112000
266 IOVA_PFN PML5E PML4E
267 0x0000000000000 | 0x0000000000000000 0x000000010df93003
268 0x0000000000001 | 0x0000000000000000 0x000000010df93003
269 0x0000000000002 | 0x0000000000000000 0x000000010df93003
270
271 PDPE PDE PTE
272 0x0000000106ae6003 0x0000000104b38003 0x0000000147c00803
273 0x0000000106ae6003 0x0000000104b38003 0x0000000147c01803
274 0x0000000106ae6003 0x0000000104b38003 0x0000000147c02803
275
276 [...]
277

3. 한국어 전문 번역

영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.

IOMMU register set dump

1-44
항목내용
What/sys/kernel/debug/iommu/intel/iommu_regset
Date2023년 12월
ContactJingqi Liu <Jingqi.liu@intel.com>
Description각 IOMMU device의 모든 register 내용을 dump합니다.

Kabylake에서의 예:

$ sudo cat /sys/kernel/debug/iommu/intel/iommu_regset

IOMMU: dmar0 Register Base Address: 26be37000

Name                    Offset          Contents
VER                     0x00            0x0000000000000010
GCMD                    0x18            0x0000000000000000
GSTS                    0x1c            0x00000000c7000000
FSTS                    0x34            0x0000000000000000
FECTL                   0x38            0x0000000000000000

[...]

IOMMU: dmar1 Register Base Address: fed90000

Name                    Offset          Contents
VER                     0x00            0x0000000000000010
GCMD                    0x18            0x0000000000000000
GSTS                    0x1c            0x00000000c7000000
FSTS                    0x34            0x0000000000000000
FECTL                   0x38            0x0000000000000000

[...]

IOMMU: dmar2 Register Base Address: fed91000

Name                    Offset          Contents
VER                     0x00            0x0000000000000010
GCMD                    0x18            0x0000000000000000
GSTS                    0x1c            0x00000000c7000000
FSTS                    0x34            0x0000000000000000
FECTL                   0x38            0x0000000000000000

[...]

Interrupt remapping·posting table

46-78
항목내용
What/sys/kernel/debug/iommu/intel/ir_translation_struct
Date2023년 12월
ContactJingqi Liu <Jingqi.liu@intel.com>
DescriptionInterrupt remapping과 Interrupt posting을 위한 table entry를 dump합니다.

Kabylake에서의 예:

$ sudo cat /sys/kernel/debug/iommu/intel/ir_translation_struct

Remapped Interrupt supported on IOMMU: dmar0
IR table address:100900000

Entry SrcID   DstID    Vct IRTE_high           IRTE_low
0     00:0a.0 00000080 24  0000000000040050    000000800024000d
1     00:0a.0 00000001 ef  0000000000040050    0000000100ef000d

Remapped Interrupt supported on IOMMU: dmar1
IR table address:100300000
Entry SrcID   DstID    Vct IRTE_high           IRTE_low
0     00:02.0 00000002 26  0000000000040010    000000020026000d

[...]

****

Posted Interrupt supported on IOMMU: dmar0
IR table address:100900000
Entry SrcID   PDA_high PDA_low  Vct IRTE_high          IRTE_low

Intel IOMMU DMA remapping table

79-115
항목내용
What/sys/kernel/debug/iommu/intel/dmar_translation_struct
Date2023년 12월
ContactJingqi Liu <Jingqi.liu@intel.com>
DescriptionRoot table, context table, PASID directory, PASID table entry 같은 Intel IOMMU DMA remapping table을 debugfs에 dump합니다.
Legacy modePASID를 지원하지 않으므로 PASID field는 기본값 -1이 되고 PASID 관련 다른 field는 유효하지 않습니다.

Kabylake에서의 예:

$ sudo cat /sys/kernel/debug/iommu/intel/dmar_translation_struct

IOMMU dmar1: Root Table Address: 0x103027000
B.D.F   Root_entry
00:02.0 0x0000000000000000:0x000000010303e001

Context_entry
0x0000000000000102:0x000000010303f005

PASID   PASID_table_entry
-1      0x0000000000000000:0x0000000000000000:0x0000000000000000

IOMMU dmar0: Root Table Address: 0x103028000
B.D.F   Root_entry
00:0a.0 0x0000000000000000:0x00000001038a7001

Context_entry
0x0000000000000000:0x0000000103220e7d

PASID   PASID_table_entry
0       0x0000000000000000:0x0000000000800002:0x00000001038a5089

[...]

IOMMU invalidation queue internals

117-151
항목내용
What/sys/kernel/debug/iommu/intel/invalidation_queue
Date2023년 12월
ContactJingqi Liu <Jingqi.liu@intel.com>
Description각 IOMMU device의 invalidation queue 내부 정보를 내보냅니다.

Kabylake에서의 예:

$ sudo cat /sys/kernel/debug/iommu/intel/invalidation_queue

Invalidation queue on IOMMU: dmar0
Base: 0x10022e000      Head: 20        Tail: 20
Index          qw0                    qw1                     qw2
    0   0000000000000014        0000000000000000        0000000000000000
    1   0000000200000025        0000000100059c04        0000000000000000
    2   0000000000000014        0000000000000000        0000000000000000

                qw3                  status
         0000000000000000        0000000000000000
         0000000000000000        0000000000000000
         0000000000000000        0000000000000000

[...]

Invalidation queue on IOMMU: dmar1
Base: 0x10026e000      Head: 32        Tail: 32
Index           qw0                     qw1                   status
    0   0000000000000004        0000000000000000         0000000000000000
    1   0000000200000025        0000000100059804         0000000000000000
    2   0000000000000011        0000000000000000         0000000000000000

[...]

DMAR latency sampling control과 결과

153-220
항목내용
What/sys/kernel/debug/iommu/intel/dmar_perf_latency
Date2023년 12월
ContactJingqi Liu <Jingqi.liu@intel.com>
DescriptionDMAR별 여러 유형의 실행 시간 범위 count를 제어하고 표시합니다.

먼저 /sys/kernel/debug/iommu/intel/dmar_perf_latency에 값을 써서 sampling을 활성화합니다. 가능한 값은 다음과 같습니다.

값동작
0모든 latency data sampling을 비활성화
1IOTLB invalidation latency data sampling을 활성화
2devTLB invalidation latency data sampling을 활성화
3Interrupt entry cache invalidation latency data sampling을 활성화

그다음 /sys/kernel/debug/iommu/intel/dmar_perf_latency를 읽으면 활성화된 모든 monitor의 sampling 결과 snapshot을 얻습니다.

Kabylake에서 모든 latency sampling을 비활성화하는 예:

$ sudo echo 0 > /sys/kernel/debug/iommu/intel/dmar_perf_latency

IOTLB invalidation latency sampling을 활성화하는 예:

$ sudo echo 1 > /sys/kernel/debug/iommu/intel/dmar_perf_latency

$ sudo cat /sys/kernel/debug/iommu/intel/dmar_perf_latency

IOMMU: dmar0 Register Base Address: 26be37000
                <0.1us   0.1us-1us    1us-10us  10us-100us   100us-1ms
inv_iotlb           0           0           0           0           0

                1ms-10ms      >=10ms     min(us)     max(us) average(us)
inv_iotlb           0           0           0           0           0

[...]

IOMMU: dmar2 Register Base Address: fed91000
                <0.1us   0.1us-1us    1us-10us  10us-100us   100us-1ms
inv_iotlb           0           0          18           0           0

                1ms-10ms      >=10ms     min(us)     max(us) average(us)
inv_iotlb           0           0           2           2           2

devTLB invalidation latency sampling을 활성화하는 예:

$ sudo echo 2 > /sys/kernel/debug/iommu/intel/dmar_perf_latency

$ sudo cat /sys/kernel/debug/iommu/intel/dmar_perf_latency

IOMMU: dmar0 Register Base Address: 26be37000
                <0.1us   0.1us-1us    1us-10us  10us-100us   100us-1ms
inv_devtlb           0           0           0           0           0

                >=10ms     min(us)     max(us) average(us)
inv_devtlb           0           0           0           0

[...]

Device·PASID domain page table dump

222-276
항목내용
What/sys/kernel/debug/iommu/intel/<bdf>/domain_translation_struct
Date2023년 12월
ContactJingqi Liu <Jingqi.liu@intel.com>
DescriptionLegacy mode 또는 scalable mode에서 지정한 Intel IOMMU page table을 dump합니다.

Legacy mode만 지원하는 device는 debugfs device directory의 파일로 page table을 dump합니다. 예: /sys/kernel/debug/iommu/intel/0000:00:02.0/domain_translation_struct.

Scalable mode를 지원하는 device는 debugfs PASID directory의 파일로 지정 PASID의 page table을 dump합니다. 예: /sys/kernel/debug/iommu/intel/0000:00:02.0/1/domain_translation_struct.

Kabylake에서 legacy mode만 지원하는 device 0000:00:02.0의 page table을 dump하는 예:

$ sudo cat /sys/kernel/debug/iommu/intel/0000:00:02.0/domain_translation_struct

Device 0000:00:02.0 @0x1017f8000
IOVA_PFN                PML5E                   PML4E
0x000000008d800 |       0x0000000000000000      0x00000001017f9003
0x000000008d801 |       0x0000000000000000      0x00000001017f9003
0x000000008d802 |       0x0000000000000000      0x00000001017f9003

PDPE                    PDE                     PTE
0x00000001017fa003      0x00000001017fb003      0x000000008d800003
0x00000001017fa003      0x00000001017fb003      0x000000008d801003
0x00000001017fa003      0x00000001017fb003      0x000000008d802003

[...]

Kabylake에서 scalable mode를 지원하는 device 0000:00:0a.0의 PASID 1 page table을 dump하는 예:

$ sudo cat /sys/kernel/debug/iommu/intel/0000:00:0a.0/1/domain_translation_struct

Device 0000:00:0a.0 with pasid 1 @0x10c112000
IOVA_PFN                PML5E                   PML4E
0x0000000000000 |       0x0000000000000000      0x000000010df93003
0x0000000000001 |       0x0000000000000000      0x000000010df93003
0x0000000000002 |       0x0000000000000000      0x000000010df93003

PDPE                    PDE                     PTE
0x0000000106ae6003      0x0000000104b38003      0x0000000147c00803
0x0000000106ae6003      0x0000000104b38003      0x0000000147c01803
0x0000000106ae6003      0x0000000104b38003      0x0000000147c02803

[...]
Intel IOMMU debugfs 관찰 지점
Intel IOMMU deviceRegister setiommu_regset
Interrupt sourceIRTE remapping or postingir_translation_struct
PCI BDFRoot and context tablesPASID table when supporteddmar_translation_struct
Invalidation requestQueue descriptor qw0..qw3invalidation_queue
Sampling mode 0..3IOTLB, devTLB, or IEC invalidationLatency histogram snapshot
Legacy BDF or scalable BDF/PASIDPML5E to PTE walkdomain_translation_struct

Hardware register에서 interrupt·DMA translation, invalidation, latency, device page table까지 서로 다른 계층을 각 파일이 노출한다.