← Documents Documentation/PCI/endpoint/pci-vntb-howto.rst GitHub 원문 ↗

Linux 6.18.37 · PCI

PCI vNTB endpoint function 사용자 안내

Configfs로 vNTB function을 만들고 PCI·NTB·virtual bus 속성을 구성한 뒤 controller에 bind해 양쪽 PCI bus와 NTB client를 확인합니다.

Source pathDocumentation/PCI/endpoint/pci-vntb-howto.rst
Source versionLinux v6.18.37
TranslationDUJINLABS 전문 번역 + 해설

요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.

1. 요약·해설

원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.

요약·해설

pci-vntb-howto.rst:1-169

EPC와 EPF를 확인하고 `func1`을 생성한 뒤 PCI ID, doorbell·scratchpad·memory-window 수, virtual bus ID를 구성합니다.

Function을 primary controller에 연결하고 link를 시작하면 host PCI bus와 EP virtual PCI bus에서 각각 device를 확인하고 표준 Linux NTB client를 사용할 수 있습니다.

2. 영어 원문 전체

번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.

원문 전체 펼치기
1 .. SPDX-License-Identifier: GPL-2.0
2
3 ===================================================================
4 PCI Non-Transparent Bridge (NTB) Endpoint Function (EPF) User Guide
5 ===================================================================
6
7 :Author: Frank Li <Frank.Li@nxp.com>
8
9 This document is a guide to help users use pci-epf-vntb function driver
10 and ntb_hw_epf host driver for NTB functionality. The list of steps to
11 be followed in the host side and EP side is given below. For the hardware
12 configuration and internals of NTB using configurable endpoints see
13 Documentation/PCI/endpoint/pci-vntb-function.rst
14
15 Endpoint Device
16 ===============
17
18 Endpoint Controller Devices
19 ---------------------------
20
21 To find the list of endpoint controller devices in the system::
22
23 # ls /sys/class/pci_epc/
24 5f010000.pcie_ep
25
26 If PCI_ENDPOINT_CONFIGFS is enabled::
27
28 # ls /sys/kernel/config/pci_ep/controllers
29 5f010000.pcie_ep
30
31 Endpoint Function Drivers
32 -------------------------
33
34 To find the list of endpoint function drivers in the system::
35
36 # ls /sys/bus/pci-epf/drivers
37 pci_epf_ntb pci_epf_test pci_epf_vntb
38
39 If PCI_ENDPOINT_CONFIGFS is enabled::
40
41 # ls /sys/kernel/config/pci_ep/functions
42 pci_epf_ntb pci_epf_test pci_epf_vntb
43
44
45 Creating pci-epf-vntb Device
46 ----------------------------
47
48 PCI endpoint function device can be created using the configfs. To create
49 pci-epf-vntb device, the following commands can be used::
50
51 # mount -t configfs none /sys/kernel/config
52 # cd /sys/kernel/config/pci_ep/
53 # mkdir functions/pci_epf_vntb/func1
54
55 The "mkdir func1" above creates the pci-epf-vntb function device that will
56 be probed by pci_epf_vntb driver.
57
58 The PCI endpoint framework populates the directory with the following
59 configurable fields::
60
61 # ls functions/pci_epf_vntb/func1
62 baseclass_code deviceid msi_interrupts pci-epf-vntb.0
63 progif_code secondary subsys_id vendorid
64 cache_line_size interrupt_pin msix_interrupts primary
65 revid subclass_code subsys_vendor_id
66
67 The PCI endpoint function driver populates these entries with default values
68 when the device is bound to the driver. The pci-epf-vntb driver populates
69 vendorid with 0xffff and interrupt_pin with 0x0001::
70
71 # cat functions/pci_epf_vntb/func1/vendorid
72 0xffff
73 # cat functions/pci_epf_vntb/func1/interrupt_pin
74 0x0001
75
76
77 Configuring pci-epf-vntb Device
78 -------------------------------
79
80 The user can configure the pci-epf-vntb device using its configfs entry. In order
81 to change the vendorid and the deviceid, the following
82 commands can be used::
83
84 # echo 0x1957 > functions/pci_epf_vntb/func1/vendorid
85 # echo 0x0809 > functions/pci_epf_vntb/func1/deviceid
86
87 The PCI endpoint framework also automatically creates a sub-directory in the
88 function attribute directory. This sub-directory has the same name as the name
89 of the function device and is populated with the following NTB specific
90 attributes that can be configured by the user::
91
92 # ls functions/pci_epf_vntb/func1/pci_epf_vntb.0/
93 ctrl_bar db_count mw1_bar mw2_bar mw3_bar mw4_bar spad_count
94 db_bar mw1 mw2 mw3 mw4 num_mws vbus_number
95 vntb_vid vntb_pid
96
97 A sample configuration for NTB function is given below::
98
99 # echo 4 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/db_count
100 # echo 128 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/spad_count
101 # echo 1 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/num_mws
102 # echo 0x100000 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/mw1
103
104 By default, each construct is assigned a BAR, as needed and in order.
105 Should a specific BAR setup be required by the platform, BAR may be assigned
106 to each construct using the related ``XYZ_bar`` entry.
107
108 A sample configuration for virtual NTB driver for virtual PCI bus::
109
110 # echo 0x1957 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vntb_vid
111 # echo 0x080A > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vntb_pid
112 # echo 0x10 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vbus_number
113
114 Binding pci-epf-vntb Device to EP Controller
115 --------------------------------------------
116
117 NTB function device should be attached to PCI endpoint controllers
118 connected to the host.
119
120 # ln -s controllers/5f010000.pcie_ep functions/pci_epf_vntb/func1/primary
121
122 Once the above step is completed, the PCI endpoint controllers are ready to
123 establish a link with the host.
124
125
126 Start the Link
127 --------------
128
129 In order for the endpoint device to establish a link with the host, the _start_
130 field should be populated with '1'. For NTB, both the PCI endpoint controllers
131 should establish link with the host (imx8 don't need this steps)::
132
133 # echo 1 > controllers/5f010000.pcie_ep/start
134
135 RootComplex Device
136 ==================
137
138 lspci Output at Host side
139 -------------------------
140
141 Note that the devices listed here correspond to the values populated in
142 "Creating pci-epf-vntb Device" section above::
143
144 # lspci
145 00:00.0 PCI bridge: Freescale Semiconductor Inc Device 0000 (rev 01)
146 01:00.0 RAM memory: Freescale Semiconductor Inc Device 0809
147
148 Endpoint Device / Virtual PCI bus
149 =================================
150
151 lspci Output at EP Side / Virtual PCI bus
152 -----------------------------------------
153
154 Note that the devices listed here correspond to the values populated in
155 "Creating pci-epf-vntb Device" section above::
156
157 # lspci
158 10:00.0 Unassigned class [ffff]: Dawicontrol Computersysteme GmbH Device 1234 (rev ff)
159
160 Using ntb_hw_epf Device
161 -----------------------
162
163 The host side software follows the standard NTB software architecture in Linux.
164 All the existing client side NTB utilities like NTB Transport Client and NTB
165 Netdev, NTB Ping Pong Test Client and NTB Tool Test Client can be used with NTB
166 function device.
167
168 For more information on NTB see
169 :doc:`Non-Transparent Bridge <../../driver-api/ntb>`
170

3. 한국어 전문 번역

영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.

안내 범위

1-14

저자는 Frank Li입니다. 이 문서는 NTB 기능을 위해 `pci-epf-vntb` function driver와 `ntb_hw_epf` host driver를 사용하는 방법을 안내합니다.

아래에는 host 측과 EP 측에서 따라야 할 절차를 차례로 제시합니다. Configurable endpoint를 이용한 NTB의 hardware 구성과 내부 구조는 `Documentation/PCI/endpoint/pci-vntb-function.rst`를 참조하십시오.

.. SPDX-License-Identifier: GPL-2.0

===================================================================
PCI Non-Transparent Bridge (NTB) Endpoint Function (EPF) User Guide
===================================================================

:Author: Frank Li <Frank.Li@nxp.com>

This document is a guide to help users use pci-epf-vntb function driver
and ntb_hw_epf host driver for NTB functionality. The list of steps to
be followed in the host side and EP side is given below. For the hardware
configuration and internals of NTB using configurable endpoints see
Documentation/PCI/endpoint/pci-vntb-function.rst

Endpoint controller 확인

15-30

System의 endpoint controller device 목록은 `/sys/class/pci_epc/`에서 확인합니다.

# ls /sys/class/pci_epc/
  5f010000.pcie_ep

`PCI_ENDPOINT_CONFIGFS`가 활성화되어 있으면 `/sys/kernel/config/pci_ep/controllers`에서도 같은 controller를 확인할 수 있습니다.

# ls /sys/kernel/config/pci_ep/controllers
  5f010000.pcie_ep
Endpoint Device
===============

Endpoint Controller Devices
---------------------------

To find the list of endpoint controller devices in the system::

        # ls /sys/class/pci_epc/
          5f010000.pcie_ep

If PCI_ENDPOINT_CONFIGFS is enabled::

        # ls /sys/kernel/config/pci_ep/controllers
          5f010000.pcie_ep

Endpoint function driver 확인

31-44

System에 등록된 endpoint function driver는 PCI EPF bus에서 확인합니다. 예시에는 `pci_epf_ntb`, `pci_epf_test`, `pci_epf_vntb`가 있습니다.

# ls /sys/bus/pci-epf/drivers
pci_epf_ntb  pci_epf_test  pci_epf_vntb

`PCI_ENDPOINT_CONFIGFS`가 활성화되어 있으면 configfs의 function 목록에도 같은 세 driver가 나타납니다.

# ls /sys/kernel/config/pci_ep/functions
pci_epf_ntb  pci_epf_test  pci_epf_vntb
Endpoint Function Drivers
-------------------------

To find the list of endpoint function drivers in the system::

        # ls /sys/bus/pci-epf/drivers
        pci_epf_ntb  pci_epf_test  pci_epf_vntb

If PCI_ENDPOINT_CONFIGFS is enabled::

        # ls /sys/kernel/config/pci_ep/functions
        pci_epf_ntb  pci_epf_test  pci_epf_vntb

pci-epf-vntb device 생성

45-76

PCI endpoint function device는 configfs로 생성할 수 있습니다. Configfs를 mount하고 PCI EP directory로 이동한 뒤 `functions/pci_epf_vntb/func1`을 만듭니다.

# mount -t configfs none /sys/kernel/config
# cd /sys/kernel/config/pci_ep/
# mkdir functions/pci_epf_vntb/func1

`mkdir func1`은 `pci_epf_vntb` driver가 probe할 pci-epf-vntb function device를 만듭니다.

PCI endpoint framework는 이 directory에 PCI identity·class·interrupt와 primary/secondary 연결을 위한 구성 field를 채웁니다.

# ls functions/pci_epf_vntb/func1
baseclass_code    deviceid          msi_interrupts    pci-epf-vntb.0
progif_code       secondary         subsys_id         vendorid
cache_line_size   interrupt_pin     msix_interrupts   primary
revid             subclass_code     subsys_vendor_id

Device가 driver에 bind되면 endpoint function driver가 기본값을 채웁니다. `pci-epf-vntb` driver의 `vendorid` 기본값은 `0xffff`, `interrupt_pin` 기본값은 `0x0001`입니다.

# cat functions/pci_epf_vntb/func1/vendorid
0xffff
# cat functions/pci_epf_vntb/func1/interrupt_pin
0x0001
Creating pci-epf-vntb Device
----------------------------

PCI endpoint function device can be created using the configfs. To create
pci-epf-vntb device, the following commands can be used::

        # mount -t configfs none /sys/kernel/config
        # cd /sys/kernel/config/pci_ep/
        # mkdir functions/pci_epf_vntb/func1

The "mkdir func1" above creates the pci-epf-vntb function device that will
be probed by pci_epf_vntb driver.

The PCI endpoint framework populates the directory with the following
configurable fields::

        # ls functions/pci_epf_vntb/func1
        baseclass_code    deviceid          msi_interrupts    pci-epf-vntb.0
        progif_code       secondary         subsys_id         vendorid
        cache_line_size   interrupt_pin     msix_interrupts   primary
        revid             subclass_code     subsys_vendor_id

The PCI endpoint function driver populates these entries with default values
when the device is bound to the driver. The pci-epf-vntb driver populates
vendorid with 0xffff and interrupt_pin with 0x0001::

        # cat functions/pci_epf_vntb/func1/vendorid
        0xffff
        # cat functions/pci_epf_vntb/func1/interrupt_pin
        0x0001

PCI·NTB·virtual bus 구성

77-113

사용자는 function의 configfs 항목으로 pci-epf-vntb device를 구성합니다. 예시는 PCI vendor ID를 `0x1957`, device ID를 `0x0809`로 설정합니다.

# echo 0x1957 > functions/pci_epf_vntb/func1/vendorid
# echo 0x0809 > functions/pci_epf_vntb/func1/deviceid

Framework는 function device와 같은 이름의 하위 directory `pci_epf_vntb.0`도 자동 생성합니다. 여기에는 control·doorbell·memory-window BAR, doorbell와 scratchpad 개수, memory-window 크기와 수, virtual bus ID를 위한 NTB 전용 속성이 있습니다.

# ls functions/pci_epf_vntb/func1/pci_epf_vntb.0/
ctrl_bar  db_count  mw1_bar  mw2_bar  mw3_bar  mw4_bar  spad_count
db_bar    mw1       mw2      mw3      mw4      num_mws  vbus_number
vntb_vid  vntb_pid

예시 NTB 구성은 doorbell 4개, scratchpad 128개, memory window 1개를 사용하고 첫 window 크기를 `0x100000`으로 설정합니다.

# echo 4 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/db_count
# echo 128 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/spad_count
# echo 1 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/num_mws
# echo 0x100000 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/mw1

기본적으로 각 구성 요소는 필요에 따라 순서대로 BAR를 배정받습니다. Platform에 특정 BAR 배치가 필요하면 해당 `XYZ_bar` 항목으로 각 구성 요소의 BAR를 지정할 수 있습니다.

Virtual PCI bus용 virtual NTB driver 예시는 vendor ID `0x1957`, product ID `0x080A`, virtual bus number `0x10`을 사용합니다.

# echo 0x1957 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vntb_vid
# echo 0x080A > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vntb_pid
# echo 0x10 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vbus_number
Configuring pci-epf-vntb Device
-------------------------------

The user can configure the pci-epf-vntb device using its configfs entry. In order
to change the vendorid and the deviceid, the following
commands can be used::

        # echo 0x1957 > functions/pci_epf_vntb/func1/vendorid
        # echo 0x0809 > functions/pci_epf_vntb/func1/deviceid

The PCI endpoint framework also automatically creates a sub-directory in the
function attribute directory. This sub-directory has the same name as the name
of the function device and is populated with the following NTB specific
attributes that can be configured by the user::

        # ls functions/pci_epf_vntb/func1/pci_epf_vntb.0/
        ctrl_bar  db_count  mw1_bar  mw2_bar  mw3_bar  mw4_bar        spad_count
        db_bar          mw1            mw2      mw3      mw4      num_mws        vbus_number
        vntb_vid  vntb_pid

A sample configuration for NTB function is given below::

        # echo 4 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/db_count
        # echo 128 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/spad_count
        # echo 1 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/num_mws
        # echo 0x100000 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/mw1

By default, each construct is assigned a BAR, as needed and in order.
Should a specific BAR setup be required by the platform, BAR may be assigned
to each construct using the related ``XYZ_bar`` entry.

A sample configuration for virtual NTB driver for virtual PCI bus::

        # echo 0x1957 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vntb_vid
        # echo 0x080A > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vntb_pid
        # echo 0x10 > functions/pci_epf_vntb/func1/pci_epf_vntb.0/vbus_number

EP controller에 bind

114-125

NTB function device는 host에 연결된 PCI endpoint controller에 붙여야 합니다. `primary` 항목에 controller symlink를 만듭니다.

# ln -s controllers/5f010000.pcie_ep functions/pci_epf_vntb/func1/primary

이 단계를 마치면 PCI endpoint controller가 host와 link를 맺을 준비를 마칩니다.

Binding pci-epf-vntb Device to EP Controller
--------------------------------------------

NTB function device should be attached to PCI endpoint controllers
connected to the host.

        # ln -s controllers/5f010000.pcie_ep functions/pci_epf_vntb/func1/primary

Once the above step is completed, the PCI endpoint controllers are ready to
establish a link with the host.

Link 시작

126-134

Endpoint device가 host와 link를 맺게 하려면 `_start_` field에 `1`을 기록해야 합니다. NTB에서는 두 PCI endpoint controller가 모두 host와 link를 맺어야 하지만, i.MX8에는 이 단계가 필요하지 않습니다.

# echo 1 > controllers/5f010000.pcie_ep/start
vNTB EP 준비 순서
EPC/EPF 확인func1 생성PCI·NTB 속성 구성primary symlinkstart = 1

Controller와 function을 찾은 뒤 생성·구성·bind·start 순으로 진행합니다.

Start the Link
--------------

In order for the endpoint device to establish a link with the host, the _start_
field should be populated with '1'. For NTB, both the PCI endpoint controllers
should establish link with the host (imx8 don't need this steps)::

        # echo 1 > controllers/5f010000.pcie_ep/start

Host 측 Root Complex 확인

135-147

Host의 `lspci` 출력에 나타나는 device는 앞의 pci-epf-vntb 생성 단계에서 채운 값에 대응합니다. 예시에서는 Freescale PCI bridge `0000` 뒤에 RAM memory device `0809`가 나타납니다.

# lspci
00:00.0 PCI bridge: Freescale Semiconductor Inc Device 0000 (rev 01)
01:00.0 RAM memory: Freescale Semiconductor Inc Device 0809
RootComplex Device
==================

lspci Output at Host side
-------------------------

Note that the devices listed here correspond to the values populated in
"Creating pci-epf-vntb Device" section above::

        # lspci
        00:00.0 PCI bridge: Freescale Semiconductor Inc Device 0000 (rev 01)
        01:00.0 RAM memory: Freescale Semiconductor Inc Device 0809

EP virtual PCI bus 확인

148-159

EP 측 virtual PCI bus의 `lspci` 출력도 앞에서 구성한 값에 대응합니다. Virtual bus number가 `0x10`이므로 예시 device는 `10:00.0`에 나타납니다.

# lspci
10:00.0 Unassigned class [ffff]: Dawicontrol Computersysteme GmbH Device 1234 (rev ff)
Endpoint Device / Virtual PCI bus
=================================

lspci Output at EP Side / Virtual PCI bus
-----------------------------------------

Note that the devices listed here correspond to the values populated in
"Creating pci-epf-vntb Device" section above::

        # lspci
        10:00.0 Unassigned class [ffff]: Dawicontrol Computersysteme GmbH Device 1234 (rev ff)

ntb_hw_epf와 NTB client 사용

160-169

Host 측 software는 Linux의 표준 NTB software architecture를 따릅니다.

NTB Transport Client와 NTB Netdev, NTB Ping Pong Test Client, NTB Tool Test Client 같은 기존 client-side NTB utility를 NTB function device와 함께 사용할 수 있습니다.

NTB에 관한 자세한 내용은 `Non-Transparent Bridge <../../driver-api/ntb>` 문서를 참조하십시오.

Using ntb_hw_epf Device
-----------------------

The host side software follows the standard NTB software architecture in Linux.
All the existing client side NTB utilities like NTB Transport Client and NTB
Netdev, NTB Ping Pong Test Client and NTB Tool Test Client can be used with NTB
function device.

For more information on NTB see
:doc:`Non-Transparent Bridge <../../driver-api/ntb>`