← Documents Documentation/networking/device_drivers/ethernet/freescale/dpaa2/overview.rst GitHub 원문 ↗

Linux 6.18.37 · Networking

DPAA2 (Data Path Acceleration Architecture Gen2) Overview

DPAA2 Management Complex의 객체 모델, DPRC 격리와 hotplug, Ethernet용 DPNI·DPIO·DPBP·DPMAC 조합, Linux bus·allocator·기능 드라이버 구조를 설명합니다.

Source pathDocumentation/networking/device_drivers/ethernet/freescale/dpaa2/overview.rst
Source versionLinux v6.18.37
TranslationDUJINLABS 전문 번역 + 해설

요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.

1. 요약·해설

원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.

요약·해설

overview.rst:1-406

DPAA2의 핵심은 하드웨어 자원을 MC가 객체로 조합한다는 점입니다. 제어 작업은 MC portal을 거치지만 패킷 fast path는 DPIO MMIO를 직접 사용합니다. DPRC가 bus와 격리 경계를 만들고, Linux에서는 MC-bus·DPRC·allocator·DPIO·Ethernet·MAC 드라이버가 객체별 책임을 나눕니다.

제어 경로와 fast path
DPAA2 driverMC portalManagement Complexcreate·discover·connect·configure·destroyDPAA2 objects
packet Tx·RxDPIO MMIOqueue·buffer hardware

MC가 중재하는 객체 수명 주기와 직접 DPIO 경로를 구분합니다.

하드웨어 자원과 객체
하드웨어 자원대표 객체
queues·queue portalsDPIO·DPNI
buffer poolsDPBP
Ethernet MAC·portsDPMAC
MC portalsDPMCP·DPRC
network interface profilesDPNI

MC가 원시 자원을 기능 객체로 추상화합니다.

DPRC 객체 구성
DPRCDPMCP ×3DPIO ×2DPBP ×1DPNI ×1DPMAC ×1

원문의 DPRC container ASCII 도식을 객체 유형별로 정리했습니다.

DPRC bus 특성
기능동작
열거·발견객체·MMIO 영역·IRQ 검색
동적 수명객체 create/destroy와 hotplug
명령매핑 가능한 MC portal
상태hotplug event interrupt
정적 구성firmware 시작 시 config file

DPRC가 OS에서 plug-and-play bus처럼 보이는 이유입니다.

격리 경계
범위isolation context
같은 DPRC 안의 객체동일 context 공유
서로 다른 DPRCcontainer 단위 격리

IOMMU 관점의 격리 단위는 개별 객체가 아닙니다.

2 CPU 네트워크 구성
CPU0DPIO0공유 DPNIDPMACport·PHY
CPU1DPIO1공유 DPNIDPMACport·PHY
DPBP·DPMCP공유 DPNI

원문의 CPU·DPIO·DPNI·DPMAC ASCII 구조도를 다시 구성했습니다.

Ethernet 핵심 객체
객체역할MMIO·IRQ
DPMACMAC·PHY 물리 송수신MMIO 없음·link change IRQ
DPNITX/RX 큐·인터페이스 구성MMIO 없음·link IRQ
DPIO큐·버퍼 접근queue/buffer MMIO·가용성 IRQ
DPBP하드웨어 버퍼 풀MMIO·IRQ 없음
DPMCPMC command portalportal MMIO·completion IRQ

객체별 기능과 주요 하드웨어 표면입니다.

객체 연결
DPNIDPMAC 또는 다른 DPNI 또는 L2 switch port
Ethernet DriverDPBP 구성RX memory pointer poolDPNI

DPNI의 기능을 완성하는 endpoint와 버퍼 풀 관계입니다.

message interrupt 구성
요소설명
device-id위조 불가능하며 같은 DPRC에서 공유
addressmessage target 주소
data valuemessage payload
ARM SoCdevice-id가 stream ID와 동일

모든 DPAA2 객체 IRQ가 공유하는 하드웨어 표현입니다.

Linux 드라이버 스택
device treeMC-busDPRC driverallocator
Network StackEthernet(DPNI)DPIO driverDPIO hardware
PHY driverMAC(DPMAC)MCDPMAC·PHY

원문의 전체 네트워킹 ASCII 도식을 드라이버 책임으로 다시 구성했습니다.

MC-bus bootstrap
단계동작
probe`fsl,qoriq-mc` device-tree node
bus 등록`fsl-mc`와 callback
IRQMSI domain 생성
root 객체device add 후 DPRC driver bind

root DPRC를 Linux bus에 노출하기까지의 핵심 작업입니다.

DPRC runtime 관리
DPRC driver bindinitial bus scanhotplug interruptDPRC re-scandevice add/remove

초기 scan과 container event 처리 흐름입니다.

Allocator 객체 풀
DPMCP·DPBP 발견allocator bindavailable object poolEthernet 등 DPAA2 driver

대체 가능한 DPMCP·DPBP를 기능 드라이버에 공급합니다.

DPIO 서비스
서비스용도
data availability callback수신 준비 알림
enqueue·dequeue하드웨어 큐 작업
buffer pool management버퍼 공급·회수
CPU별 DPIO동시 queue 처리

모든 DPAA2 기능 드라이버가 공유하는 fast-path 서비스입니다.

MAC proxy
PHYPHY driver·MDIOMAC driverDPMAC commandMC
interface up/downMC interruptDPMAC driverlink action

board별 PHY 드라이버와 MC 사이의 링크 상태 전달 경로입니다.

2. 영어 원문 전체

번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.

원문 전체 펼치기
1 .. include:: <isonum.txt>
2
3 =========================================================
4 DPAA2 (Data Path Acceleration Architecture Gen2) Overview
5 =========================================================
6
7 :Copyright: |copy| 2015 Freescale Semiconductor Inc.
8 :Copyright: |copy| 2018 NXP
9
10 This document provides an overview of the Freescale DPAA2 architecture
11 and how it is integrated into the Linux kernel.
12
13 Introduction
14 ============
15
16 DPAA2 is a hardware architecture designed for high-speeed network
17 packet processing. DPAA2 consists of sophisticated mechanisms for
18 processing Ethernet packets, queue management, buffer management,
19 autonomous L2 switching, virtual Ethernet bridging, and accelerator
20 (e.g. crypto) sharing.
21
22 A DPAA2 hardware component called the Management Complex (or MC) manages the
23 DPAA2 hardware resources. The MC provides an object-based abstraction for
24 software drivers to use the DPAA2 hardware.
25 The MC uses DPAA2 hardware resources such as queues, buffer pools, and
26 network ports to create functional objects/devices such as network
27 interfaces, an L2 switch, or accelerator instances.
28 The MC provides memory-mapped I/O command interfaces (MC portals)
29 which DPAA2 software drivers use to operate on DPAA2 objects.
30
31 The diagram below shows an overview of the DPAA2 resource management
32 architecture::
33
34 +--------------------------------------+
35 | OS |
36 | DPAA2 drivers |
37 | | |
38 +-----------------------------|--------+
39 |
40 | (create,discover,connect
41 | config,use,destroy)
42 |
43 DPAA2 |
44 +------------------------| mc portal |-+
45 | | |
46 | +- - - - - - - - - - - - -V- - -+ |
47 | | | |
48 | | Management Complex (MC) | |
49 | | | |
50 | +- - - - - - - - - - - - - - - -+ |
51 | |
52 | Hardware Hardware |
53 | Resources Objects |
54 | --------- ------- |
55 | -queues -DPRC |
56 | -buffer pools -DPMCP |
57 | -Eth MACs/ports -DPIO |
58 | -network interface -DPNI |
59 | profiles -DPMAC |
60 | -queue portals -DPBP |
61 | -MC portals ... |
62 | ... |
63 | |
64 +--------------------------------------+
65
66
67 The MC mediates operations such as create, discover,
68 connect, configuration, and destroy. Fast-path operations
69 on data, such as packet transmit/receive, are not mediated by
70 the MC and are done directly using memory mapped regions in
71 DPIO objects.
72
73 Overview of DPAA2 Objects
74 =========================
75
76 The section provides a brief overview of some key DPAA2 objects.
77 A simple scenario is described illustrating the objects involved
78 in creating a network interfaces.
79
80 DPRC (Datapath Resource Container)
81 ----------------------------------
82
83 A DPRC is a container object that holds all the other
84 types of DPAA2 objects. In the example diagram below there
85 are 8 objects of 5 types (DPMCP, DPIO, DPBP, DPNI, and DPMAC)
86 in the container.
87
88 ::
89
90 +---------------------------------------------------------+
91 | DPRC |
92 | |
93 | +-------+ +-------+ +-------+ +-------+ +-------+ |
94 | | DPMCP | | DPIO | | DPBP | | DPNI | | DPMAC | |
95 | +-------+ +-------+ +-------+ +---+---+ +---+---+ |
96 | | DPMCP | | DPIO | |
97 | +-------+ +-------+ |
98 | | DPMCP | |
99 | +-------+ |
100 | |
101 +---------------------------------------------------------+
102
103 From the point of view of an OS, a DPRC behaves similar to a plug and
104 play bus, like PCI. DPRC commands can be used to enumerate the contents
105 of the DPRC, discover the hardware objects present (including mappable
106 regions and interrupts).
107
108 ::
109
110 DPRC.1 (bus)
111 |
112 +--+--------+-------+-------+-------+
113 | | | | |
114 DPMCP.1 DPIO.1 DPBP.1 DPNI.1 DPMAC.1
115 DPMCP.2 DPIO.2
116 DPMCP.3
117
118 Hardware objects can be created and destroyed dynamically, providing
119 the ability to hot plug/unplug objects in and out of the DPRC.
120
121 A DPRC has a mappable MMIO region (an MC portal) that can be used
122 to send MC commands. It has an interrupt for status events (like
123 hotplug).
124 All objects in a container share the same hardware "isolation context".
125 This means that with respect to an IOMMU the isolation granularity
126 is at the DPRC (container) level, not at the individual object
127 level.
128
129 DPRCs can be defined statically and populated with objects
130 via a config file passed to the MC when firmware starts it.
131
132 DPAA2 Objects for an Ethernet Network Interface
133 -----------------------------------------------
134
135 A typical Ethernet NIC is monolithic-- the NIC device contains TX/RX
136 queuing mechanisms, configuration mechanisms, buffer management,
137 physical ports, and interrupts. DPAA2 uses a more granular approach
138 utilizing multiple hardware objects. Each object provides specialized
139 functions. Groups of these objects are used by software to provide
140 Ethernet network interface functionality. This approach provides
141 efficient use of finite hardware resources, flexibility, and
142 performance advantages.
143
144 The diagram below shows the objects needed for a simple
145 network interface configuration on a system with 2 CPUs.
146
147 ::
148
149 +---+---+ +---+---+
150 CPU0 CPU1
151 +---+---+ +---+---+
152 | |
153 +---+---+ +---+---+
154 DPIO DPIO
155 +---+---+ +---+---+
156 \ /
157 \ /
158 \ /
159 +---+---+
160 DPNI --- DPBP,DPMCP
161 +---+---+
162 |
163 |
164 +---+---+
165 DPMAC
166 +---+---+
167 |
168 port/PHY
169
170 Below the objects are described. For each object a brief description
171 is provided along with a summary of the kinds of operations the object
172 supports and a summary of key resources of the object (MMIO regions
173 and IRQs).
174
175 DPMAC (Datapath Ethernet MAC)
176 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
177 Represents an Ethernet MAC, a hardware device that connects to an Ethernet
178 PHY and allows physical transmission and reception of Ethernet frames.
179
180 - MMIO regions: none
181 - IRQs: DPNI link change
182 - commands: set link up/down, link config, get stats,
183 IRQ config, enable, reset
184
185 DPNI (Datapath Network Interface)
186 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
187 Contains TX/RX queues, network interface configuration, and RX buffer pool
188 configuration mechanisms. The TX/RX queues are in memory and are identified
189 by queue number.
190
191 - MMIO regions: none
192 - IRQs: link state
193 - commands: port config, offload config, queue config,
194 parse/classify config, IRQ config, enable, reset
195
196 DPIO (Datapath I/O)
197 ~~~~~~~~~~~~~~~~~~~
198 Provides interfaces to enqueue and dequeue
199 packets and do hardware buffer pool management operations. The DPAA2
200 architecture separates the mechanism to access queues (the DPIO object)
201 from the queues themselves. The DPIO provides an MMIO interface to
202 enqueue/dequeue packets. To enqueue something a descriptor is written
203 to the DPIO MMIO region, which includes the target queue number.
204 There will typically be one DPIO assigned to each CPU. This allows all
205 CPUs to simultaneously perform enqueue/dequeued operations. DPIOs are
206 expected to be shared by different DPAA2 drivers.
207
208 - MMIO regions: queue operations, buffer management
209 - IRQs: data availability, congestion notification, buffer
210 pool depletion
211 - commands: IRQ config, enable, reset
212
213 DPBP (Datapath Buffer Pool)
214 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
215 Represents a hardware buffer pool.
216
217 - MMIO regions: none
218 - IRQs: none
219 - commands: enable, reset
220
221 DPMCP (Datapath MC Portal)
222 ~~~~~~~~~~~~~~~~~~~~~~~~~~
223 Provides an MC command portal.
224 Used by drivers to send commands to the MC to manage
225 objects.
226
227 - MMIO regions: MC command portal
228 - IRQs: command completion
229 - commands: IRQ config, enable, reset
230
231 Object Connections
232 ==================
233 Some objects have explicit relationships that must
234 be configured:
235
236 - DPNI <--> DPMAC
237 - DPNI <--> DPNI
238 - DPNI <--> L2-switch-port
239
240 A DPNI must be connected to something such as a DPMAC,
241 another DPNI, or L2 switch port. The DPNI connection
242 is made via a DPRC command.
243
244 ::
245
246 +-------+ +-------+
247 | DPNI | | DPMAC |
248 +---+---+ +---+---+
249 | |
250 +==========+
251
252 - DPNI <--> DPBP
253
254 A network interface requires a 'buffer pool' (DPBP
255 object) which provides a list of pointers to memory
256 where received Ethernet data is to be copied. The
257 Ethernet driver configures the DPBPs associated with
258 the network interface.
259
260 Interrupts
261 ==========
262 All interrupts generated by DPAA2 objects are message
263 interrupts. At the hardware level message interrupts
264 generated by devices will normally have 3 components--
265 1) a non-spoofable 'device-id' expressed on the hardware
266 bus, 2) an address, 3) a data value.
267
268 In the case of DPAA2 devices/objects, all objects in the
269 same container/DPRC share the same 'device-id'.
270 For ARM-based SoC this is the same as the stream ID.
271
272
273 DPAA2 Linux Drivers Overview
274 ============================
275
276 This section provides an overview of the Linux kernel drivers for
277 DPAA2-- 1) the bus driver and associated "DPAA2 infrastructure"
278 drivers and 2) functional object drivers (such as Ethernet).
279
280 As described previously, a DPRC is a container that holds the other
281 types of DPAA2 objects. It is functionally similar to a plug-and-play
282 bus controller.
283 Each object in the DPRC is a Linux "device" and is bound to a driver.
284 The diagram below shows the Linux drivers involved in a networking
285 scenario and the objects bound to each driver. A brief description
286 of each driver follows.
287
288 ::
289
290 +------------+
291 | OS Network |
292 | Stack |
293 +------------+ +------------+
294 | Allocator |. . . . . . . | Ethernet |
295 |(DPMCP,DPBP)| | (DPNI) |
296 +-.----------+ +---+---+----+
297 . . ^ |
298 . . <data avail, | | <enqueue,
299 . . tx confirm> | | dequeue>
300 +-------------+ . | |
301 | DPRC driver | . +---+---V----+ +---------+
302 | (DPRC) | . . . . . .| DPIO driver| | MAC |
303 +----------+--+ | (DPIO) | | (DPMAC) |
304 | +------+-----+ +-----+---+
305 |<dev add/remove> | |
306 | | |
307 +--------+----------+ | +--+---+
308 | MC-bus driver | | | PHY |
309 | | | |driver|
310 | /bus/fsl-mc | | +--+---+
311 +-------------------+ | |
312 | |
313 ========================= HARDWARE =========|=================|======
314 DPIO |
315 | |
316 DPNI---DPBP |
317 | |
318 DPMAC |
319 | |
320 PHY ---------------+
321 ============================================|========================
322
323 A brief description of each driver is provided below.
324
325 MC-bus driver
326 -------------
327 The MC-bus driver is a platform driver and is probed from a
328 node in the device tree (compatible "fsl,qoriq-mc") passed in by boot
329 firmware. It is responsible for bootstrapping the DPAA2 kernel
330 infrastructure.
331 Key functions include:
332
333 - registering a new bus type named "fsl-mc" with the kernel,
334 and implementing bus call-backs (e.g. match/uevent/dev_groups)
335 - implementing APIs for DPAA2 driver registration and for device
336 add/remove
337 - creates an MSI IRQ domain
338 - doing a 'device add' to expose the 'root' DPRC, in turn triggering
339 a bind of the root DPRC to the DPRC driver
340
341 The binding for the MC-bus device-tree node can be consulted at
342 *Documentation/devicetree/bindings/misc/fsl,qoriq-mc.yaml*.
343 The sysfs bind/unbind interfaces for the MC-bus can be consulted at
344 *Documentation/ABI/testing/sysfs-bus-fsl-mc*.
345
346 DPRC driver
347 -----------
348 The DPRC driver is bound to DPRC objects and does runtime management
349 of a bus instance. It performs the initial bus scan of the DPRC
350 and handles interrupts for container events such as hot plug by
351 re-scanning the DPRC.
352
353 Allocator
354 ---------
355 Certain objects such as DPMCP and DPBP are generic and fungible,
356 and are intended to be used by other drivers. For example,
357 the DPAA2 Ethernet driver needs:
358
359 - DPMCPs to send MC commands, to configure network interfaces
360 - DPBPs for network buffer pools
361
362 The allocator driver registers for these allocatable object types
363 and those objects are bound to the allocator when the bus is probed.
364 The allocator maintains a pool of objects that are available for
365 allocation by other DPAA2 drivers.
366
367 DPIO driver
368 -----------
369 The DPIO driver is bound to DPIO objects and provides services that allow
370 other drivers such as the Ethernet driver to enqueue and dequeue data for
371 their respective objects.
372 Key services include:
373
374 - data availability notifications
375 - hardware queuing operations (enqueue and dequeue of data)
376 - hardware buffer pool management
377
378 To transmit a packet the Ethernet driver puts data on a queue and
379 invokes a DPIO API. For receive, the Ethernet driver registers
380 a data availability notification callback. To dequeue a packet
381 a DPIO API is used.
382 There is typically one DPIO object per physical CPU for optimum
383 performance, allowing different CPUs to simultaneously enqueue
384 and dequeue data.
385
386 The DPIO driver operates on behalf of all DPAA2 drivers
387 active in the kernel-- Ethernet, crypto, compression,
388 etc.
389
390 Ethernet driver
391 ---------------
392 The Ethernet driver is bound to a DPNI and implements the kernel
393 interfaces needed to connect the DPAA2 network interface to
394 the network stack.
395 Each DPNI corresponds to a Linux network interface.
396
397 MAC driver
398 ----------
399 An Ethernet PHY is an off-chip, board specific component and is managed
400 by the appropriate PHY driver via an mdio bus. The MAC driver
401 plays a role of being a proxy between the PHY driver and the
402 MC. It does this proxy via the MC commands to a DPMAC object.
403 If the PHY driver signals a link change, the MAC driver notifies
404 the MC via a DPMAC command. If a network interface is brought
405 up or down, the MC notifies the DPMAC driver via an interrupt and
406 the driver can take appropriate action.
407

3. 한국어 전문 번역

영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.

DPAA2 소개와 Management Complex

1-72

이 문서는 `isonum.txt`를 포함해 저작권 기호를 사용하며 저작권은 © 2015 Freescale Semiconductor Inc. 및 © 2018 NXP에 있습니다.

DPAA2(Data Path Acceleration Architecture Gen2) 개요

이 문서는 Freescale DPAA2 아키텍처와 Linux 커널 통합 방식을 개괄합니다.

소개

DPAA2는 고속 네트워크 패킷 처리를 위해 설계한 하드웨어 아키텍처입니다. Ethernet 패킷 처리, 큐 관리, 버퍼 관리, 자율 L2 switching, 가상 Ethernet bridging, crypto 같은 accelerator 공유를 위한 정교한 메커니즘으로 구성됩니다.

Management Complex(`MC`)라는 DPAA2 하드웨어 구성 요소가 DPAA2 하드웨어 자원을 관리합니다. MC는 소프트웨어 드라이버가 하드웨어를 사용할 수 있도록 객체 기반 추상화를 제공합니다.

MC는 큐, 버퍼 풀, 네트워크 port 같은 자원을 조합해 네트워크 인터페이스, L2 switch, accelerator instance 같은 기능 객체와 장치를 만듭니다.

DPAA2 소프트웨어 드라이버는 memory-mapped I/O 명령 인터페이스인 MC portal을 통해 DPAA2 객체를 조작합니다.

DPAA2 자원 관리 아키텍처

MC 자원 관리 구조
OS·DPAA2 driversMC portalManagement Complexcreate·discover·connect·configure·use·destroyDPAA2 objects
queues·buffer pools·Ethernet MAC/portsManagement ComplexDPRC·DPMCP·DPIO·DPNI·DPMAC·DPBP

원문의 첫 번째 ASCII 도식을 OS, MC portal, Management Complex, 자원과 객체 계층으로 다시 구성했습니다.

MC는 객체의 생성, 발견, 연결, 구성, 파괴 같은 제어 작업을 중재합니다.

패킷 송수신 같은 데이터 fast path 작업은 MC를 거치지 않습니다. DPIO 객체의 memory-mapped 영역을 직접 사용합니다.

.. include:: <isonum.txt>

=========================================================
DPAA2 (Data Path Acceleration Architecture Gen2) Overview
=========================================================

:Copyright: |copy| 2015 Freescale Semiconductor Inc.
:Copyright: |copy| 2018 NXP

This document provides an overview of the Freescale DPAA2 architecture
and how it is integrated into the Linux kernel.

Introduction
============

DPAA2 is a hardware architecture designed for high-speeed network
packet processing.  DPAA2 consists of sophisticated mechanisms for
processing Ethernet packets, queue management, buffer management,
autonomous L2 switching, virtual Ethernet bridging, and accelerator
(e.g. crypto) sharing.

A DPAA2 hardware component called the Management Complex (or MC) manages the
DPAA2 hardware resources.  The MC provides an object-based abstraction for
software drivers to use the DPAA2 hardware.
The MC uses DPAA2 hardware resources such as queues, buffer pools, and
network ports to create functional objects/devices such as network
interfaces, an L2 switch, or accelerator instances.
The MC provides memory-mapped I/O command interfaces (MC portals)
which DPAA2 software drivers use to operate on DPAA2 objects.

The diagram below shows an overview of the DPAA2 resource management
architecture::

        +--------------------------------------+
        |                  OS                  |
        |                        DPAA2 drivers |
        |                             |        |
        +-----------------------------|--------+
                                      |
                                      | (create,discover,connect
                                      |  config,use,destroy)
                                      |
                         DPAA2        |
        +------------------------| mc portal |-+
        |                             |        |
        |   +- - - - - - - - - - - - -V- - -+  |
        |   |                               |  |
        |   |   Management Complex (MC)     |  |
        |   |                               |  |
        |   +- - - - - - - - - - - - - - - -+  |
        |                                      |
        | Hardware                  Hardware   |
        | Resources                 Objects    |
        | ---------                 -------    |
        | -queues                   -DPRC      |
        | -buffer pools             -DPMCP     |
        | -Eth MACs/ports           -DPIO      |
        | -network interface        -DPNI      |
        |  profiles                 -DPMAC     |
        | -queue portals            -DPBP      |
        | -MC portals                ...       |
        |  ...                                 |
        |                                      |
        +--------------------------------------+


The MC mediates operations such as create, discover,
connect, configuration, and destroy.  Fast-path operations
on data, such as packet transmit/receive, are not mediated by
the MC and are done directly using memory mapped regions in
DPIO objects.

DPRC 컨테이너, bus 모델과 격리

73-131

DPAA2 객체 개요

이 절은 핵심 DPAA2 객체를 간략히 설명하고, 네트워크 인터페이스를 만드는 데 참여하는 객체를 단순한 시나리오로 보여 줍니다.

DPRC(Datapath Resource Container)

DPRC는 다른 모든 유형의 DPAA2 객체를 담는 container 객체입니다. 예제 container에는 DPMCP, DPIO, DPBP, DPNI, DPMAC의 다섯 유형 객체가 모두 여덟 개 있습니다.

DPRC 객체 구성
DPRCDPMCP.1·DPMCP.2·DPMCP.3DPIO.1·DPIO.2DPBP.1DPNI.1DPMAC.1

원문의 container ASCII 도식을 객체 수와 유형 관계로 다시 구성했습니다.

OS 관점에서 DPRC는 PCI 같은 plug-and-play bus처럼 동작합니다. DPRC 명령으로 container 내용을 열거하고, 매핑 가능한 영역과 인터럽트를 포함해 존재하는 하드웨어 객체를 발견할 수 있습니다.

DPRC bus 트리
DPRC.1 busDPMCP.1·2·3DPIO.1·2DPBP.1DPNI.1DPMAC.1

원문의 두 번째 DPRC ASCII 도식을 Linux bus 관점으로 다시 구성했습니다.

하드웨어 객체를 동적으로 만들고 파괴할 수 있으므로 DPRC 안팎으로 객체를 hot plug하거나 hot unplug할 수 있습니다.

DPRC에는 MC 명령을 보내는 데 사용할 수 있는 매핑 가능한 MMIO 영역, 즉 MC portal이 있습니다. hotplug 같은 상태 event를 알리는 인터럽트도 가집니다.

한 container의 모든 객체는 같은 하드웨어 isolation context를 공유합니다. 따라서 IOMMU 격리 단위는 개별 객체가 아니라 DPRC container 수준입니다.

DPRC는 정적으로 정의할 수 있으며 firmware가 MC를 시작할 때 전달하는 구성 파일로 객체를 채울 수 있습니다.

Overview of DPAA2 Objects
=========================

The section provides a brief overview of some key DPAA2 objects.
A simple scenario is described illustrating the objects involved
in creating a network interfaces.

DPRC (Datapath Resource Container)
----------------------------------

A DPRC is a container object that holds all the other
types of DPAA2 objects.  In the example diagram below there
are 8 objects of 5 types (DPMCP, DPIO, DPBP, DPNI, and DPMAC)
in the container.

::

        +---------------------------------------------------------+
        | DPRC                                                    |
        |                                                         |
        |  +-------+  +-------+  +-------+  +-------+  +-------+  |
        |  | DPMCP |  | DPIO  |  | DPBP  |  | DPNI  |  | DPMAC |  |
        |  +-------+  +-------+  +-------+  +---+---+  +---+---+  |
        |  | DPMCP |  | DPIO  |                                   |
        |  +-------+  +-------+                                   |
        |  | DPMCP |                                              |
        |  +-------+                                              |
        |                                                         |
        +---------------------------------------------------------+

From the point of view of an OS, a DPRC behaves similar to a plug and
play bus, like PCI.  DPRC commands can be used to enumerate the contents
of the DPRC, discover the hardware objects present (including mappable
regions and interrupts).

::

        DPRC.1 (bus)
           |
           +--+--------+-------+-------+-------+
              |        |       |       |       |
            DPMCP.1  DPIO.1  DPBP.1  DPNI.1  DPMAC.1
            DPMCP.2  DPIO.2
            DPMCP.3

Hardware objects can be created and destroyed dynamically, providing
the ability to hot plug/unplug objects in and out of the DPRC.

A DPRC has a mappable MMIO region (an MC portal) that can be used
to send MC commands.  It has an interrupt for status events (like
hotplug).
All objects in a container share the same hardware "isolation context".
This means that with respect to an IOMMU the isolation granularity
is at the DPRC (container) level, not at the individual object
level.

DPRCs can be defined statically and populated with objects
via a config file passed to the MC when firmware starts it.

Ethernet 인터페이스용 객체 구성

132-174

Ethernet 네트워크 인터페이스용 DPAA2 객체

일반 Ethernet NIC는 단일 장치 안에 TX/RX 큐, 구성 메커니즘, 버퍼 관리, 물리 port, 인터럽트를 모두 포함하는 일체형 구조입니다.

DPAA2는 여러 하드웨어 객체를 사용하는 더 세분화된 방식을 택합니다. 각 객체가 특화 기능을 제공하고, 소프트웨어가 객체 그룹을 조합해 Ethernet 인터페이스 기능을 만듭니다.

이 방식은 한정된 하드웨어 자원을 효율적으로 사용하고 유연성과 성능상의 이점을 제공합니다.

다음은 CPU 두 개가 있는 시스템에서 단순한 네트워크 인터페이스를 구성하는 데 필요한 객체입니다.

2 CPU Ethernet 객체 토폴로지
CPU0DPIO0DPNIDPMACport·PHY
CPU1DPIO1DPNIDPMACport·PHY
DPBPDPNI
DPMCPDPNI

원문의 네 번째 ASCII 도식을 CPU별 DPIO와 공유 DPNI·DPBP·DPMCP·DPMAC 연결로 다시 구성했습니다.

이하에서는 각 객체의 간단한 설명, 지원 연산 종류, 핵심 자원인 MMIO 영역과 IRQ를 요약합니다.

DPAA2 Objects for an Ethernet Network Interface
-----------------------------------------------

A typical Ethernet NIC is monolithic-- the NIC device contains TX/RX
queuing mechanisms, configuration mechanisms, buffer management,
physical ports, and interrupts.  DPAA2 uses a more granular approach
utilizing multiple hardware objects.  Each object provides specialized
functions. Groups of these objects are used by software to provide
Ethernet network interface functionality.  This approach provides
efficient use of finite hardware resources, flexibility, and
performance advantages.

The diagram below shows the objects needed for a simple
network interface configuration on a system with 2 CPUs.

::

        +---+---+ +---+---+
           CPU0     CPU1
        +---+---+ +---+---+
            |         |
        +---+---+ +---+---+
           DPIO     DPIO
        +---+---+ +---+---+
            \     /
             \   /
              \ /
           +---+---+
              DPNI  --- DPBP,DPMCP
           +---+---+
               |
               |
           +---+---+
             DPMAC
           +---+---+
               |
           port/PHY

Below the objects are described.  For each object a brief description
is provided along with a summary of the kinds of operations the object
supports and a summary of key resources of the object (MMIO regions
and IRQs).

DPMAC·DPNI·DPIO·DPBP·DPMCP 세부 사항

175-230

DPMAC(Datapath Ethernet MAC)

Ethernet PHY에 연결되어 Ethernet frame을 물리적으로 송수신하는 하드웨어 장치인 Ethernet MAC을 나타냅니다.

  • MMIO 영역: 없음
  • IRQ: DPNI link change
  • 명령: link up/down, link 구성, 통계 조회, IRQ 구성, enable, reset

DPNI(Datapath Network Interface)

TX/RX 큐, 네트워크 인터페이스 구성, RX 버퍼 풀 구성 메커니즘을 포함합니다. TX/RX 큐는 memory에 있고 queue number로 식별합니다.

  • MMIO 영역: 없음
  • IRQ: link state
  • 명령: port 구성, offload 구성, queue 구성, parse/classify 구성, IRQ 구성, enable, reset

DPIO(Datapath I/O)

패킷을 enqueue/dequeue하고 하드웨어 버퍼 풀 관리 작업을 수행하는 인터페이스를 제공합니다.

DPAA2는 큐에 접근하는 메커니즘인 DPIO 객체와 큐 자체를 분리합니다. DPIO는 packet enqueue/dequeue용 MMIO 인터페이스를 제공합니다.

enqueue하려면 목표 queue number를 포함한 descriptor를 DPIO MMIO 영역에 기록합니다.

일반적으로 CPU마다 DPIO 하나를 할당해 모든 CPU가 enqueue/dequeue 작업을 동시에 수행하게 합니다. DPIO는 서로 다른 DPAA2 드라이버가 공유할 것으로 예상합니다.

  • MMIO 영역: queue 작업, buffer 관리
  • IRQ: 데이터 가용성, congestion 알림, buffer pool depletion
  • 명령: IRQ 구성, enable, reset

DPBP(Datapath Buffer Pool)

하드웨어 버퍼 풀을 나타냅니다.

  • MMIO 영역: 없음
  • IRQ: 없음
  • 명령: enable, reset

DPMCP(Datapath MC Portal)

드라이버가 MC에 명령을 보내 객체를 관리할 때 사용하는 MC command portal을 제공합니다.

  • MMIO 영역: MC command portal
  • IRQ: command completion
  • 명령: IRQ 구성, enable, reset
DPMAC (Datapath Ethernet MAC)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Represents an Ethernet MAC, a hardware device that connects to an Ethernet
PHY and allows physical transmission and reception of Ethernet frames.

- MMIO regions: none
- IRQs: DPNI link change
- commands: set link up/down, link config, get stats,
  IRQ config, enable, reset

DPNI (Datapath Network Interface)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Contains TX/RX queues, network interface configuration, and RX buffer pool
configuration mechanisms.  The TX/RX queues are in memory and are identified
by queue number.

- MMIO regions: none
- IRQs: link state
- commands: port config, offload config, queue config,
  parse/classify config, IRQ config, enable, reset

DPIO (Datapath I/O)
~~~~~~~~~~~~~~~~~~~
Provides interfaces to enqueue and dequeue
packets and do hardware buffer pool management operations.  The DPAA2
architecture separates the mechanism to access queues (the DPIO object)
from the queues themselves.  The DPIO provides an MMIO interface to
enqueue/dequeue packets.  To enqueue something a descriptor is written
to the DPIO MMIO region, which includes the target queue number.
There will typically be one DPIO assigned to each CPU.  This allows all
CPUs to simultaneously perform enqueue/dequeued operations.  DPIOs are
expected to be shared by different DPAA2 drivers.

- MMIO regions: queue operations, buffer management
- IRQs: data availability, congestion notification, buffer
  pool depletion
- commands: IRQ config, enable, reset

DPBP (Datapath Buffer Pool)
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Represents a hardware buffer pool.

- MMIO regions: none
- IRQs: none
- commands: enable, reset

DPMCP (Datapath MC Portal)
~~~~~~~~~~~~~~~~~~~~~~~~~~
Provides an MC command portal.
Used by drivers to send commands to the MC to manage
objects.

- MMIO regions: MC command portal
- IRQs: command completion
- commands: IRQ config, enable, reset

객체 연결과 message interrupt

231-272

객체 연결

일부 객체에는 명시적으로 구성해야 하는 관계가 있습니다.

  • DPNI ↔ DPMAC
  • DPNI ↔ DPNI
  • DPNI ↔ L2 switch port

DPNI는 DPMAC, 다른 DPNI, L2 switch port 중 하나에 연결해야 합니다. DPNI 연결은 DPRC 명령으로 만듭니다.

DPNI-DPMAC 연결
DPNIDPRC connection commandDPMAC

원문의 다섯 번째 ASCII 도식을 객체 endpoint 연결로 다시 구성했습니다.

DPNI ↔ DPBP

네트워크 인터페이스에는 수신 Ethernet 데이터를 복사할 memory 위치의 pointer 목록을 제공하는 버퍼 풀, 즉 DPBP 객체가 필요합니다.

Ethernet 드라이버는 네트워크 인터페이스에 연결된 DPBP를 구성합니다.

인터럽트

DPAA2 객체가 생성하는 모든 인터럽트는 message interrupt입니다. 하드웨어 수준의 message interrupt는 일반적으로 세 요소로 구성됩니다.

  • 하드웨어 bus에 표현되는 위조 불가능한 `device-id`
  • address
  • data value

같은 container/DPRC의 모든 DPAA2 객체는 같은 `device-id`를 공유합니다. ARM 기반 SoC에서는 이 값이 stream ID와 같습니다.

Object Connections
==================
Some objects have explicit relationships that must
be configured:

- DPNI <--> DPMAC
- DPNI <--> DPNI
- DPNI <--> L2-switch-port

    A DPNI must be connected to something such as a DPMAC,
    another DPNI, or L2 switch port.  The DPNI connection
    is made via a DPRC command.

::

              +-------+  +-------+
              | DPNI  |  | DPMAC |
              +---+---+  +---+---+
                  |          |
                  +==========+

- DPNI <--> DPBP

    A network interface requires a 'buffer pool' (DPBP
    object) which provides a list of pointers to memory
    where received Ethernet data is to be copied.  The
    Ethernet driver configures the DPBPs associated with
    the network interface.

Interrupts
==========
All interrupts generated by DPAA2 objects are message
interrupts.  At the hardware level message interrupts
generated by devices will normally have 3 components--
1) a non-spoofable 'device-id' expressed on the hardware
bus, 2) an address, 3) a data value.

In the case of DPAA2 devices/objects, all objects in the
same container/DPRC share the same 'device-id'.
For ARM-based SoC this is the same as the stream ID.

DPAA2 Linux 드라이버 전체 구조

273-324

DPAA2 Linux 드라이버 개요

Linux 커널의 DPAA2 드라이버는 크게 두 그룹입니다. 첫째는 bus 드라이버와 관련 DPAA2 infrastructure 드라이버이고, 둘째는 Ethernet 같은 기능 객체 드라이버입니다.

DPRC는 다른 DPAA2 객체를 담는 container이며 기능적으로 plug-and-play bus controller와 비슷합니다. DPRC 안의 각 객체는 Linux `device`이고 대응 드라이버에 bind됩니다.

다음 구조는 네트워킹 시나리오에 참여하는 Linux 드라이버와 각 드라이버에 bind된 객체를 보여 줍니다.

DPAA2 Linux 네트워킹 스택
`/bus/fsl-mc` MC-bus driver`dev add/remove`DPRC driverAllocator `(DPMCP, DPBP)`
OS Network StackEthernet driver `(DPNI)`DPIO driverDPIO hardwareDPNI·DPBP
Ethernet driverenqueue·dequeue·data available·Tx confirmationDPIO driver
MAC driver `(DPMAC)`PHY driverPHYDPMACDPNI

원문의 여섯 번째 ASCII 도식을 bus, allocator, DPIO, Ethernet, MAC·PHY 경로로 다시 구성했습니다.

이하에서는 각 드라이버를 간략히 설명합니다.

DPAA2 Linux Drivers Overview
============================

This section provides an overview of the Linux kernel drivers for
DPAA2-- 1) the bus driver and associated "DPAA2 infrastructure"
drivers and 2) functional object drivers (such as Ethernet).

As described previously, a DPRC is a container that holds the other
types of DPAA2 objects.  It is functionally similar to a plug-and-play
bus controller.
Each object in the DPRC is a Linux "device" and is bound to a driver.
The diagram below shows the Linux drivers involved in a networking
scenario and the objects bound to each driver.  A brief description
of each driver follows.

::

                                             +------------+
                                             | OS Network |
                                             |   Stack    |
                 +------------+              +------------+
                 | Allocator  |. . . . . . . |  Ethernet  |
                 |(DPMCP,DPBP)|              |   (DPNI)   |
                 +-.----------+              +---+---+----+
                  .          .                   ^   |
                 .            .     <data avail, |   | <enqueue,
                .              .     tx confirm> |   | dequeue>
        +-------------+         .                |   |
        | DPRC driver |          .           +---+---V----+     +---------+
        |   (DPRC)    |           . . . . . .| DPIO driver|     |   MAC   |
        +----------+--+                      |  (DPIO)    |     | (DPMAC) |
                   |                         +------+-----+     +-----+---+
                   |<dev add/remove>                |                 |
                   |                                |                 |
          +--------+----------+                     |              +--+---+
          |   MC-bus driver   |                     |              | PHY  |
          |                   |                     |              |driver|
          |   /bus/fsl-mc     |                     |              +--+---+
          +-------------------+                     |                 |
                                                    |                 |
        ========================= HARDWARE =========|=================|======
                                                  DPIO                |
                                                    |                 |
                                                  DPNI---DPBP         |
                                                    |                 |
                                                  DPMAC               |
                                                    |                 |
                                                   PHY ---------------+
        ============================================|========================

A brief description of each driver is provided below.

MC-bus·DPRC·Allocator 드라이버

325-366

MC-bus 드라이버

MC-bus 드라이버는 platform driver입니다. boot firmware가 전달한 device tree의 `compatible = "fsl,qoriq-mc"` node에서 probe되며 DPAA2 커널 infrastructure를 bootstrap합니다.

주요 기능은 다음과 같습니다.

  • 커널에 `fsl-mc`라는 새 bus type을 등록하고 `match`, `uevent`, `dev_groups` 같은 bus callback 구현
  • DPAA2 드라이버 등록 및 device add/remove API 구현
  • MSI IRQ domain 생성
  • `device add`로 root DPRC를 노출해 root DPRC가 DPRC 드라이버에 bind되도록 유도

MC-bus device-tree node binding은 `Documentation/devicetree/bindings/misc/fsl,qoriq-mc.yaml`에서 확인할 수 있습니다.

MC-bus의 sysfs bind/unbind 인터페이스는 `Documentation/ABI/testing/sysfs-bus-fsl-mc`에서 확인할 수 있습니다.

DPRC 드라이버

DPRC 드라이버는 DPRC 객체에 bind되어 bus instance를 실행 중 관리합니다. DPRC의 초기 bus scan을 수행하고 hot plug 같은 container event 인터럽트를 처리하며 DPRC를 다시 scan합니다.

Allocator

DPMCP와 DPBP 같은 일부 객체는 범용이고 서로 대체 가능하며 다른 드라이버가 사용하도록 설계되었습니다.

예를 들어 DPAA2 Ethernet 드라이버에는 다음 객체가 필요합니다.

  • 네트워크 인터페이스를 구성하는 MC 명령 전송용 DPMCP
  • 네트워크 버퍼 풀용 DPBP

allocator 드라이버는 이러한 할당 가능 객체 유형에 등록됩니다. bus를 probe하면 객체가 allocator에 bind됩니다.

allocator는 다른 DPAA2 드라이버가 할당할 수 있는 객체 pool을 유지합니다.

MC-bus driver
-------------
The MC-bus driver is a platform driver and is probed from a
node in the device tree (compatible "fsl,qoriq-mc") passed in by boot
firmware.  It is responsible for bootstrapping the DPAA2 kernel
infrastructure.
Key functions include:

- registering a new bus type named "fsl-mc" with the kernel,
  and implementing bus call-backs (e.g. match/uevent/dev_groups)
- implementing APIs for DPAA2 driver registration and for device
  add/remove
- creates an MSI IRQ domain
- doing a 'device add' to expose the 'root' DPRC, in turn triggering
  a bind of the root DPRC to the DPRC driver

The binding for the MC-bus device-tree node can be consulted at
*Documentation/devicetree/bindings/misc/fsl,qoriq-mc.yaml*.
The sysfs bind/unbind interfaces for the MC-bus can be consulted at
*Documentation/ABI/testing/sysfs-bus-fsl-mc*.

DPRC driver
-----------
The DPRC driver is bound to DPRC objects and does runtime management
of a bus instance.  It performs the initial bus scan of the DPRC
and handles interrupts for container events such as hot plug by
re-scanning the DPRC.

Allocator
---------
Certain objects such as DPMCP and DPBP are generic and fungible,
and are intended to be used by other drivers.  For example,
the DPAA2 Ethernet driver needs:

- DPMCPs to send MC commands, to configure network interfaces
- DPBPs for network buffer pools

The allocator driver registers for these allocatable object types
and those objects are bound to the allocator when the bus is probed.
The allocator maintains a pool of objects that are available for
allocation by other DPAA2 drivers.

DPIO·Ethernet·MAC 드라이버

367-406

DPIO 드라이버

DPIO 드라이버는 DPIO 객체에 bind되며 Ethernet 드라이버 같은 다른 드라이버가 각자의 객체에 데이터를 enqueue하고 dequeue하도록 서비스를 제공합니다.

핵심 서비스는 다음과 같습니다.

  • 데이터 가용성 알림
  • 하드웨어 큐 작업, 즉 데이터 enqueue와 dequeue
  • 하드웨어 버퍼 풀 관리

패킷을 송신할 때 Ethernet 드라이버는 데이터를 큐에 넣고 DPIO API를 호출합니다. 수신을 위해서는 데이터 가용성 알림 callback을 등록하고, 패킷을 dequeue할 때도 DPIO API를 사용합니다.

최적 성능을 위해 일반적으로 physical CPU마다 DPIO 객체 하나를 둡니다. 서로 다른 CPU가 동시에 데이터를 enqueue/dequeue할 수 있습니다.

DPIO 드라이버는 커널에서 활성화된 Ethernet, crypto, compression 등 모든 DPAA2 드라이버를 대신해 동작합니다.

Ethernet 드라이버

Ethernet 드라이버는 DPNI에 bind되고 DPAA2 네트워크 인터페이스를 network stack에 연결하는 데 필요한 커널 인터페이스를 구현합니다. DPNI 하나가 Linux 네트워크 인터페이스 하나에 대응합니다.

MAC 드라이버

Ethernet PHY는 board별 off-chip 구성 요소이며 적절한 PHY 드라이버가 MDIO bus를 통해 관리합니다.

MAC 드라이버는 PHY 드라이버와 MC 사이의 proxy 역할을 하며 DPMAC 객체에 대한 MC 명령으로 이를 수행합니다.

PHY 드라이버가 link change를 알리면 MAC 드라이버가 DPMAC 명령으로 MC에 통지합니다.

네트워크 인터페이스를 up 또는 down하면 MC가 interrupt로 DPMAC 드라이버에 알리고 드라이버가 적절히 처리합니다.

DPIO driver
-----------
The DPIO driver is bound to DPIO objects and provides services that allow
other drivers such as the Ethernet driver to enqueue and dequeue data for
their respective objects.
Key services include:

- data availability notifications
- hardware queuing operations (enqueue and dequeue of data)
- hardware buffer pool management

To transmit a packet the Ethernet driver puts data on a queue and
invokes a DPIO API.  For receive, the Ethernet driver registers
a data availability notification callback.  To dequeue a packet
a DPIO API is used.
There is typically one DPIO object per physical CPU for optimum
performance, allowing different CPUs to simultaneously enqueue
and dequeue data.

The DPIO driver operates on behalf of all DPAA2 drivers
active in the kernel--  Ethernet, crypto, compression,
etc.

Ethernet driver
---------------
The Ethernet driver is bound to a DPNI and implements the kernel
interfaces needed to connect the DPAA2 network interface to
the network stack.
Each DPNI corresponds to a Linux network interface.

MAC driver
----------
An Ethernet PHY is an off-chip, board specific component and is managed
by the appropriate PHY driver via an mdio bus.  The MAC driver
plays a role of being a proxy between the PHY driver and the
MC.  It does this proxy via the MC commands to a DPMAC object.
If the PHY driver signals a link change, the MAC driver notifies
the MC via a DPMAC command.  If a network interface is brought
up or down, the MC notifies the DPMAC driver via an interrupt and
the driver can take appropriate action.