요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
========================================================
OpenCAPI (Open Coherent Accelerator Processor Interface)
========================================================
OpenCAPI is an interface between processors and accelerators. It aims
at being low-latency and high-bandwidth.
The specification was developed by the OpenCAPI Consortium, and is now
available from the `Compute Express Link Consortium
<https://computeexpresslink.org/resource/opencapi-specification-archive/>`_.
It allows an accelerator (which could be an FPGA, ASICs, ...) to access
the host memory coherently, using virtual addresses. An OpenCAPI
device can also host its own memory, that can be accessed from the
host.
OpenCAPI is known in linux as 'ocxl', as the open, processor-agnostic
evolution of 'cxl' (the driver for the IBM CAPI interface for
powerpc), which was named that way to avoid confusion with the ISDN
CAPI subsystem.
High-level view
===============
OpenCAPI defines a Data Link Layer (DL) and Transaction Layer (TL), to
be implemented on top of a physical link. Any processor or device
implementing the DL and TL can start sharing memory.
::
+-----------+ +-------------+
| | | |
| | | Accelerated |
| Processor | | Function |
| | +--------+ | Unit | +--------+
| |--| Memory | | (AFU) |--| Memory |
| | +--------+ | | +--------+
+-----------+ +-------------+
| |
+-----------+ +-------------+
| TL | | TLX |
+-----------+ +-------------+
| |
+-----------+ +-------------+
| DL | | DLX |
+-----------+ +-------------+
| |
| PHY |
+---------------------------------------+
Device discovery
================
OpenCAPI relies on a PCI-like configuration space, implemented on the
device. So the host can discover AFUs by querying the config space.
OpenCAPI devices in Linux are treated like PCI devices (with a few
caveats). The firmware is expected to abstract the hardware as if it
was a PCI link. A lot of the existing PCI infrastructure is reused:
devices are scanned and BARs are assigned during the standard PCI
enumeration. Commands like 'lspci' can therefore be used to see what
devices are available.
The configuration space defines the AFU(s) that can be found on the
physical adapter, such as its name, how many memory contexts it can
work with, the size of its MMIO areas, ...
MMIO
====
OpenCAPI defines two MMIO areas for each AFU:
* the global MMIO area, with registers pertinent to the whole AFU.
* a per-process MMIO area, which has a fixed size for each context.
AFU interrupts
==============
OpenCAPI includes the possibility for an AFU to send an interrupt to a
host process. It is done through a 'intrp_req' defined in the
Transaction Layer, specifying a 64-bit object handle which defines the
interrupt.
The driver allows a process to allocate an interrupt and obtain its
64-bit object handle, that can be passed to the AFU.
char devices
============
The driver creates one char device per AFU found on the physical
device. A physical device may have multiple functions and each
function can have multiple AFUs. At the time of this writing though,
it has only been tested with devices exporting only one AFU.
Char devices can be found in /dev/ocxl/ and are named as:
/dev/ocxl/<AFU name>.<location>.<index>
where <AFU name> is a max 20-character long name, as found in the
config space of the AFU.
<location> is added by the driver and can help distinguish devices
when a system has more than one instance of the same OpenCAPI device.
<index> is also to help distinguish AFUs in the unlikely case where a
device carries multiple copies of the same AFU.
Sysfs class
===========
An ocxl class is added for the devices representing the AFUs. See
/sys/class/ocxl. The layout is described in
Documentation/ABI/testing/sysfs-class-ocxl
User API
========
open
----
Based on the AFU definition found in the config space, an AFU may
support working with more than one memory context, in which case the
associated char device may be opened multiple times by different
processes.
ioctl
-----
OCXL_IOCTL_ATTACH:
Attach the memory context of the calling process to the AFU so that
the AFU can access its memory.
OCXL_IOCTL_IRQ_ALLOC:
Allocate an AFU interrupt and return an identifier.
OCXL_IOCTL_IRQ_FREE:
Free a previously allocated AFU interrupt.
OCXL_IOCTL_IRQ_SET_FD:
Associate an event fd to an AFU interrupt so that the user process
can be notified when the AFU sends an interrupt.
OCXL_IOCTL_GET_METADATA:
Obtains configuration information from the card, such at the size of
MMIO areas, the AFU version, and the PASID for the current context.
OCXL_IOCTL_ENABLE_P9_WAIT:
Allows the AFU to wake a userspace thread executing 'wait'. Returns
information to userspace to allow it to configure the AFU. Note that
this is only available on POWER9.
OCXL_IOCTL_GET_FEATURES:
Reports on which CPU features that affect OpenCAPI are usable from
userspace.
mmap
----
A process can mmap the per-process MMIO area for interactions with the
AFU.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
OpenCAPI 소개
1-21OpenCAPI(Open Coherent Accelerator Processor Interface)는 processor와 accelerator 사이의 low-latency, high-bandwidth interface입니다.
specification은 OpenCAPI Consortium이 개발했으며 현재 Compute Express Link Consortium의 `https://computeexpresslink.org/resource/opencapi-specification-archive/`에서 제공합니다.
FPGA, ASIC 같은 accelerator가 virtual address를 사용해 host memory에 coherent하게 접근할 수 있습니다. OpenCAPI device는 자체 memory를 가질 수도 있고 host가 그 memory에 접근할 수 있습니다.
Linux에서는 OpenCAPI를 `ocxl`이라고 부릅니다. powerpc의 IBM CAPI interface driver인 `cxl`이 ISDN CAPI subsystem과 혼동을 피하려고 그렇게 명명되었고, ocxl은 그 개방적이고 processor-agnostic한 진화라는 뜻입니다.
processor와 accelerator가 공유하는 memory model입니다.
========================================================
OpenCAPI (Open Coherent Accelerator Processor Interface)
========================================================
OpenCAPI is an interface between processors and accelerators. It aims
at being low-latency and high-bandwidth.
The specification was developed by the OpenCAPI Consortium, and is now
available from the `Compute Express Link Consortium
<https://computeexpresslink.org/resource/opencapi-specification-archive/>`_.
It allows an accelerator (which could be an FPGA, ASICs, ...) to access
the host memory coherently, using virtual addresses. An OpenCAPI
device can also host its own memory, that can be accessed from the
host.
OpenCAPI is known in linux as 'ocxl', as the open, processor-agnostic
evolution of 'cxl' (the driver for the IBM CAPI interface for
powerpc), which was named that way to avoid confusion with the ISDN
CAPI subsystem.
High-level view
22-53OpenCAPI는 physical link 위에 구현할 Data Link Layer(DL)와 Transaction Layer(TL)를 정의합니다. DL과 TL을 구현한 processor나 device는 memory sharing을 시작할 수 있습니다.
processor 쪽 memory는 TL과 DL을 거쳐 PHY에 연결되고, accelerator 쪽 AFU(Accelerated Function Unit)와 memory는 TLX와 DLX를 거쳐 같은 PHY link에 연결됩니다.
원문의 양쪽 ASCII stack을 같은 연결 관계로 구조화했습니다.
processor와 AFU 측 layer 명칭입니다.
High-level view
===============
OpenCAPI defines a Data Link Layer (DL) and Transaction Layer (TL), to
be implemented on top of a physical link. Any processor or device
implementing the DL and TL can start sharing memory.
::
+-----------+ +-------------+
| | | |
| | | Accelerated |
| Processor | | Function |
| | +--------+ | Unit | +--------+
| |--| Memory | | (AFU) |--| Memory |
| | +--------+ | | +--------+
+-----------+ +-------------+
| |
+-----------+ +-------------+
| TL | | TLX |
+-----------+ +-------------+
| |
+-----------+ +-------------+
| DL | | DLX |
+-----------+ +-------------+
| |
| PHY |
+---------------------------------------+
Device discovery
54-72OpenCAPI는 device가 구현하는 PCI-like configuration space에 의존합니다. host는 config space를 query해 AFU를 발견합니다.
Linux에서 OpenCAPI device는 몇 가지 예외를 빼면 PCI device처럼 취급합니다. firmware는 hardware를 PCI link인 것처럼 추상화해야 합니다.
기존 PCI infrastructure를 많이 재사용하여 standard PCI enumeration 중 device를 scan하고 BAR를 할당합니다. 따라서 `lspci`로 사용 가능한 장치를 확인할 수 있습니다.
configuration space는 physical adapter에 있는 AFU의 이름, 사용할 수 있는 memory context 수, MMIO area 크기 등을 정의합니다.
firmware abstraction부터 사용자 확인까지의 경로입니다.
Device discovery
================
OpenCAPI relies on a PCI-like configuration space, implemented on the
device. So the host can discover AFUs by querying the config space.
OpenCAPI devices in Linux are treated like PCI devices (with a few
caveats). The firmware is expected to abstract the hardware as if it
was a PCI link. A lot of the existing PCI infrastructure is reused:
devices are scanned and BARs are assigned during the standard PCI
enumeration. Commands like 'lspci' can therefore be used to see what
devices are available.
The configuration space defines the AFU(s) that can be found on the
physical adapter, such as its name, how many memory contexts it can
work with, the size of its MMIO areas, ...
MMIO 영역
73-82OpenCAPI는 각 AFU에 두 MMIO area를 정의합니다.
global MMIO area에는 AFU 전체와 관련된 register가 있고, per-process MMIO area는 context마다 고정 크기를 갖습니다.
scope에 따른 두 register 영역입니다.
MMIO
====
OpenCAPI defines two MMIO areas for each AFU:
* the global MMIO area, with registers pertinent to the whole AFU.
* a per-process MMIO area, which has a fixed size for each context.
AFU interrupt
83-95OpenCAPI AFU는 host process에 interrupt를 보낼 수 있습니다. Transaction Layer의 `intrp_req`가 interrupt를 정의하는 64-bit object handle을 지정해 전달합니다.
driver는 process가 interrupt를 allocate하고 그 64-bit object handle을 얻도록 하며, process는 handle을 AFU에 전달할 수 있습니다.
사용자 process가 handle을 설정하고 AFU가 알림을 보내는 관계입니다.
AFU interrupts
==============
OpenCAPI includes the possibility for an AFU to send an interrupt to a
host process. It is done through a 'intrp_req' defined in the
Transaction Layer, specifying a 64-bit object handle which defines the
interrupt.
The driver allows a process to allocate an interrupt and obtain its
64-bit object handle, that can be passed to the AFU.
Character device
96-115driver는 physical device에서 발견한 AFU마다 character device 하나를 만듭니다. physical device는 여러 function을 가질 수 있고 function마다 여러 AFU가 있을 수 있지만, 문서 작성 시점에는 AFU 하나만 export하는 장치에서만 시험했습니다.
character device는 `/dev/ocxl/` 아래 `/dev/ocxl/<AFU name>.<location>.<index>` 형식으로 나타납니다.
`<AFU name>`은 AFU config space의 최대 20-character 이름입니다. `<location>`은 동일 OpenCAPI device가 여러 개 있을 때 구분하도록 driver가 붙입니다. `<index>`는 한 device에 같은 AFU copy가 여러 개 있는 드문 경우를 구분합니다.
AFU character-device 경로 구성 요소입니다.
char devices
============
The driver creates one char device per AFU found on the physical
device. A physical device may have multiple functions and each
function can have multiple AFUs. At the time of this writing though,
it has only been tested with devices exporting only one AFU.
Char devices can be found in /dev/ocxl/ and are named as:
/dev/ocxl/<AFU name>.<location>.<index>
where <AFU name> is a max 20-character long name, as found in the
config space of the AFU.
<location> is added by the driver and can help distinguish devices
when a system has more than one instance of the same OpenCAPI device.
<index> is also to help distinguish AFUs in the unlikely case where a
device carries multiple copies of the same AFU.
Sysfs class
116-124AFU를 나타내는 device를 위해 `ocxl` class가 추가되며 `/sys/class/ocxl`에서 확인합니다.
layout은 `Documentation/ABI/testing/sysfs-class-ocxl`에 설명되어 있습니다.
class path와 ABI 문서입니다.
Sysfs class
===========
An ocxl class is added for the devices representing the AFUs. See
/sys/class/ocxl. The layout is described in
Documentation/ABI/testing/sysfs-class-ocxl
User API: open
125-136config space의 AFU 정의에 따라 AFU가 여러 memory context와 함께 동작할 수 있습니다.
그 경우 서로 다른 process가 연결된 character device를 여러 번 open할 수 있습니다.
AFU context capability가 open 수를 결정합니다.
User API
========
open
----
Based on the AFU definition found in the config space, an AFU may
support working with more than one memory context, in which case the
associated char device may be opened multiple times by different
processes.
User API: ioctl
137-174`OCXL_IOCTL_ATTACH`는 calling process의 memory context를 AFU에 attach하여 AFU가 그 memory에 접근하게 합니다.
`OCXL_IOCTL_IRQ_ALLOC`은 AFU interrupt를 allocate하고 identifier를 반환하며, `OCXL_IOCTL_IRQ_FREE`는 이전에 allocate한 interrupt를 free합니다.
`OCXL_IOCTL_IRQ_SET_FD`는 event fd를 AFU interrupt에 연결하여 AFU가 interrupt를 보낼 때 user process가 notification을 받게 합니다.
`OCXL_IOCTL_GET_METADATA`는 MMIO area 크기, AFU version, 현재 context의 PASID 같은 card configuration 정보를 얻습니다.
`OCXL_IOCTL_ENABLE_P9_WAIT`는 AFU가 `wait`를 실행 중인 userspace thread를 깨우게 하고 AFU 설정에 필요한 정보를 userspace에 반환합니다. POWER9에서만 사용할 수 있습니다.
`OCXL_IOCTL_GET_FEATURES`는 OpenCAPI에 영향을 주는 CPU feature 중 userspace에서 사용할 수 있는 항목을 보고합니다.
AFU context, interrupt와 metadata 제어 command입니다.
ioctl
-----
OCXL_IOCTL_ATTACH:
Attach the memory context of the calling process to the AFU so that
the AFU can access its memory.
OCXL_IOCTL_IRQ_ALLOC:
Allocate an AFU interrupt and return an identifier.
OCXL_IOCTL_IRQ_FREE:
Free a previously allocated AFU interrupt.
OCXL_IOCTL_IRQ_SET_FD:
Associate an event fd to an AFU interrupt so that the user process
can be notified when the AFU sends an interrupt.
OCXL_IOCTL_GET_METADATA:
Obtains configuration information from the card, such at the size of
MMIO areas, the AFU version, and the PASID for the current context.
OCXL_IOCTL_ENABLE_P9_WAIT:
Allows the AFU to wake a userspace thread executing 'wait'. Returns
information to userspace to allow it to configure the AFU. Note that
this is only available on POWER9.
OCXL_IOCTL_GET_FEATURES:
Reports on which CPU features that affect OpenCAPI are usable from
userspace.
User API: mmap
175-179process는 AFU와 상호작용하기 위해 per-process MMIO area를 `mmap`할 수 있습니다.
context attach 후 MMIO mapping과 interrupt를 함께 사용합니다.
mmap
----
A process can mmap the per-process MMIO area for interactions with the
AFU.
요약·해설
ocxl.rst:1-179OpenCAPI의 coherent memory architecture, AFU discovery, MMIO, interrupt와 ocxl character-device ioctl ABI를 설명합니다.
원문 ABI name, ioctl, sysfs path, 수치, code symbol과 줄 좌표를 유지해 모든 절을 한국어로 옮겼습니다.