요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
MHI channel과 DMA Bridge
aic100.rst:162-417Boot·control·status·telemetry channel map과 host-backed DBC request/response FIFO, bit field, semaphore·doorbell·MSI ordering을 정리합니다.
NNC, SSR, RAS와 telemetry
aic100.rst:418-517NNC message framing과 transaction, per-workload restart recovery, internal error reporting과 physical telemetry channel을 설명합니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0-only
===============================
Qualcomm Cloud AI 100 (AIC100)
===============================
Overview
========
The Qualcomm Cloud AI 100/AIC100 family of products (including SA9000P - part of
Snapdragon Ride) are PCIe adapter cards which contain a dedicated SoC ASIC for
the purpose of efficiently running Artificial Intelligence (AI) Deep Learning
inference workloads. They are AI accelerators.
The PCIe interface of AIC100 is capable of PCIe Gen4 speeds over eight lanes
(x8). An individual SoC on a card can have up to 16 NSPs for running workloads.
Each SoC has an A53 management CPU. On card, there can be up to 32 GB of DDR.
Multiple AIC100 cards can be hosted in a single system to scale overall
performance. AIC100 cards are multi-user capable and able to execute workloads
from multiple users in a concurrent manner.
Hardware Description
====================
An AIC100 card consists of an AIC100 SoC, on-card DDR, and a set of misc
peripherals (PMICs, etc).
An AIC100 card can either be a PCIe HHHL form factor (a traditional PCIe card),
or a Dual M.2 card. Both use PCIe to connect to the host system.
As a PCIe endpoint/adapter, AIC100 uses the standard VendorID(VID)/
DeviceID(DID) combination to uniquely identify itself to the host. AIC100
uses the standard Qualcomm VID (0x17cb). All AIC100 SKUs use the same
AIC100 DID (0xa100).
AIC100 does not implement FLR (function level reset).
AIC100 implements MSI but does not implement MSI-X. AIC100 prefers 17 MSIs to
operate (1 for MHI, 16 for the DMA Bridge). Falling back to 1 MSI is possible in
scenarios where reserving 32 MSIs isn't feasible.
As a PCIe device, AIC100 utilizes BARs to provide host interfaces to the device
hardware. AIC100 provides 3, 64-bit BARs.
* The first BAR is 4K in size, and exposes the MHI interface to the host.
* The second BAR is 2M in size, and exposes the DMA Bridge interface to the
host.
* The third BAR is variable in size based on an individual AIC100's
configuration, but defaults to 64K. This BAR currently has no purpose.
From the host perspective, AIC100 has several key hardware components -
* MHI (Modem Host Interface)
* QSM (QAIC Service Manager)
* NSPs (Neural Signal Processor)
* DMA Bridge
* DDR
MHI
---
AIC100 has one MHI interface over PCIe. MHI itself is documented at
Documentation/mhi/index.rst MHI is the mechanism the host uses to communicate
with the QSM. Except for workload data via the DMA Bridge, all interaction with
the device occurs via MHI.
QSM
---
QAIC Service Manager. This is an ARM A53 CPU that runs the primary
firmware of the card and performs on-card management tasks. It also
communicates with the host via MHI. Each AIC100 has one of
these.
NSP
---
Neural Signal Processor. Each AIC100 has up to 16 of these. These are
the processors that run the workloads on AIC100. Each NSP is a Qualcomm Hexagon
(Q6) DSP with HVX and HMX. Each NSP can only run one workload at a time, but
multiple NSPs may be assigned to a single workload. Since each NSP can only run
one workload, AIC100 is limited to 16 concurrent workloads. Workload
"scheduling" is under the purview of the host. AIC100 does not automatically
timeslice.
DMA Bridge
----------
The DMA Bridge is custom DMA engine that manages the flow of data
in and out of workloads. AIC100 has one of these. The DMA Bridge has 16
channels, each consisting of a set of request/response FIFOs. Each active
workload is assigned a single DMA Bridge channel. The DMA Bridge exposes
hardware registers to manage the FIFOs (head/tail pointers), but requires host
memory to store the FIFOs.
DDR
---
AIC100 has on-card DDR. In total, an AIC100 can have up to 32 GB of DDR.
This DDR is used to store workloads, data for the workloads, and is used by the
QSM for managing the device. NSPs are granted access to sections of the DDR by
the QSM. The host does not have direct access to the DDR, and must make
requests to the QSM to transfer data to the DDR.
High-level Use Flow
===================
AIC100 is a multi-user, programmable accelerator typically used for running
neural networks in inferencing mode to efficiently perform AI operations.
AIC100 is not intended for training neural networks. AIC100 can be utilized
for generic compute workloads.
Assuming a user wants to utilize AIC100, they would follow these steps:
1. Compile the workload into an ELF targeting the NSP(s)
2. Make requests to the QSM to load the workload and related artifacts into the
device DDR
3. Make a request to the QSM to activate the workload onto a set of idle NSPs
4. Make requests to the DMA Bridge to send input data to the workload to be
processed, and other requests to receive processed output data from the
workload.
5. Once the workload is no longer required, make a request to the QSM to
deactivate the workload, thus putting the NSPs back into an idle state.
6. Once the workload and related artifacts are no longer needed for future
sessions, make requests to the QSM to unload the data from DDR. This frees
the DDR to be used by other users.
Boot Flow
=========
AIC100 uses a flashless boot flow, derived from Qualcomm MSMs.
When AIC100 is first powered on, it begins executing PBL (Primary Bootloader)
from ROM. PBL enumerates the PCIe link, and initializes the BHI (Boot Host
Interface) component of MHI.
Using BHI, the host points PBL to the location of the SBL (Secondary Bootloader)
image. The PBL pulls the image from the host, validates it, and begins
execution of SBL.
SBL initializes MHI, and uses MHI to notify the host that the device has entered
the SBL stage. SBL performs a number of operations:
* SBL initializes the majority of hardware (anything PBL left uninitialized),
including DDR.
* SBL offloads the bootlog to the host.
* SBL synchronizes timestamps with the host for future logging.
* SBL uses the Sahara protocol to obtain the runtime firmware images from the
host.
Once SBL has obtained and validated the runtime firmware, it brings the NSPs out
of reset, and jumps into the QSM.
The QSM uses MHI to notify the host that the device has entered the QSM stage
(AMSS in MHI terms). At this point, the AIC100 device is fully functional, and
ready to process workloads.
Userspace components
====================
Compiler
--------
An open compiler for AIC100 based on upstream LLVM can be found at:
https://github.com/quic/software-kit-for-qualcomm-cloud-ai-100-cc
Usermode Driver (UMD)
---------------------
An open UMD that interfaces with the qaic kernel driver can be found at:
https://github.com/quic/software-kit-for-qualcomm-cloud-ai-100
Sahara loader
-------------
An open implementation of the Sahara protocol called kickstart can be found at:
https://github.com/andersson/qdl
MHI Channels
============
AIC100 defines a number of MHI channels for different purposes. This is a list
of the defined channels, and their uses.
+----------------+---------+----------+----------------------------------------+
| Channel name | IDs | EEs | Purpose |
+================+=========+==========+========================================+
| QAIC_LOOPBACK | 0 & 1 | AMSS | Any data sent to the device on this |
| | | | channel is sent back to the host. |
+----------------+---------+----------+----------------------------------------+
| QAIC_SAHARA | 2 & 3 | SBL | Used by SBL to obtain the runtime |
| | | | firmware from the host. |
+----------------+---------+----------+----------------------------------------+
| QAIC_DIAG | 4 & 5 | AMSS | Used to communicate with QSM via the |
| | | | DIAG protocol. |
+----------------+---------+----------+----------------------------------------+
| QAIC_SSR | 6 & 7 | AMSS | Used to notify the host of subsystem |
| | | | restart events, and to offload SSR |
| | | | crashdumps. |
+----------------+---------+----------+----------------------------------------+
| QAIC_QDSS | 8 & 9 | AMSS | Used for the Qualcomm Debug Subsystem. |
+----------------+---------+----------+----------------------------------------+
| QAIC_CONTROL | 10 & 11 | AMSS | Used for the Neural Network Control |
| | | | (NNC) protocol. This is the primary |
| | | | channel between host and QSM for |
| | | | managing workloads. |
+----------------+---------+----------+----------------------------------------+
| QAIC_LOGGING | 12 & 13 | SBL | Used by the SBL to send the bootlog to |
| | | | the host. |
+----------------+---------+----------+----------------------------------------+
| QAIC_STATUS | 14 & 15 | AMSS | Used to notify the host of Reliability,|
| | | | Accessibility, Serviceability (RAS) |
| | | | events. |
+----------------+---------+----------+----------------------------------------+
| QAIC_TELEMETRY | 16 & 17 | AMSS | Used to get/set power/thermal/etc |
| | | | attributes. |
+----------------+---------+----------+----------------------------------------+
| QAIC_DEBUG | 18 & 19 | AMSS | Not used. |
+----------------+---------+----------+----------------------------------------+
| QAIC_TIMESYNC | 20 & 21 | SBL | Used to synchronize timestamps in the |
| | | | device side logs with the host time |
| | | | source. |
+----------------+---------+----------+----------------------------------------+
| QAIC_TIMESYNC | 22 & 23 | AMSS | Used to periodically synchronize |
| _PERIODIC | | | timestamps in the device side logs with|
| | | | the host time source. |
+----------------+---------+----------+----------------------------------------+
| IPCR | 24 & 25 | AMSS | AF_QIPCRTR clients and servers. |
+----------------+---------+----------+----------------------------------------+
DMA Bridge
==========
Overview
--------
The DMA Bridge is one of the main interfaces to the host from the device
(the other being MHI). As part of activating a workload to run on NSPs, the QSM
assigns that network a DMA Bridge channel. A workload's DMA Bridge channel
(DBC for short) is solely for the use of that workload and is not shared with
other workloads.
Each DBC is a pair of FIFOs that manage data in and out of the workload. One
FIFO is the request FIFO. The other FIFO is the response FIFO.
Each DBC contains 4 registers in hardware:
* Request FIFO head pointer (offset 0x0). Read only by the host. Indicates the
latest item in the FIFO the device has consumed.
* Request FIFO tail pointer (offset 0x4). Read/write by the host. Host
increments this register to add new items to the FIFO.
* Response FIFO head pointer (offset 0x8). Read/write by the host. Indicates
the latest item in the FIFO the host has consumed.
* Response FIFO tail pointer (offset 0xc). Read only by the host. Device
increments this register to add new items to the FIFO.
The values in each register are indexes in the FIFO. To get the location of the
FIFO element pointed to by the register: FIFO base address + register * element
size.
DBC registers are exposed to the host via the second BAR. Each DBC consumes
4KB of space in the BAR.
The actual FIFOs are backed by host memory. When sending a request to the QSM
to activate a network, the host must donate memory to be used for the FIFOs.
Due to internal mapping limitations of the device, a single contiguous chunk of
memory must be provided per DBC, which hosts both FIFOs. The request FIFO will
consume the beginning of the memory chunk, and the response FIFO will consume
the end of the memory chunk.
Request FIFO
------------
A request FIFO element has the following structure:
.. code-block:: c
struct request_elem {
u16 req_id;
u8 seq_id;
u8 pcie_dma_cmd;
u32 reserved;
u64 pcie_dma_source_addr;
u64 pcie_dma_dest_addr;
u32 pcie_dma_len;
u32 reserved;
u64 doorbell_addr;
u8 doorbell_attr;
u8 reserved;
u16 reserved;
u32 doorbell_data;
u32 sem_cmd0;
u32 sem_cmd1;
u32 sem_cmd2;
u32 sem_cmd3;
};
Request field descriptions:
req_id
request ID. A request FIFO element and a response FIFO element with
the same request ID refer to the same command.
seq_id
sequence ID within a request. Ignored by the DMA Bridge.
pcie_dma_cmd
describes the DMA element of this request.
* Bit(7) is the force msi flag, which overrides the DMA Bridge MSI logic
and generates a MSI when this request is complete, and QSM
configures the DMA Bridge to look at this bit.
* Bits(6:5) are reserved.
* Bit(4) is the completion code flag, and indicates that the DMA Bridge
shall generate a response FIFO element when this request is
complete.
* Bit(3) indicates if this request is a linked list transfer(0) or a bulk
transfer(1).
* Bit(2) is reserved.
* Bits(1:0) indicate the type of transfer. No transfer(0), to device(1),
from device(2). Value 3 is illegal.
pcie_dma_source_addr
source address for a bulk transfer, or the address of the linked list.
pcie_dma_dest_addr
destination address for a bulk transfer.
pcie_dma_len
length of the bulk transfer. Note that the size of this field
limits transfers to 4G in size.
doorbell_addr
address of the doorbell to ring when this request is complete.
doorbell_attr
doorbell attributes.
* Bit(7) indicates if a write to a doorbell is to occur.
* Bits(6:2) are reserved.
* Bits(1:0) contain the encoding of the doorbell length. 0 is 32-bit,
1 is 16-bit, 2 is 8-bit, 3 is reserved. The doorbell address
must be naturally aligned to the specified length.
doorbell_data
data to write to the doorbell. Only the bits corresponding to
the doorbell length are valid.
sem_cmdN
semaphore command.
* Bit(31) indicates this semaphore command is enabled.
* Bit(30) is the to-device DMA fence. Block this request until all
to-device DMA transfers are complete.
* Bit(29) is the from-device DMA fence. Block this request until all
from-device DMA transfers are complete.
* Bits(28:27) are reserved.
* Bits(26:24) are the semaphore command. 0 is NOP. 1 is init with the
specified value. 2 is increment. 3 is decrement. 4 is wait
until the semaphore is equal to the specified value. 5 is wait
until the semaphore is greater or equal to the specified value.
6 is "P", wait until semaphore is greater than 0, then
decrement by 1. 7 is reserved.
* Bit(23) is reserved.
* Bit(22) is the semaphore sync. 0 is post sync, which means that the
semaphore operation is done after the DMA transfer. 1 is
presync, which gates the DMA transfer. Only one presync is
allowed per request.
* Bit(21) is reserved.
* Bits(20:16) is the index of the semaphore to operate on.
* Bits(15:12) are reserved.
* Bits(11:0) are the semaphore value to use in operations.
Overall, a request is processed in 4 steps:
1. If specified, the presync semaphore condition must be true
2. If enabled, the DMA transfer occurs
3. If specified, the postsync semaphore conditions must be true
4. If enabled, the doorbell is written
By using the semaphores in conjunction with the workload running on the NSPs,
the data pipeline can be synchronized such that the host can queue multiple
requests of data for the workload to process, but the DMA Bridge will only copy
the data into the memory of the workload when the workload is ready to process
the next input.
Response FIFO
-------------
Once a request is fully processed, a response FIFO element is generated if
specified in pcie_dma_cmd. The structure of a response FIFO element:
.. code-block:: c
struct response_elem {
u16 req_id;
u16 completion_code;
};
req_id
matches the req_id of the request that generated this element.
completion_code
status of this request. 0 is success. Non-zero is an error.
The DMA Bridge will generate a MSI to the host as a reaction to activity in the
response FIFO of a DBC. The DMA Bridge hardware has an IRQ storm mitigation
algorithm, where it will only generate a MSI when the response FIFO transitions
from empty to non-empty (unless force MSI is enabled and triggered). In
response to this MSI, the host is expected to drain the response FIFO, and must
take care to handle any race conditions between draining the FIFO, and the
device inserting elements into the FIFO.
Neural Network Control (NNC) Protocol
=====================================
The NNC protocol is how the host makes requests to the QSM to manage workloads.
It uses the QAIC_CONTROL MHI channel.
Each NNC request is packaged into a message. Each message is a series of
transactions. A passthrough type transaction can contain elements known as
commands.
QSM requires NNC messages be little endian encoded and the fields be naturally
aligned. Since there are 64-bit elements in some NNC messages, 64-bit alignment
must be maintained.
A message contains a header and then a series of transactions. A message may be
at most 4K in size from QSM to the host. From the host to the QSM, a message
can be at most 64K (maximum size of a single MHI packet), but there is a
continuation feature where message N+1 can be marked as a continuation of
message N. This is used for exceedingly large DMA xfer transactions.
Transaction descriptions
------------------------
passthrough
Allows userspace to send an opaque payload directly to the QSM.
This is used for NNC commands. Userspace is responsible for managing
the QSM message requirements in the payload.
dma_xfer
DMA transfer. Describes an object that the QSM should DMA into the
device via address and size tuples.
activate
Activate a workload onto NSPs. The host must provide memory to be
used by the DBC.
deactivate
Deactivate an active workload and return the NSPs to idle.
status
Query the QSM about it's NNC implementation. Returns the NNC version,
and if CRC is used.
terminate
Release a user's resources.
dma_xfer_cont
Continuation of a previous DMA transfer. If a DMA transfer
cannot be specified in a single message (highly fragmented), this
transaction can be used to specify more ranges.
validate_partition
Query to QSM to determine if a partition identifier is valid.
Each message is tagged with a user id, and a partition id. The user id allows
QSM to track resources, and release them when the user goes away (eg the process
crashes). A partition id identifies the resource partition that QSM manages,
which this message applies to.
Messages may have CRCs. Messages should have CRCs applied until the QSM
reports via the status transaction that CRCs are not needed. The QSM on the
SA9000P requires CRCs for black channel safing.
Subsystem Restart (SSR)
=======================
SSR is the concept of limiting the impact of an error. An AIC100 device may
have multiple users, each with their own workload running. If the workload of
one user crashes, the fallout of that should be limited to that workload and not
impact other workloads. SSR accomplishes this.
If a particular workload crashes, QSM notifies the host via the QAIC_SSR MHI
channel. This notification identifies the workload by it's assigned DBC. A
multi-stage recovery process is then used to cleanup both sides, and get the
DBC/NSPs into a working state.
When SSR occurs, any state in the workload is lost. Any inputs that were in
process, or queued by not yet serviced, are lost. The loaded artifacts will
remain in on-card DDR, but the host will need to re-activate the workload if
it desires to recover the workload.
Reliability, Accessibility, Serviceability (RAS)
================================================
AIC100 is expected to be deployed in server systems where RAS ideology is
applied. Simply put, RAS is the concept of detecting, classifying, and
reporting errors. While PCIe has AER (Advanced Error Reporting) which factors
into RAS, AER does not allow for a device to report details about internal
errors. Therefore, AIC100 implements a custom RAS mechanism. When a RAS event
occurs, QSM will report the event with appropriate details via the QAIC_STATUS
MHI channel. A sysadmin may determine that a particular device needs
additional service based on RAS reports.
Telemetry
=========
QSM has the ability to report various physical attributes of the device, and in
some cases, to allow the host to control them. Examples include thermal limits,
thermal readings, and power readings. These items are communicated via the
QAIC_TELEMETRY MHI channel.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
문서 표제와 license
1-6이 문서는 `GPL-2.0-only`로 배포되는 Qualcomm Cloud AI 100(AIC100) hardware 설명서입니다.
.. SPDX-License-Identifier: GPL-2.0-only
Qualcomm Cloud AI 100 (AIC100)개요
7-22Qualcomm Cloud AI 100/AIC100 제품군은 Snapdragon Ride의 일부인 SA9000P를 포함합니다. Artificial Intelligence(AI) Deep Learning inference workload를 효율적으로 실행하기 위한 전용 SoC ASIC을 탑재한 PCIe adapter card이며 AI accelerator입니다.
AIC100 PCIe interface는 lane 8개(`x8`)에서 PCIe Gen4 속도를 지원합니다. Card의 SoC 하나에는 workload 실행용 NSP가 최대 16개, management용 A53 CPU 하나, DDR이 최대 32 GB 있습니다. 한 system에 card 여러 개를 장착해 전체 성능을 확장할 수 있고, multi-user 환경에서 여러 사용자의 workload를 동시에 실행할 수 있습니다.
Host link, compute, management와 memory capacity를 한눈에 정리합니다.
Card form factor, PCI identity와 BAR
23-61AIC100 card는 AIC100 SoC, on-card DDR, PMIC 등의 주변 장치로 구성됩니다. Form factor는 전통적인 PCIe card인 PCIe HHHL 또는 Dual M.2이며 둘 다 PCIe로 host system에 연결합니다.
PCIe endpoint/adapter인 AIC100은 표준 VendorID(VID)/DeviceID(DID) 조합으로 host에 자신을 고유하게 식별합니다. Qualcomm VID `0x17cb`를 사용하며 모든 AIC100 SKU의 DID는 `0xa100`입니다. AIC100은 FLR(function level reset)을 구현하지 않습니다.
AIC100은 MSI를 구현하지만 MSI-X는 구현하지 않습니다. 정상 동작에는 MSI 17개, 즉 MHI용 1개와 DMA Bridge용 16개를 선호합니다. MSI 32개를 reserve하기 어려운 경우 MSI 1개로 fallback할 수 있습니다.
Host interface용으로 64-bit BAR 세 개를 제공합니다. 첫 번째 BAR는 4K이고 MHI interface를 노출합니다. 두 번째 BAR는 2M이고 DMA Bridge interface를 노출합니다. 세 번째 BAR 크기는 개별 AIC100 구성에 따라 달라지며 default는 64K이고 현재 용도는 없습니다. Host 관점의 핵심 hardware component는 MHI(Modem Host Interface), QSM(QAIC Service Manager), NSP(Neural Signal Processor), DMA Bridge와 DDR입니다.
Host enumeration과 interrupt allocation 특성입니다.
세 BAR의 default size와 exposed host interface입니다.
MHI
62-69AIC100에는 PCIe를 통한 MHI interface가 하나 있습니다. MHI 자체 문서는 `Documentation/mhi/index.rst`에 있습니다. Host는 MHI로 QSM과 통신합니다. DMA Bridge를 통한 workload data를 제외하면 device와의 모든 상호 작용은 MHI를 거칩니다.
QAIC Service Manager
70-77QSM(QAIC Service Manager)은 card의 primary firmware를 실행하고 on-card management task를 수행하는 ARM A53 CPU입니다. MHI로 host와 통신하며 AIC100마다 하나씩 있습니다.
Neural Signal Processor
78-88AIC100에는 workload를 실행하는 NSP(Neural Signal Processor)가 최대 16개 있습니다. 각 NSP는 HVX와 HMX를 갖춘 Qualcomm Hexagon(Q6) DSP입니다. NSP 하나는 한 번에 workload 하나만 실행하지만 workload 하나에 NSP 여러 개를 배정할 수 있습니다.
NSP 하나당 workload 하나라는 제약으로 AIC100은 concurrent workload를 최대 16개 실행합니다. Workload scheduling은 host가 담당하며 AIC100은 자동으로 timeslice하지 않습니다.
DMA Bridge hardware
89-98DMA Bridge는 workload로 들어가고 나오는 data flow를 관리하는 custom DMA engine이며 AIC100마다 하나 있습니다. Channel 16개가 있고 각 channel은 request FIFO와 response FIFO 한 set으로 구성됩니다. Active workload마다 DMA Bridge channel 하나를 배정합니다.
DMA Bridge는 FIFO head/tail pointer를 관리하는 hardware register를 노출하지만 실제 FIFO 저장에는 host memory가 필요합니다.
On-card DDR
99-107AIC100의 on-card DDR 총량은 최대 32 GB입니다. Workload와 그 data를 저장하고 QSM의 device 관리에도 사용합니다. QSM은 NSP에 DDR 일부 구간의 access를 부여합니다. Host는 DDR에 직접 접근할 수 없으며 DDR data transfer를 QSM에 요청해야 합니다.
High-level workload 사용 흐름
108-131AIC100은 일반적으로 neural network inference로 AI operation을 효율적으로 수행하는 multi-user programmable accelerator입니다. Neural network training용으로 의도된 장치는 아니지만 generic compute workload에도 사용할 수 있습니다.
사용 순서는 다음과 같습니다. (1) Workload를 NSP용 ELF로 compile합니다. (2) QSM에 workload와 관련 artifact를 device DDR에 load하도록 요청합니다. (3) Idle NSP set에 workload를 activate하도록 QSM에 요청합니다. (4) DMA Bridge에 input data를 workload로 보내고 처리된 output data를 받도록 요청합니다.
(5) Workload가 더 필요하지 않으면 QSM에 deactivate를 요청해 NSP를 idle state로 되돌립니다. (6) 향후 session에도 workload와 artifact가 필요 없으면 QSM에 DDR data unload를 요청해 다른 사용자가 쓸 DDR을 확보합니다.
Compilation부터 DDR 회수까지 host가 명시적으로 관리합니다.
Flashless boot flow
132-161AIC100은 Qualcomm MSM에서 파생한 flashless boot flow를 사용합니다. 처음 power-on되면 ROM의 PBL(Primary Bootloader)을 실행합니다. PBL은 PCIe link를 enumerate하고 MHI의 BHI(Boot Host Interface)를 initialize합니다.
Host는 BHI를 이용해 PBL에 SBL(Secondary Bootloader) image 위치를 알려 줍니다. PBL은 host에서 image를 가져와 validate한 뒤 SBL 실행을 시작합니다. SBL은 MHI를 initialize하고 device가 SBL stage에 들어왔음을 host에 알립니다.
SBL은 PBL이 남겨 둔 DDR 등 대부분의 hardware를 initialize하고, bootlog를 host로 offload하고, 이후 logging을 위해 host와 timestamp를 synchronize하고, Sahara protocol로 host에서 runtime firmware image를 가져옵니다. Runtime firmware를 획득하고 validate하면 NSP를 reset에서 해제하고 QSM으로 jump합니다.
QSM은 MHI를 사용해 device가 QSM stage, 즉 MHI 용어로 AMSS에 진입했음을 host에 알립니다. 이 시점부터 AIC100은 완전히 동작하며 workload를 처리할 준비가 됩니다.
ROM PBL에서 host-provided runtime firmware와 QSM으로 이어지는 단계입니다.
Userspace components
162-182Upstream LLVM 기반 AIC100 open compiler는 `https://github.com/quic/software-kit-for-qualcomm-cloud-ai-100-cc`에서 제공합니다. `qaic` kernel driver와 interface하는 open UMD는 `https://github.com/quic/software-kit-for-qualcomm-cloud-ai-100`에서 제공합니다. `kickstart`라는 Sahara protocol open implementation은 `https://github.com/andersson/qdl`에서 제공합니다.
Compilation, runtime와 boot-loading components입니다.
MHI channel map
183-234AIC100은 목적별 MHI channel을 다음과 같이 정의합니다. ID는 양방향 channel pair이고 EE는 해당 channel을 사용하는 execution environment입니다.
원문의 ASCII 표를 channel ID, execution environment와 용도로 구조화했습니다.
DMA Bridge channel과 FIFO memory
235-274DMA Bridge는 MHI와 함께 device에서 host로 향하는 주요 interface입니다. QSM은 NSP에서 실행할 workload를 activate할 때 그 network에 DMA Bridge channel(DBC)을 배정합니다. Workload의 DBC는 해당 workload 전용이며 다른 workload와 공유하지 않습니다.
DBC 하나는 workload로 들어가고 나오는 data를 관리하는 FIFO 두 개, 즉 request FIFO와 response FIFO로 구성됩니다. 각 DBC에는 hardware register 네 개가 있습니다. Request FIFO head(offset `0x0`)는 host read-only이며 device가 consume한 최신 항목을 가리킵니다. Request FIFO tail(offset `0x4`)은 host read/write이며 host가 새 항목을 추가할 때 증가시킵니다.
Response FIFO head(offset `0x8`)는 host read/write이며 host가 consume한 최신 항목을 가리킵니다. Response FIFO tail(offset `0xc`)은 host read-only이며 device가 새 항목을 추가할 때 증가시킵니다. Register 값은 FIFO index이고 해당 element 위치는 `FIFO base address + register * element size`로 계산합니다.
DBC register는 두 번째 BAR로 host에 노출되며 DBC 하나가 BAR 공간 4KB를 사용합니다. 실제 FIFO backing은 host memory입니다. Network activation request 때 host가 FIFO용 memory를 제공해야 합니다. Device 내부 mapping 제한 때문에 DBC마다 두 FIFO를 모두 담는 contiguous memory chunk 하나가 필요합니다. Request FIFO는 chunk 앞부분, response FIFO는 끝부분을 사용합니다.
Host access direction과 pointer ownership을 구분합니다.
Host가 제공한 memory chunk의 양 끝을 두 FIFO가 나누어 사용합니다.
Request FIFO element와 command bit fields
275-390Request FIFO element는 다음 C structure를 사용합니다. Type과 field order는 UAPI를 해석하는 데 중요하므로 원문 그대로 보존합니다.
struct request_elem {
u16 req_id;
u8 seq_id;
u8 pcie_dma_cmd;
u32 reserved;
u64 pcie_dma_source_addr;
u64 pcie_dma_dest_addr;
u32 pcie_dma_len;
u32 reserved;
u64 doorbell_addr;
u8 doorbell_attr;
u8 reserved;
u16 reserved;
u32 doorbell_data;
u32 sem_cmd0;
u32 sem_cmd1;
u32 sem_cmd2;
u32 sem_cmd3;
};Identifier, DMA address/length와 doorbell fields의 의미입니다.
`req_id`가 같은 request FIFO element와 response FIFO element는 같은 command를 가리킵니다. `seq_id`는 request 내부 sequence ID지만 DMA Bridge가 무시합니다. `pcie_dma_source_addr`는 bulk transfer source 또는 linked list address, `pcie_dma_dest_addr`는 bulk destination입니다. `pcie_dma_len`은 bulk length이며 field 크기 때문에 transfer는 최대 `4G`입니다.
DMA 동작, completion response와 MSI 생성을 제어합니다.
`doorbell_addr`는 request 완료 후 ring할 doorbell address입니다. `doorbell_attr` bit 7은 doorbell write 수행 여부, bit 6:2는 reserved, bit 1:0은 doorbell length encoding입니다. 값 `0`은 32-bit, `1`은 16-bit, `2`는 8-bit, `3`은 reserved입니다. Doorbell address는 지정 길이에 natural alignment되어야 합니다. `doorbell_data`에서는 doorbell length에 해당하는 bit만 유효합니다.
Doorbell write enable과 access width를 나타냅니다.
`sem_cmdN`은 semaphore command입니다. Bit 31은 command enable, bit 30은 모든 to-device DMA가 끝날 때까지 이 request를 block하는 to-device DMA fence, bit 29는 모든 from-device DMA가 끝날 때까지 block하는 from-device DMA fence입니다. Bit 28:27은 reserved입니다.
Bit 26:24의 semaphore opcode는 `0` NOP, `1` specified value로 initialize, `2` increment, `3` decrement, `4` semaphore가 specified value와 같을 때까지 wait, `5` specified value 이상일 때까지 wait, `6`은 `P` operation으로 0보다 클 때까지 wait한 뒤 1 decrement, `7` reserved입니다.
Bit 23은 reserved입니다. Bit 22는 semaphore sync로 `0` post sync이면 DMA transfer 뒤에 semaphore operation을 수행하고 `1` presync이면 DMA transfer를 gate합니다. Request 하나에는 presync 하나만 허용됩니다. Bit 21은 reserved, bit 20:16은 대상 semaphore index, bit 15:12는 reserved, bit 11:0은 operation에 사용할 semaphore value입니다.
Fence, operation, synchronization point, index와 value를 보존합니다.
Request는 네 단계로 처리합니다. (1) 지정했다면 presync semaphore condition이 true여야 합니다. (2) Enable했다면 DMA transfer를 수행합니다. (3) 지정했다면 postsync semaphore condition이 true여야 합니다. (4) Enable했다면 doorbell을 씁니다.
NSP에서 실행되는 workload와 semaphore를 함께 사용하면 host가 처리할 input request 여러 개를 queue할 수 있습니다. DMA Bridge는 workload가 다음 input을 처리할 준비가 됐을 때만 data를 workload memory에 copy하도록 pipeline을 synchronize합니다.
Semaphore와 DMA, doorbell의 ordering입니다.
Response FIFO와 MSI
391-417Request 처리가 모두 끝나면 `pcie_dma_cmd`에서 요청한 경우 response FIFO element를 생성합니다. Structure는 다음과 같습니다.
struct response_elem {
u16 req_id;
u16 completion_code;
};`req_id`는 이 element를 생성한 request의 `req_id`와 일치합니다. `completion_code`는 request status이며 `0`은 success, non-zero는 error입니다.
DMA Bridge는 DBC response FIFO activity에 반응해 host에 MSI를 생성합니다. IRQ storm mitigation algorithm 때문에 force MSI가 enable되어 trigger된 경우를 제외하면 response FIFO가 empty에서 non-empty로 바뀔 때만 MSI를 생성합니다.
Host는 MSI를 받으면 response FIFO를 drain해야 하며, FIFO를 drain하는 동안 device가 새 element를 insert하는 race condition을 올바르게 처리해야 합니다.
Empty-to-non-empty transition과 host drain 사이의 race를 고려합니다.
Neural Network Control protocol framing
418-437NNC protocol은 host가 QSM에 workload management request를 보내는 방법이며 `QAIC_CONTROL` MHI channel을 사용합니다. 각 NNC request는 message 하나로 packaging되고 message는 transaction series입니다. Passthrough type transaction에는 command라고 부르는 element가 들어갈 수 있습니다.
QSM은 NNC message를 little endian으로 encode하고 field를 natural alignment할 것을 요구합니다. 일부 message에는 64-bit element가 있으므로 64-bit alignment를 유지해야 합니다.
Message는 header 뒤에 transaction series를 둡니다. QSM에서 host로 보내는 message는 최대 `4K`입니다. Host에서 QSM으로 보내는 message는 single MHI packet 최대 크기인 `64K`까지 가능하지만, message N+1을 message N의 continuation으로 표시하는 기능이 있습니다. 매우 큰 DMA xfer transaction에 사용합니다.
Direction별 size와 common encoding requirements입니다.
NNC transaction types와 identity
438-480`passthrough`는 userspace가 opaque payload를 QSM에 직접 보내게 하며 NNC command에 사용합니다. Payload의 QSM message requirement는 userspace가 관리합니다. `dma_xfer`는 address와 size tuple로 QSM이 device에 DMA할 object를 설명합니다. `activate`는 workload를 NSP에서 활성화하며 host가 DBC memory를 제공해야 합니다. `deactivate`는 active workload를 비활성화하고 NSP를 idle로 돌립니다.
`status`는 QSM의 NNC implementation을 query해 NNC version과 CRC 사용 여부를 반환합니다. `terminate`는 사용자의 resource를 release합니다. `dma_xfer_cont`는 이전 DMA transfer의 continuation이며 fragmentation이 심해 한 message로 지정할 수 없을 때 추가 range를 기술합니다. `validate_partition`은 partition identifier가 valid한지 QSM에 query합니다.
Workload management message에서 사용하는 transaction semantics입니다.
각 message에는 user ID와 partition ID를 tag합니다. User ID를 통해 QSM이 resource를 추적하고 process crash 등으로 사용자가 사라질 때 release할 수 있습니다. Partition ID는 QSM이 관리하는 resource partition 중 이 message가 적용될 대상을 식별합니다.
Message에는 CRC가 있을 수 있습니다. QSM이 `status` transaction으로 CRC가 불필요하다고 보고할 때까지 CRC를 적용해야 합니다. SA9000P의 QSM은 black channel safing을 위해 CRC를 요구합니다.
Subsystem Restart
481-498SSR은 error impact를 제한하는 개념입니다. AIC100의 여러 사용자가 각자 workload를 실행할 때 한 사용자의 workload crash가 다른 workload에 영향을 주지 않도록 격리합니다.
특정 workload가 crash하면 QSM은 `QAIC_SSR` MHI channel로 host에 알립니다. Notification은 assigned DBC로 workload를 식별합니다. 이후 multi-stage recovery process가 양쪽을 cleanup하고 DBC/NSP를 working state로 되돌립니다.
SSR이 발생하면 workload state와 처리 중이거나 queue되었지만 아직 service되지 않은 input은 모두 사라집니다. Load된 artifact는 on-card DDR에 남지만 workload를 복구하려면 host가 다시 activate해야 합니다.
Crash 영향을 assigned DBC와 NSP 범위로 제한합니다.
Reliability, Accessibility, Serviceability
499-510AIC100은 RAS ideology를 적용하는 server system 배포를 전제로 합니다. 간단히 말해 RAS는 error를 detect, classify, report하는 개념입니다. PCIe AER(Advanced Error Reporting)도 RAS에 기여하지만 device 내부 error의 상세 정보를 보고할 수 없습니다.
그래서 AIC100은 custom RAS mechanism을 구현합니다. RAS event가 발생하면 QSM이 적절한 detail과 함께 `QAIC_STATUS` MHI channel로 보고합니다. Sysadmin은 RAS report를 근거로 특정 device에 추가 service가 필요한지 판단할 수 있습니다.
PCIe transport error와 device-internal detail의 reporting 범위를 구분합니다.
Telemetry
511-517QSM은 device의 여러 physical attribute를 보고하고 일부는 host가 제어할 수 있게 합니다. 예로 thermal limit, thermal reading, power reading이 있으며 `QAIC_TELEMETRY` MHI channel로 전달합니다.
QSM이 보고하거나 host control을 허용하는 physical attributes입니다.
Card hardware와 workload lifecycle
aic100.rst:1-161PCIe Gen4 x8 card의 QSM, 최대 16개 NSP, 32 GB DDR와 DMA Bridge를 살펴보고 compile·load·activate·transfer·deactivate·unload 흐름을 설명합니다.