요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
1
.. SPDX-License-Identifier: GPL-2.0+
3
==============================================================
4
Linux kernel driver for Compute Engine Virtual Ethernet (gve):
5
==============================================================
7
Supported Hardware
8
===================
9
The GVE driver binds to a single PCI device id used by the virtual
10
Ethernet device found in some Compute Engine VMs.
12
+--------------+----------+---------+
13
|Field | Value | Comments|
14
+==============+==========+=========+
15
|Vendor ID | `0x1AE0` | Google |
16
+--------------+----------+---------+
17
|Device ID | `0x0042` | |
18
+--------------+----------+---------+
19
|Sub-vendor ID | `0x1AE0` | Google |
20
+--------------+----------+---------+
21
|Sub-device ID | `0x0058` | |
22
+--------------+----------+---------+
23
|Revision ID | `0x0` | |
24
+--------------+----------+---------+
25
|Device Class | `0x200` | Ethernet|
26
+--------------+----------+---------+
28
PCI Bars
29
========
30
The gVNIC PCI device exposes three 32-bit memory BARS:
31
- Bar0 - Device configuration and status registers.
32
- Bar1 - MSI-X vector table
33
- Bar2 - IRQ, RX and TX doorbells
35
Device Interactions
36
===================
37
The driver interacts with the device in the following ways:
38
- Registers
39
- A block of MMIO registers
40
- See gve_register.h for more detail
41
- Admin Queue
42
- See description below
43
- Reset
44
- At any time the device can be reset
45
- Interrupts
46
- See supported interrupts below
47
- Transmit and Receive Queues
48
- See description below
50
Descriptor Formats
51
------------------
52
GVE supports two descriptor formats: GQI and DQO. These two formats have
53
entirely different descriptors, which will be described below.
55
Addressing Mode
56
------------------
57
GVE supports two addressing modes: QPL and RDA.
58
QPL ("queue-page-list") mode communicates data through a set of
59
pre-registered pages.
61
For RDA ("raw DMA addressing") mode, the set of pages is dynamic.
62
Therefore, the packet buffers can be anywhere in guest memory.
64
Registers
65
---------
66
All registers are MMIO.
68
The registers are used for initializing and configuring the device as well as
69
querying device status in response to management interrupts.
71
Endianness
72
----------
73
- Admin Queue messages and registers are all Big Endian.
74
- GQI descriptors and datapath registers are Big Endian.
75
- DQO descriptors and datapath registers are Little Endian.
77
Admin Queue (AQ)
78
----------------
79
The Admin Queue is a PAGE_SIZE memory block, treated as an array of AQ
80
commands, used by the driver to issue commands to the device and set up
81
resources.The driver and the device maintain a count of how many commands
82
have been submitted and executed. To issue AQ commands, the driver must do
83
the following (with proper locking):
85
1) Copy new commands into next available slots in the AQ array
86
2) Increment its counter by he number of new commands
87
3) Write the counter into the GVE_ADMIN_QUEUE_DOORBELL register
88
4) Poll the ADMIN_QUEUE_EVENT_COUNTER register until it equals
89
the value written to the doorbell, or until a timeout.
91
The device will update the status field in each AQ command reported as
92
executed through the ADMIN_QUEUE_EVENT_COUNTER register.
94
Device Resets
95
-------------
96
A device reset is triggered by writing 0x0 to the AQ PFN register.
97
This causes the device to release all resources allocated by the
98
driver, including the AQ itself.
100
Interrupts
101
----------
102
The following interrupts are supported by the driver:
104
Management Interrupt
105
~~~~~~~~~~~~~~~~~~~~
106
The management interrupt is used by the device to tell the driver to
107
look at the GVE_DEVICE_STATUS register.
109
The handler for the management irq simply queues the service task in
110
the workqueue to check the register and acks the irq.
112
Notification Block Interrupts
113
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
114
The notification block interrupts are used to tell the driver to poll
115
the queues associated with that interrupt.
117
The handler for these irqs schedule the napi for that block to run
118
and poll the queues.
120
GQI Traffic Queues
121
------------------
122
GQI queues are composed of a descriptor ring and a buffer and are assigned to a
123
notification block.
125
The descriptor rings are power-of-two-sized ring buffers consisting of
126
fixed-size descriptors. They advance their head pointer using a __be32
127
doorbell located in Bar2. The tail pointers are advanced by consuming
128
descriptors in-order and updating a __be32 counter. Both the doorbell
129
and the counter overflow to zero.
131
Each queue's buffers must be registered in advance with the device as a
132
queue page list, and packet data can only be put in those pages.
134
Transmit
135
~~~~~~~~
136
gve maps the buffers for transmit rings into a FIFO and copies the packets
137
into the FIFO before sending them to the NIC.
139
Receive
140
~~~~~~~
141
The buffers for receive rings are put into a data ring that is the same
142
length as the descriptor ring and the head and tail pointers advance over
143
the rings together.
145
DQO Traffic Queues
146
------------------
147
- Every TX and RX queue is assigned a notification block.
149
- TX and RX buffers queues, which send descriptors to the device, use MMIO
150
doorbells to notify the device of new descriptors.
152
- RX and TX completion queues, which receive descriptors from the device, use a
153
"generation bit" to know when a descriptor was populated by the device. The
154
driver initializes all bits with the "current generation". The device will
155
populate received descriptors with the "next generation" which is inverted
156
from the current generation. When the ring wraps, the current/next generation
157
are swapped.
159
- It's the driver's responsibility to ensure that the RX and TX completion
160
queues are not overrun. This can be accomplished by limiting the number of
161
descriptors posted to HW.
163
- TX packets have a 16 bit completion_tag and RX buffers have a 16 bit
164
buffer_id. These will be returned on the TX completion and RX queues
165
respectively to let the driver know which packet/buffer was completed.
167
Transmit
168
~~~~~~~~
169
A packet's buffers are DMA mapped for the device to access before transmission.
170
After the packet was successfully transmitted, the buffers are unmapped.
172
Receive
173
~~~~~~~
174
The driver posts fixed sized buffers to HW on the RX buffer queue. The packet
175
received on the associated RX queue may span multiple descriptors.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
지원 하드웨어와 PCI BAR
1-34이 문서는 `GPL-2.0+` 라이선스를 따릅니다.
Compute Engine Virtual Ethernet(gve)용 Linux 커널 드라이버
지원 하드웨어
GVE 드라이버는 일부 Compute Engine VM에서 제공되는 가상 Ethernet 장치가 사용하는 단일 PCI device ID에 bind됩니다.
필드값설명
Vendor ID`0x1AE0`Google
Device ID`0x0042`
Sub-vendor ID`0x1AE0`Google
Sub-device ID`0x0058`
Revision ID`0x0`
Device Class`0x200`Ethernet
gVNIC가 노출하는 PCI 식별자와 장치 분류입니다.
PCI BAR
gVNIC PCI 장치는 세 개의 32-bit memory BAR를 노출합니다.
- `Bar0`: 장치 구성 및 상태 register
- `Bar1`: MSI-X vector table
- `Bar2`: IRQ, RX 및 TX doorbell
.. SPDX-License-Identifier: GPL-2.0+
==============================================================
Linux kernel driver for Compute Engine Virtual Ethernet (gve):
==============================================================
Supported Hardware
===================
The GVE driver binds to a single PCI device id used by the virtual
Ethernet device found in some Compute Engine VMs.
+--------------+----------+---------+
|Field | Value | Comments|
+==============+==========+=========+
|Vendor ID | `0x1AE0` | Google |
+--------------+----------+---------+
|Device ID | `0x0042` | |
+--------------+----------+---------+
|Sub-vendor ID | `0x1AE0` | Google |
+--------------+----------+---------+
|Sub-device ID | `0x0058` | |
+--------------+----------+---------+
|Revision ID | `0x0` | |
+--------------+----------+---------+
|Device Class | `0x200` | Ethernet|
+--------------+----------+---------+
PCI Bars
========
The gVNIC PCI device exposes three 32-bit memory BARS:
- Bar0 - Device configuration and status registers.
- Bar1 - MSI-X vector table
- Bar2 - IRQ, RX and TX doorbells
장치 상호작용, descriptor 형식과 byte order
35-76장치 상호작용
드라이버는 다음 경로를 통해 장치와 상호작용합니다.
- Register: MMIO register block을 사용하며 자세한 내용은 `gve_register.h`를 참고합니다.
- Admin Queue: 아래에서 설명하는 관리 명령 경로입니다.
- Reset: 장치는 언제든 reset될 수 있습니다.
- Interrupt: 아래에서 지원 interrupt를 설명합니다.
- Transmit 및 Receive Queue: 아래에서 queue 동작을 설명합니다.
Descriptor 형식
GVE는 `GQI`와 `DQO`라는 두 descriptor 형식을 지원합니다. 두 형식의 descriptor 구조는 완전히 다릅니다.
Addressing mode
GVE는 `QPL`과 `RDA`라는 두 addressing mode를 지원합니다. QPL(`queue-page-list`) mode는 미리 등록한 page 집합을 통해 data를 주고받습니다.
RDA(`raw DMA addressing`) mode에서는 page 집합이 동적으로 바뀌므로 packet buffer가 guest memory의 어느 위치에나 있을 수 있습니다.
Register
모든 register는 MMIO입니다. register는 장치를 초기화하고 구성하는 데 사용하며, management interrupt에 응답해 장치 상태를 조회할 때도 사용합니다.
Endianness
- Admin Queue message와 register는 모두 Big Endian입니다.
- GQI descriptor와 datapath register는 Big Endian입니다.
- DQO descriptor와 datapath register는 Little Endian입니다.
Device Interactions
===================
The driver interacts with the device in the following ways:
- Registers
- A block of MMIO registers
- See gve_register.h for more detail
- Admin Queue
- See description below
- Reset
- At any time the device can be reset
- Interrupts
- See supported interrupts below
- Transmit and Receive Queues
- See description below
Descriptor Formats
------------------
GVE supports two descriptor formats: GQI and DQO. These two formats have
entirely different descriptors, which will be described below.
Addressing Mode
------------------
GVE supports two addressing modes: QPL and RDA.
QPL ("queue-page-list") mode communicates data through a set of
pre-registered pages.
For RDA ("raw DMA addressing") mode, the set of pages is dynamic.
Therefore, the packet buffers can be anywhere in guest memory.
Registers
---------
All registers are MMIO.
The registers are used for initializing and configuring the device as well as
querying device status in response to management interrupts.
Endianness
----------
- Admin Queue messages and registers are all Big Endian.
- GQI descriptors and datapath registers are Big Endian.
- DQO descriptors and datapath registers are Little Endian.
Admin Queue와 장치 reset
77-99Admin Queue(AQ)
Admin Queue는 `PAGE_SIZE` 크기의 memory block입니다. AQ command 배열로 취급되며, 드라이버가 장치에 명령을 내리고 resource를 설정하는 데 사용합니다. 드라이버와 장치는 제출된 명령 수와 실행된 명령 수를 각각 유지합니다.
적절한 lock을 잡은 상태에서 AQ command를 발행하려면 다음 절차를 수행해야 합니다.
- 새 command를 AQ 배열의 다음 가용 slot에 복사합니다.
- 새 command 수만큼 드라이버 counter를 증가시킵니다.
- counter를 `GVE_ADMIN_QUEUE_DOORBELL` register에 기록합니다.
- `ADMIN_QUEUE_EVENT_COUNTER` register가 doorbell에 쓴 값과 같아질 때까지, 또는 timeout이 날 때까지 poll합니다.
장치는 `ADMIN_QUEUE_EVENT_COUNTER` register를 통해 실행 완료로 보고한 각 AQ command의 status field를 갱신합니다.
장치 reset
AQ PFN register에 `0x0`을 쓰면 장치 reset이 시작됩니다. 이 동작은 AQ 자체를 포함해 드라이버가 할당한 모든 resource를 장치가 해제하게 합니다.
Admin Queue (AQ)
----------------
The Admin Queue is a PAGE_SIZE memory block, treated as an array of AQ
commands, used by the driver to issue commands to the device and set up
resources.The driver and the device maintain a count of how many commands
have been submitted and executed. To issue AQ commands, the driver must do
the following (with proper locking):
1) Copy new commands into next available slots in the AQ array
2) Increment its counter by he number of new commands
3) Write the counter into the GVE_ADMIN_QUEUE_DOORBELL register
4) Poll the ADMIN_QUEUE_EVENT_COUNTER register until it equals
the value written to the doorbell, or until a timeout.
The device will update the status field in each AQ command reported as
executed through the ADMIN_QUEUE_EVENT_COUNTER register.
Device Resets
-------------
A device reset is triggered by writing 0x0 to the AQ PFN register.
This causes the device to release all resources allocated by the
driver, including the AQ itself.
Management 및 notification block interrupt
100-119Interrupt
드라이버는 다음 interrupt를 지원합니다.
Management interrupt
장치는 management interrupt를 사용해 드라이버가 `GVE_DEVICE_STATUS` register를 확인해야 함을 알립니다.
management IRQ handler는 register를 확인할 service task를 workqueue에 넣고 IRQ에 ack하는 작업만 수행합니다.
Notification block interrupt
notification block interrupt는 해당 interrupt에 연결된 queue를 poll하라고 드라이버에 알립니다.
이 IRQ의 handler는 해당 block의 NAPI가 실행되어 queue를 poll하도록 schedule합니다.
Interrupts
----------
The following interrupts are supported by the driver:
Management Interrupt
~~~~~~~~~~~~~~~~~~~~
The management interrupt is used by the device to tell the driver to
look at the GVE_DEVICE_STATUS register.
The handler for the management irq simply queues the service task in
the workqueue to check the register and acks the irq.
Notification Block Interrupts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The notification block interrupts are used to tell the driver to poll
the queues associated with that interrupt.
The handler for these irqs schedule the napi for that block to run
and poll the queues.
GQI traffic queue
120-144GQI traffic queue
GQI queue는 descriptor ring과 buffer로 구성되며 notification block 하나에 배정됩니다.
descriptor ring은 고정 크기 descriptor로 이뤄진, 크기가 2의 거듭제곱인 ring buffer입니다. head pointer는 `Bar2`에 있는 `__be32` doorbell을 사용해 전진합니다. tail pointer는 descriptor를 순서대로 소비하고 `__be32` counter를 갱신하면서 전진합니다. doorbell과 counter는 모두 overflow되면 0으로 돌아갑니다.
각 queue의 buffer는 queue page list로 장치에 미리 등록해야 하며 packet data는 그 page들에만 둘 수 있습니다.
Transmit
gve는 transmit ring용 buffer를 FIFO에 map하고 packet을 NIC로 보내기 전에 FIFO로 복사합니다.
Receive
receive ring의 buffer는 descriptor ring과 길이가 같은 data ring에 배치되며 head pointer와 tail pointer는 두 ring에서 함께 전진합니다.
GQI Traffic Queues
------------------
GQI queues are composed of a descriptor ring and a buffer and are assigned to a
notification block.
The descriptor rings are power-of-two-sized ring buffers consisting of
fixed-size descriptors. They advance their head pointer using a __be32
doorbell located in Bar2. The tail pointers are advanced by consuming
descriptors in-order and updating a __be32 counter. Both the doorbell
and the counter overflow to zero.
Each queue's buffers must be registered in advance with the device as a
queue page list, and packet data can only be put in those pages.
Transmit
~~~~~~~~
gve maps the buffers for transmit rings into a FIFO and copies the packets
into the FIFO before sending them to the NIC.
Receive
~~~~~~~
The buffers for receive rings are put into a data ring that is the same
length as the descriptor ring and the head and tail pointers advance over
the rings together.
DQO traffic queue
145-175DQO traffic queue
- 모든 TX queue와 RX queue는 notification block 하나에 배정됩니다.
- 장치에 descriptor를 보내는 TX 및 RX buffer queue는 MMIO doorbell을 사용해 새 descriptor를 장치에 알립니다.
- 장치에서 descriptor를 받는 RX 및 TX completion queue는 `generation bit`로 장치가 descriptor를 채웠는지 판별합니다.
- 드라이버는 RX 및 TX completion queue가 넘치지 않도록 보장해야 합니다. 이를 위해 hardware에 게시하는 descriptor 수를 제한할 수 있습니다.
- TX packet에는 16-bit `completion_tag`, RX buffer에는 16-bit `buffer_id`가 있습니다. 이 값은 각각 TX completion queue와 RX queue에서 반환되어 어느 packet 또는 buffer의 처리가 끝났는지 드라이버에 알립니다.
generation bit는 모두 `current generation`으로 초기화됩니다. 장치는 수신 descriptor를 그 반전값인 `next generation`으로 채웁니다. ring이 한 바퀴 돌아 wrap되면 current generation과 next generation을 서로 바꿉니다.
Transmit
전송 전에 장치가 접근할 수 있도록 packet buffer를 DMA map합니다. packet 전송이 성공적으로 끝나면 buffer의 map을 해제합니다.
Receive
드라이버는 고정 크기 buffer를 RX buffer queue를 통해 hardware에 게시합니다. 연결된 RX queue에서 수신한 packet 하나가 여러 descriptor에 걸칠 수 있습니다.
DQO Traffic Queues
------------------
- Every TX and RX queue is assigned a notification block.
- TX and RX buffers queues, which send descriptors to the device, use MMIO
doorbells to notify the device of new descriptors.
- RX and TX completion queues, which receive descriptors from the device, use a
"generation bit" to know when a descriptor was populated by the device. The
driver initializes all bits with the "current generation". The device will
populate received descriptors with the "next generation" which is inverted
from the current generation. When the ring wraps, the current/next generation
are swapped.
- It's the driver's responsibility to ensure that the RX and TX completion
queues are not overrun. This can be accomplished by limiting the number of
descriptors posted to HW.
- TX packets have a 16 bit completion_tag and RX buffers have a 16 bit
buffer_id. These will be returned on the TX completion and RX queues
respectively to let the driver know which packet/buffer was completed.
Transmit
~~~~~~~~
A packet's buffers are DMA mapped for the device to access before transmission.
After the packet was successfully transmitted, the buffers are unmapped.
Receive
~~~~~~~
The driver posts fixed sized buffers to HW on the RX buffer queue. The packet
received on the associated RX queue may span multiple descriptors.
요약·해설
gve.rst:1-175GVE는 Google Compute Engine의 gVNIC를 위한 가상 Ethernet 드라이버입니다. 제어 경로는 MMIO register와 Admin Queue, data 경로는 GQI 또는 DQO queue 형식으로 나뉩니다. 선택한 형식에 따라 descriptor 구조와 byte order, buffer 수명 관리가 달라집니다.
장치가 노출하는 세 memory BAR의 책임을 구분합니다.
descriptor 형식과 addressing mode는 서로 다른 축입니다.
제어 경로와 각 data 경로의 endianness입니다.
doorbell과 event counter가 명령 제출과 완료를 동기화합니다.
management와 data queue notification의 후속 처리가 다릅니다.
descriptor와 buffer ring의 pointer가 함께 진행되는 방식을 요약합니다.
completion ring은 generation bit로 새 descriptor를 식별합니다.
TX와 RX에서 buffer를 식별하고 회수하는 방식입니다.