요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
===============
fwctl subsystem
===============
:Author: Jason Gunthorpe
Overview
========
Modern devices contain extensive amounts of FW, and in many cases, are largely
software-defined pieces of hardware. The evolution of this approach is largely a
reaction to Moore's Law where a chip tape out is now highly expensive, and the
chip design is extremely large. Replacing fixed HW logic with a flexible and
tightly coupled FW/HW combination is an effective risk mitigation against chip
respin. Problems in the HW design can be counteracted in device FW. This is
especially true for devices which present a stable and backwards compatible
interface to the operating system driver (such as NVMe).
The FW layer in devices has grown to incredible size and devices frequently
integrate clusters of fast processors to run it. For example, mlx5 devices have
over 30MB of FW code, and big configurations operate with over 1GB of FW managed
runtime state.
The availability of such a flexible layer has created quite a variety in the
industry where single pieces of silicon are now configurable software-defined
devices and can operate in substantially different ways depending on the need.
Further, we often see cases where specific sites wish to operate devices in ways
that are highly specialized and require applications that have been tailored to
their unique configuration.
Further, devices have become multi-functional and integrated to the point they
no longer fit neatly into the kernel's division of subsystems. Modern
multi-functional devices have drivers, such as bnxt/ice/mlx5/pds, that span many
subsystems while sharing the underlying hardware using the auxiliary device
system.
All together this creates a challenge for the operating system, where devices
have an expansive FW environment that needs robust device-specific debugging
support, and FW-driven functionality that is not well suited to “generic”
interfaces. fwctl seeks to allow access to the full device functionality from
user space in the areas of debuggability, management, and first-boot/nth-boot
provisioning.
fwctl is aimed at the common device design pattern where the OS and FW
communicate via an RPC message layer constructed with a queue or mailbox scheme.
In this case the driver will typically have some layer to deliver RPC messages
and collect RPC responses from device FW. The in-kernel subsystem drivers that
operate the device for its primary purposes will use these RPCs to build their
drivers, but devices also usually have a set of ancillary RPCs that don't really
fit into any specific subsystem. For example, a HW RAID controller is primarily
operated by the block layer but also comes with a set of RPCs to administer the
construction of drives within the HW RAID.
In the past when devices were more single function, individual subsystems would
grow different approaches to solving some of these common problems. For instance,
monitoring device health, manipulating its FLASH, debugging the FW,
provisioning, all have various unique interfaces across the kernel.
fwctl's purpose is to define a common set of limited rules, described below,
that allow user space to securely construct and execute RPCs inside device FW.
The rules serve as an agreement between the operating system and FW on how to
correctly design the RPC interface. As a uAPI the subsystem provides a thin
layer of discovery and a generic uAPI to deliver the RPCs and collect the
response. It supports a system of user space libraries and tools which will
use this interface to control the device using the device native protocols.
Scope of Action
---------------
fwctl drivers are strictly restricted to being a way to operate the device FW.
It is not an avenue to access random kernel internals, or other operating system
SW states.
fwctl instances must operate on a well-defined device function, and the device
should have a well-defined security model for what scope within the physical
device the function is permitted to access. For instance, the most complex PCIe
device today may broadly have several function-level scopes:
1. A privileged function with full access to the on-device global state and
configuration
2. Multiple hypervisor functions with control over itself and child functions
used with VMs
3. Multiple VM functions tightly scoped within the VM
The device may create a logical parent/child relationship between these scopes.
For instance, a child VM's FW may be within the scope of the hypervisor FW. It is
quite common in the VFIO world that the hypervisor environment has a complex
provisioning/profiling/configuration responsibility for the function VFIO
assigns to the VM.
Further, within the function, devices often have RPC commands that fall within
some general scopes of action (see enum fwctl_rpc_scope):
1. Access to function & child configuration, FLASH, etc. that becomes live at a
function reset. Access to function & child runtime configuration that is
transparent or non-disruptive to any driver or VM.
2. Read-only access to function debug information that may report on FW objects
in the function & child, including FW objects owned by other kernel
subsystems.
3. Write access to function & child debug information strictly compatible with
the principles of kernel lockdown and kernel integrity protection. Triggers
a kernel taint.
4. Full debug device access. Triggers a kernel taint, requires CAP_SYS_RAWIO.
User space will provide a scope label on each RPC and the kernel must enforce the
above CAPs and taints based on that scope. A combination of kernel and FW can
enforce that RPCs are placed in the correct scope by user space.
Disallowed behavior
-------------------
There are many things this interface must not allow user space to do (without a
taint or CAP), broadly derived from the principles of kernel lockdown. Some
examples:
1. DMA to/from arbitrary memory, hang the system, compromise FW integrity with
untrusted code, or otherwise compromise device or system security and
integrity.
2. Provide an abnormal “back door” to kernel drivers. No manipulation of kernel
objects owned by kernel drivers.
3. Directly configure or otherwise control kernel drivers. A subsystem kernel
driver can react to the device configuration at function reset/driver load
time, but otherwise must not be coupled to fwctl.
4. Operate the HW in a way that overlaps with the core purpose of another
primary kernel subsystem, such as read/write to LBAs, send/receive of
network packets, or operate an accelerator's data plane.
fwctl is not a replacement for device direct access subsystems like uacce or
VFIO.
Operations exposed through fwctl's non-tainting interfaces should be fully
sharable with other users of the device. For instance, exposing a RPC through
fwctl should never prevent a kernel subsystem from also concurrently using that
same RPC or hardware unit down the road. In such cases fwctl will be less
important than proper kernel subsystems that eventually emerge. Mistakes in this
area resulting in clashes will be resolved in favour of a kernel implementation.
fwctl User API
==============
.. kernel-doc:: include/uapi/fwctl/fwctl.h
.. kernel-doc:: include/uapi/fwctl/mlx5.h
.. kernel-doc:: include/uapi/fwctl/pds.h
sysfs Class
-----------
fwctl has a sysfs class (/sys/class/fwctl/fwctlNN/) and character devices
(/dev/fwctl/fwctlNN) with a simple numbered scheme. The character device
operates the iotcl uAPI described above.
fwctl devices can be related to driver components in other subsystems through
sysfs::
$ ls /sys/class/fwctl/fwctl0/device/infiniband/
ibp0s10f0
$ ls /sys/class/infiniband/ibp0s10f0/device/fwctl/
fwctl0/
$ ls /sys/devices/pci0000:00/0000:00:0a.0/fwctl/fwctl0
dev device power subsystem uevent
User space Community
--------------------
Drawing inspiration from nvme-cli, participating in the kernel side must come
with a user space in a common TBD git tree, at a minimum to usefully operate the
kernel driver. Providing such an implementation is a pre-condition to merging a
kernel driver.
The goal is to build user space community around some of the shared problems
we all have, and ideally develop some common user space programs with some
starting themes of:
- Device in-field debugging
- HW provisioning
- VFIO child device profiling before VM boot
- Confidential Compute topics (attestation, secure provisioning)
that stretch across all subsystems in the kernel. fwupd is a great example of
how an excellent user space experience can emerge out of kernel-side diversity.
fwctl Kernel API
================
.. kernel-doc:: drivers/fwctl/main.c
:export:
.. kernel-doc:: include/linux/fwctl.h
fwctl Driver design
-------------------
In many cases a fwctl driver is going to be part of a larger cross-subsystem
device possibly using the auxiliary_device mechanism. In that case several
subsystems are going to be sharing the same device and FW interface layer so the
device design must already provide for isolation and cooperation between kernel
subsystems. fwctl should fit into that same model.
Part of the driver should include a description of how its scope restrictions
and security model work. The driver and FW together must ensure that RPCs
provided by user space are mapped to the appropriate scope. If the validation is
done in the driver then the validation can read a 'command effects' report from
the device, or hardwire the enforcement. If the validation is done in the FW,
then the driver should pass the fwctl_rpc_scope to the FW along with the command.
The driver and FW must cooperate to ensure that either fwctl cannot allocate
any FW resources, or any resources it does allocate are freed on FD closure. A
driver primarily constructed around FW RPCs may find that its core PCI function
and RPC layer belongs under fwctl with auxiliary devices connecting to other
subsystems.
Each device type must be mindful of Linux's philosophy for stable ABI. The FW
RPC interface does not have to meet a strictly stable ABI, but it does need to
meet an expectation that user space tools that are deployed and in significant
use don't needlessly break. FW upgrade and kernel upgrade should keep widely
deployed tooling working.
Development and debugging focused RPCs under more permissive scopes can have
less stability if the tools using them are only run under exceptional
circumstances and not for every day use of the device. Debugging tools may even
require exact version matching as they may require something similar to DWARF
debug information from the FW binary.
Security Response
=================
The kernel remains the gatekeeper for this interface. If violations of the
scopes, security or isolation principles are found, we have options to let
devices fix them with a FW update, push a kernel patch to parse and block RPC
commands or push a kernel patch to block entire firmware versions/devices.
While the kernel can always directly parse and restrict RPCs, it is expected
that the existing kernel pattern of allowing drivers to delegate validation to
FW to be a useful design.
Existing Similar Examples
=========================
The approach described in this document is not a new idea. Direct, or near
direct device access has been offered by the kernel in different areas for
decades. With more devices wanting to follow this design pattern it is becoming
clear that it is not entirely well understood and, more importantly, the
security considerations are not well defined or agreed upon.
Some examples:
- HW RAID controllers. This includes RPCs to do things like compose drives into
a RAID volume, configure RAID parameters, monitor the HW and more.
- Baseboard managers. RPCs for configuring settings in the device and more.
- NVMe vendor command capsules. nvme-cli provides access to some monitoring
functions that different products have defined, but more exist.
- CXL also has a NVMe-like vendor command system.
- DRM allows user space drivers to send commands to the device via kernel
mediation.
- RDMA allows user space drivers to directly push commands to the device
without kernel involvement.
- Various “raw” APIs, raw HID (SDL2), raw USB, NVMe Generic Interface, etc.
The first 4 are examples of areas that fwctl intends to cover. The latter three
are examples of disallowed behavior as they fully overlap with the primary purpose
of a kernel subsystem.
Some key lessons learned from these past efforts are the importance of having a
common user space project to use as a pre-condition for obtaining a kernel
driver. Developing good community around useful software in user space is key to
getting companies to fund participation to enable their products.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
배경과 목적
1-68현대 device는 방대한 firmware를 포함하며 상당 부분 software-defined hardware로 구성됩니다. Chip tape-out 비용과 설계 규모가 커진 상황에서 fixed hardware logic을 유연하고 긴밀히 결합한 FW/HW로 바꾸면 chip respin 위험을 줄이고 hardware 설계 문제를 device firmware로 보완할 수 있습니다. NVMe처럼 operating-system driver에 stable하고 backward-compatible한 interface를 제공하는 device에서 특히 유효합니다.
Device firmware는 매우 커졌고 빠른 processor cluster를 통합해 실행하는 경우가 많습니다. 예를 들어 mlx5 device에는 30MB가 넘는 firmware code가 있고 큰 configuration은 1GB가 넘는 firmware-managed runtime state로 동작합니다.
이 유연성 때문에 하나의 silicon이 요구에 따라 크게 다른 방식으로 동작하는 configurable software-defined device가 되었습니다. 특정 site가 고유 configuration에 맞춘 application으로 device를 전문적인 방식으로 운영하는 경우도 많습니다.
Device는 multi-functional하고 통합되어 kernel subsystem 구분에 깔끔하게 들어맞지 않습니다. `bnxt`, `ice`, `mlx5`, `pds` 같은 driver는 auxiliary device system으로 underlying hardware를 공유하면서 여러 subsystem에 걸쳐 있습니다.
따라서 operating system은 넓은 firmware environment를 위한 device-specific debugging, generic interface에 맞지 않는 firmware-driven function, management와 first-boot/nth-boot provisioning을 지원해야 합니다. `fwctl`은 이런 기능을 userspace에 제공하려 합니다.
fwctl은 OS와 firmware가 queue 또는 mailbox 기반 RPC message layer로 통신하는 일반적인 device design을 대상으로 합니다. Kernel subsystem driver가 primary function을 구현할 때도 이 RPC를 쓰지만, hardware RAID의 drive 구성 관리처럼 어느 특정 subsystem에도 속하지 않는 ancillary RPC도 있습니다.
과거 single-function device 시절에는 health monitoring, FLASH 조작, firmware debugging, provisioning에 subsystem별로 서로 다른 interface가 생겼습니다.
fwctl의 목적은 userspace가 device firmware 안에서 RPC를 안전하게 구성하고 실행하도록 제한된 공통 rule을 정하는 것입니다. UAPI는 discovery와 RPC request/response 전달을 위한 얇은 generic layer를 제공하고, userspace library와 tool은 device-native protocol로 device를 제어합니다.
Device-native RPC를 kernel이 중재해 userspace에 연결합니다.
.. SPDX-License-Identifier: GPL-2.0
===============
fwctl subsystem
===============
:Author: Jason Gunthorpe
Overview
========
Modern devices contain extensive amounts of FW, and in many cases, are largely
software-defined pieces of hardware. The evolution of this approach is largely a
reaction to Moore's Law where a chip tape out is now highly expensive, and the
chip design is extremely large. Replacing fixed HW logic with a flexible and
tightly coupled FW/HW combination is an effective risk mitigation against chip
respin. Problems in the HW design can be counteracted in device FW. This is
especially true for devices which present a stable and backwards compatible
interface to the operating system driver (such as NVMe).
The FW layer in devices has grown to incredible size and devices frequently
integrate clusters of fast processors to run it. For example, mlx5 devices have
over 30MB of FW code, and big configurations operate with over 1GB of FW managed
runtime state.
The availability of such a flexible layer has created quite a variety in the
industry where single pieces of silicon are now configurable software-defined
devices and can operate in substantially different ways depending on the need.
Further, we often see cases where specific sites wish to operate devices in ways
that are highly specialized and require applications that have been tailored to
their unique configuration.
Further, devices have become multi-functional and integrated to the point they
no longer fit neatly into the kernel's division of subsystems. Modern
multi-functional devices have drivers, such as bnxt/ice/mlx5/pds, that span many
subsystems while sharing the underlying hardware using the auxiliary device
system.
All together this creates a challenge for the operating system, where devices
have an expansive FW environment that needs robust device-specific debugging
support, and FW-driven functionality that is not well suited to “generic”
interfaces. fwctl seeks to allow access to the full device functionality from
user space in the areas of debuggability, management, and first-boot/nth-boot
provisioning.
fwctl is aimed at the common device design pattern where the OS and FW
communicate via an RPC message layer constructed with a queue or mailbox scheme.
In this case the driver will typically have some layer to deliver RPC messages
and collect RPC responses from device FW. The in-kernel subsystem drivers that
operate the device for its primary purposes will use these RPCs to build their
drivers, but devices also usually have a set of ancillary RPCs that don't really
fit into any specific subsystem. For example, a HW RAID controller is primarily
operated by the block layer but also comes with a set of RPCs to administer the
construction of drives within the HW RAID.
In the past when devices were more single function, individual subsystems would
grow different approaches to solving some of these common problems. For instance,
monitoring device health, manipulating its FLASH, debugging the FW,
provisioning, all have various unique interfaces across the kernel.
fwctl's purpose is to define a common set of limited rules, described below,
that allow user space to securely construct and execute RPCs inside device FW.
The rules serve as an agreement between the operating system and FW on how to
correctly design the RPC interface. As a uAPI the subsystem provides a thin
layer of discovery and a generic uAPI to deliver the RPCs and collect the
response. It supports a system of user space libraries and tools which will
use this interface to control the device using the device native protocols.
동작 범위와 RPC scope
69-115fwctl driver는 device firmware를 운영하는 수단으로 엄격히 제한됩니다. 임의의 kernel internal이나 다른 operating-system software state에 접근하는 통로가 아닙니다.
각 fwctl instance는 잘 정의된 device function에서 동작해야 하고, physical device 안에서 그 function이 접근할 수 있는 범위를 정하는 security model이 있어야 합니다.
복잡한 PCIe device의 function-level scope에는 on-device global state와 configuration 전체에 접근하는 privileged function, 자신과 VM용 child function을 제어하는 여러 hypervisor function, 각 VM 안으로 엄격히 제한된 여러 VM function이 있을 수 있습니다.
Device는 이런 scope 사이에 logical parent/child relation을 만들 수 있습니다. Child VM firmware가 hypervisor firmware scope 안에 들어갈 수 있고, VFIO 환경에서는 hypervisor가 VM에 assign할 function의 provisioning, profiling, configuration을 담당하는 경우가 흔합니다.
Function 안의 RPC command는 `enum fwctl_rpc_scope`가 나타내는 일반 action scope로 나뉩니다.
첫 범주는 function reset 때 활성화되는 function/child configuration과 FLASH access, 또는 driver와 VM에 transparent하거나 non-disruptive한 runtime configuration입니다.
둘째는 function과 child의 firmware object를 보고할 수 있는 read-only debug information이며, 다른 kernel subsystem이 소유한 object도 포함할 수 있습니다.
셋째는 kernel lockdown과 kernel integrity protection 원칙에 엄격히 맞는 function/child debug write access이며 kernel taint를 일으킵니다. 넷째 full debug device access도 kernel taint를 일으키고 `CAP_SYS_RAWIO`가 필요합니다.
Userspace는 모든 RPC에 scope label을 제공하고 kernel은 그 scope에 맞는 capability와 taint를 강제해야 합니다. Kernel과 firmware를 함께 사용해 userspace RPC가 올바른 scope에 놓였는지 검증할 수 있습니다.
Physical device 안의 권한 계층 예시입니다.
권한이 커질수록 taint와 capability 요구가 강해집니다.
Scope of Action
---------------
fwctl drivers are strictly restricted to being a way to operate the device FW.
It is not an avenue to access random kernel internals, or other operating system
SW states.
fwctl instances must operate on a well-defined device function, and the device
should have a well-defined security model for what scope within the physical
device the function is permitted to access. For instance, the most complex PCIe
device today may broadly have several function-level scopes:
1. A privileged function with full access to the on-device global state and
configuration
2. Multiple hypervisor functions with control over itself and child functions
used with VMs
3. Multiple VM functions tightly scoped within the VM
The device may create a logical parent/child relationship between these scopes.
For instance, a child VM's FW may be within the scope of the hypervisor FW. It is
quite common in the VFIO world that the hypervisor environment has a complex
provisioning/profiling/configuration responsibility for the function VFIO
assigns to the VM.
Further, within the function, devices often have RPC commands that fall within
some general scopes of action (see enum fwctl_rpc_scope):
1. Access to function & child configuration, FLASH, etc. that becomes live at a
function reset. Access to function & child runtime configuration that is
transparent or non-disruptive to any driver or VM.
2. Read-only access to function debug information that may report on FW objects
in the function & child, including FW objects owned by other kernel
subsystems.
3. Write access to function & child debug information strictly compatible with
the principles of kernel lockdown and kernel integrity protection. Triggers
a kernel taint.
4. Full debug device access. Triggers a kernel taint, requires CAP_SYS_RAWIO.
User space will provide a scope label on each RPC and the kernel must enforce the
above CAPs and taints based on that scope. A combination of kernel and FW can
enforce that RPCs are placed in the correct scope by user space.
허용하지 않는 동작
116-147Kernel lockdown 원칙에 따라 taint나 capability 없이 userspace가 해서는 안 되는 동작이 많습니다.
임의 memory와 DMA를 주고받거나 system을 hang시키고, untrusted code로 firmware integrity를 훼손하거나 device/system security와 integrity를 침해해서는 안 됩니다.
Kernel driver로 향하는 비정상적인 back door를 제공하거나 kernel driver가 소유한 kernel object를 조작해서는 안 됩니다.
Kernel driver를 직접 configure하거나 control해서는 안 됩니다. Subsystem driver는 function reset 또는 driver load 시 device configuration에 반응할 수 있지만 그 외에는 fwctl과 결합되면 안 됩니다.
LBA read/write, network packet send/receive, accelerator data plane operation처럼 다른 primary kernel subsystem의 core purpose와 겹치게 hardware를 운영해서는 안 됩니다. fwctl은 `uacce`나 `VFIO` 같은 direct-access subsystem의 대체물이 아닙니다.
Non-tainting interface로 노출한 operation은 device의 다른 사용자와 완전히 공유할 수 있어야 합니다. fwctl RPC가 같은 RPC나 hardware unit을 kernel subsystem이 동시에 사용하는 일을 막아서는 안 되며, 충돌이 생기면 적절한 kernel implementation을 우선합니다.
fwctl이 넘지 않아야 할 subsystem 및 security 경계입니다.
Disallowed behavior
-------------------
There are many things this interface must not allow user space to do (without a
taint or CAP), broadly derived from the principles of kernel lockdown. Some
examples:
1. DMA to/from arbitrary memory, hang the system, compromise FW integrity with
untrusted code, or otherwise compromise device or system security and
integrity.
2. Provide an abnormal “back door” to kernel drivers. No manipulation of kernel
objects owned by kernel drivers.
3. Directly configure or otherwise control kernel drivers. A subsystem kernel
driver can react to the device configuration at function reset/driver load
time, but otherwise must not be coupled to fwctl.
4. Operate the HW in a way that overlaps with the core purpose of another
primary kernel subsystem, such as read/write to LBAs, send/receive of
network packets, or operate an accelerator's data plane.
fwctl is not a replacement for device direct access subsystems like uacce or
VFIO.
Operations exposed through fwctl's non-tainting interfaces should be fully
sharable with other users of the device. For instance, exposing a RPC through
fwctl should never prevent a kernel subsystem from also concurrently using that
same RPC or hardware unit down the road. In such cases fwctl will be less
important than proper kernel subsystems that eventually emerge. Mistakes in this
area resulting in clashes will be resolved in favour of a kernel implementation.
Userspace UAPI
148-154fwctl userspace API 문서는 `include/uapi/fwctl/fwctl.h`, `include/uapi/fwctl/mlx5.h`, `include/uapi/fwctl/pds.h`의 kernel-doc에서 생성됩니다.
공통 및 device-specific ABI source입니다.
fwctl User API
==============
.. kernel-doc:: include/uapi/fwctl/fwctl.h
.. kernel-doc:: include/uapi/fwctl/mlx5.h
.. kernel-doc:: include/uapi/fwctl/pds.h
sysfs class와 character device
155-173fwctl은 단순 번호 체계의 sysfs class `/sys/class/fwctl/fwctlNN/`와 character device `/dev/fwctl/fwctlNN`을 제공합니다. Character device는 앞에서 설명한 ioctl UAPI를 실행합니다.
sysfs link를 통해 fwctl device와 다른 subsystem의 driver component 관계를 추적할 수 있습니다. 예제는 `/sys/class/fwctl/fwctl0/device/infiniband/`, `/sys/class/infiniband/ibp0s10f0/device/fwctl/`, `/sys/devices/pci0000:00/0000:00:0a.0/fwctl/fwctl0`의 양방향 관계와 device attribute를 보여 줍니다.
fwctl instance와 subsystem device를 양쪽에서 찾을 수 있습니다.
sysfs Class
-----------
fwctl has a sysfs class (/sys/class/fwctl/fwctlNN/) and character devices
(/dev/fwctl/fwctlNN) with a simple numbered scheme. The character device
operates the iotcl uAPI described above.
fwctl devices can be related to driver components in other subsystems through
sysfs::
$ ls /sys/class/fwctl/fwctl0/device/infiniband/
ibp0s10f0
$ ls /sys/class/infiniband/ibp0s10f0/device/fwctl/
fwctl0/
$ ls /sys/devices/pci0000:00/0000:00:0a.0/fwctl/fwctl0
dev device power subsystem uevent
Userspace community
174-196`nvme-cli`에서 영감을 받아 kernel driver를 제안하려면 최소한 driver를 유용하게 운영하는 userspace implementation을 공통 TBD git tree에 제공해야 합니다. 이것이 kernel driver merge의 선행 조건입니다.
목표는 여러 subsystem에 걸친 공통 문제를 중심으로 userspace community와 program을 만드는 것입니다. 시작 주제는 in-field device debugging, hardware provisioning, VM boot 전 VFIO child-device profiling, attestation과 secure provisioning 같은 Confidential Compute입니다.
`fwupd`는 kernel 쪽 다양성 위에서 훌륭한 userspace experience가 만들어질 수 있음을 보여 주는 예입니다.
fwctl project가 함께 다루려는 영역입니다.
User space Community
--------------------
Drawing inspiration from nvme-cli, participating in the kernel side must come
with a user space in a common TBD git tree, at a minimum to usefully operate the
kernel driver. Providing such an implementation is a pre-condition to merging a
kernel driver.
The goal is to build user space community around some of the shared problems
we all have, and ideally develop some common user space programs with some
starting themes of:
- Device in-field debugging
- HW provisioning
- VFIO child device profiling before VM boot
- Confidential Compute topics (attestation, secure provisioning)
that stretch across all subsystems in the kernel. fwupd is a great example of
how an excellent user space experience can emerge out of kernel-side diversity.
Kernel API
197-203Kernel API reference는 `drivers/fwctl/main.c`의 exported kernel-doc과 `include/linux/fwctl.h` declaration에서 생성됩니다.
Implementation과 public kernel header입니다.
fwctl Kernel API
================
.. kernel-doc:: drivers/fwctl/main.c
:export:
.. kernel-doc:: include/linux/fwctl.h
Driver 설계
204-237fwctl driver는 `auxiliary_device` mechanism을 쓰는 큰 cross-subsystem device의 일부인 경우가 많습니다. 여러 subsystem이 같은 device와 firmware interface layer를 공유하므로 device 설계가 이미 isolation과 cooperation을 제공해야 하며 fwctl도 같은 model에 들어맞아야 합니다.
Driver 문서에는 scope restriction과 security model의 동작 방식을 포함해야 합니다. Driver와 firmware는 userspace RPC가 적절한 scope에 mapping되도록 함께 보장해야 합니다.
Driver가 validation을 수행한다면 device의 command-effects report를 읽거나 enforcement를 hardwire할 수 있습니다. Firmware가 validation한다면 driver는 command와 함께 `fwctl_rpc_scope`를 firmware에 전달해야 합니다.
Driver와 firmware는 fwctl이 firmware resource를 allocate할 수 없게 하거나, allocate한 모든 resource를 file descriptor close 때 free하도록 협력해야 합니다. Firmware RPC 중심 driver라면 core PCI function과 RPC layer를 fwctl 아래 두고 auxiliary device로 다른 subsystem에 연결할 수 있습니다.
각 device type은 Linux stable ABI 철학을 고려해야 합니다. Firmware RPC interface가 엄격히 stable할 필요는 없지만 널리 배포되어 많이 쓰는 userspace tool이 불필요하게 깨져서는 안 됩니다. Firmware와 kernel upgrade 뒤에도 그런 tool이 동작해야 합니다.
더 permissive한 scope 아래 development/debugging RPC는 일상 사용이 아니라 exceptional circumstance에서만 쓰인다면 안정성이 낮아도 됩니다. Firmware binary의 DWARF-like debug information에 의존하는 debugging tool은 exact version matching을 요구할 수도 있습니다.
Driver 또는 firmware가 command scope를 검증합니다.
사용 빈도와 목적에 따른 안정성 차이입니다.
fwctl Driver design
-------------------
In many cases a fwctl driver is going to be part of a larger cross-subsystem
device possibly using the auxiliary_device mechanism. In that case several
subsystems are going to be sharing the same device and FW interface layer so the
device design must already provide for isolation and cooperation between kernel
subsystems. fwctl should fit into that same model.
Part of the driver should include a description of how its scope restrictions
and security model work. The driver and FW together must ensure that RPCs
provided by user space are mapped to the appropriate scope. If the validation is
done in the driver then the validation can read a 'command effects' report from
the device, or hardwire the enforcement. If the validation is done in the FW,
then the driver should pass the fwctl_rpc_scope to the FW along with the command.
The driver and FW must cooperate to ensure that either fwctl cannot allocate
any FW resources, or any resources it does allocate are freed on FD closure. A
driver primarily constructed around FW RPCs may find that its core PCI function
and RPC layer belongs under fwctl with auxiliary devices connecting to other
subsystems.
Each device type must be mindful of Linux's philosophy for stable ABI. The FW
RPC interface does not have to meet a strictly stable ABI, but it does need to
meet an expectation that user space tools that are deployed and in significant
use don't needlessly break. FW upgrade and kernel upgrade should keep widely
deployed tooling working.
Development and debugging focused RPCs under more permissive scopes can have
less stability if the tools using them are only run under exceptional
circumstances and not for every day use of the device. Debugging tools may even
require exact version matching as they may require something similar to DWARF
debug information from the FW binary.
Security response
238-249Kernel은 이 interface의 gatekeeper입니다. Scope, security, isolation 원칙 위반이 발견되면 device firmware update로 수정하거나, kernel patch로 특정 RPC command를 parse하고 block하거나, firmware version 또는 device 전체를 block할 수 있습니다.
Kernel이 RPC를 직접 parse하고 제한할 수 있지만, 기존 driver pattern처럼 validation을 firmware에 위임하는 설계도 유용할 것으로 봅니다.
문제 범위에 따라 firmware와 kernel 차단을 단계적으로 적용합니다.
Security Response
=================
The kernel remains the gatekeeper for this interface. If violations of the
scopes, security or isolation principles are found, we have options to let
devices fix them with a FW update, push a kernel patch to parse and block RPC
commands or push a kernel patch to block entire firmware versions/devices.
While the kernel can always directly parse and restrict RPCs, it is expected
that the existing kernel pattern of allowing drivers to delegate validation to
FW to be a useful design.
기존 유사 사례와 교훈
250-286직접 또는 거의 직접적인 device access는 새로운 개념이 아니며 kernel은 수십 년 동안 여러 영역에서 제공해 왔습니다. 더 많은 device가 이 pattern을 따르면서 design과 security consideration을 명확히 정의하고 합의할 필요가 커졌습니다.
fwctl이 다루려는 사례는 RAID volume 구성과 parameter/health 관리 RPC를 가진 hardware RAID controller, device setting을 구성하는 baseboard manager, `nvme-cli`가 일부 monitoring function을 제공하는 NVMe vendor command capsule, NVMe와 비슷한 vendor command system을 가진 CXL입니다.
반대로 DRM userspace driver command, kernel 개입 없이 직접 command를 넣는 RDMA userspace driver, raw HID/USB와 NVMe Generic Interface 같은 raw API는 primary kernel subsystem의 목적과 완전히 겹치므로 fwctl에서 허용하지 않는 동작의 사례입니다.
과거 사례의 핵심 교훈은 kernel driver를 얻기 위한 선행 조건으로 공통 userspace project가 중요하다는 것입니다. 유용한 userspace software 주위에 좋은 community를 만들어야 회사가 제품 지원 참여에 투자할 동기가 생깁니다.
앞 네 사례는 대상이고 뒤 세 사례는 subsystem 중복으로 제외됩니다.
Existing Similar Examples
=========================
The approach described in this document is not a new idea. Direct, or near
direct device access has been offered by the kernel in different areas for
decades. With more devices wanting to follow this design pattern it is becoming
clear that it is not entirely well understood and, more importantly, the
security considerations are not well defined or agreed upon.
Some examples:
- HW RAID controllers. This includes RPCs to do things like compose drives into
a RAID volume, configure RAID parameters, monitor the HW and more.
- Baseboard managers. RPCs for configuring settings in the device and more.
- NVMe vendor command capsules. nvme-cli provides access to some monitoring
functions that different products have defined, but more exist.
- CXL also has a NVMe-like vendor command system.
- DRM allows user space drivers to send commands to the device via kernel
mediation.
- RDMA allows user space drivers to directly push commands to the device
without kernel involvement.
- Various “raw” APIs, raw HID (SDL2), raw USB, NVMe Generic Interface, etc.
The first 4 are examples of areas that fwctl intends to cover. The latter three
are examples of disallowed behavior as they fully overlap with the primary purpose
of a kernel subsystem.
Some key lessons learned from these past efforts are the importance of having a
common user space project to use as a pre-condition for obtaining a kernel
driver. Developing good community around useful software in user space is key to
getting companies to fund participation to enable their products.
요약·해설
fwctl.rst:1-286Device-native firmware RPC를 안전하게 중재하는 fwctl의 scope, 금지 경계, UAPI, driver 및 security-response 설계를 설명합니다.
원문의 RPC scope, capability, taint, ABI symbol, sysfs path, code와 줄 좌표를 보존해 전문을 번역했습니다.