요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
===========================
Hypercall Op-codes (hcalls)
===========================
Overview
=========
Virtualization on 64-bit Power Book3S Platforms is based on the PAPR
specification [1]_ which describes the run-time environment for a guest
operating system and how it should interact with the hypervisor for
privileged operations. Currently there are two PAPR compliant hypervisors:
- **IBM PowerVM (PHYP)**: IBM's proprietary hypervisor that supports AIX,
IBM-i and Linux as supported guests (termed as Logical Partitions
or LPARS). It supports the full PAPR specification.
- **Qemu/KVM**: Supports PPC64 linux guests running on a PPC64 linux host.
Though it only implements a subset of PAPR specification called LoPAPR [2]_.
On PPC64 arch a guest kernel running on top of a PAPR hypervisor is called
a *pSeries guest*. A pseries guest runs in a supervisor mode (HV=0) and must
issue hypercalls to the hypervisor whenever it needs to perform an action
that is hypervisor privileged [3]_ or for other services managed by the
hypervisor.
Hence a Hypercall (hcall) is essentially a request by the pseries guest
asking hypervisor to perform a privileged operation on behalf of the guest. The
guest issues a with necessary input operands. The hypervisor after performing
the privilege operation returns a status code and output operands back to the
guest.
HCALL ABI
=========
The ABI specification for a hcall between a pseries guest and PAPR hypervisor
is covered in section 14.5.3 of ref [2]_. Switch to the Hypervisor context is
done via the instruction **HVCS** that expects the Opcode for hcall is set in *r3*
and any in-arguments for the hcall are provided in registers *r4-r12*. If values
have to be passed through a memory buffer, the data stored in that buffer should be
in Big-endian byte order.
Once control returns back to the guest after hypervisor has serviced the
'HVCS' instruction the return value of the hcall is available in *r3* and any
out values are returned in registers *r4-r12*. Again like in case of in-arguments,
any out values stored in a memory buffer will be in Big-endian byte order.
Powerpc arch code provides convenient wrappers named **plpar_hcall_xxx** defined
in a arch specific header [4]_ to issue hcalls from the linux kernel
running as pseries guest.
Register Conventions
====================
Any hcall should follow same register convention as described in section 2.2.1.1
of "64-Bit ELF V2 ABI Specification: Power Architecture"[5]_. Table below
summarizes these conventions:
+----------+----------+-------------------------------------------+
| Register |Volatile | Purpose |
| Range |(Y/N) | |
+==========+==========+===========================================+
| r0 | Y | Optional-usage |
+----------+----------+-------------------------------------------+
| r1 | N | Stack Pointer |
+----------+----------+-------------------------------------------+
| r2 | N | TOC |
+----------+----------+-------------------------------------------+
| r3 | Y | hcall opcode/return value |
+----------+----------+-------------------------------------------+
| r4-r10 | Y | in and out values |
+----------+----------+-------------------------------------------+
| r11 | Y | Optional-usage/Environmental pointer |
+----------+----------+-------------------------------------------+
| r12 | Y | Optional-usage/Function entry address at |
| | | global entry point |
+----------+----------+-------------------------------------------+
| r13 | N | Thread-Pointer |
+----------+----------+-------------------------------------------+
| r14-r31 | N | Local Variables |
+----------+----------+-------------------------------------------+
| LR | Y | Link Register |
+----------+----------+-------------------------------------------+
| CTR | Y | Loop Counter |
+----------+----------+-------------------------------------------+
| XER | Y | Fixed-point exception register. |
+----------+----------+-------------------------------------------+
| CR0-1 | Y | Condition register fields. |
+----------+----------+-------------------------------------------+
| CR2-4 | N | Condition register fields. |
+----------+----------+-------------------------------------------+
| CR5-7 | Y | Condition register fields. |
+----------+----------+-------------------------------------------+
| Others | N | |
+----------+----------+-------------------------------------------+
DRC & DRC Indexes
=================
::
DR1 Guest
+--+ +------------+ +---------+
| | <----> | | | User |
+--+ DRC1 | | DRC | Space |
| PAPR | Index +---------+
DR2 | Hypervisor | | |
+--+ | | <-----> | Kernel |
| | <----> | | Hcall | |
+--+ DRC2 +------------+ +---------+
PAPR hypervisor terms shared hardware resources like PCI devices, NVDIMMs etc
available for use by LPARs as Dynamic Resource (DR). When a DR is allocated to
an LPAR, PHYP creates a data-structure called Dynamic Resource Connector (DRC)
to manage LPAR access. An LPAR refers to a DRC via an opaque 32-bit number
called DRC-Index. The DRC-index value is provided to the LPAR via device-tree
where its present as an attribute in the device tree node associated with the
DR.
HCALL Return-values
===================
After servicing the hcall, hypervisor sets the return-value in *r3* indicating
success or failure of the hcall. In case of a failure an error code indicates
the cause for error. These codes are defined and documented in arch specific
header [4]_.
In some cases a hcall can potentially take a long time and need to be issued
multiple times in order to be completely serviced. These hcalls will usually
accept an opaque value *continue-token* within there argument list and a
return value of *H_CONTINUE* indicates that hypervisor hasn't still finished
servicing the hcall yet.
To make such hcalls the guest need to set *continue-token == 0* for the
initial call and use the hypervisor returned value of *continue-token*
for each subsequent hcall until hypervisor returns a non *H_CONTINUE*
return value.
HCALL Op-codes
==============
Below is a partial list of HCALLs that are supported by PHYP. For the
corresponding opcode values please look into the arch specific header [4]_:
**H_SCM_READ_METADATA**
| Input: *drcIndex, offset, buffer-address, numBytesToRead*
| Out: *numBytesRead*
| Return Value: *H_Success, H_Parameter, H_P2, H_P3, H_Hardware*
Given a DRC Index of an NVDIMM, read N-bytes from the metadata area
associated with it, at a specified offset and copy it to provided buffer.
The metadata area stores configuration information such as label information,
bad-blocks etc. The metadata area is located out-of-band of NVDIMM storage
area hence a separate access semantics is provided.
**H_SCM_WRITE_METADATA**
| Input: *drcIndex, offset, data, numBytesToWrite*
| Out: *None*
| Return Value: *H_Success, H_Parameter, H_P2, H_P4, H_Hardware*
Given a DRC Index of an NVDIMM, write N-bytes to the metadata area
associated with it, at the specified offset and from the provided buffer.
**H_SCM_BIND_MEM**
| Input: *drcIndex, startingScmBlockIndex, numScmBlocksToBind,*
| *targetLogicalMemoryAddress, continue-token*
| Out: *continue-token, targetLogicalMemoryAddress, numScmBlocksToBound*
| Return Value: *H_Success, H_Parameter, H_P2, H_P3, H_P4, H_Overlap,*
| *H_Too_Big, H_P5, H_Busy*
Given a DRC-Index of an NVDIMM, map a continuous SCM blocks range
*(startingScmBlockIndex, startingScmBlockIndex+numScmBlocksToBind)* to the guest
at *targetLogicalMemoryAddress* within guest physical address space. In
case *targetLogicalMemoryAddress == 0xFFFFFFFF_FFFFFFFF* then hypervisor
assigns a target address to the guest. The HCALL can fail if the Guest has
an active PTE entry to the SCM block being bound.
**H_SCM_UNBIND_MEM**
| Input: drcIndex, startingScmLogicalMemoryAddress, numScmBlocksToUnbind
| Out: numScmBlocksUnbound
| Return Value: *H_Success, H_Parameter, H_P2, H_P3, H_In_Use, H_Overlap,*
| *H_Busy, H_LongBusyOrder1mSec, H_LongBusyOrder10mSec*
Given a DRC-Index of an NVDimm, unmap *numScmBlocksToUnbind* SCM blocks starting
at *startingScmLogicalMemoryAddress* from guest physical address space. The
HCALL can fail if the Guest has an active PTE entry to the SCM block being
unbound.
**H_SCM_QUERY_BLOCK_MEM_BINDING**
| Input: *drcIndex, scmBlockIndex*
| Out: *Guest-Physical-Address*
| Return Value: *H_Success, H_Parameter, H_P2, H_NotFound*
Given a DRC-Index and an SCM Block index return the guest physical address to
which the SCM block is mapped to.
**H_SCM_QUERY_LOGICAL_MEM_BINDING**
| Input: *Guest-Physical-Address*
| Out: *drcIndex, scmBlockIndex*
| Return Value: *H_Success, H_Parameter, H_P2, H_NotFound*
Given a guest physical address return which DRC Index and SCM block is mapped
to that address.
**H_SCM_UNBIND_ALL**
| Input: *scmTargetScope, drcIndex*
| Out: *None*
| Return Value: *H_Success, H_Parameter, H_P2, H_P3, H_In_Use, H_Busy,*
| *H_LongBusyOrder1mSec, H_LongBusyOrder10mSec*
Depending on the Target scope unmap all SCM blocks belonging to all NVDIMMs
or all SCM blocks belonging to a single NVDIMM identified by its drcIndex
from the LPAR memory.
**H_SCM_HEALTH**
| Input: drcIndex
| Out: *health-bitmap (r4), health-bit-valid-bitmap (r5)*
| Return Value: *H_Success, H_Parameter, H_Hardware*
Given a DRC Index return the info on predictive failure and overall health of
the PMEM device. The asserted bits in the health-bitmap indicate one or more states
(described in table below) of the PMEM device and health-bit-valid-bitmap indicate
which bits in health-bitmap are valid. The bits are reported in
reverse bit ordering for example a value of 0xC400000000000000
indicates bits 0, 1, and 5 are valid.
Health Bitmap Flags:
+------+-----------------------------------------------------------------------+
| Bit | Definition |
+======+=======================================================================+
| 00 | PMEM device is unable to persist memory contents. |
| | If the system is powered down, nothing will be saved. |
+------+-----------------------------------------------------------------------+
| 01 | PMEM device failed to persist memory contents. Either contents were |
| | not saved successfully on power down or were not restored properly on |
| | power up. |
+------+-----------------------------------------------------------------------+
| 02 | PMEM device contents are persisted from previous IPL. The data from |
| | the last boot were successfully restored. |
+------+-----------------------------------------------------------------------+
| 03 | PMEM device contents are not persisted from previous IPL. There was no|
| | data to restore from the last boot. |
+------+-----------------------------------------------------------------------+
| 04 | PMEM device memory life remaining is critically low |
+------+-----------------------------------------------------------------------+
| 05 | PMEM device will be garded off next IPL due to failure |
+------+-----------------------------------------------------------------------+
| 06 | PMEM device contents cannot persist due to current platform health |
| | status. A hardware failure may prevent data from being saved or |
| | restored. |
+------+-----------------------------------------------------------------------+
| 07 | PMEM device is unable to persist memory contents in certain conditions|
+------+-----------------------------------------------------------------------+
| 08 | PMEM device is encrypted |
+------+-----------------------------------------------------------------------+
| 09 | PMEM device has successfully completed a requested erase or secure |
| | erase procedure. |
+------+-----------------------------------------------------------------------+
|10:63 | Reserved / Unused |
+------+-----------------------------------------------------------------------+
**H_SCM_PERFORMANCE_STATS**
| Input: drcIndex, resultBuffer Addr
| Out: None
| Return Value: *H_Success, H_Parameter, H_Unsupported, H_Hardware, H_Authority, H_Privilege*
Given a DRC Index collect the performance statistics for NVDIMM and copy them
to the resultBuffer.
**H_SCM_FLUSH**
| Input: *drcIndex, continue-token*
| Out: *continue-token*
| Return Value: *H_SUCCESS, H_Parameter, H_P2, H_BUSY*
Given a DRC Index Flush the data to backend NVDIMM device.
The hcall returns H_BUSY when the flush takes longer time and the hcall needs
to be issued multiple times in order to be completely serviced. The
*continue-token* from the output to be passed in the argument list of
subsequent hcalls to the hypervisor until the hcall is completely serviced
at which point H_SUCCESS or other error is returned by the hypervisor.
**H_HTM**
| Input: flags, target, operation (op), op-param1, op-param2, op-param3
| Out: *dumphtmbufferdata*
| Return Value: *H_Success,H_Busy,H_LongBusyOrder,H_Partial,H_Parameter,
H_P2,H_P3,H_P4,H_P5,H_P6,H_State,H_Not_Available,H_Authority*
H_HTM supports setup, configuration, control and dumping of Hardware Trace
Macro (HTM) function and its data. HTM buffer stores tracing data for functions
like core instruction, core LLAT and nest.
References
==========
.. [1] "Power Architecture Platform Reference"
https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference
.. [2] "Linux on Power Architecture Platform Reference"
https://members.openpowerfoundation.org/document/dl/469
.. [3] "Definitions and Notation" Book III-Section 14.5.3
https://openpowerfoundation.org/?resource_lib=power-isa-version-3-0
.. [4] arch/powerpc/include/asm/hvcall.h
.. [5] "64-Bit ELF V2 ABI Specification: Power Architecture"
https://openpowerfoundation.org/?resource_lib=64-bit-elf-v2-abi-specification-power-architecture
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
Hypercall opcode
1-6이 문서는 64-bit Power Book3S의 PAPR hypercall(hcall) ABI와 주요 opcode를 설명합니다.
PAPR virtualization 개요
7-3364-bit Power Book3S virtualization은 guest operating system의 runtime environment와 privileged operation 요청 방식을 정의한 PAPR(Power Architecture Platform Reference)을 기반으로 합니다.
| Hypervisor | 지원 범위 |
|---|---|
| IBM PowerVM (PHYP) | AIX, IBM-i, Linux LPAR를 지원하는 proprietary hypervisor이며 전체 PAPR specification 구현 |
| Qemu/KVM | PPC64 Linux host의 PPC64 Linux guest를 지원하며 PAPR subset인 LoPAPR 구현 |
PAPR hypervisor 위의 PPC64 guest kernel은 pSeries guest라 부릅니다. Supervisor mode(`HV=0`)에서 실행되므로 hypervisor-privileged action이나 hypervisor service가 필요할 때 hcall을 발행합니다.
Hcall은 guest가 필요한 input operand를 넘겨 privileged operation을 대신 수행해 달라고 요청하는 것입니다. Hypervisor는 처리 뒤 status code와 output operand를 guest에 반환합니다.
HCALL ABI
34-51PSeries guest는 `HVCS` instruction으로 hypervisor context에 진입합니다. HCALL opcode는 `r3`, input argument는 `r4-r12`에 둡니다. Memory buffer를 사용하면 data는 Big-endian byte order여야 합니다.
처리가 끝나 guest로 돌아오면 `r3`에 return value, `r4-r12`에 output value가 있습니다. Output memory buffer도 Big-endian입니다.
PowerPC architecture code는 pSeries Linux kernel에서 hcall을 호출하기 위한 `plpar_hcall_xxx` wrapper를 architecture-specific header에 제공합니다.
Guest가 HVCS로 진입하고 같은 register set으로 결과를 돌려받습니다.
Register convention
52-96HCALL은 64-Bit ELF V2 ABI의 Power Architecture register convention을 따릅니다. Volatile register는 hypervisor call 뒤 보존된다고 가정할 수 없습니다.
| Register | Volatile | 목적 |
|---|---|---|
| `r0` | Y | 선택적 사용 |
| `r1` | N | Stack pointer |
| `r2` | N | TOC |
| `r3` | Y | HCALL opcode / return value |
| `r4-r10` | Y | 입력 및 출력 값 |
| `r11` | Y | 선택적 사용 / environmental pointer |
| `r12` | Y | 선택적 사용 / global entry point의 function entry address |
| `r13` | N | Thread pointer |
| `r14-r31` | N | Local variable |
| `LR` | Y | Link register |
| `CTR` | Y | Loop counter |
| `XER` | Y | Fixed-point exception register |
| `CR0-1` | Y | Condition register field |
| `CR2-4` | N | Condition register field |
| `CR5-7` | Y | Condition register field |
| 기타 | N | 보존 |
Opcode와 operand register, ABI-preserved register를 구분합니다.
Dynamic Resource Connector와 index
97-118PAPR hypervisor는 PCI device, NVDIMM 같은 shared hardware resource를 Dynamic Resource(DR)라 부릅니다. DR을 LPAR에 할당할 때 PHYP가 접근을 관리하는 Dynamic Resource Connector(DRC)를 만듭니다.
LPAR는 DRC를 opaque 32-bit `DRC-Index`로 참조합니다. 이 값은 해당 DR의 device tree node attribute로 전달됩니다.
각 hardware resource의 connector를 opaque DRC index로 guest kernel에 노출합니다.
Return value와 continue token
119-137Hypervisor는 HCALL 처리 뒤 `r3`에 success 또는 failure status를 둡니다. Failure code는 architecture-specific header에 정의되고 문서화됩니다.
오래 걸리는 HCALL은 여러 번 호출해야 할 수 있습니다. 첫 호출은 `continue-token == 0`으로 시작하고, `H_CONTINUE`가 반환되면 hypervisor가 준 token을 다음 호출에 전달합니다. `H_CONTINUE`가 아닌 값이 나올 때까지 반복합니다.
Opaque continue token으로 중단 가능한 작업을 이어갑니다.
SCM metadata와 memory binding HCALL
138-219다음은 PHYP가 지원하는 주요 Storage Class Memory HCALL입니다. Opcode 숫자는 `arch/powerpc/include/asm/hvcall.h`에서 확인합니다.
| HCALL | Input | Output | Return value | 동작 |
|---|---|---|---|---|
| `H_SCM_READ_METADATA` | `drcIndex, offset, buffer-address, numBytesToRead` | `numBytesRead` | `H_Success, H_Parameter, H_P2, H_P3, H_Hardware` | NVDIMM storage 밖의 metadata area에서 label, bad-block 등 N byte를 읽어 buffer로 복사 |
| `H_SCM_WRITE_METADATA` | `drcIndex, offset, data, numBytesToWrite` | 없음 | `H_Success, H_Parameter, H_P2, H_P4, H_Hardware` | 지정 offset의 NVDIMM metadata area에 N byte 기록 |
| `H_SCM_BIND_MEM` | `drcIndex, startingScmBlockIndex, numScmBlocksToBind, targetLogicalMemoryAddress, continue-token` | `continue-token, targetLogicalMemoryAddress, numScmBlocksToBound` | `H_Success, H_Parameter, H_P2, H_P3, H_P4, H_Overlap, H_Too_Big, H_P5, H_Busy` | 연속 SCM block을 guest physical address에 map. Target이 `0xFFFFFFFF_FFFFFFFF`이면 hypervisor가 address 배정 |
| `H_SCM_UNBIND_MEM` | `drcIndex, startingScmLogicalMemoryAddress, numScmBlocksToUnbind` | `numScmBlocksUnbound` | `H_Success, H_Parameter, H_P2, H_P3, H_In_Use, H_Overlap, H_Busy, H_LongBusyOrder1mSec, H_LongBusyOrder10mSec` | 지정 logical address부터 SCM block을 guest physical address space에서 unmap |
| `H_SCM_QUERY_BLOCK_MEM_BINDING` | `drcIndex, scmBlockIndex` | `Guest-Physical-Address` | `H_Success, H_Parameter, H_P2, H_NotFound` | SCM block이 map된 guest physical address 조회 |
| `H_SCM_QUERY_LOGICAL_MEM_BINDING` | `Guest-Physical-Address` | `drcIndex, scmBlockIndex` | `H_Success, H_Parameter, H_P2, H_NotFound` | Guest physical address에 map된 DRC index와 SCM block 조회 |
| `H_SCM_UNBIND_ALL` | `scmTargetScope, drcIndex` | 없음 | `H_Success, H_Parameter, H_P2, H_P3, H_In_Use, H_Busy, H_LongBusyOrder1mSec, H_LongBusyOrder10mSec` | Scope에 따라 모든 NVDIMM 또는 한 NVDIMM의 SCM block을 LPAR memory에서 unmap |
`H_SCM_BIND_MEM`과 `H_SCM_UNBIND_MEM`은 대상 SCM block에 active PTE가 있으면 실패할 수 있습니다. Query HCALL은 block→guest address와 guest address→block 양방향 mapping을 제공합니다.
DRC index와 block range를 guest physical address에 연결하거나 해제합니다.
`H_SCM_HEALTH`
220-268`H_SCM_HEALTH`는 `drcIndex`를 입력받아 R4의 `health-bitmap`과 R5의 `health-bit-valid-bitmap`을 반환합니다. Return value는 `H_Success`, `H_Parameter`, `H_Hardware`입니다.
Set된 health bit는 predictive failure와 PMEM 상태를 나타내며 valid bitmap은 어떤 bit가 유효한지 알려 줍니다. Bit ordering은 reverse이므로 `0xC400000000000000`은 bit 0, 1, 5가 유효하다는 뜻입니다.
| Bit | 정의 |
|---|---|
| 00 | PMEM이 memory content를 persist할 수 없어 power down 시 저장되지 않음 |
| 01 | Power down 저장 또는 power up 복원 실패 |
| 02 | 이전 IPL의 content를 성공적으로 복원 |
| 03 | 이전 IPL에서 복원할 content가 없었음 |
| 04 | 남은 PMEM 수명이 임계 수준으로 낮음 |
| 05 | Failure 때문에 다음 IPL에서 PMEM을 gard off할 예정 |
| 06 | 현재 platform health 상태 때문에 content persistence 불가 |
| 07 | 특정 조건에서 memory content를 persist할 수 없음 |
| 08 | PMEM device가 encrypted 상태 |
| 09 | 요청한 erase 또는 secure erase 절차를 성공적으로 완료 |
| 10:63 | Reserved / unused |
Persistence, lifetime, platform health, encryption과 erase 결과를 bit로 보고합니다.
Performance, flush와 HTM HCALL
269-302| HCALL | Input | Output | Return value | 동작 |
|---|---|---|---|---|
| `H_SCM_PERFORMANCE_STATS` | `drcIndex, resultBuffer Addr` | 없음 | `H_Success, H_Parameter, H_Unsupported, H_Hardware, H_Authority, H_Privilege` | NVDIMM performance statistics를 수집해 result buffer로 복사 |
| `H_SCM_FLUSH` | `drcIndex, continue-token` | `continue-token` | `H_SUCCESS, H_Parameter, H_P2, H_BUSY` | Backend NVDIMM으로 data flush. 오래 걸리면 `H_BUSY`와 token으로 반복 |
| `H_HTM` | `flags, target, operation, op-param1, op-param2, op-param3` | `dumphtmbufferdata` | `H_Success, H_Busy, H_LongBusyOrder, H_Partial, H_Parameter, H_P2, H_P3, H_P4, H_P5, H_P6, H_State, H_Not_Available, H_Authority` | Hardware Trace Macro setup, configuration, control과 core instruction/core LLAT/nest trace dump |
`H_SCM_FLUSH`가 오래 걸리면 `H_BUSY`와 output continue token을 반환합니다. 완료되어 `H_SUCCESS` 또는 다른 error가 나올 때까지 token을 다음 호출에 전달합니다.
`H_HTM`은 Hardware Trace Macro의 setup, configuration, control, dump를 지원합니다. HTM buffer에는 core instruction, core LLAT와 nest 같은 trace data가 저장됩니다.
참고 문서
303-313- Power Architecture Platform Reference: `https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference`
- Linux on Power Architecture Platform Reference: `https://members.openpowerfoundation.org/document/dl/469`
- Power ISA v3.0 Book III, Definitions and Notation section 14.5.3: `https://openpowerfoundation.org/?resource_lib=power-isa-version-3-0`
- Kernel HCALL definitions: `arch/powerpc/include/asm/hvcall.h`
- 64-Bit ELF V2 ABI Specification: `https://openpowerfoundation.org/?resource_lib=64-bit-elf-v2-abi-specification-power-architecture`
요약과 해설
papr_hcalls.rst:1-313PSeries guest는 `HVCS`와 `r3`/`r4-r12` convention으로 privileged service를 요청합니다. Long-running HCALL은 opaque continue token으로 재개하며 DRC index가 NVDIMM 같은 dynamic resource를 식별합니다.