요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
=================
KVM-specific MSRs
=================
:Author: Glauber Costa <glommer@redhat.com>, Red Hat Inc, 2010
KVM makes use of some custom MSRs to service some requests.
Custom MSRs have a range reserved for them, that goes from
0x4b564d00 to 0x4b564dff. There are MSRs outside this area,
but they are deprecated and their use is discouraged.
Custom MSR list
---------------
The current supported Custom MSR list is:
MSR_KVM_WALL_CLOCK_NEW:
0x4b564d00
data:
4-byte alignment physical address of a memory area which must be
in guest RAM. This memory is expected to hold a copy of the following
structure::
struct pvclock_wall_clock {
u32 version;
u32 sec;
u32 nsec;
} __attribute__((__packed__));
whose data will be filled in by the hypervisor. The hypervisor is only
guaranteed to update this data at the moment of MSR write.
Users that want to reliably query this information more than once have
to write more than once to this MSR. Fields have the following meanings:
version:
guest has to check version before and after grabbing
time information and check that they are both equal and even.
An odd version indicates an in-progress update.
sec:
number of seconds for wallclock at time of boot.
nsec:
number of nanoseconds for wallclock at time of boot.
In order to get the current wallclock time, the system_time from
MSR_KVM_SYSTEM_TIME_NEW needs to be added.
Note that although MSRs are per-CPU entities, the effect of this
particular MSR is global.
Availability of this MSR must be checked via bit 3 in 0x4000001 cpuid
leaf prior to usage.
MSR_KVM_SYSTEM_TIME_NEW:
0x4b564d01
data:
4-byte aligned physical address of a memory area which must be in
guest RAM, plus an enable bit in bit 0. This memory is expected to hold
a copy of the following structure::
struct pvclock_vcpu_time_info {
u32 version;
u32 pad0;
u64 tsc_timestamp;
u64 system_time;
u32 tsc_to_system_mul;
s8 tsc_shift;
u8 flags;
u8 pad[2];
} __attribute__((__packed__)); /* 32 bytes */
whose data will be filled in by the hypervisor periodically. Only one
write, or registration, is needed for each VCPU. The interval between
updates of this structure is arbitrary and implementation-dependent.
The hypervisor may update this structure at any time it sees fit until
anything with bit0 == 0 is written to it.
Fields have the following meanings:
version:
guest has to check version before and after grabbing
time information and check that they are both equal and even.
An odd version indicates an in-progress update.
tsc_timestamp:
the tsc value at the current VCPU at the time
of the update of this structure. Guests can subtract this value
from current tsc to derive a notion of elapsed time since the
structure update.
system_time:
a host notion of monotonic time, including sleep
time at the time this structure was last updated. Unit is
nanoseconds.
tsc_to_system_mul:
multiplier to be used when converting
tsc-related quantity to nanoseconds
tsc_shift:
shift to be used when converting tsc-related
quantity to nanoseconds. This shift will ensure that
multiplication with tsc_to_system_mul does not overflow.
A positive value denotes a left shift, a negative value
a right shift.
The conversion from tsc to nanoseconds involves an additional
right shift by 32 bits. With this information, guests can
derive per-CPU time by doing::
time = (current_tsc - tsc_timestamp)
if (tsc_shift >= 0)
time <<= tsc_shift;
else
time >>= -tsc_shift;
time = (time * tsc_to_system_mul) >> 32
time = time + system_time
flags:
bits in this field indicate extended capabilities
coordinated between the guest and the hypervisor. Availability
of specific flags has to be checked in 0x40000001 cpuid leaf.
Current flags are:
+-----------+--------------+----------------------------------+
| flag bit | cpuid bit | meaning |
+-----------+--------------+----------------------------------+
| | | time measures taken across |
| 0 | 24 | multiple cpus are guaranteed to |
| | | be monotonic |
+-----------+--------------+----------------------------------+
| | | guest vcpu has been paused by |
| 1 | N/A | the host |
| | | See 4.70 in api.txt |
+-----------+--------------+----------------------------------+
Availability of this MSR must be checked via bit 3 in 0x4000001 cpuid
leaf prior to usage.
MSR_KVM_WALL_CLOCK:
0x11
data and functioning:
same as MSR_KVM_WALL_CLOCK_NEW. Use that instead.
This MSR falls outside the reserved KVM range and may be removed in the
future. Its usage is deprecated.
Availability of this MSR must be checked via bit 0 in 0x4000001 cpuid
leaf prior to usage.
MSR_KVM_SYSTEM_TIME:
0x12
data and functioning:
same as MSR_KVM_SYSTEM_TIME_NEW. Use that instead.
This MSR falls outside the reserved KVM range and may be removed in the
future. Its usage is deprecated.
Availability of this MSR must be checked via bit 0 in 0x4000001 cpuid
leaf prior to usage.
The suggested algorithm for detecting kvmclock presence is then::
if (!kvm_para_available()) /* refer to cpuid.txt */
return NON_PRESENT;
flags = cpuid_eax(0x40000001);
if (flags & 3) {
msr_kvm_system_time = MSR_KVM_SYSTEM_TIME_NEW;
msr_kvm_wall_clock = MSR_KVM_WALL_CLOCK_NEW;
return PRESENT;
} else if (flags & 0) {
msr_kvm_system_time = MSR_KVM_SYSTEM_TIME;
msr_kvm_wall_clock = MSR_KVM_WALL_CLOCK;
return PRESENT;
} else
return NON_PRESENT;
MSR_KVM_ASYNC_PF_EN:
0x4b564d02
data:
Asynchronous page fault (APF) control MSR.
Bits 63-6 hold 64-byte aligned physical address of a 64 byte memory area
which must be in guest RAM. This memory is expected to hold the
following structure::
struct kvm_vcpu_pv_apf_data {
/* Used for 'page not present' events delivered via #PF */
__u32 flags;
/* Used for 'page ready' events delivered via interrupt notification */
__u32 token;
__u8 pad[56];
};
Bits 5-4 of the MSR are reserved and should be zero. Bit 0 is set to 1
when asynchronous page faults are enabled on the vcpu, 0 when disabled.
Bit 1 is 1 if asynchronous page faults can be injected when vcpu is in
cpl == 0. Bit 2 is 1 if asynchronous page faults are delivered to L1 as
#PF vmexits. Bit 2 can be set only if KVM_FEATURE_ASYNC_PF_VMEXIT is
present in CPUID. Bit 3 enables interrupt based delivery of 'page ready'
events. Bit 3 can only be set if KVM_FEATURE_ASYNC_PF_INT is present in
CPUID.
'Page not present' events are currently always delivered as synthetic
#PF exception. During delivery of these events APF CR2 register contains
a token that will be used to notify the guest when missing page becomes
available. Also, to make it possible to distinguish between real #PF and
APF, first 4 bytes of 64 byte memory location ('flags') will be written
to by the hypervisor at the time of injection. Only first bit of 'flags'
is currently supported, when set, it indicates that the guest is dealing
with asynchronous 'page not present' event. If during a page fault APF
'flags' is '0' it means that this is regular page fault. Guest is
supposed to clear 'flags' when it is done handling #PF exception so the
next event can be delivered.
Note, since APF 'page not present' events use the same exception vector
as regular page fault, guest must reset 'flags' to '0' before it does
something that can generate normal page fault.
Bytes 4-7 of 64 byte memory location ('token') will be written to by the
hypervisor at the time of APF 'page ready' event injection. The content
of these bytes is a token which was previously delivered in CR2 as
'page not present' event. The event indicates the page is now available.
Guest is supposed to write '0' to 'token' when it is done handling
'page ready' event and to write '1' to MSR_KVM_ASYNC_PF_ACK after
clearing the location; writing to the MSR forces KVM to re-scan its
queue and deliver the next pending notification.
Note, MSR_KVM_ASYNC_PF_INT MSR specifying the interrupt vector for 'page
ready' APF delivery needs to be written to before enabling APF mechanism
in MSR_KVM_ASYNC_PF_EN or interrupt #0 can get injected. The MSR is
available if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
Note, previously, 'page ready' events were delivered via the same #PF
exception as 'page not present' events but this is now deprecated. If
bit 3 (interrupt based delivery) is not set APF events are not delivered.
If APF is disabled while there are outstanding APFs, they will
not be delivered.
Currently 'page ready' APF events will be always delivered on the
same vcpu as 'page not present' event was, but guest should not rely on
that.
MSR_KVM_STEAL_TIME:
0x4b564d03
data:
64-byte alignment physical address of a memory area which must be
in guest RAM, plus an enable bit in bit 0. This memory is expected to
hold a copy of the following structure::
struct kvm_steal_time {
__u64 steal;
__u32 version;
__u32 flags;
__u8 preempted;
__u8 u8_pad[3];
__u32 pad[11];
}
whose data will be filled in by the hypervisor periodically. Only one
write, or registration, is needed for each VCPU. The interval between
updates of this structure is arbitrary and implementation-dependent.
The hypervisor may update this structure at any time it sees fit until
anything with bit0 == 0 is written to it. Guest is required to make sure
this structure is initialized to zero.
Fields have the following meanings:
version:
a sequence counter. In other words, guest has to check
this field before and after grabbing time information and make
sure they are both equal and even. An odd version indicates an
in-progress update.
flags:
At this point, always zero. May be used to indicate
changes in this structure in the future.
steal:
the amount of time in which this vCPU did not run, in
nanoseconds. Time during which the vcpu is idle, will not be
reported as steal time.
preempted:
indicate the vCPU who owns this struct is running or
not. Non-zero values mean the vCPU has been preempted. Zero
means the vCPU is not preempted. NOTE, it is always zero if the
the hypervisor doesn't support this field.
MSR_KVM_EOI_EN:
0x4b564d04
data:
Bit 0 is 1 when PV end of interrupt is enabled on the vcpu; 0
when disabled. Bit 1 is reserved and must be zero. When PV end of
interrupt is enabled (bit 0 set), bits 63-2 hold a 4-byte aligned
physical address of a 4 byte memory area which must be in guest RAM and
must be zeroed.
The first, least significant bit of 4 byte memory location will be
written to by the hypervisor, typically at the time of interrupt
injection. Value of 1 means that guest can skip writing EOI to the apic
(using MSR or MMIO write); instead, it is sufficient to signal
EOI by clearing the bit in guest memory - this location will
later be polled by the hypervisor.
Value of 0 means that the EOI write is required.
It is always safe for the guest to ignore the optimization and perform
the APIC EOI write anyway.
Hypervisor is guaranteed to only modify this least
significant bit while in the current VCPU context, this means that
guest does not need to use either lock prefix or memory ordering
primitives to synchronise with the hypervisor.
However, hypervisor can set and clear this memory bit at any time:
therefore to make sure hypervisor does not interrupt the
guest and clear the least significant bit in the memory area
in the window between guest testing it to detect
whether it can skip EOI apic write and between guest
clearing it to signal EOI to the hypervisor,
guest must both read the least significant bit in the memory area and
clear it using a single CPU instruction, such as test and clear, or
compare and exchange.
MSR_KVM_POLL_CONTROL:
0x4b564d05
Control host-side polling.
data:
Bit 0 enables (1) or disables (0) host-side HLT polling logic.
KVM guests can request the host not to poll on HLT, for example if
they are performing polling themselves.
MSR_KVM_ASYNC_PF_INT:
0x4b564d06
data:
Second asynchronous page fault (APF) control MSR.
Bits 0-7: APIC vector for delivery of 'page ready' APF events.
Bits 8-63: Reserved
Interrupt vector for asynchnonous 'page ready' notifications delivery.
The vector has to be set up before asynchronous page fault mechanism
is enabled in MSR_KVM_ASYNC_PF_EN. The MSR is only available if
KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
MSR_KVM_ASYNC_PF_ACK:
0x4b564d07
data:
Asynchronous page fault (APF) acknowledgment.
When the guest is done processing 'page ready' APF event and 'token'
field in 'struct kvm_vcpu_pv_apf_data' is cleared it is supposed to
write '1' to bit 0 of the MSR, this causes the host to re-scan its queue
and check if there are more notifications pending. The MSR is available
if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
MSR_KVM_MIGRATION_CONTROL:
0x4b564d08
data:
This MSR is available if KVM_FEATURE_MIGRATION_CONTROL is present in
CPUID. Bit 0 represents whether live migration of the guest is allowed.
When a guest is started, bit 0 will be 0 if the guest has encrypted
memory and 1 if the guest does not have encrypted memory. If the
guest is communicating page encryption status to the host using the
``KVM_HC_MAP_GPA_RANGE`` hypercall, it can set bit 0 in this MSR to
allow live migration of the guest.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
KVM 전용 MSR 범위
1-19KVM은 게스트 요청을 처리하기 위해 자체 model-specific register(MSR)를 사용합니다. KVM 전용으로 예약된 범위는 `0x4b564d00`부터 `0x4b564dff`까지입니다.
이 범위 밖에도 예전 KVM MSR이 남아 있지만 deprecated 상태이며 새 구현에서는 사용하지 않는 것이 좋습니다. 기능을 사용하기 전에는 해당 KVM CPUID feature bit가 게스트에 실제로 노출되었는지 확인해야 합니다.
신규 인터페이스와 호환성용 인터페이스를 구분합니다.
.. SPDX-License-Identifier: GPL-2.0
=================
KVM-specific MSRs
=================
:Author: Glauber Costa <glommer@redhat.com>, Red Hat Inc, 2010
KVM makes use of some custom MSRs to service some requests.
Custom MSRs have a range reserved for them, that goes from
0x4b564d00 to 0x4b564dff. There are MSRs outside this area,
but they are deprecated and their use is discouraged.
Custom MSR list
---------------
The current supported Custom MSR list is:
MSR_KVM_WALL_CLOCK_NEW
20-58`MSR_KVM_WALL_CLOCK_NEW (0x4b564d00)`에는 게스트 RAM 안의 4-byte 정렬 물리 주소를 기록합니다. 그 위치에는 packed `struct pvclock_wall_clock`이 놓이며 hypervisor가 `version`, `sec`, `nsec`를 채웁니다.
Hypervisor가 자료를 갱신한다고 보장하는 시점은 이 MSR에 쓸 때뿐입니다. wall clock을 두 번 이상 확실하게 질의하려면 질의할 때마다 MSR을 다시 써야 합니다.
게스트는 시간 필드를 읽기 전과 후에 `version`을 읽어 두 값이 같고 짝수인지 검사합니다. 홀수 version은 hypervisor가 갱신 중임을 뜻하므로 다시 읽어야 합니다.
`sec`와 `nsec`는 부팅 시점의 wall clock입니다. 현재 wall clock을 얻으려면 여기에 `MSR_KVM_SYSTEM_TIME_NEW`에서 계산한 `system_time`을 더합니다. MSR 자체는 per-CPU지만 이 MSR의 효과는 전역입니다.
부팅 기준 wall clock 자료와 일관성 표식입니다.
일관된 부팅 시간과 vCPU 시간을 결합합니다.
MSR_KVM_WALL_CLOCK_NEW:
0x4b564d00
data:
4-byte alignment physical address of a memory area which must be
in guest RAM. This memory is expected to hold a copy of the following
structure::
struct pvclock_wall_clock {
u32 version;
u32 sec;
u32 nsec;
} __attribute__((__packed__));
whose data will be filled in by the hypervisor. The hypervisor is only
guaranteed to update this data at the moment of MSR write.
Users that want to reliably query this information more than once have
to write more than once to this MSR. Fields have the following meanings:
version:
guest has to check version before and after grabbing
time information and check that they are both equal and even.
An odd version indicates an in-progress update.
sec:
number of seconds for wallclock at time of boot.
nsec:
number of nanoseconds for wallclock at time of boot.
In order to get the current wallclock time, the system_time from
MSR_KVM_SYSTEM_TIME_NEW needs to be added.
Note that although MSRs are per-CPU entities, the effect of this
particular MSR is global.
Availability of this MSR must be checked via bit 3 in 0x4000001 cpuid
leaf prior to usage.
MSR_KVM_SYSTEM_TIME_NEW
59-147`MSR_KVM_SYSTEM_TIME_NEW (0x4b564d01)`에는 게스트 RAM의 4-byte 정렬 주소와 bit 0의 enable 값을 함께 기록합니다. 대상은 32-byte packed `struct pvclock_vcpu_time_info`입니다.
vCPU마다 한 번만 등록하면 hypervisor가 구현 의존적인 임의 간격으로 구조체를 갱신합니다. bit 0이 0인 값을 쓸 때까지 hypervisor는 필요하다고 판단한 시점에 언제든 갱신할 수 있습니다.
`version`은 wall-clock 구조체와 같은 sequence 규칙을 사용합니다. 읽기 전후 값이 같고 짝수인 snapshot만 유효하며 홀수면 갱신 중입니다.
`tsc_timestamp`는 구조체 갱신 당시 현재 vCPU의 TSC입니다. 현재 TSC에서 이를 빼면 마지막 갱신 이후의 TSC 경과량을 얻습니다. `system_time`은 sleep 시간을 포함하는 host monotonic time이며 단위는 nanoseconds입니다.
경과 TSC는 `tsc_shift`가 양수면 왼쪽, 음수면 오른쪽으로 이동합니다. 이어서 `tsc_to_system_mul`을 곱하고 32 bits 오른쪽 이동한 뒤 `system_time`을 더하면 vCPU별 시간을 nanoseconds로 얻습니다. shift는 곱셈 overflow를 피하도록 정해집니다.
`flags` bit 0은 여러 CPU에서 잰 시간이 monotonic임을 보장하며 CPUID의 `KVM_FEATURE_CLOCKSOURCE_STABLE_BIT`(bit 24)로 가용성을 확인합니다. flags bit 1은 host가 guest vCPU를 일시 정지했다는 표시이고 별도 CPUID bit는 없습니다.
32-byte per-vCPU 시간 변환 구조체입니다.
flag 사용 전 대응 CPUID capability를 확인합니다.
원문의 정수 연산 순서를 그대로 적용합니다.
MSR_KVM_SYSTEM_TIME_NEW:
0x4b564d01
data:
4-byte aligned physical address of a memory area which must be in
guest RAM, plus an enable bit in bit 0. This memory is expected to hold
a copy of the following structure::
struct pvclock_vcpu_time_info {
u32 version;
u32 pad0;
u64 tsc_timestamp;
u64 system_time;
u32 tsc_to_system_mul;
s8 tsc_shift;
u8 flags;
u8 pad[2];
} __attribute__((__packed__)); /* 32 bytes */
whose data will be filled in by the hypervisor periodically. Only one
write, or registration, is needed for each VCPU. The interval between
updates of this structure is arbitrary and implementation-dependent.
The hypervisor may update this structure at any time it sees fit until
anything with bit0 == 0 is written to it.
Fields have the following meanings:
version:
guest has to check version before and after grabbing
time information and check that they are both equal and even.
An odd version indicates an in-progress update.
tsc_timestamp:
the tsc value at the current VCPU at the time
of the update of this structure. Guests can subtract this value
from current tsc to derive a notion of elapsed time since the
structure update.
system_time:
a host notion of monotonic time, including sleep
time at the time this structure was last updated. Unit is
nanoseconds.
tsc_to_system_mul:
multiplier to be used when converting
tsc-related quantity to nanoseconds
tsc_shift:
shift to be used when converting tsc-related
quantity to nanoseconds. This shift will ensure that
multiplication with tsc_to_system_mul does not overflow.
A positive value denotes a left shift, a negative value
a right shift.
The conversion from tsc to nanoseconds involves an additional
right shift by 32 bits. With this information, guests can
derive per-CPU time by doing::
time = (current_tsc - tsc_timestamp)
if (tsc_shift >= 0)
time <<= tsc_shift;
else
time >>= -tsc_shift;
time = (time * tsc_to_system_mul) >> 32
time = time + system_time
flags:
bits in this field indicate extended capabilities
coordinated between the guest and the hypervisor. Availability
of specific flags has to be checked in 0x40000001 cpuid leaf.
Current flags are:
+-----------+--------------+----------------------------------+
| flag bit | cpuid bit | meaning |
+-----------+--------------+----------------------------------+
| | | time measures taken across |
| 0 | 24 | multiple cpus are guaranteed to |
| | | be monotonic |
+-----------+--------------+----------------------------------+
| | | guest vcpu has been paused by |
| 1 | N/A | the host |
| | | See 4.70 in api.txt |
+-----------+--------------+----------------------------------+
Availability of this MSR must be checked via bit 3 in 0x4000001 cpuid
leaf prior to usage.
Deprecated MSR_KVM_WALL_CLOCK
148-159`MSR_KVM_WALL_CLOCK (0x11)`의 자료와 동작은 `MSR_KVM_WALL_CLOCK_NEW`와 같습니다. 다만 KVM 예약 범위 밖에 있고 앞으로 제거될 수 있으므로 deprecated이며 NEW 인터페이스를 사용해야 합니다.
호환성 코드가 이 legacy MSR을 사용하려면 CPUID leaf `0x40000001`의 bit 0을 먼저 확인합니다.
이전 주소와 대체 주소입니다.
MSR_KVM_WALL_CLOCK:
0x11
data and functioning:
same as MSR_KVM_WALL_CLOCK_NEW. Use that instead.
This MSR falls outside the reserved KVM range and may be removed in the
future. Its usage is deprecated.
Availability of this MSR must be checked via bit 0 in 0x4000001 cpuid
leaf prior to usage.
Deprecated MSR_KVM_SYSTEM_TIME
160-188`MSR_KVM_SYSTEM_TIME (0x12)`도 자료와 동작이 `MSR_KVM_SYSTEM_TIME_NEW`와 같지만 예약 범위 밖의 deprecated 인터페이스입니다. legacy 지원에는 CPUID bit 0을 사용하고 새 guest는 bit 3의 NEW MSR을 선택합니다.
원문 감지 예제는 먼저 `kvm_para_available()`을 검사한 뒤 `cpuid_eax(0x40000001)` 결과를 분기합니다. 예제의 두 번째 조건은 문자 그대로 `else if (flags & 0)`이며, 이 번역은 원문 보존을 위해 이를 임의로 고치지 않습니다.
실제 구현에서는 사용하는 커널 header와 ABI 정의를 기준으로 CPUID bit를 명시적으로 검사해야 합니다. NEW clocksource가 있으면 `MSR_KVM_SYSTEM_TIME_NEW`와 `MSR_KVM_WALL_CLOCK_NEW`를 한 쌍으로 선택합니다.
원문 예제의 선택 결과를 정리합니다.
호출자는 CPUID 결과에 맞는 MSR 쌍을 사용합니다.
MSR_KVM_SYSTEM_TIME:
0x12
data and functioning:
same as MSR_KVM_SYSTEM_TIME_NEW. Use that instead.
This MSR falls outside the reserved KVM range and may be removed in the
future. Its usage is deprecated.
Availability of this MSR must be checked via bit 0 in 0x4000001 cpuid
leaf prior to usage.
The suggested algorithm for detecting kvmclock presence is then::
if (!kvm_para_available()) /* refer to cpuid.txt */
return NON_PRESENT;
flags = cpuid_eax(0x40000001);
if (flags & 3) {
msr_kvm_system_time = MSR_KVM_SYSTEM_TIME_NEW;
msr_kvm_wall_clock = MSR_KVM_WALL_CLOCK_NEW;
return PRESENT;
} else if (flags & 0) {
msr_kvm_system_time = MSR_KVM_SYSTEM_TIME;
msr_kvm_wall_clock = MSR_KVM_WALL_CLOCK;
return PRESENT;
} else
return NON_PRESENT;
MSR_KVM_ASYNC_PF_EN
189-258`MSR_KVM_ASYNC_PF_EN (0x4b564d02)`은 asynchronous page fault(APF) 제어 MSR입니다. bits 63-6은 게스트 RAM 안의 64-byte 정렬된 64-byte 영역 주소이며 그곳에 `struct kvm_vcpu_pv_apf_data`를 둡니다.
구조체의 첫 4 bytes인 `flags`는 #PF로 전달되는 page-not-present event에 쓰이고 다음 4 bytes인 `token`은 interrupt로 전달되는 page-ready event에 쓰입니다. 나머지 56 bytes는 padding입니다.
MSR bits 5-4는 예약되어 0이어야 합니다. bit 0은 APF 활성화, bit 1은 CPL0에서 APF 주입 허용, bit 2는 L1에 #PF VM exit로 전달, bit 3은 page-ready의 interrupt 전달을 뜻합니다.
bit 2는 CPUID에 `KVM_FEATURE_ASYNC_PF_VMEXIT`이 있을 때만 설정할 수 있고 bit 3은 `KVM_FEATURE_ASYNC_PF_INT`가 있을 때만 설정할 수 있습니다.
Page-not-present는 synthetic #PF로 전달됩니다. 이때 APF CR2에는 나중의 page-ready 알림과 연결할 token이 들어가고 hypervisor는 `flags` bit 0을 설정합니다. flags가 0이면 일반 page fault입니다.
게스트는 #PF 처리를 끝내면 `flags`를 0으로 지워야 합니다. APF와 일반 page fault가 같은 exception vector를 쓰므로 일반 page fault를 일으킬 수 있는 작업을 하기 전에도 반드시 flags를 0으로 재설정해야 합니다.
Page-ready interrupt 때 hypervisor는 앞서 CR2로 보낸 값을 `token`에 기록합니다. 게스트는 event 처리를 마친 후 token을 0으로 지우고 `MSR_KVM_ASYNC_PF_ACK` bit 0에 1을 써서 KVM이 queue를 다시 훑고 다음 알림을 전달하게 합니다.
`MSR_KVM_ASYNC_PF_INT`로 interrupt vector를 먼저 설정하지 않고 APF를 활성화하면 interrupt #0이 주입될 수 있습니다. 예전의 #PF 기반 page-ready 전달은 deprecated이며 bit 3이 없으면 APF event가 전달되지 않습니다.
Outstanding APF가 있는 동안 APF를 비활성화하면 남은 event는 전달되지 않습니다. 현재 page-ready는 대응 page-not-present와 같은 vCPU에 전달되지만 게스트는 이 구현 특성에 의존해서는 안 됩니다.
64-byte APF 공유 영역의 배치입니다.
기능 bit와 선행 capability입니다.
page-not-present token을 page-ready ACK까지 연결합니다.
MSR_KVM_ASYNC_PF_EN:
0x4b564d02
data:
Asynchronous page fault (APF) control MSR.
Bits 63-6 hold 64-byte aligned physical address of a 64 byte memory area
which must be in guest RAM. This memory is expected to hold the
following structure::
struct kvm_vcpu_pv_apf_data {
/* Used for 'page not present' events delivered via #PF */
__u32 flags;
/* Used for 'page ready' events delivered via interrupt notification */
__u32 token;
__u8 pad[56];
};
Bits 5-4 of the MSR are reserved and should be zero. Bit 0 is set to 1
when asynchronous page faults are enabled on the vcpu, 0 when disabled.
Bit 1 is 1 if asynchronous page faults can be injected when vcpu is in
cpl == 0. Bit 2 is 1 if asynchronous page faults are delivered to L1 as
#PF vmexits. Bit 2 can be set only if KVM_FEATURE_ASYNC_PF_VMEXIT is
present in CPUID. Bit 3 enables interrupt based delivery of 'page ready'
events. Bit 3 can only be set if KVM_FEATURE_ASYNC_PF_INT is present in
CPUID.
'Page not present' events are currently always delivered as synthetic
#PF exception. During delivery of these events APF CR2 register contains
a token that will be used to notify the guest when missing page becomes
available. Also, to make it possible to distinguish between real #PF and
APF, first 4 bytes of 64 byte memory location ('flags') will be written
to by the hypervisor at the time of injection. Only first bit of 'flags'
is currently supported, when set, it indicates that the guest is dealing
with asynchronous 'page not present' event. If during a page fault APF
'flags' is '0' it means that this is regular page fault. Guest is
supposed to clear 'flags' when it is done handling #PF exception so the
next event can be delivered.
Note, since APF 'page not present' events use the same exception vector
as regular page fault, guest must reset 'flags' to '0' before it does
something that can generate normal page fault.
Bytes 4-7 of 64 byte memory location ('token') will be written to by the
hypervisor at the time of APF 'page ready' event injection. The content
of these bytes is a token which was previously delivered in CR2 as
'page not present' event. The event indicates the page is now available.
Guest is supposed to write '0' to 'token' when it is done handling
'page ready' event and to write '1' to MSR_KVM_ASYNC_PF_ACK after
clearing the location; writing to the MSR forces KVM to re-scan its
queue and deliver the next pending notification.
Note, MSR_KVM_ASYNC_PF_INT MSR specifying the interrupt vector for 'page
ready' APF delivery needs to be written to before enabling APF mechanism
in MSR_KVM_ASYNC_PF_EN or interrupt #0 can get injected. The MSR is
available if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
Note, previously, 'page ready' events were delivered via the same #PF
exception as 'page not present' events but this is now deprecated. If
bit 3 (interrupt based delivery) is not set APF events are not delivered.
If APF is disabled while there are outstanding APFs, they will
not be delivered.
Currently 'page ready' APF events will be always delivered on the
same vcpu as 'page not present' event was, but guest should not rely on
that.
MSR_KVM_STEAL_TIME
259-305`MSR_KVM_STEAL_TIME (0x4b564d03)`에는 게스트 RAM의 64-byte 정렬 주소와 bit 0의 enable 값을 기록합니다. 대상 `struct kvm_steal_time`은 게스트가 먼저 모두 0으로 초기화해야 합니다.
vCPU마다 한 번 등록하면 hypervisor가 구현 의존적인 임의 간격으로 갱신합니다. bit 0이 0인 값을 쓸 때까지 언제든 갱신될 수 있습니다.
`version`은 sequence counter이므로 자료를 읽기 전후 값이 같고 짝수인지 검사해야 합니다. 홀수는 갱신 중이라는 뜻입니다. `flags`는 현재 항상 0이며 미래 구조 변경을 알리는 데 쓰일 수 있습니다.
`steal`은 이 vCPU가 실행되지 못한 시간을 nanoseconds로 나타내며 vCPU가 idle이었던 시간은 포함하지 않습니다. `preempted`가 0이 아니면 vCPU가 선점된 상태이고 0이면 선점되지 않았습니다. Hypervisor가 이 필드를 지원하지 않으면 항상 0입니다.
64-byte 공유 구조체의 의미입니다.
MSR_KVM_STEAL_TIME:
0x4b564d03
data:
64-byte alignment physical address of a memory area which must be
in guest RAM, plus an enable bit in bit 0. This memory is expected to
hold a copy of the following structure::
struct kvm_steal_time {
__u64 steal;
__u32 version;
__u32 flags;
__u8 preempted;
__u8 u8_pad[3];
__u32 pad[11];
}
whose data will be filled in by the hypervisor periodically. Only one
write, or registration, is needed for each VCPU. The interval between
updates of this structure is arbitrary and implementation-dependent.
The hypervisor may update this structure at any time it sees fit until
anything with bit0 == 0 is written to it. Guest is required to make sure
this structure is initialized to zero.
Fields have the following meanings:
version:
a sequence counter. In other words, guest has to check
this field before and after grabbing time information and make
sure they are both equal and even. An odd version indicates an
in-progress update.
flags:
At this point, always zero. May be used to indicate
changes in this structure in the future.
steal:
the amount of time in which this vCPU did not run, in
nanoseconds. Time during which the vcpu is idle, will not be
reported as steal time.
preempted:
indicate the vCPU who owns this struct is running or
not. Non-zero values mean the vCPU has been preempted. Zero
means the vCPU is not preempted. NOTE, it is always zero if the
the hypervisor doesn't support this field.
MSR_KVM_EOI_EN
306-341`MSR_KVM_EOI_EN (0x4b564d04)` bit 0은 vCPU의 paravirtual end-of-interrupt(PV EOI)를 켜고 끕니다. bit 1은 예약되어 0이어야 하며, 활성화할 때 bits 63-2에는 게스트 RAM의 4-byte 정렬된 4-byte 영역 주소를 넣습니다. 게스트는 이 영역을 먼저 0으로 만들어야 합니다.
Hypervisor는 공유 영역의 최하위 bit를 주로 interrupt 주입 시점에 기록합니다. 값이 1이면 게스트는 APIC EOI MSR/MMIO write를 생략하고 공유 bit를 지우는 것만으로 EOI를 알릴 수 있습니다. 값이 0이면 APIC EOI write가 필요합니다.
게스트가 최적화를 무시하고 항상 APIC EOI를 써도 안전합니다. Hypervisor는 현재 vCPU context에서만 이 bit를 바꾸므로 lock prefix나 별도 memory-ordering primitive로 hypervisor와 동기화할 필요는 없습니다.
다만 bit 검사와 clear 사이에 interrupt가 끼어 hypervisor의 변경을 잃지 않도록 두 동작은 하나의 CPU instruction으로 수행해야 합니다. test-and-clear 또는 compare-and-exchange를 사용합니다.
값에 따른 guest EOI 동작입니다.
검사와 clear 사이의 interrupt race를 막습니다.
MSR_KVM_EOI_EN:
0x4b564d04
data:
Bit 0 is 1 when PV end of interrupt is enabled on the vcpu; 0
when disabled. Bit 1 is reserved and must be zero. When PV end of
interrupt is enabled (bit 0 set), bits 63-2 hold a 4-byte aligned
physical address of a 4 byte memory area which must be in guest RAM and
must be zeroed.
The first, least significant bit of 4 byte memory location will be
written to by the hypervisor, typically at the time of interrupt
injection. Value of 1 means that guest can skip writing EOI to the apic
(using MSR or MMIO write); instead, it is sufficient to signal
EOI by clearing the bit in guest memory - this location will
later be polled by the hypervisor.
Value of 0 means that the EOI write is required.
It is always safe for the guest to ignore the optimization and perform
the APIC EOI write anyway.
Hypervisor is guaranteed to only modify this least
significant bit while in the current VCPU context, this means that
guest does not need to use either lock prefix or memory ordering
primitives to synchronise with the hypervisor.
However, hypervisor can set and clear this memory bit at any time:
therefore to make sure hypervisor does not interrupt the
guest and clear the least significant bit in the memory area
in the window between guest testing it to detect
whether it can skip EOI apic write and between guest
clearing it to signal EOI to the hypervisor,
guest must both read the least significant bit in the memory area and
clear it using a single CPU instruction, such as test and clear, or
compare and exchange.
MSR_KVM_POLL_CONTROL
342-352`MSR_KVM_POLL_CONTROL (0x4b564d05)`은 host 쪽 HLT polling을 제어합니다. bit 0이 1이면 polling을 활성화하고 0이면 비활성화합니다.
게스트가 자체 polling을 수행하는 경우처럼 host의 추가 polling이 불필요할 때 KVM guest가 HLT에서 poll하지 말도록 요청할 수 있습니다.
bit 0의 동작입니다.
MSR_KVM_POLL_CONTROL:
0x4b564d05
Control host-side polling.
data:
Bit 0 enables (1) or disables (0) host-side HLT polling logic.
KVM guests can request the host not to poll on HLT, for example if
they are performing polling themselves.
MSR_KVM_ASYNC_PF_INT
353-366`MSR_KVM_ASYNC_PF_INT (0x4b564d06)`은 두 번째 APF 제어 MSR입니다. bits 0-7에 page-ready APF event를 전달할 APIC vector를 넣고 bits 8-63은 예약합니다.
이 vector는 `MSR_KVM_ASYNC_PF_EN`에서 APF를 활성화하기 전에 설정해야 합니다. CPUID에 `KVM_FEATURE_ASYNC_PF_INT`가 있을 때만 이 MSR을 사용할 수 있습니다.
page-ready 알림 vector의 배치입니다.
MSR_KVM_ASYNC_PF_INT:
0x4b564d06
data:
Second asynchronous page fault (APF) control MSR.
Bits 0-7: APIC vector for delivery of 'page ready' APF events.
Bits 8-63: Reserved
Interrupt vector for asynchnonous 'page ready' notifications delivery.
The vector has to be set up before asynchronous page fault mechanism
is enabled in MSR_KVM_ASYNC_PF_EN. The MSR is only available if
KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
MSR_KVM_ASYNC_PF_ACK
367-378`MSR_KVM_ASYNC_PF_ACK (0x4b564d07)`은 APF acknowledgment MSR입니다. 게스트가 page-ready event 처리를 마치고 `struct kvm_vcpu_pv_apf_data`의 `token`을 지운 뒤 MSR bit 0에 1을 씁니다.
ACK write를 받으면 host가 APF queue를 다시 훑어 대기 중인 다음 notification이 있는지 확인합니다. 이 MSR도 CPUID의 `KVM_FEATURE_ASYNC_PF_INT`가 있을 때 사용할 수 있습니다.
공유 token clear와 MSR write의 순서입니다.
다음 pending notification을 요청합니다.
MSR_KVM_ASYNC_PF_ACK:
0x4b564d07
data:
Asynchronous page fault (APF) acknowledgment.
When the guest is done processing 'page ready' APF event and 'token'
field in 'struct kvm_vcpu_pv_apf_data' is cleared it is supposed to
write '1' to bit 0 of the MSR, this causes the host to re-scan its queue
and check if there are more notifications pending. The MSR is available
if KVM_FEATURE_ASYNC_PF_INT is present in CPUID.
MSR_KVM_MIGRATION_CONTROL
379-390`MSR_KVM_MIGRATION_CONTROL (0x4b564d08)`은 CPUID에 `KVM_FEATURE_MIGRATION_CONTROL`이 있을 때 사용할 수 있습니다. bit 0은 guest live migration 허용 여부를 나타냅니다.
게스트 시작 시 encrypted memory를 사용하는 guest는 bit 0이 0이고 encrypted memory를 사용하지 않는 guest는 1입니다.
게스트가 `KVM_HC_MAP_GPA_RANGE` hypercall로 page encryption 상태를 host에 전달한다면 이 MSR bit 0을 1로 설정해 live migration을 허용할 수 있습니다.
초기값과 migration 허용 조건입니다.
MSR_KVM_MIGRATION_CONTROL:
0x4b564d08
data:
This MSR is available if KVM_FEATURE_MIGRATION_CONTROL is present in
CPUID. Bit 0 represents whether live migration of the guest is allowed.
When a guest is started, bit 0 will be 0 if the guest has encrypted
memory and 1 if the guest does not have encrypted memory. If the
guest is communicating page encryption status to the host using the
``KVM_HC_MAP_GPA_RANGE`` hypercall, it can set bit 0 in this MSR to
allow live migration of the guest.
요약·해설
msr.rst:1-390x86 KVM 전용 MSR의 주소, 공유 구조체, CPUID 선행 조건과 guest-host 동기화 절차를 설명합니다.
시간 snapshot의 sequence 검사, APF token과 ACK 순서, PV EOI의 원자적 test-and-clear, encrypted guest의 migration 허용 조건을 구조화했습니다.