요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
1
.. SPDX-License-Identifier: GPL-2.0
2
.. include:: <isonum.txt>
4
.. |intel_pstate| replace:: :doc:`intel_pstate <intel_pstate>`
6
=======================
7
CPU Performance Scaling
8
=======================
10
:Copyright: |copy| 2017 Intel Corporation
12
:Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
15
The Concept of CPU Performance Scaling
16
======================================
18
The majority of modern processors are capable of operating in a number of
19
different clock frequency and voltage configurations, often referred to as
20
Operating Performance Points or P-states (in ACPI terminology). As a rule,
21
the higher the clock frequency and the higher the voltage, the more instructions
22
can be retired by the CPU over a unit of time, but also the higher the clock
23
frequency and the higher the voltage, the more energy is consumed over a unit of
24
time (or the more power is drawn) by the CPU in the given P-state. Therefore
25
there is a natural tradeoff between the CPU capacity (the number of instructions
26
that can be executed over a unit of time) and the power drawn by the CPU.
28
In some situations it is desirable or even necessary to run the program as fast
29
as possible and then there is no reason to use any P-states different from the
30
highest one (i.e. the highest-performance frequency/voltage configuration
31
available). In some other cases, however, it may not be necessary to execute
32
instructions so quickly and maintaining the highest available CPU capacity for a
33
relatively long time without utilizing it entirely may be regarded as wasteful.
34
It also may not be physically possible to maintain maximum CPU capacity for too
35
long for thermal or power supply capacity reasons or similar. To cover those
36
cases, there are hardware interfaces allowing CPUs to be switched between
37
different frequency/voltage configurations or (in the ACPI terminology) to be
38
put into different P-states.
40
Typically, they are used along with algorithms to estimate the required CPU
41
capacity, so as to decide which P-states to put the CPUs into. Of course, since
42
the utilization of the system generally changes over time, that has to be done
43
repeatedly on a regular basis. The activity by which this happens is referred
44
to as CPU performance scaling or CPU frequency scaling (because it involves
45
adjusting the CPU clock frequency).
48
CPU Performance Scaling in Linux
49
================================
51
The Linux kernel supports CPU performance scaling by means of the ``CPUFreq``
52
(CPU Frequency scaling) subsystem that consists of three layers of code: the
53
core, scaling governors and scaling drivers.
55
The ``CPUFreq`` core provides the common code infrastructure and user space
56
interfaces for all platforms that support CPU performance scaling. It defines
57
the basic framework in which the other components operate.
59
Scaling governors implement algorithms to estimate the required CPU capacity.
60
As a rule, each governor implements one, possibly parametrized, scaling
61
algorithm.
63
Scaling drivers talk to the hardware. They provide scaling governors with
64
information on the available P-states (or P-state ranges in some cases) and
65
access platform-specific hardware interfaces to change CPU P-states as requested
66
by scaling governors.
68
In principle, all available scaling governors can be used with every scaling
69
driver. That design is based on the observation that the information used by
70
performance scaling algorithms for P-state selection can be represented in a
71
platform-independent form in the majority of cases, so it should be possible
72
to use the same performance scaling algorithm implemented in exactly the same
73
way regardless of which scaling driver is used. Consequently, the same set of
74
scaling governors should be suitable for every supported platform.
76
However, that observation may not hold for performance scaling algorithms
77
based on information provided by the hardware itself, for example through
78
feedback registers, as that information is typically specific to the hardware
79
interface it comes from and may not be easily represented in an abstract,
80
platform-independent way. For this reason, ``CPUFreq`` allows scaling drivers
81
to bypass the governor layer and implement their own performance scaling
82
algorithms. That is done by the |intel_pstate| scaling driver.
85
``CPUFreq`` Policy Objects
86
==========================
88
In some cases the hardware interface for P-state control is shared by multiple
89
CPUs. That is, for example, the same register (or set of registers) is used to
90
control the P-state of multiple CPUs at the same time and writing to it affects
91
all of those CPUs simultaneously.
93
Sets of CPUs sharing hardware P-state control interfaces are represented by
94
``CPUFreq`` as struct cpufreq_policy objects. For consistency,
95
struct cpufreq_policy is also used when there is only one CPU in the given
96
set.
98
The ``CPUFreq`` core maintains a pointer to a struct cpufreq_policy object for
99
every CPU in the system, including CPUs that are currently offline. If multiple
100
CPUs share the same hardware P-state control interface, all of the pointers
101
corresponding to them point to the same struct cpufreq_policy object.
103
``CPUFreq`` uses struct cpufreq_policy as its basic data type and the design
104
of its user space interface is based on the policy concept.
107
CPU Initialization
108
==================
110
First of all, a scaling driver has to be registered for ``CPUFreq`` to work.
111
It is only possible to register one scaling driver at a time, so the scaling
112
driver is expected to be able to handle all CPUs in the system.
114
The scaling driver may be registered before or after CPU registration. If
115
CPUs are registered earlier, the driver core invokes the ``CPUFreq`` core to
116
take a note of all of the already registered CPUs during the registration of the
117
scaling driver. In turn, if any CPUs are registered after the registration of
118
the scaling driver, the ``CPUFreq`` core will be invoked to take note of them
119
at their registration time.
121
In any case, the ``CPUFreq`` core is invoked to take note of any logical CPU it
122
has not seen so far as soon as it is ready to handle that CPU. [Note that the
123
logical CPU may be a physical single-core processor, or a single core in a
124
multicore processor, or a hardware thread in a physical processor or processor
125
core. In what follows "CPU" always means "logical CPU" unless explicitly stated
126
otherwise and the word "processor" is used to refer to the physical part
127
possibly including multiple logical CPUs.]
129
Once invoked, the ``CPUFreq`` core checks if the policy pointer is already set
130
for the given CPU and if so, it skips the policy object creation. Otherwise,
131
a new policy object is created and initialized, which involves the creation of
132
a new policy directory in ``sysfs``, and the policy pointer corresponding to
133
the given CPU is set to the new policy object's address in memory.
135
Next, the scaling driver's ``->init()`` callback is invoked with the policy
136
pointer of the new CPU passed to it as the argument. That callback is expected
137
to initialize the performance scaling hardware interface for the given CPU (or,
138
more precisely, for the set of CPUs sharing the hardware interface it belongs
139
to, represented by its policy object) and, if the policy object it has been
140
called for is new, to set parameters of the policy, like the minimum and maximum
141
frequencies supported by the hardware, the table of available frequencies (if
142
the set of supported P-states is not a continuous range), and the mask of CPUs
143
that belong to the same policy (including both online and offline CPUs). That
144
mask is then used by the core to populate the policy pointers for all of the
145
CPUs in it.
147
The next major initialization step for a new policy object is to attach a
148
scaling governor to it (to begin with, that is the default scaling governor
149
determined by the kernel command line or configuration, but it may be changed
150
later via ``sysfs``). First, a pointer to the new policy object is passed to
151
the governor's ``->init()`` callback which is expected to initialize all of the
152
data structures necessary to handle the given policy and, possibly, to add
153
a governor ``sysfs`` interface to it. Next, the governor is started by
154
invoking its ``->start()`` callback.
156
That callback is expected to register per-CPU utilization update callbacks for
157
all of the online CPUs belonging to the given policy with the CPU scheduler.
158
The utilization update callbacks will be invoked by the CPU scheduler on
159
important events, like task enqueue and dequeue, on every iteration of the
160
scheduler tick or generally whenever the CPU utilization may change (from the
161
scheduler's perspective). They are expected to carry out computations needed
162
to determine the P-state to use for the given policy going forward and to
163
invoke the scaling driver to make changes to the hardware in accordance with
164
the P-state selection. The scaling driver may be invoked directly from
165
scheduler context or asynchronously, via a kernel thread or workqueue, depending
166
on the configuration and capabilities of the scaling driver and the governor.
168
Similar steps are taken for policy objects that are not new, but were "inactive"
169
previously, meaning that all of the CPUs belonging to them were offline. The
170
only practical difference in that case is that the ``CPUFreq`` core will attempt
171
to use the scaling governor previously used with the policy that became
172
"inactive" (and is re-initialized now) instead of the default governor.
174
In turn, if a previously offline CPU is being brought back online, but some
175
other CPUs sharing the policy object with it are online already, there is no
176
need to re-initialize the policy object at all. In that case, it only is
177
necessary to restart the scaling governor so that it can take the new online CPU
178
into account. That is achieved by invoking the governor's ``->stop`` and
179
``->start()`` callbacks, in this order, for the entire policy.
181
As mentioned before, the |intel_pstate| scaling driver bypasses the scaling
182
governor layer of ``CPUFreq`` and provides its own P-state selection algorithms.
183
Consequently, if |intel_pstate| is used, scaling governors are not attached to
184
new policy objects. Instead, the driver's ``->setpolicy()`` callback is invoked
185
to register per-CPU utilization update callbacks for each policy. These
186
callbacks are invoked by the CPU scheduler in the same way as for scaling
187
governors, but in the |intel_pstate| case they both determine the P-state to
188
use and change the hardware configuration accordingly in one go from scheduler
189
context.
191
The policy objects created during CPU initialization and other data structures
192
associated with them are torn down when the scaling driver is unregistered
193
(which happens when the kernel module containing it is unloaded, for example) or
194
when the last CPU belonging to the given policy in unregistered.
197
Policy Interface in ``sysfs``
198
=============================
200
During the initialization of the kernel, the ``CPUFreq`` core creates a
201
``sysfs`` directory (kobject) called ``cpufreq`` under
202
:file:`/sys/devices/system/cpu/`.
204
That directory contains a ``policyX`` subdirectory (where ``X`` represents an
205
integer number) for every policy object maintained by the ``CPUFreq`` core.
206
Each ``policyX`` directory is pointed to by ``cpufreq`` symbolic links
207
under :file:`/sys/devices/system/cpu/cpuY/` (where ``Y`` represents an integer
208
that may be different from the one represented by ``X``) for all of the CPUs
209
associated with (or belonging to) the given policy. The ``policyX`` directories
210
in :file:`/sys/devices/system/cpu/cpufreq` each contain policy-specific
211
attributes (files) to control ``CPUFreq`` behavior for the corresponding policy
212
objects (that is, for all of the CPUs associated with them).
214
Some of those attributes are generic. They are created by the ``CPUFreq`` core
215
and their behavior generally does not depend on what scaling driver is in use
216
and what scaling governor is attached to the given policy. Some scaling drivers
217
also add driver-specific attributes to the policy directories in ``sysfs`` to
218
control policy-specific aspects of driver behavior.
220
The generic attributes under :file:`/sys/devices/system/cpu/cpufreq/policyX/`
221
are the following:
223
``affected_cpus``
224
List of online CPUs belonging to this policy (i.e. sharing the hardware
225
performance scaling interface represented by the ``policyX`` policy
226
object).
228
``bios_limit``
229
If the platform firmware (BIOS) tells the OS to apply an upper limit to
230
CPU frequencies, that limit will be reported through this attribute (if
231
present).
233
The existence of the limit may be a result of some (often unintentional)
234
BIOS settings, restrictions coming from a service processor or other
235
BIOS/HW-based mechanisms.
237
This does not cover ACPI thermal limitations which can be discovered
238
through a generic thermal driver.
240
This attribute is not present if the scaling driver in use does not
241
support it.
243
``cpuinfo_cur_freq``
244
Current frequency of the CPUs belonging to this policy as obtained from
245
the hardware (in KHz).
247
This is expected to be the frequency the hardware actually runs at.
248
If that frequency cannot be determined, this attribute should not
249
be present.
251
``cpuinfo_avg_freq``
252
An average frequency (in KHz) of all CPUs belonging to a given policy,
253
derived from a hardware provided feedback and reported on a time frame
254
spanning at most few milliseconds.
256
This is expected to be based on the frequency the hardware actually runs
257
at and, as such, might require specialised hardware support (such as AMU
258
extension on ARM). If one cannot be determined, this attribute should
259
not be present.
261
Note that failed attempt to retrieve current frequency for a given
262
CPU(s) will result in an appropriate error, i.e.: EAGAIN for CPU that
263
remains idle (raised on ARM).
265
``cpuinfo_max_freq``
266
Maximum possible operating frequency the CPUs belonging to this policy
267
can run at (in kHz).
269
``cpuinfo_min_freq``
270
Minimum possible operating frequency the CPUs belonging to this policy
271
can run at (in kHz).
273
``cpuinfo_transition_latency``
274
The time it takes to switch the CPUs belonging to this policy from one
275
P-state to another, in nanoseconds.
277
``related_cpus``
278
List of all (online and offline) CPUs belonging to this policy.
280
``scaling_available_frequencies``
281
List of available frequencies of the CPUs belonging to this policy
282
(in kHz).
284
``scaling_available_governors``
285
List of ``CPUFreq`` scaling governors present in the kernel that can
286
be attached to this policy or (if the |intel_pstate| scaling driver is
287
in use) list of scaling algorithms provided by the driver that can be
288
applied to this policy.
290
[Note that some governors are modular and it may be necessary to load a
291
kernel module for the governor held by it to become available and be
292
listed by this attribute.]
294
``scaling_cur_freq``
295
Current frequency of all of the CPUs belonging to this policy (in kHz).
297
In the majority of cases, this is the frequency of the last P-state
298
requested by the scaling driver from the hardware using the scaling
299
interface provided by it, which may or may not reflect the frequency
300
the CPU is actually running at (due to hardware design and other
301
limitations).
303
Some architectures (e.g. ``x86``) may attempt to provide information
304
more precisely reflecting the current CPU frequency through this
305
attribute, but that still may not be the exact current CPU frequency as
306
seen by the hardware at the moment. This behavior though, is only
307
available via c:macro:``CPUFREQ_ARCH_CUR_FREQ`` option.
309
``scaling_driver``
310
The scaling driver currently in use.
312
``scaling_governor``
313
The scaling governor currently attached to this policy or (if the
314
|intel_pstate| scaling driver is in use) the scaling algorithm
315
provided by the driver that is currently applied to this policy.
317
This attribute is read-write and writing to it will cause a new scaling
318
governor to be attached to this policy or a new scaling algorithm
319
provided by the scaling driver to be applied to it (in the
320
|intel_pstate| case), as indicated by the string written to this
321
attribute (which must be one of the names listed by the
322
``scaling_available_governors`` attribute described above).
324
``scaling_max_freq``
325
Maximum frequency the CPUs belonging to this policy are allowed to be
326
running at (in kHz).
328
This attribute is read-write and writing a string representing an
329
integer to it will cause a new limit to be set (it must not be lower
330
than the value of the ``scaling_min_freq`` attribute).
332
``scaling_min_freq``
333
Minimum frequency the CPUs belonging to this policy are allowed to be
334
running at (in kHz).
336
This attribute is read-write and writing a string representing a
337
non-negative integer to it will cause a new limit to be set (it must not
338
be higher than the value of the ``scaling_max_freq`` attribute).
340
``scaling_setspeed``
341
This attribute is functional only if the `userspace`_ scaling governor
342
is attached to the given policy.
344
It returns the last frequency requested by the governor (in kHz) or can
345
be written to in order to set a new frequency for the policy.
348
Generic Scaling Governors
349
=========================
351
``CPUFreq`` provides generic scaling governors that can be used with all
352
scaling drivers. As stated before, each of them implements a single, possibly
353
parametrized, performance scaling algorithm.
355
Scaling governors are attached to policy objects and different policy objects
356
can be handled by different scaling governors at the same time (although that
357
may lead to suboptimal results in some cases).
359
The scaling governor for a given policy object can be changed at any time with
360
the help of the ``scaling_governor`` policy attribute in ``sysfs``.
362
Some governors expose ``sysfs`` attributes to control or fine-tune the scaling
363
algorithms implemented by them. Those attributes, referred to as governor
364
tunables, can be either global (system-wide) or per-policy, depending on the
365
scaling driver in use. If the driver requires governor tunables to be
366
per-policy, they are located in a subdirectory of each policy directory.
367
Otherwise, they are located in a subdirectory under
368
:file:`/sys/devices/system/cpu/cpufreq/`. In either case the name of the
369
subdirectory containing the governor tunables is the name of the governor
370
providing them.
372
``performance``
373
---------------
375
When attached to a policy object, this governor causes the highest frequency,
376
within the ``scaling_max_freq`` policy limit, to be requested for that policy.
378
The request is made once at that time the governor for the policy is set to
379
``performance`` and whenever the ``scaling_max_freq`` or ``scaling_min_freq``
380
policy limits change after that.
382
``powersave``
383
-------------
385
When attached to a policy object, this governor causes the lowest frequency,
386
within the ``scaling_min_freq`` policy limit, to be requested for that policy.
388
The request is made once at that time the governor for the policy is set to
389
``powersave`` and whenever the ``scaling_max_freq`` or ``scaling_min_freq``
390
policy limits change after that.
392
``userspace``
393
-------------
395
This governor does not do anything by itself. Instead, it allows user space
396
to set the CPU frequency for the policy it is attached to by writing to the
397
``scaling_setspeed`` attribute of that policy. Though the intention may be to
398
set an exact frequency for the policy, the actual frequency may vary depending
399
on hardware coordination, thermal and power limits, and other factors.
401
``schedutil``
402
-------------
404
This governor uses CPU utilization data available from the CPU scheduler. It
405
generally is regarded as a part of the CPU scheduler, so it can access the
406
scheduler's internal data structures directly.
408
It runs entirely in scheduler context, although in some cases it may need to
409
invoke the scaling driver asynchronously when it decides that the CPU frequency
410
should be changed for a given policy (that depends on whether or not the driver
411
is capable of changing the CPU frequency from scheduler context).
413
The actions of this governor for a particular CPU depend on the scheduling class
414
invoking its utilization update callback for that CPU. If it is invoked by the
415
RT or deadline scheduling classes, the governor will increase the frequency to
416
the allowed maximum (that is, the ``scaling_max_freq`` policy limit). In turn,
417
if it is invoked by the CFS scheduling class, the governor will use the
418
Per-Entity Load Tracking (PELT) metric for the root control group of the
419
given CPU as the CPU utilization estimate (see the *Per-entity load tracking*
420
LWN.net article [1]_ for a description of the PELT mechanism). Then, the new
421
CPU frequency to apply is computed in accordance with the formula
423
f = 1.25 * ``f_0`` * ``util`` / ``max``
425
where ``util`` is the PELT number, ``max`` is the theoretical maximum of
426
``util``, and ``f_0`` is either the maximum possible CPU frequency for the given
427
policy (if the PELT number is frequency-invariant), or the current CPU frequency
428
(otherwise).
430
This governor also employs a mechanism allowing it to temporarily bump up the
431
CPU frequency for tasks that have been waiting on I/O most recently, called
432
"IO-wait boosting". That happens when the :c:macro:`SCHED_CPUFREQ_IOWAIT` flag
433
is passed by the scheduler to the governor callback which causes the frequency
434
to go up to the allowed maximum immediately and then draw back to the value
435
returned by the above formula over time.
437
This governor exposes only one tunable:
439
``rate_limit_us``
440
Minimum time (in microseconds) that has to pass between two consecutive
441
runs of governor computations (default: 1.5 times the scaling driver's
442
transition latency or the maximum 2ms).
444
The purpose of this tunable is to reduce the scheduler context overhead
445
of the governor which might be excessive without it.
447
This governor generally is regarded as a replacement for the older `ondemand`_
448
and `conservative`_ governors (described below), as it is simpler and more
449
tightly integrated with the CPU scheduler, its overhead in terms of CPU context
450
switches and similar is less significant, and it uses the scheduler's own CPU
451
utilization metric, so in principle its decisions should not contradict the
452
decisions made by the other parts of the scheduler.
454
``ondemand``
455
------------
457
This governor uses CPU load as a CPU frequency selection metric.
459
In order to estimate the current CPU load, it measures the time elapsed between
460
consecutive invocations of its worker routine and computes the fraction of that
461
time in which the given CPU was not idle. The ratio of the non-idle (active)
462
time to the total CPU time is taken as an estimate of the load.
464
If this governor is attached to a policy shared by multiple CPUs, the load is
465
estimated for all of them and the greatest result is taken as the load estimate
466
for the entire policy.
468
The worker routine of this governor has to run in process context, so it is
469
invoked asynchronously (via a workqueue) and CPU P-states are updated from
470
there if necessary. As a result, the scheduler context overhead from this
471
governor is minimum, but it causes additional CPU context switches to happen
472
relatively often and the CPU P-state updates triggered by it can be relatively
473
irregular. Also, it affects its own CPU load metric by running code that
474
reduces the CPU idle time (even though the CPU idle time is only reduced very
475
slightly by it).
477
It generally selects CPU frequencies proportional to the estimated load, so that
478
the value of the ``cpuinfo_max_freq`` policy attribute corresponds to the load of
479
1 (or 100%), and the value of the ``cpuinfo_min_freq`` policy attribute
480
corresponds to the load of 0, unless when the load exceeds a (configurable)
481
speedup threshold, in which case it will go straight for the highest frequency
482
it is allowed to use (the ``scaling_max_freq`` policy limit).
484
This governor exposes the following tunables:
486
``sampling_rate``
487
This is how often the governor's worker routine should run, in
488
microseconds.
490
Typically, it is set to values of the order of 2000 (2 ms). Its
491
default value is to add a 50% breathing room
492
to ``cpuinfo_transition_latency`` on each policy this governor is
493
attached to. The minimum is typically the length of two scheduler
494
ticks.
496
If this tunable is per-policy, the following shell command sets the time
497
represented by it to be 1.5 times as high as the transition latency
498
(the default)::
500
# echo `$(($(cat cpuinfo_transition_latency) * 3 / 2))` > ondemand/sampling_rate
502
``up_threshold``
503
If the estimated CPU load is above this value (in percent), the governor
504
will set the frequency to the maximum value allowed for the policy.
505
Otherwise, the selected frequency will be proportional to the estimated
506
CPU load.
508
``ignore_nice_load``
509
If set to 1 (default 0), it will cause the CPU load estimation code to
510
treat the CPU time spent on executing tasks with "nice" levels greater
511
than 0 as CPU idle time.
513
This may be useful if there are tasks in the system that should not be
514
taken into account when deciding what frequency to run the CPUs at.
515
Then, to make that happen it is sufficient to increase the "nice" level
516
of those tasks above 0 and set this attribute to 1.
518
``sampling_down_factor``
519
Temporary multiplier, between 1 (default) and 100 inclusive, to apply to
520
the ``sampling_rate`` value if the CPU load goes above ``up_threshold``.
522
This causes the next execution of the governor's worker routine (after
523
setting the frequency to the allowed maximum) to be delayed, so the
524
frequency stays at the maximum level for a longer time.
526
Frequency fluctuations in some bursty workloads may be avoided this way
527
at the cost of additional energy spent on maintaining the maximum CPU
528
capacity.
530
``powersave_bias``
531
Reduction factor to apply to the original frequency target of the
532
governor (including the maximum value used when the ``up_threshold``
533
value is exceeded by the estimated CPU load) or sensitivity threshold
534
for the AMD frequency sensitivity powersave bias driver
535
(:file:`drivers/cpufreq/amd_freq_sensitivity.c`), between 0 and 1000
536
inclusive.
538
If the AMD frequency sensitivity powersave bias driver is not loaded,
539
the effective frequency to apply is given by
541
f * (1 - ``powersave_bias`` / 1000)
543
where f is the governor's original frequency target. The default value
544
of this attribute is 0 in that case.
546
If the AMD frequency sensitivity powersave bias driver is loaded, the
547
value of this attribute is 400 by default and it is used in a different
548
way.
550
On Family 16h (and later) AMD processors there is a mechanism to get a
551
measured workload sensitivity, between 0 and 100% inclusive, from the
552
hardware. That value can be used to estimate how the performance of the
553
workload running on a CPU will change in response to frequency changes.
555
The performance of a workload with the sensitivity of 0 (memory-bound or
556
IO-bound) is not expected to increase at all as a result of increasing
557
the CPU frequency, whereas workloads with the sensitivity of 100%
558
(CPU-bound) are expected to perform much better if the CPU frequency is
559
increased.
561
If the workload sensitivity is less than the threshold represented by
562
the ``powersave_bias`` value, the sensitivity powersave bias driver
563
will cause the governor to select a frequency lower than its original
564
target, so as to avoid over-provisioning workloads that will not benefit
565
from running at higher CPU frequencies.
567
``conservative``
568
----------------
570
This governor uses CPU load as a CPU frequency selection metric.
572
It estimates the CPU load in the same way as the `ondemand`_ governor described
573
above, but the CPU frequency selection algorithm implemented by it is different.
575
Namely, it avoids changing the frequency significantly over short time intervals
576
which may not be suitable for systems with limited power supply capacity (e.g.
577
battery-powered). To achieve that, it changes the frequency in relatively
578
small steps, one step at a time, up or down - depending on whether or not a
579
(configurable) threshold has been exceeded by the estimated CPU load.
581
This governor exposes the following tunables:
583
``freq_step``
584
Frequency step in percent of the maximum frequency the governor is
585
allowed to set (the ``scaling_max_freq`` policy limit), between 0 and
586
100 (5 by default).
588
This is how much the frequency is allowed to change in one go. Setting
589
it to 0 will cause the default frequency step (5 percent) to be used
590
and setting it to 100 effectively causes the governor to periodically
591
switch the frequency between the ``scaling_min_freq`` and
592
``scaling_max_freq`` policy limits.
594
``down_threshold``
595
Threshold value (in percent, 20 by default) used to determine the
596
frequency change direction.
598
If the estimated CPU load is greater than this value, the frequency will
599
go up (by ``freq_step``). If the load is less than this value (and the
600
``sampling_down_factor`` mechanism is not in effect), the frequency will
601
go down. Otherwise, the frequency will not be changed.
603
``sampling_down_factor``
604
Frequency decrease deferral factor, between 1 (default) and 10
605
inclusive.
607
It effectively causes the frequency to go down ``sampling_down_factor``
608
times slower than it ramps up.
611
Frequency Boost Support
612
=======================
614
Background
615
----------
617
Some processors support a mechanism to raise the operating frequency of some
618
cores in a multicore package temporarily (and above the sustainable frequency
619
threshold for the whole package) under certain conditions, for example if the
620
whole chip is not fully utilized and below its intended thermal or power budget.
622
Different names are used by different vendors to refer to this functionality.
623
For Intel processors it is referred to as "Turbo Boost", AMD calls it
624
"Turbo-Core" or (in technical documentation) "Core Performance Boost" and so on.
625
As a rule, it also is implemented differently by different vendors. The simple
626
term "frequency boost" is used here for brevity to refer to all of those
627
implementations.
629
The frequency boost mechanism may be either hardware-based or software-based.
630
If it is hardware-based (e.g. on x86), the decision to trigger the boosting is
631
made by the hardware (although in general it requires the hardware to be put
632
into a special state in which it can control the CPU frequency within certain
633
limits). If it is software-based (e.g. on ARM), the scaling driver decides
634
whether or not to trigger boosting and when to do that.
636
The ``boost`` File in ``sysfs``
637
-------------------------------
639
This file is located under :file:`/sys/devices/system/cpu/cpufreq/` and controls
640
the "boost" setting for the whole system. It is not present if the underlying
641
scaling driver does not support the frequency boost mechanism (or supports it,
642
but provides a driver-specific interface for controlling it, like
643
|intel_pstate|).
645
If the value in this file is 1, the frequency boost mechanism is enabled. This
646
means that either the hardware can be put into states in which it is able to
647
trigger boosting (in the hardware-based case), or the software is allowed to
648
trigger boosting (in the software-based case). It does not mean that boosting
649
is actually in use at the moment on any CPUs in the system. It only means a
650
permission to use the frequency boost mechanism (which still may never be used
651
for other reasons).
653
If the value in this file is 0, the frequency boost mechanism is disabled and
654
cannot be used at all.
656
The only values that can be written to this file are 0 and 1.
658
Rationale for Boost Control Knob
659
--------------------------------
661
The frequency boost mechanism is generally intended to help to achieve optimum
662
CPU performance on time scales below software resolution (e.g. below the
663
scheduler tick interval) and it is demonstrably suitable for many workloads, but
664
it may lead to problems in certain situations.
666
For this reason, many systems make it possible to disable the frequency boost
667
mechanism in the platform firmware (BIOS) setup, but that requires the system to
668
be restarted for the setting to be adjusted as desired, which may not be
669
practical at least in some cases. For example:
671
1. Boosting means overclocking the processor, although under controlled
672
conditions. Generally, the processor's energy consumption increases
673
as a result of increasing its frequency and voltage, even temporarily.
674
That may not be desirable on systems that switch to power sources of
675
limited capacity, such as batteries, so the ability to disable the boost
676
mechanism while the system is running may help there (but that depends on
677
the workload too).
679
2. In some situations deterministic behavior is more important than
680
performance or energy consumption (or both) and the ability to disable
681
boosting while the system is running may be useful then.
683
3. To examine the impact of the frequency boost mechanism itself, it is useful
684
to be able to run tests with and without boosting, preferably without
685
restarting the system in the meantime.
687
4. Reproducible results are important when running benchmarks. Since
688
the boosting functionality depends on the load of the whole package,
689
single-thread performance may vary because of it which may lead to
690
unreproducible results sometimes. That can be avoided by disabling the
691
frequency boost mechanism before running benchmarks sensitive to that
692
issue.
694
Legacy AMD ``cpb`` Knob
695
-----------------------
697
The AMD powernow-k8 scaling driver supports a ``sysfs`` knob very similar to
698
the global ``boost`` one. It is used for disabling/enabling the "Core
699
Performance Boost" feature of some AMD processors.
701
If present, that knob is located in every ``CPUFreq`` policy directory in
702
``sysfs`` (:file:`/sys/devices/system/cpu/cpufreq/policyX/`) and is called
703
``cpb``, which indicates a more fine grained control interface. The actual
704
implementation, however, works on the system-wide basis and setting that knob
705
for one policy causes the same value of it to be set for all of the other
706
policies at the same time.
708
That knob is still supported on AMD processors that support its underlying
709
hardware feature, but it may be configured out of the kernel (via the
710
:c:macro:`CONFIG_X86_ACPI_CPUFREQ_CPB` configuration option) and the global
711
``boost`` knob is present regardless. Thus it is always possible use the
712
``boost`` knob instead of the ``cpb`` one which is highly recommended, as that
713
is more consistent with what all of the other systems do (and the ``cpb`` knob
714
may not be supported any more in the future).
716
The ``cpb`` knob is never present for any processors without the underlying
717
hardware feature (e.g. all Intel ones), even if the
718
:c:macro:`CONFIG_X86_ACPI_CPUFREQ_CPB` configuration option is set.
721
References
722
==========
724
.. [1] Jonathan Corbet, *Per-entity load tracking*,
725
https://lwn.net/Articles/531853/
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
문서 정보
1-14이 문서는 `SPDX-License-Identifier: GPL-2.0`을 따르고 2017 Intel Corporation 저작물이며, 작성자는 Rafael J. Wysocki `<rafael.j.wysocki@intel.com>`입니다.
문서 안의 `intel_pstate` 표기는 별도 `intel_pstate` 문서로 연결됩니다.
CPU performance scaling 개념
15-47현대 processor 대부분은 여러 clock frequency와 voltage 조합, 즉 Operating Performance Point 또는 ACPI P-state로 동작할 수 있습니다. Frequency와 voltage가 높을수록 단위 시간에 더 많은 instruction을 완료하지만 energy 소비와 power draw도 커집니다.
따라서 CPU capacity와 power 사이에는 자연스러운 절충이 있습니다. 최대한 빨리 끝내야 하는 작업은 최고 P-state가 적합하지만, capacity를 오래 사용하지 않으면서 최고 상태를 유지하면 낭비이고 thermal·power supply 한계 때문에 물리적으로 지속 불가능할 수도 있습니다.
Hardware interface는 CPU를 frequency/voltage 조합 사이에서 전환합니다. 필요한 capacity를 추정하는 algorithm이 P-state를 선택하고, utilization이 계속 바뀌므로 이를 정기적으로 반복합니다. 이 활동을 CPU performance scaling 또는 CPU frequency scaling이라 합니다.
Linux CPUFreq 세 계층
48-84Linux kernel의 `CPUFreq` subsystem은 core, scaling governor, scaling driver 세 계층으로 CPU performance scaling을 지원합니다.
| 계층 | 책임 |
|---|---|
| CPUFreq core | 공통 infrastructure와 userspace interface |
| Scaling governor | 필요 CPU capacity를 추정하는 algorithm |
| Scaling driver | P-state 정보 제공과 hardware interface 제어 |
원칙적으로 platform-independent 정보로 P-state 선택을 표현할 수 있어 모든 governor를 모든 driver와 조합할 수 있습니다. 같은 algorithm을 hardware driver와 무관하게 재사용하는 설계입니다.
다만 feedback register처럼 hardware 고유 정보에 기반한 algorithm은 추상화하기 어렵습니다. 그래서 CPUFreq는 driver가 governor 계층을 우회해 자체 algorithm을 구현할 수 있게 하며 `intel_pstate`가 이 방식을 사용합니다.
struct cpufreq_policy
85-106여러 CPU가 같은 P-state control register를 공유하면 하나를 쓰는 순간 모두가 동시에 영향을 받습니다. CPUFreq는 이런 CPU 집합을 `struct cpufreq_policy` object로 표현하며, CPU 하나뿐인 집합에도 일관되게 같은 type을 씁니다.
Core는 offline CPU까지 system의 모든 CPU에 policy pointer를 유지합니다. Hardware interface를 공유하는 CPU들의 pointer는 같은 object를 가리킵니다. 이 object가 CPUFreq의 기본 data type이며 userspace interface도 policy 개념을 기반으로 합니다.
CPU와 policy 초기화
107-196CPUFreq가 동작하려면 먼저 scaling driver 하나가 등록돼야 합니다. 한 번에 하나만 등록할 수 있으므로 system의 모든 CPU를 처리할 수 있어야 합니다. Driver와 CPU 중 어느 쪽이 먼저 등록돼도 core가 이미 등록된 CPU 또는 새 CPU를 발견해 기록합니다.
이 문서에서 CPU는 별도 언급이 없으면 logical CPU를 뜻하며 single-core processor, multicore의 core, hardware thread일 수 있습니다. Processor는 여러 logical CPU를 포함할 수 있는 물리 부품을 뜻합니다.
새 CPU에 policy pointer가 이미 있으면 object 생성을 건너뜁니다. 없으면 새 object와 sysfs policy directory를 만들고 CPU pointer를 설정합니다.
그다음 driver의 `->init()` callback이 호출됩니다. Callback은 공유 hardware interface를 초기화하고, 새 policy라면 hardware 최소·최대 frequency, discrete P-state의 frequency table, online/offline을 포함한 공유 CPU mask를 설정합니다. Core는 mask의 모든 CPU pointer를 채웁니다.
새 policy에는 kernel command line/configuration이 정한 기본 governor를 연결합니다. Governor `->init()`가 data structure와 선택적 sysfs interface를 준비하고 `->start()`가 시작합니다.
`->start()`는 online CPU마다 scheduler utilization update callback을 등록합니다. Task enqueue/dequeue, scheduler tick 등 utilization 변화 때 callback이 P-state 계산을 수행하고 driver에 hardware 변경을 요청합니다. Driver는 capability에 따라 scheduler context에서 직접 또는 kernel thread/workqueue로 비동기 호출됩니다.
모든 CPU가 offline이어서 inactive였다가 되살아난 policy는 기본 governor 대신 이전 governor를 재사용하려 합니다. 공유 policy의 다른 CPU가 이미 online이면 object 재초기화 없이 전체 policy에 governor `->stop()` 후 `->start()`를 호출해 새 CPU만 반영합니다.
`intel_pstate`는 governor를 연결하지 않고 driver `->setpolicy()`가 policy별 utilization callback을 등록합니다. Callback이 scheduler context에서 P-state 결정과 hardware 변경을 함께 수행합니다.
Scaling driver가 unregister되거나 policy의 마지막 CPU가 unregister되면 policy object와 연결 data structure를 해제합니다.
Policy sysfs ABI
197-347Kernel 초기화 때 CPUFreq core는 `/sys/devices/system/cpu/cpufreq/`를 만들고 policy마다 `policyX` subdirectory를 둡니다. 관련 CPU의 `/sys/devices/system/cpu/cpuY/cpufreq` symlink가 이 directory를 가리킵니다.
Policy directory의 generic attribute는 core가 만들며 driver/governor와 대체로 무관합니다. Driver는 policy 고유 동작을 위한 attribute를 추가할 수 있습니다.
| Attribute | 의미 |
|---|---|
| `affected_cpus` | Policy의 online CPU |
| `bios_limit` | Firmware가 요구한 frequency 상한 |
| `cpuinfo_cur_freq` | Hardware에서 얻은 실제 현재 frequency |
| `cpuinfo_avg_freq` | 수 ms 이하 구간의 hardware feedback 평균 |
| `cpuinfo_max_freq` | 가능한 최대 operating frequency |
| `cpuinfo_min_freq` | 가능한 최소 operating frequency |
| `cpuinfo_transition_latency` | P-state 전환 시간(ns) |
| `related_cpus` | Policy의 online/offline CPU 전체 |
| `scaling_available_frequencies` | 사용 가능한 frequency 목록 |
| `scaling_available_governors` | 연결 가능한 governor/driver algorithm |
| `scaling_cur_freq` | 마지막 요청값 또는 architecture가 추정한 현재값 |
| `scaling_driver` | 현재 scaling driver |
| `scaling_governor` | 현재 governor/driver algorithm; read-write |
| `scaling_max_freq` | 허용 최대 frequency; read-write |
| `scaling_min_freq` | 허용 최소 frequency; read-write |
| `scaling_setspeed` | `userspace` governor의 요청 frequency |
`bios_limit`은 BIOS setting, service processor 등 firmware/HW mechanism의 상한이며 generic thermal driver가 찾는 ACPI thermal limit은 포함하지 않습니다. Driver가 지원하지 않으면 파일도 없습니다.
`cpuinfo_avg_freq`는 AMU 같은 전용 hardware가 필요할 수 있고 current frequency 조회 실패 시 ARM idle CPU처럼 `EAGAIN`이 날 수 있습니다.
`scaling_cur_freq`는 보통 hardware에 마지막으로 요청한 P-state frequency여서 실제 값과 다를 수 있습니다. X86 등은 :c:macro:`CPUFREQ_ARCH_CUR_FREQ`로 더 가까운 값을 제공할 수 있지만 순간 hardware 값과 정확히 같다는 보장은 없습니다.
`scaling_governor`에는 `scaling_available_governors`의 이름만 쓸 수 있습니다. `scaling_max_freq`는 `scaling_min_freq`보다 낮을 수 없고, 최소값은 최대값보다 높을 수 없습니다. `scaling_setspeed`는 `userspace` governor에서만 동작합니다.
Generic governor와 tunable 위치
348-371CPUFreq의 generic governor는 모든 scaling driver와 사용할 수 있으며 하나의, 경우에 따라 parameter화된 algorithm을 구현합니다. Policy마다 다른 governor를 동시에 쓸 수 있지만 결과가 최적이 아닐 수 있습니다.
`scaling_governor`로 언제든 바꿀 수 있습니다. Driver가 per-policy tunable을 요구하면 각 policy 아래 governor 이름 directory에, 그렇지 않으면 `/sys/devices/system/cpu/cpufreq/<governor>/`에 system-wide tunable을 둡니다.
| Governor | 선택 방식 |
|---|---|
| `performance` | `scaling_max_freq` 안의 최고값 |
| `powersave` | `scaling_min_freq` 안의 최저값 |
| `userspace` | `scaling_setspeed`에 userspace가 쓰는 값 |
| `schedutil` | Scheduler utilization과 PELT |
| `ondemand` | 비동기 worker가 계산한 active-time load |
| `conservative` | Load threshold와 작은 단계의 점진적 변경 |
performance, powersave와 userspace
372-400`performance`는 policy 설정 시와 min/max limit 변경 때 `scaling_max_freq` 범위의 최고 frequency를 한 번 요청합니다.
`powersave`는 같은 시점에 `scaling_min_freq` 범위의 최저 frequency를 요청합니다.
`userspace`는 스스로 결정하지 않고 userspace가 `scaling_setspeed`에 써서 frequency를 지정하게 합니다. 정확한 값을 의도해도 hardware coordination, thermal·power limit 등으로 실제 frequency는 달라질 수 있습니다.
schedutil
401-453`schedutil`은 CPU scheduler의 utilization data를 사용하고 scheduler 내부 구조에 직접 접근합니다. 전부 scheduler context에서 실행하지만 driver가 그 context에서 frequency를 바꾸지 못하면 비동기로 호출합니다.
RT 또는 deadline class가 callback을 부르면 허용 최대 `scaling_max_freq`로 올립니다. CFS는 root control group의 Per-Entity Load Tracking(PELT) metric을 utilization 추정값으로 사용합니다.
f = 1.25 * ``f_0`` * ``util`` / ``max``| 항목 | 수식에서의 의미 |
|---|---|
| `util` | Root control group의 PELT utilization |
| `max` | `util`의 이론적 최대값 |
| `f_0` | Frequency-invariant이면 policy 최대값, 아니면 현재 frequency |
| `rate_limit_us` | 연속 계산 사이 최소 시간; 기본 transition latency x 1.5 또는 최대 2 ms |
IO-wait boosting은 scheduler가 :c:macro:`SCHED_CPUFREQ_IOWAIT`를 넘길 때 즉시 최대 frequency로 올린 뒤 수식 값으로 서서히 되돌립니다.
`rate_limit_us`는 연속 계산 사이 최소 시간으로 scheduler context overhead를 줄입니다. `schedutil`은 단순하고 scheduler와 긴밀하며 context switch overhead가 작고 scheduler와 같은 utilization metric을 써서 `ondemand`와 `conservative`의 대체로 간주됩니다.
ondemand
454-567`ondemand`는 worker 연속 실행 사이의 전체 시간 중 non-idle 비율을 CPU load로 봅니다. 여러 CPU가 policy를 공유하면 가장 큰 load를 policy 추정값으로 사용합니다.
Worker는 process context의 workqueue에서 비동기로 실행됩니다. Scheduler overhead는 작지만 context switch가 자주 생기고 P-state update가 불규칙할 수 있으며, worker 자신도 idle time을 조금 줄여 metric에 영향을 줍니다.
일반적으로 load에 비례해 frequency를 선택하되 load가 `up_threshold`를 넘으면 즉시 `scaling_max_freq`로 갑니다.
| Tunable | 동작 |
|---|---|
| `sampling_rate` | Worker 실행 주기(us) |
| `up_threshold` | 초과 시 허용 최고 frequency로 이동하는 load 비율 |
| `ignore_nice_load` | 양수 nice task 시간을 idle로 취급 |
| `sampling_down_factor` | 최고 frequency 유지 시간을 늘리는 임시 배수 1-100 |
| `powersave_bias` | 목표 감소율 또는 AMD sensitivity threshold 0-1000 |
`sampling_rate` 기본값은 각 policy `cpuinfo_transition_latency`에 50% 여유를 더한 값이고 최소값은 보통 scheduler tick 두 번 길이입니다. Per-policy일 때 기본 비율로 설정하는 원문 명령입니다.
# echo `$(($(cat cpuinfo_transition_latency) * 3 / 2))` > ondemand/sampling_rate`sampling_down_factor`는 bursty workload의 frequency fluctuation을 줄이는 대신 최고 capacity 유지 energy를 더 씁니다.
AMD frequency sensitivity driver가 없으면 원 목표 `f`에 다음 감소율을 적용하며 기본 bias는 0입니다.
f * (1 - ``powersave_bias`` / 1000)`drivers/cpufreq/amd_freq_sensitivity.c` driver가 load되면 기본 bias는 400입니다. AMD Family 16h 이후 hardware의 workload sensitivity 0-100%를 이용합니다. 0인 memory/IO-bound workload는 frequency 증가 이득이 없고, 100% CPU-bound workload는 큰 이득이 예상됩니다. Sensitivity가 threshold보다 낮으면 목표를 내려 over-provisioning을 피합니다.
conservative
568-610`conservative`도 `ondemand`와 같은 방식으로 load를 추정하지만 짧은 시간에 큰 frequency 변화를 피합니다. Battery system처럼 power supply capacity가 제한된 환경을 위해 작은 step으로 한 번씩 올리거나 내립니다.
| Tunable | 동작 |
|---|---|
| `freq_step` | 한 번에 바꿀 수 있는 최대 frequency 비율; 기본 5% |
| `down_threshold` | 증가·감소 방향을 정하는 load threshold; 기본 20% |
| `sampling_down_factor` | 감소를 증가보다 1-10배 늦추는 factor |
`freq_step=0`은 기본 5%를 사용하고 100은 사실상 min/max 사이를 주기적으로 전환합니다. Load가 `down_threshold`보다 크면 한 step 올리고 작으면 감소하며, 중간이면 유지합니다. `sampling_down_factor`는 감소 속도를 증가보다 느리게 합니다.
Frequency boost와 전역 knob
611-694일부 multicore processor는 chip 전체가 충분히 idle이고 thermal/power budget 아래일 때 일부 core를 package 지속 가능 frequency보다 잠시 높입니다. Intel은 Turbo Boost, AMD는 Turbo-Core 또는 Core Performance Boost라 부르며 여기서는 frequency boost로 통칭합니다.
X86 같은 hardware-based boost는 hardware가 결정하고, ARM 같은 software-based boost는 scaling driver가 trigger 여부와 시점을 정합니다.
전역 `/sys/devices/system/cpu/cpufreq/boost`는 system 전체 boost를 제어합니다. Driver가 boost를 지원하지 않거나 `intel_pstate`처럼 별도 interface를 제공하면 파일이 없습니다.
| 값 | 의미 |
|---|---|
| `1` | Boost 사용 허용; 실제 boost 중이라는 뜻은 아님 |
| `0` | Boost 완전 비활성화 |
`1`은 boost 사용 권한일 뿐 현재 어떤 CPU가 실제 boosting 중이라는 뜻은 아닙니다. 쓸 수 있는 값은 `0`과 `1`뿐입니다.
Boost는 scheduler tick보다 짧은 시간 scale에서 성능을 최적화하지만 문제도 일으킬 수 있습니다. BIOS에서 끄려면 reboot가 필요하므로 runtime knob가 유용합니다.
이유는 네 가지입니다. 제한된 battery 전원에서 일시 overclock의 energy 증가를 피하고, 성능보다 deterministic behavior가 중요한 상황을 만들고, reboot 없이 boost 효과를 비교하며, package 전체 load에 따른 single-thread benchmark 변동을 제거해 재현성을 높일 수 있습니다.
Legacy AMD cpb와 참고 문헌
695-725AMD `powernow-k8`의 policy별 `cpb` sysfs knob는 Core Performance Boost를 켜고 끕니다. `/sys/devices/system/cpu/cpufreq/policyX/cpb`에 있지만 실제 구현은 system-wide라 한 policy를 바꾸면 모두 같은 값으로 바뀝니다.
Hardware 지원 AMD processor에서는 계속 제공되지만 :c:macro:`CONFIG_X86_ACPI_CPUFREQ_CPB`로 제외할 수 있고 전역 `boost`는 항상 있습니다. 다른 system과 일관되고 향후 `cpb`가 사라질 수 있으므로 `boost` 사용을 강하게 권장합니다.
기반 hardware 기능이 없는 Intel 등의 processor에는 config가 켜져 있어도 `cpb`가 나타나지 않습니다.
참고 문헌 [1]은 Jonathan Corbet의 `Per-entity load tracking`입니다: `https://lwn.net/Articles/531853/`.
운영 요약
cpufreq.rst:1-725CPUFreq는 공유 hardware interface를 policy object로 묶고 governor 결정과 driver 제어를 연결합니다. Policy sysfs 한계, governor tunable, boost permission을 함께 확인해야 실제 frequency 동작을 해석할 수 있습니다.