요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
Performance domain과 root domain
sched-energy.rst:68-127performance domain은 같은 주파수 상태를 공유하는 CPU 집합이고 root domain은 scheduler가 admission과 balancing에 사용하는 CPU 집합입니다. 두 경계는 반드시 일치하지 않습니다.
Energy Model의 performance domain은 한 CPU의 부하가 같은 domain의 다른 CPU 주파수와 전력에도 영향을 준다는 사실을 표현합니다. EAS는 후보 CPU 하나만 계산하지 않고 그 CPU가 속한 domain의 합산 utilization과 선택될 performance state를 계산합니다.
후보 CPU별 에너지 추정
sched-energy.rst:128-277표의 수치는 개념 예입니다. 실제 비용은 Energy Model performance state와 domain utilization에서 계산합니다.
wake-up task를 각 후보 CPU에 더한 가상 utilization을 만들고, 모든 performance domain에 대해 필요한 frequency와 energy cost를 계산합니다. 후보 배치의 총 에너지 차이가 migration 비용과 정책상 margin을 넘어설 때 더 효율적인 CPU를 선택합니다.
prev_cpu에 그대로 두는 경우, Little CPU1으로 옮기는 경우, Big CPU3으로 옮기는 경우의 performance-domain 부하를 나란히 비교합니다.
각 CPU의 utilization을 domain OPP power에 비례 배분한 원문 계산을 후보별 총합으로 정리했습니다.
utilization은 capacity scale로 정규화되어야 합니다. CPU frequency invariance와 micro-architecture capacity invariance가 맞지 않으면 같은 숫자가 CPU마다 다른 실제 작업량을 뜻하게 되어 에너지 비교가 왜곡됩니다.
Over-utilization과 fallback
sched-energy.rst:278-316root domain의 가용 capacity에 비해 utilization이 너무 높아지면 에너지 최소화보다 처리량과 latency 보장이 우선입니다. scheduler는 overutilized 표시를 세우고 EAS의 세밀한 energy comparison 대신 일반 load balancing을 사용합니다.
EAS 활성 조건
sched-energy.rst:317-405- CPU capacity가 서로 다른 asymmetric topology여야 합니다.
- CPU performance domain을 설명하는 Energy Model이 등록되어야 합니다.
- Energy Model 계산 복잡도가 scheduler hot path에서 감당 가능한 수준이어야 합니다.
- cpufreq governor로 schedutil을 사용해 scheduler utilization과 DVFS 결정을 연결해야 합니다.
- frequency와 CPU capacity에 대해 scale-invariant utilization 신호가 제공되어야 합니다.
- SMT topology는 현재 EAS 가정과 충돌할 수 있으므로 지원 조건을 별도로 확인해야 합니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
=======================
Energy Aware Scheduling
=======================
1. Introduction
---------------
Energy Aware Scheduling (or EAS) gives the scheduler the ability to predict
the impact of its decisions on the energy consumed by CPUs. EAS relies on an
Energy Model (EM) of the CPUs to select an energy efficient CPU for each task,
with a minimal impact on throughput. This document aims at providing an
introduction on how EAS works, what are the main design decisions behind it, and
details what is needed to get it to run.
Before going any further, please note that at the time of writing::
/!\ EAS does not support platforms with symmetric CPU topologies /!\
EAS operates only on heterogeneous CPU topologies (such as Arm big.LITTLE)
because this is where the potential for saving energy through scheduling is
the highest.
The actual EM used by EAS is _not_ maintained by the scheduler, but by a
dedicated framework. For details about this framework and what it provides,
please refer to its documentation (see Documentation/power/energy-model.rst).
2. Background and Terminology
-----------------------------
To make it clear from the start:
- energy = [joule] (resource like a battery on powered devices)
- power = energy/time = [joule/second] = [watt]
The goal of EAS is to minimize energy, while still getting the job done. That
is, we want to maximize::
performance [inst/s]
--------------------
power [W]
which is equivalent to minimizing::
energy [J]
-----------
instruction
while still getting 'good' performance. It is essentially an alternative
optimization objective to the current performance-only objective for the
scheduler. This alternative considers two objectives: energy-efficiency and
performance.
The idea behind introducing an EM is to allow the scheduler to evaluate the
implications of its decisions rather than blindly applying energy-saving
techniques that may have positive effects only on some platforms. At the same
time, the EM must be as simple as possible to minimize the scheduler latency
impact.
In short, EAS changes the way CFS tasks are assigned to CPUs. When it is time
for the scheduler to decide where a task should run (during wake-up), the EM
is used to break the tie between several good CPU candidates and pick the one
that is predicted to yield the best energy consumption without harming the
system's throughput. The predictions made by EAS rely on specific elements of
knowledge about the platform's topology, which include the 'capacity' of CPUs,
and their respective energy costs.
3. Topology information
-----------------------
EAS (as well as the rest of the scheduler) uses the notion of 'capacity' to
differentiate CPUs with different computing throughput. The 'capacity' of a CPU
represents the amount of work it can absorb when running at its highest
frequency compared to the most capable CPU of the system. Capacity values are
normalized in a 1024 range, and are comparable with the utilization signals of
tasks and CPUs computed by the Per-Entity Load Tracking (PELT) mechanism. Thanks
to capacity and utilization values, EAS is able to estimate how big/busy a
task/CPU is, and to take this into consideration when evaluating performance vs
energy trade-offs. The capacity of CPUs is provided via arch-specific code
through the arch_scale_cpu_capacity() callback.
The rest of platform knowledge used by EAS is directly read from the Energy
Model (EM) framework. The EM of a platform is composed of a power cost table
per 'performance domain' in the system (see Documentation/power/energy-model.rst
for further details about performance domains).
The scheduler manages references to the EM objects in the topology code when the
scheduling domains are built, or re-built. For each root domain (rd), the
scheduler maintains a singly linked list of all performance domains intersecting
the current rd->span. Each node in the list contains a pointer to a struct
em_perf_domain as provided by the EM framework.
The lists are attached to the root domains in order to cope with exclusive
cpuset configurations. Since the boundaries of exclusive cpusets do not
necessarily match those of performance domains, the lists of different root
domains can contain duplicate elements.
Example 1.
Let us consider a platform with 12 CPUs, split in 3 performance domains
(pd0, pd4 and pd8), organized as follows::
CPUs: 0 1 2 3 4 5 6 7 8 9 10 11
PDs: |--pd0--|--pd4--|---pd8---|
RDs: |----rd1----|-----rd2-----|
Now, consider that userspace decided to split the system with two
exclusive cpusets, hence creating two independent root domains, each
containing 6 CPUs. The two root domains are denoted rd1 and rd2 in the
above figure. Since pd4 intersects with both rd1 and rd2, it will be
present in the linked list '->pd' attached to each of them:
* rd1->pd: pd0 -> pd4
* rd2->pd: pd4 -> pd8
Please note that the scheduler will create two duplicate list nodes for
pd4 (one for each list). However, both just hold a pointer to the same
shared data structure of the EM framework.
Since the access to these lists can happen concurrently with hotplug and other
things, they are protected by RCU, like the rest of topology structures
manipulated by the scheduler.
EAS also maintains a static key (sched_energy_present) which is enabled when at
least one root domain meets all conditions for EAS to start. Those conditions
are summarized in Section 6.
4. Energy-Aware task placement
------------------------------
EAS overrides the CFS task wake-up balancing code. It uses the EM of the
platform and the PELT signals to choose an energy-efficient target CPU during
wake-up balance. When EAS is enabled, select_task_rq_fair() calls
find_energy_efficient_cpu() to do the placement decision. This function looks
for the CPU with the highest spare capacity (CPU capacity - CPU utilization) in
each performance domain since it is the one which will allow us to keep the
frequency the lowest. Then, the function checks if placing the task there could
save energy compared to leaving it on prev_cpu, i.e. the CPU where the task ran
in its previous activation.
find_energy_efficient_cpu() uses compute_energy() to estimate what will be the
energy consumed by the system if the waking task was migrated. compute_energy()
looks at the current utilization landscape of the CPUs and adjusts it to
'simulate' the task migration. The EM framework provides the em_pd_energy() API
which computes the expected energy consumption of each performance domain for
the given utilization landscape.
An example of energy-optimized task placement decision is detailed below.
Example 2.
Let us consider a (fake) platform with 2 independent performance domains
composed of two CPUs each. CPU0 and CPU1 are little CPUs; CPU2 and CPU3
are big.
The scheduler must decide where to place a task P whose util_avg = 200
and prev_cpu = 0.
The current utilization landscape of the CPUs is depicted on the graph
below. CPUs 0-3 have a util_avg of 400, 100, 600 and 500 respectively
Each performance domain has three Operating Performance Points (OPPs).
The CPU capacity and power cost associated with each OPP is listed in
the Energy Model table. The util_avg of P is shown on the figures
below as 'PP'::
CPU util.
1024 - - - - - - - Energy Model
+-----------+-------------+
| Little | Big |
768 ============= +-----+-----+------+------+
| Cap | Pwr | Cap | Pwr |
+-----+-----+------+------+
512 =========== - ##- - - - - | 170 | 50 | 512 | 400 |
## ## | 341 | 150 | 768 | 800 |
341 -PP - - - - ## ## | 512 | 300 | 1024 | 1700 |
PP ## ## +-----+-----+------+------+
170 -## - - - - ## ##
## ## ## ##
------------ -------------
CPU0 CPU1 CPU2 CPU3
Current OPP: ===== Other OPP: - - - util_avg (100 each): ##
find_energy_efficient_cpu() will first look for the CPUs with the
maximum spare capacity in the two performance domains. In this example,
CPU1 and CPU3. Then it will estimate the energy of the system if P was
placed on either of them, and check if that would save some energy
compared to leaving P on CPU0. EAS assumes that OPPs follow utilization
(which is coherent with the behaviour of the schedutil CPUFreq
governor, see Section 6. for more details on this topic).
**Case 1. P is migrated to CPU1**::
1024 - - - - - - -
Energy calculation:
768 ============= * CPU0: 200 / 341 * 150 = 88
* CPU1: 300 / 341 * 150 = 131
* CPU2: 600 / 768 * 800 = 625
512 - - - - - - - ##- - - - - * CPU3: 500 / 768 * 800 = 520
## ## => total_energy = 1364
341 =========== ## ##
PP ## ##
170 -## - - PP- ## ##
## ## ## ##
------------ -------------
CPU0 CPU1 CPU2 CPU3
**Case 2. P is migrated to CPU3**::
1024 - - - - - - -
Energy calculation:
768 ============= * CPU0: 200 / 341 * 150 = 88
* CPU1: 100 / 341 * 150 = 43
PP * CPU2: 600 / 768 * 800 = 625
512 - - - - - - - ##- - -PP - * CPU3: 700 / 768 * 800 = 729
## ## => total_energy = 1485
341 =========== ## ##
## ##
170 -## - - - - ## ##
## ## ## ##
------------ -------------
CPU0 CPU1 CPU2 CPU3
**Case 3. P stays on prev_cpu / CPU 0**::
1024 - - - - - - -
Energy calculation:
768 ============= * CPU0: 400 / 512 * 300 = 234
* CPU1: 100 / 512 * 300 = 58
* CPU2: 600 / 768 * 800 = 625
512 =========== - ##- - - - - * CPU3: 500 / 768 * 800 = 520
## ## => total_energy = 1437
341 -PP - - - - ## ##
PP ## ##
170 -## - - - - ## ##
## ## ## ##
------------ -------------
CPU0 CPU1 CPU2 CPU3
From these calculations, the Case 1 has the lowest total energy. So CPU 1
is be the best candidate from an energy-efficiency standpoint.
Big CPUs are generally more power hungry than the little ones and are thus used
mainly when a task doesn't fit the littles. However, little CPUs aren't always
necessarily more energy-efficient than big CPUs. For some systems, the high OPPs
of the little CPUs can be less energy-efficient than the lowest OPPs of the
bigs, for example. So, if the little CPUs happen to have enough utilization at
a specific point in time, a small task waking up at that moment could be better
of executing on the big side in order to save energy, even though it would fit
on the little side.
And even in the case where all OPPs of the big CPUs are less energy-efficient
than those of the little, using the big CPUs for a small task might still, under
specific conditions, save energy. Indeed, placing a task on a little CPU can
result in raising the OPP of the entire performance domain, and that will
increase the cost of the tasks already running there. If the waking task is
placed on a big CPU, its own execution cost might be higher than if it was
running on a little, but it won't impact the other tasks of the little CPUs
which will keep running at a lower OPP. So, when considering the total energy
consumed by CPUs, the extra cost of running that one task on a big core can be
smaller than the cost of raising the OPP on the little CPUs for all the other
tasks.
The examples above would be nearly impossible to get right in a generic way, and
for all platforms, without knowing the cost of running at different OPPs on all
CPUs of the system. Thanks to its EM-based design, EAS should cope with them
correctly without too many troubles. However, in order to ensure a minimal
impact on throughput for high-utilization scenarios, EAS also implements another
mechanism called 'over-utilization'.
5. Over-utilization
-------------------
From a general standpoint, the use-cases where EAS can help the most are those
involving a light/medium CPU utilization. Whenever long CPU-bound tasks are
being run, they will require all of the available CPU capacity, and there isn't
much that can be done by the scheduler to save energy without severely harming
throughput. In order to avoid hurting performance with EAS, CPUs are flagged as
'over-utilized' as soon as they are used at more than 80% of their compute
capacity. As long as no CPUs are over-utilized in a root domain, load balancing
is disabled and EAS overridess the wake-up balancing code. EAS is likely to load
the most energy efficient CPUs of the system more than the others if that can be
done without harming throughput. So, the load-balancer is disabled to prevent
it from breaking the energy-efficient task placement found by EAS. It is safe to
do so when the system isn't overutilized since being below the 80% tipping point
implies that:
a. there is some idle time on all CPUs, so the utilization signals used by
EAS are likely to accurately represent the 'size' of the various tasks
in the system;
b. all tasks should already be provided with enough CPU capacity,
regardless of their nice values;
c. since there is spare capacity all tasks must be blocking/sleeping
regularly and balancing at wake-up is sufficient.
As soon as one CPU goes above the 80% tipping point, at least one of the three
assumptions above becomes incorrect. In this scenario, the 'overutilized' flag
is raised for the entire root domain, EAS is disabled, and the load-balancer is
re-enabled. By doing so, the scheduler falls back onto load-based algorithms for
wake-up and load balance under CPU-bound conditions. This provides a better
respect of the nice values of tasks.
Since the notion of overutilization largely relies on detecting whether or not
there is some idle time in the system, the CPU capacity 'stolen' by higher
(than CFS) scheduling classes (as well as IRQ) must be taken into account. As
such, the detection of overutilization accounts for the capacity used not only
by CFS tasks, but also by the other scheduling classes and IRQ.
6. Dependencies and requirements for EAS
----------------------------------------
Energy Aware Scheduling depends on the CPUs of the system having specific
hardware properties and on other features of the kernel being enabled. This
section lists these dependencies and provides hints as to how they can be met.
6.1 - Asymmetric CPU topology
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
As mentioned in the introduction, EAS is only supported on platforms with
asymmetric CPU topologies for now. This requirement is checked at run-time by
looking for the presence of the SD_ASYM_CPUCAPACITY_FULL flag when the scheduling
domains are built.
See Documentation/scheduler/sched-capacity.rst for requirements to be met for this
flag to be set in the sched_domain hierarchy.
Please note that EAS is not fundamentally incompatible with SMP, but no
significant savings on SMP platforms have been observed yet. This restriction
could be amended in the future if proven otherwise.
6.2 - Energy Model presence
^^^^^^^^^^^^^^^^^^^^^^^^^^^
EAS uses the EM of a platform to estimate the impact of scheduling decisions on
energy. So, your platform must provide power cost tables to the EM framework in
order to make EAS start. To do so, please refer to documentation of the
independent EM framework in Documentation/power/energy-model.rst.
Please also note that the scheduling domains need to be re-built after the
EM has been registered in order to start EAS.
EAS uses the EM to make a forecasting decision on energy usage and thus it is
more focused on the difference when checking possible options for task
placement. For EAS it doesn't matter whether the EM power values are expressed
in milli-Watts or in an 'abstract scale'.
6.3 - Energy Model complexity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
EAS does not impose any complexity limit on the number of PDs/OPPs/CPUs but
restricts the number of CPUs to EM_MAX_NUM_CPUS to prevent overflows during
the energy estimation.
6.4 - Schedutil governor
^^^^^^^^^^^^^^^^^^^^^^^^
EAS tries to predict at which OPP will the CPUs be running in the close future
in order to estimate their energy consumption. To do so, it is assumed that OPPs
of CPUs follow their utilization.
Although it is very difficult to provide hard guarantees regarding the accuracy
of this assumption in practice (because the hardware might not do what it is
told to do, for example), schedutil as opposed to other CPUFreq governors at
least _requests_ frequencies calculated using the utilization signals.
Consequently, the only sane governor to use together with EAS is schedutil,
because it is the only one providing some degree of consistency between
frequency requests and energy predictions.
Using EAS with any other governor than schedutil is not supported.
6.5 Scale-invariant utilization signals
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In order to make accurate prediction across CPUs and for all performance
states, EAS needs frequency-invariant and CPU-invariant PELT signals. These can
be obtained using the architecture-defined arch_scale{cpu,freq}_capacity()
callbacks.
Using EAS on a platform that doesn't implement these two callbacks is not
supported.
6.6 Multithreading (SMT)
^^^^^^^^^^^^^^^^^^^^^^^^
EAS in its current form is SMT unaware and is not able to leverage
multithreaded hardware to save energy. EAS considers threads as independent
CPUs, which can actually be counter-productive for both performance and energy.
EAS on SMT is not supported.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
EAS의 목적과 지원 범위
1-27Energy Aware Scheduling(EAS)은 scheduler가 task placement 결정이 CPU energy 소비에 미칠 영향을 예측하게 한다. CPU Energy Model(EM)을 이용해 throughput 손실을 최소화하면서 각 task가 실행될 energy-efficient CPU를 선택한다.
문서 작성 시점의 EAS는 symmetric CPU topology를 지원하지 않는다. scheduling으로 energy를 절약할 가능성이 큰 Arm big.LITTLE 같은 heterogeneous topology에서만 동작한다.
EAS가 사용하는 EM은 scheduler가 직접 유지하지 않고 독립 Energy Model framework가 제공한다. power/energy-model 문서에서 performance domain과 power cost table 등록 방식을 확인할 수 있다.
energy, power와 최적화 목표
28-67energy는 joule로 측정하는 유한 자원이고, power는 단위 시간당 energy인 joule/second 또는 watt다. EAS는 필요한 작업을 완료하면서 총 energy를 최소화한다.
maximize: performance [instructions/s] / power [W]
same objective: minimize energy [J] / instruction기존 performance-only scheduler 목표에 energy efficiency를 추가한 다목적 최적화다. EM이 없으면 특정 platform에서만 이득인 절전 기법을 scheduler가 맹목적으로 적용할 수 있다. 반대로 EM이 너무 복잡하면 wakeup latency를 늘리므로 필요한 차이를 예측할 만큼 단순해야 한다.
EAS는 CFS task가 wakeup할 때 여러 성능상 적합한 CPU candidate 사이의 tie를 energy 예측으로 푼다. throughput을 해치지 않는 후보 중 예상 energy가 가장 낮은 CPU를 고르며, platform별 CPU capacity와 energy cost를 입력으로 사용한다.
capacity, performance domain과 root domain
68-127CPU capacity는 system에서 가장 강한 CPU가 최고 frequency로 처리할 수 있는 양을 1024로 정규화한 상대 throughput이다. task와 CPU utilization은 PELT가 같은 1024 scale로 계산하므로 capacity와 직접 비교해 task 크기와 CPU 여유를 판단할 수 있다. architecture는 arch_scale_cpu_capacity() callback으로 값을 제공한다.
나머지 platform 정보는 EM framework의 performance domain별 power cost table에서 읽는다. scheduler가 scheduling domain을 build 또는 rebuild할 때 각 root domain(rd)의 span과 교차하는 모든 performance domain을 singly linked list로 보관하며, node는 EM의 struct em_perf_domain을 가리킨다.
exclusive cpuset 경계와 performance domain 경계는 일치하지 않을 수 있다. 12 CPU가 pd0={0-3}, pd4={4-7}, pd8={8-11}이고 root domain이 rd1={0-5}, rd2={6-11}이면 pd4가 두 rd와 모두 교차한다.
| root domain | CPU span | 연결된 EM PD list |
|---|---|---|
| rd1 | CPU 0-5 | pd0 → pd4 |
| rd2 | CPU 6-11 | pd4 → pd8 |
pd4 list node는 rd마다 별도로 생기지만 두 node가 가리키는 struct em_perf_domain은 같은 EM object다. hotplug과 topology 변경이 list 접근과 동시에 일어날 수 있어 다른 scheduler topology와 마찬가지로 RCU로 보호한다.
sched_energy_present static key는 root domain 하나 이상이 EAS 시작 조건을 모두 만족할 때 활성화된다.
pd4는 두 root domain에 걸치므로 list node는 중복되지만 EM data는 공유한다.
energy-aware wakeup placement
128-277EAS가 켜지면 CFS wakeup balancing의 select_task_rq_fair()가 find_energy_efficient_cpu()를 호출한다. 각 performance domain에서 spare capacity=capacity-utilization이 가장 큰 CPU를 candidate로 선택한다. domain의 최대 utilization이 낮아져 더 낮은 OPP를 유지할 가능성이 가장 큰 CPU이기 때문이다.
그 다음 waking task를 candidate로 옮긴 경우와 이전 activation의 prev_cpu에 남긴 경우를 비교한다. compute_energy()가 현재 CPU utilization landscape에 task migration을 가상 반영하고, em_pd_energy()가 각 performance domain의 예상 energy를 계산한다.
성능상 맞지 않는 CPU를 먼저 제외하고, domain별 여유가 가장 큰 후보만 EM으로 비교한다.
예제 platform은 CPU0-1 little PD와 CPU2-3 big PD로 구성된다. waking task P는 util_avg=200이고 prev_cpu=CPU0다. 현재 CPU utilization은 각각 400, 100, 600, 500이다.
| PD | OPP capacity/power 1 | OPP capacity/power 2 | OPP capacity/power 3 |
|---|---|---|---|
| Little CPU0-1 | 170 / 50 | 341 / 150 | 512 / 300 |
| Big CPU2-3 | 512 / 400 | 768 / 800 | 1024 / 1700 |
| CPU | 현재 util | PD 내 spare가 큰 후보 | P 배치 전 상태 |
|---|---|---|---|
| CPU0 little | 400 | 아니오 | prev_cpu |
| CPU1 little | 100 | 예 | little candidate |
| CPU2 big | 600 | 아니오 | 실행 중 |
| CPU3 big | 500 | 예 | big candidate |
EAS는 utilization에 맞춰 OPP가 선택된다고 가정하며 이는 schedutil CPUFreq governor의 동작과 일치한다. CPU1에 P를 두면 little PD 최대 util은 300이 되어 capacity 341 OPP를 쓰고, big PD는 capacity 768 OPP를 쓴다.
| case | CPU별 energy 계산 | total energy |
|---|---|---|
| 1: P → CPU1 | CPU0 200/341*150=88; CPU1 300/341*150=131; CPU2 600/768*800=625; CPU3 500/768*800=520 | 1364 |
| 2: P → CPU3 | CPU0 200/341*150=88; CPU1 100/341*150=43; CPU2 600/768*800=625; CPU3 700/768*800=729 | 1485 |
| 3: P → prev CPU0 | CPU0 400/512*300=234; CPU1 100/512*300=58; CPU2 600/768*800=625; CPU3 500/768*800=520 | 1437 |
case 1의 total energy 1364가 가장 낮아 CPU1이 선택된다. 원문 표에서는 migration 뒤 CPU0 util을 200으로 표시한다. 이는 waking P의 200을 이전 CPU utilization에서 빼고 candidate에 더해 simulation한다는 의미다.
big CPU가 보통 더 많은 power를 쓰므로 task가 little capacity에 맞지 않을 때 주로 사용하지만, 모든 상황에서 little이 더 energy-efficient한 것은 아니다. little의 높은 OPP가 big의 낮은 OPP보다 나쁠 수 있다.
big의 모든 OPP 효율이 더 낮아도 small task를 big에 두는 편이 system total energy를 줄일 수 있다. little에 task를 더해 domain 전체 OPP가 올라가면 이미 그 domain에서 실행 중인 모든 task 비용이 증가한다. big에서 waking task 하나가 추가로 쓰는 비용이 little domain 전체 OPP 상승 비용보다 작을 수 있다.
이런 platform별 trade-off를 일반 scheduler heuristic만으로 정확히 풀기 어렵기 때문에 EM이 필요하다. 다만 utilization이 매우 높을 때 energy placement가 throughput을 해치지 않도록 over-utilization mechanism을 별도로 사용한다.
over-utilization과 load balancer 전환
278-316EAS의 이득은 CPU utilization이 가볍거나 중간 정도일 때 가장 크다. 긴 CPU-bound task가 전체 capacity를 요구하면 throughput을 크게 낮추지 않고 절약할 energy가 거의 없다.
CPU 사용량이 compute capacity의 80%를 넘으면 over-utilized로 판단한다. root domain에 over-utilized CPU가 하나도 없으면 일반 load balancing을 끄고 EAS가 wakeup balancing을 대체한다. EAS는 energy-efficient CPU에 의도적으로 task를 더 모을 수 있으므로 일반 balancer가 그 배치를 다시 흩뜨리지 않게 한다.
- 모든 CPU가 80% 아래면 idle time이 있어 PELT utilization이 task 크기를 비교적 정확히 나타낸다.
- 모든 task가 nice 값과 무관하게 필요한 CPU capacity를 이미 받을 수 있다.
- spare capacity가 있으므로 task가 주기적으로 block/sleep하고 wakeup 시점 balancing만으로 충분하다.
CPU 하나라도 80%를 넘으면 가정 중 적어도 하나가 깨진다. root domain 전체에 overutilized flag를 세우고 EAS를 끄며 load balancer를 다시 켠다. CPU-bound 조건에서는 load 기반 wakeup과 periodic balancing이 task nice 값을 더 잘 반영한다.
over-utilization은 system에 idle time이 있는지 판단하는 개념이므로 CFS보다 높은 scheduling class와 IRQ가 가져간 CPU capacity도 포함한다. CFS task utilization만 보고 spare를 계산하면 실제로 사용할 수 없는 capacity를 있다고 오판할 수 있다.
root domain의 CPU 하나가 capacity 80%를 넘는 순간 throughput 중심 알고리즘으로 돌아간다.
EAS 활성화 조건
317-404EAS는 hardware topology와 kernel 기능이 특정 조건을 만족해야 시작한다. scheduler domain build 시 SD_ASYM_CPUCAPACITY_FULL flag를 찾아 asymmetric CPU topology인지 확인한다. EAS가 본질적으로 SMP와 양립할 수 없는 것은 아니지만 symmetric platform에서 의미 있는 절감이 관측되지 않아 현재 지원하지 않는다.
platform은 EM framework에 power cost table을 등록해야 한다. EM 등록 뒤 scheduling domain을 rebuild해야 EAS가 시작된다. EAS는 placement 후보 사이 energy 차이만 비교하므로 power 값은 milli-watt 같은 실제 단위여도 되고 일관된 abstract scale이어도 된다.
performance domain, OPP와 CPU 수의 알고리즘 복잡도에 별도 제한을 두지는 않지만 energy 계산 overflow를 막기 위해 CPU 수를 EM_MAX_NUM_CPUS로 제한한다.
EAS는 가까운 미래 CPU OPP가 utilization을 따라간다고 가정한다. hardware가 frequency request를 정확히 따르지 않을 수 있어 완전한 보장은 어렵지만, schedutil은 PELT utilization으로 frequency를 요청하므로 energy prediction과 일관성을 제공하는 유일하게 지원되는 governor다. 다른 CPUFreq governor와 EAS 조합은 지원하지 않는다.
서로 다른 CPU와 모든 performance state를 비교하려면 PELT signal이 frequency-invariant이고 CPU-invariant여야 한다. architecture가 arch_scale_cpu_capacity()와 arch_scale_freq_capacity() callback을 구현해야 하며, 둘 중 하나라도 없으면 EAS를 지원하지 않는다.
현재 EAS는 SMT를 인식하지 못하고 hardware thread를 독립 CPU로 취급한다. 이 모델은 performance와 energy 모두에 역효과를 낼 수 있으므로 SMT platform에서는 지원하지 않는다.
| 요구 조건 | 검사 또는 제공 요소 |
|---|---|
| Asymmetric topology | SD_ASYM_CPUCAPACITY_FULL |
| Energy Model | performance domain별 power cost table, 등록 뒤 sched domain rebuild |
| CPUFreq governor | schedutil |
| Scale invariance | arch_scale_cpu_capacity(), arch_scale_freq_capacity() |
| CPU 수 | EM_MAX_NUM_CPUS 이하 |
| SMT | 지원하지 않음 |
EAS가 해결하려는 문제
sched-energy.rst:1-67Energy Aware Scheduling은 비대칭 CPU 시스템에서 task를 어느 CPU에 둘지 결정할 때 처리 용량과 예상 에너지를 함께 비교합니다. 단순히 가장 한가한 CPU를 고르면 작은 task가 big CPU를 깨우거나 performance domain의 주파수를 올려 시스템 전체 에너지가 증가할 수 있습니다.
EAS는 CFS wake-up placement 경로에서 사용되며 과부하 상태에서는 일반 load balancing으로 돌아갑니다. 따라서 EAS는 항상 적용되는 별도 scheduler가 아니라 특정 조건에서 CFS의 CPU 선택을 보강하는 정책입니다.