요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
=========
Workqueue
=========
:Date: September, 2010
:Author: Tejun Heo <tj@kernel.org>
:Author: Florian Mickler <florian@mickler.org>
Introduction
============
There are many cases where an asynchronous process execution context
is needed and the workqueue (wq) API is the most commonly used
mechanism for such cases.
When such an asynchronous execution context is needed, a work item
describing which function to execute is put on a queue. An
independent thread serves as the asynchronous execution context. The
queue is called workqueue and the thread is called worker.
While there are work items on the workqueue the worker executes the
functions associated with the work items one after the other. When
there is no work item left on the workqueue the worker becomes idle.
When a new work item gets queued, the worker begins executing again.
Why Concurrency Managed Workqueue?
==================================
In the original wq implementation, a multi threaded (MT) wq had one
worker thread per CPU and a single threaded (ST) wq had one worker
thread system-wide. A single MT wq needed to keep around the same
number of workers as the number of CPUs. The kernel grew a lot of MT
wq users over the years and with the number of CPU cores continuously
rising, some systems saturated the default 32k PID space just booting
up.
Although MT wq wasted a lot of resource, the level of concurrency
provided was unsatisfactory. The limitation was common to both ST and
MT wq albeit less severe on MT. Each wq maintained its own separate
worker pool. An MT wq could provide only one execution context per CPU
while an ST wq one for the whole system. Work items had to compete for
those very limited execution contexts leading to various problems
including proneness to deadlocks around the single execution context.
The tension between the provided level of concurrency and resource
usage also forced its users to make unnecessary tradeoffs like libata
choosing to use ST wq for polling PIOs and accepting an unnecessary
limitation that no two polling PIOs can progress at the same time. As
MT wq don't provide much better concurrency, users which require
higher level of concurrency, like async or fscache, had to implement
their own thread pool.
Concurrency Managed Workqueue (cmwq) is a reimplementation of wq with
focus on the following goals.
* Maintain compatibility with the original workqueue API.
* Use per-CPU unified worker pools shared by all wq to provide
flexible level of concurrency on demand without wasting a lot of
resource.
* Automatically regulate worker pool and level of concurrency so that
the API users don't need to worry about such details.
The Design
==========
In order to ease the asynchronous execution of functions a new
abstraction, the work item, is introduced.
A work item is a simple struct that holds a pointer to the function
that is to be executed asynchronously. Whenever a driver or subsystem
wants a function to be executed asynchronously it has to set up a work
item pointing to that function and queue that work item on a
workqueue.
A work item can be executed in either a thread or the BH (softirq) context.
For threaded workqueues, special purpose threads, called [k]workers, execute
the functions off of the queue, one after the other. If no work is queued,
the worker threads become idle. These worker threads are managed in
worker-pools.
The cmwq design differentiates between the user-facing workqueues that
subsystems and drivers queue work items on and the backend mechanism
which manages worker-pools and processes the queued work items.
There are two worker-pools, one for normal work items and the other
for high priority ones, for each possible CPU and some extra
worker-pools to serve work items queued on unbound workqueues - the
number of these backing pools is dynamic.
BH workqueues use the same framework. However, as there can only be one
concurrent execution context, there's no need to worry about concurrency.
Each per-CPU BH worker pool contains only one pseudo worker which represents
the BH execution context. A BH workqueue can be considered a convenience
interface to softirq.
Subsystems and drivers can create and queue work items through special
workqueue API functions as they see fit. They can influence some
aspects of the way the work items are executed by setting flags on the
workqueue they are putting the work item on. These flags include
things like CPU locality, concurrency limits, priority and more. To
get a detailed overview refer to the API description of
``alloc_workqueue()`` below.
When a work item is queued to a workqueue, the target worker-pool is
determined according to the queue parameters and workqueue attributes
and appended on the shared worklist of the worker-pool. For example,
unless specifically overridden, a work item of a bound workqueue will
be queued on the worklist of either normal or highpri worker-pool that
is associated to the CPU the issuer is running on.
For any thread pool implementation, managing the concurrency level
(how many execution contexts are active) is an important issue. cmwq
tries to keep the concurrency at a minimal but sufficient level.
Minimal to save resources and sufficient in that the system is used at
its full capacity.
Each worker-pool bound to an actual CPU implements concurrency
management by hooking into the scheduler. The worker-pool is notified
whenever an active worker wakes up or sleeps and keeps track of the
number of the currently runnable workers. Generally, work items are
not expected to hog a CPU and consume many cycles. That means
maintaining just enough concurrency to prevent work processing from
stalling should be optimal. As long as there are one or more runnable
workers on the CPU, the worker-pool doesn't start execution of a new
work, but, when the last running worker goes to sleep, it immediately
schedules a new worker so that the CPU doesn't sit idle while there
are pending work items. This allows using a minimal number of workers
without losing execution bandwidth.
Keeping idle workers around doesn't cost other than the memory space
for kthreads, so cmwq holds onto idle ones for a while before killing
them.
For unbound workqueues, the number of backing pools is dynamic.
Unbound workqueue can be assigned custom attributes using
``apply_workqueue_attrs()`` and workqueue will automatically create
backing worker pools matching the attributes. The responsibility of
regulating concurrency level is on the users. There is also a flag to
mark a bound wq to ignore the concurrency management. Please refer to
the API section for details.
Forward progress guarantee relies on that workers can be created when
more execution contexts are necessary, which in turn is guaranteed
through the use of rescue workers. All work items which might be used
on code paths that handle memory reclaim are required to be queued on
wq's that have a rescue-worker reserved for execution under memory
pressure. Else it is possible that the worker-pool deadlocks waiting
for execution contexts to free up.
Application Programming Interface (API)
=======================================
``alloc_workqueue()`` allocates a wq. The original
``create_*workqueue()`` functions are deprecated and scheduled for
removal. ``alloc_workqueue()`` takes three arguments - ``@name``,
``@flags`` and ``@max_active``. ``@name`` is the name of the wq and
also used as the name of the rescuer thread if there is one.
A wq no longer manages execution resources but serves as a domain for
forward progress guarantee, flush and work item attributes. ``@flags``
and ``@max_active`` control how work items are assigned execution
resources, scheduled and executed.
``flags``
---------
``WQ_BH``
BH workqueues can be considered a convenience interface to softirq. BH
workqueues are always per-CPU and all BH work items are executed in the
queueing CPU's softirq context in the queueing order.
All BH workqueues must have 0 ``max_active`` and ``WQ_HIGHPRI`` is the
only allowed additional flag.
BH work items cannot sleep. All other features such as delayed queueing,
flushing and canceling are supported.
``WQ_PERCPU``
Work items queued to a per-cpu wq are bound to a specific CPU.
This flag is the right choice when cpu locality is important.
This flag is the complement of ``WQ_UNBOUND``.
``WQ_UNBOUND``
Work items queued to an unbound wq are served by the special
worker-pools which host workers which are not bound to any
specific CPU. This makes the wq behave as a simple execution
context provider without concurrency management. The unbound
worker-pools try to start execution of work items as soon as
possible. Unbound wq sacrifices locality but is useful for
the following cases.
* Wide fluctuation in the concurrency level requirement is
expected and using bound wq may end up creating large number
of mostly unused workers across different CPUs as the issuer
hops through different CPUs.
* Long running CPU intensive workloads which can be better
managed by the system scheduler.
``WQ_FREEZABLE``
A freezable wq participates in the freeze phase of the system
suspend operations. Work items on the wq are drained and no
new work item starts execution until thawed.
``WQ_MEM_RECLAIM``
All wq which might be used in the memory reclaim paths **MUST**
have this flag set. The wq is guaranteed to have at least one
execution context regardless of memory pressure.
``WQ_HIGHPRI``
Work items of a highpri wq are queued to the highpri
worker-pool of the target cpu. Highpri worker-pools are
served by worker threads with elevated nice level.
Note that normal and highpri worker-pools don't interact with
each other. Each maintains its separate pool of workers and
implements concurrency management among its workers.
``WQ_CPU_INTENSIVE``
Work items of a CPU intensive wq do not contribute to the
concurrency level. In other words, runnable CPU intensive
work items will not prevent other work items in the same
worker-pool from starting execution. This is useful for bound
work items which are expected to hog CPU cycles so that their
execution is regulated by the system scheduler.
Although CPU intensive work items don't contribute to the
concurrency level, start of their executions is still
regulated by the concurrency management and runnable
non-CPU-intensive work items can delay execution of CPU
intensive work items.
This flag is meaningless for unbound wq.
``max_active``
--------------
``@max_active`` determines the maximum number of execution contexts per
CPU which can be assigned to the work items of a wq. For example, with
``@max_active`` of 16, at most 16 work items of the wq can be executing
at the same time per CPU. This is always a per-CPU attribute, even for
unbound workqueues.
The maximum limit for ``@max_active`` is 2048 and the default value used
when 0 is specified is 1024. These values are chosen sufficiently high
such that they are not the limiting factor while providing protection in
runaway cases.
The number of active work items of a wq is usually regulated by the
users of the wq, more specifically, by how many work items the users
may queue at the same time. Unless there is a specific need for
throttling the number of active work items, specifying '0' is
recommended.
Some users depend on strict execution ordering where only one work item
is in flight at any given time and the work items are processed in
queueing order. While the combination of ``@max_active`` of 1 and
``WQ_UNBOUND`` used to achieve this behavior, this is no longer the
case. Use alloc_ordered_workqueue() instead.
Example Execution Scenarios
===========================
The following example execution scenarios try to illustrate how cmwq
behave under different configurations.
Work items w0, w1, w2 are queued to a bound wq q0 on the same CPU.
w0 burns CPU for 5ms then sleeps for 10ms then burns CPU for 5ms
again before finishing. w1 and w2 burn CPU for 5ms then sleep for
10ms.
Ignoring all other tasks, works and processing overhead, and assuming
simple FIFO scheduling, the following is one highly simplified version
of possible sequences of events with the original wq. ::
TIME IN MSECS EVENT
0 w0 starts and burns CPU
5 w0 sleeps
15 w0 wakes up and burns CPU
20 w0 finishes
20 w1 starts and burns CPU
25 w1 sleeps
35 w1 wakes up and finishes
35 w2 starts and burns CPU
40 w2 sleeps
50 w2 wakes up and finishes
And with cmwq with ``@max_active`` >= 3, ::
TIME IN MSECS EVENT
0 w0 starts and burns CPU
5 w0 sleeps
5 w1 starts and burns CPU
10 w1 sleeps
10 w2 starts and burns CPU
15 w2 sleeps
15 w0 wakes up and burns CPU
20 w0 finishes
20 w1 wakes up and finishes
25 w2 wakes up and finishes
If ``@max_active`` == 2, ::
TIME IN MSECS EVENT
0 w0 starts and burns CPU
5 w0 sleeps
5 w1 starts and burns CPU
10 w1 sleeps
15 w0 wakes up and burns CPU
20 w0 finishes
20 w1 wakes up and finishes
20 w2 starts and burns CPU
25 w2 sleeps
35 w2 wakes up and finishes
Now, let's assume w1 and w2 are queued to a different wq q1 which has
``WQ_CPU_INTENSIVE`` set, ::
TIME IN MSECS EVENT
0 w0 starts and burns CPU
5 w0 sleeps
5 w1 and w2 start and burn CPU
10 w1 sleeps
15 w2 sleeps
15 w0 wakes up and burns CPU
20 w0 finishes
20 w1 wakes up and finishes
25 w2 wakes up and finishes
Guidelines
==========
* Do not forget to use ``WQ_MEM_RECLAIM`` if a wq may process work
items which are used during memory reclaim. Each wq with
``WQ_MEM_RECLAIM`` set has an execution context reserved for it. If
there is dependency among multiple work items used during memory
reclaim, they should be queued to separate wq each with
``WQ_MEM_RECLAIM``.
* Unless strict ordering is required, there is no need to use ST wq.
* Unless there is a specific need, using 0 for @max_active is
recommended. In most use cases, concurrency level usually stays
well under the default limit.
* A wq serves as a domain for forward progress guarantee
(``WQ_MEM_RECLAIM``, flush and work item attributes. Work items
which are not involved in memory reclaim and don't need to be
flushed as a part of a group of work items, and don't require any
special attribute, can use one of the system wq. There is no
difference in execution characteristics between using a dedicated wq
and a system wq.
Note: If something may generate more than @max_active outstanding
work items (do stress test your producers), it may saturate a system
wq and potentially lead to deadlock. It should utilize its own
dedicated workqueue rather than the system wq.
* Unless work items are expected to consume a huge amount of CPU
cycles, using a bound wq is usually beneficial due to the increased
level of locality in wq operations and work item execution.
Affinity Scopes
===============
An unbound workqueue groups CPUs according to its affinity scope to improve
cache locality. For example, if a workqueue is using the default affinity
scope of "cache", it will group CPUs according to last level cache
boundaries. A work item queued on the workqueue will be assigned to a worker
on one of the CPUs which share the last level cache with the issuing CPU.
Once started, the worker may or may not be allowed to move outside the scope
depending on the ``affinity_strict`` setting of the scope.
Workqueue currently supports the following affinity scopes.
``default``
Use the scope in module parameter ``workqueue.default_affinity_scope``
which is always set to one of the scopes below.
``cpu``
CPUs are not grouped. A work item issued on one CPU is processed by a
worker on the same CPU. This makes unbound workqueues behave as per-cpu
workqueues without concurrency management.
``smt``
CPUs are grouped according to SMT boundaries. This usually means that the
logical threads of each physical CPU core are grouped together.
``cache``
CPUs are grouped according to cache boundaries. Which specific cache
boundary is used is determined by the arch code. L3 is used in a lot of
cases. This is the default affinity scope.
``numa``
CPUs are grouped according to NUMA boundaries.
``system``
All CPUs are put in the same group. Workqueue makes no effort to process a
work item on a CPU close to the issuing CPU.
The default affinity scope can be changed with the module parameter
``workqueue.default_affinity_scope`` and a specific workqueue's affinity
scope can be changed using ``apply_workqueue_attrs()``.
If ``WQ_SYSFS`` is set, the workqueue will have the following affinity scope
related interface files under its ``/sys/devices/virtual/workqueue/WQ_NAME/``
directory.
``affinity_scope``
Read to see the current affinity scope. Write to change.
When default is the current scope, reading this file will also show the
current effective scope in parentheses, for example, ``default (cache)``.
``affinity_strict``
0 by default indicating that affinity scopes are not strict. When a work
item starts execution, workqueue makes a best-effort attempt to ensure
that the worker is inside its affinity scope, which is called
repatriation. Once started, the scheduler is free to move the worker
anywhere in the system as it sees fit. This enables benefiting from scope
locality while still being able to utilize other CPUs if necessary and
available.
If set to 1, all workers of the scope are guaranteed always to be in the
scope. This may be useful when crossing affinity scopes has other
implications, for example, in terms of power consumption or workload
isolation. Strict NUMA scope can also be used to match the workqueue
behavior of older kernels.
Affinity Scopes and Performance
===============================
It'd be ideal if an unbound workqueue's behavior is optimal for vast
majority of use cases without further tuning. Unfortunately, in the current
kernel, there exists a pronounced trade-off between locality and utilization
necessitating explicit configurations when workqueues are heavily used.
Higher locality leads to higher efficiency where more work is performed for
the same number of consumed CPU cycles. However, higher locality may also
cause lower overall system utilization if the work items are not spread
enough across the affinity scopes by the issuers. The following performance
testing with dm-crypt clearly illustrates this trade-off.
The tests are run on a CPU with 12-cores/24-threads split across four L3
caches (AMD Ryzen 9 3900x). CPU clock boost is turned off for consistency.
``/dev/dm-0`` is a dm-crypt device created on NVME SSD (Samsung 990 PRO) and
opened with ``cryptsetup`` with default settings.
Scenario 1: Enough issuers and work spread across the machine
-------------------------------------------------------------
The command used: ::
$ fio --filename=/dev/dm-0 --direct=1 --rw=randrw --bs=32k --ioengine=libaio \
--iodepth=64 --runtime=60 --numjobs=24 --time_based --group_reporting \
--name=iops-test-job --verify=sha512
There are 24 issuers, each issuing 64 IOs concurrently. ``--verify=sha512``
makes ``fio`` generate and read back the content each time which makes
execution locality matter between the issuer and ``kcryptd``. The following
are the read bandwidths and CPU utilizations depending on different affinity
scope settings on ``kcryptd`` measured over five runs. Bandwidths are in
MiBps, and CPU util in percents.
.. list-table::
:widths: 16 20 20
:header-rows: 1
* - Affinity
- Bandwidth (MiBps)
- CPU util (%)
* - system
- 1159.40 ±1.34
- 99.31 ±0.02
* - cache
- 1166.40 ±0.89
- 99.34 ±0.01
* - cache (strict)
- 1166.00 ±0.71
- 99.35 ±0.01
With enough issuers spread across the system, there is no downside to
"cache", strict or otherwise. All three configurations saturate the whole
machine but the cache-affine ones outperform by 0.6% thanks to improved
locality.
Scenario 2: Fewer issuers, enough work for saturation
-----------------------------------------------------
The command used: ::
$ fio --filename=/dev/dm-0 --direct=1 --rw=randrw --bs=32k \
--ioengine=libaio --iodepth=64 --runtime=60 --numjobs=8 \
--time_based --group_reporting --name=iops-test-job --verify=sha512
The only difference from the previous scenario is ``--numjobs=8``. There are
a third of the issuers but is still enough total work to saturate the
system.
.. list-table::
:widths: 16 20 20
:header-rows: 1
* - Affinity
- Bandwidth (MiBps)
- CPU util (%)
* - system
- 1155.40 ±0.89
- 97.41 ±0.05
* - cache
- 1154.40 ±1.14
- 96.15 ±0.09
* - cache (strict)
- 1112.00 ±4.64
- 93.26 ±0.35
This is more than enough work to saturate the system. Both "system" and
"cache" are nearly saturating the machine but not fully. "cache" is using
less CPU but the better efficiency puts it at the same bandwidth as
"system".
Eight issuers moving around over four L3 cache scope still allow "cache
(strict)" to mostly saturate the machine but the loss of work conservation
is now starting to hurt with 3.7% bandwidth loss.
Scenario 3: Even fewer issuers, not enough work to saturate
-----------------------------------------------------------
The command used: ::
$ fio --filename=/dev/dm-0 --direct=1 --rw=randrw --bs=32k \
--ioengine=libaio --iodepth=64 --runtime=60 --numjobs=4 \
--time_based --group_reporting --name=iops-test-job --verify=sha512
Again, the only difference is ``--numjobs=4``. With the number of issuers
reduced to four, there now isn't enough work to saturate the whole system
and the bandwidth becomes dependent on completion latencies.
.. list-table::
:widths: 16 20 20
:header-rows: 1
* - Affinity
- Bandwidth (MiBps)
- CPU util (%)
* - system
- 993.60 ±1.82
- 75.49 ±0.06
* - cache
- 973.40 ±1.52
- 74.90 ±0.07
* - cache (strict)
- 828.20 ±4.49
- 66.84 ±0.29
Now, the tradeoff between locality and utilization is clearer. "cache" shows
2% bandwidth loss compared to "system" and "cache (struct)" whopping 20%.
Conclusion and Recommendations
------------------------------
In the above experiments, the efficiency advantage of the "cache" affinity
scope over "system" is, while consistent and noticeable, small. However, the
impact is dependent on the distances between the scopes and may be more
pronounced in processors with more complex topologies.
While the loss of work-conservation in certain scenarios hurts, it is a lot
better than "cache (strict)" and maximizing workqueue utilization is
unlikely to be the common case anyway. As such, "cache" is the default
affinity scope for unbound pools.
* As there is no one option which is great for most cases, workqueue usages
that may consume a significant amount of CPU are recommended to configure
the workqueues using ``apply_workqueue_attrs()`` and/or enable
``WQ_SYSFS``.
* An unbound workqueue with strict "cpu" affinity scope behaves the same as
``WQ_CPU_INTENSIVE`` per-cpu workqueue. There is no real advanage to the
latter and an unbound workqueue provides a lot more flexibility.
* Affinity scopes are introduced in Linux v6.5. To emulate the previous
behavior, use strict "numa" affinity scope.
* The loss of work-conservation in non-strict affinity scopes is likely
originating from the scheduler. There is no theoretical reason why the
kernel wouldn't be able to do the right thing and maintain
work-conservation in most cases. As such, it is possible that future
scheduler improvements may make most of these tunables unnecessary.
Examining Configuration
=======================
Use tools/workqueue/wq_dump.py to examine unbound CPU affinity
configuration, worker pools and how workqueues map to the pools: ::
$ tools/workqueue/wq_dump.py
Affinity Scopes
===============
wq_unbound_cpumask=0000000f
CPU
nr_pods 4
pod_cpus [0]=00000001 [1]=00000002 [2]=00000004 [3]=00000008
pod_node [0]=0 [1]=0 [2]=1 [3]=1
cpu_pod [0]=0 [1]=1 [2]=2 [3]=3
SMT
nr_pods 4
pod_cpus [0]=00000001 [1]=00000002 [2]=00000004 [3]=00000008
pod_node [0]=0 [1]=0 [2]=1 [3]=1
cpu_pod [0]=0 [1]=1 [2]=2 [3]=3
CACHE (default)
nr_pods 2
pod_cpus [0]=00000003 [1]=0000000c
pod_node [0]=0 [1]=1
cpu_pod [0]=0 [1]=0 [2]=1 [3]=1
NUMA
nr_pods 2
pod_cpus [0]=00000003 [1]=0000000c
pod_node [0]=0 [1]=1
cpu_pod [0]=0 [1]=0 [2]=1 [3]=1
SYSTEM
nr_pods 1
pod_cpus [0]=0000000f
pod_node [0]=-1
cpu_pod [0]=0 [1]=0 [2]=0 [3]=0
Worker Pools
============
pool[00] ref= 1 nice= 0 idle/workers= 4/ 4 cpu= 0
pool[01] ref= 1 nice=-20 idle/workers= 2/ 2 cpu= 0
pool[02] ref= 1 nice= 0 idle/workers= 4/ 4 cpu= 1
pool[03] ref= 1 nice=-20 idle/workers= 2/ 2 cpu= 1
pool[04] ref= 1 nice= 0 idle/workers= 4/ 4 cpu= 2
pool[05] ref= 1 nice=-20 idle/workers= 2/ 2 cpu= 2
pool[06] ref= 1 nice= 0 idle/workers= 3/ 3 cpu= 3
pool[07] ref= 1 nice=-20 idle/workers= 2/ 2 cpu= 3
pool[08] ref=42 nice= 0 idle/workers= 6/ 6 cpus=0000000f
pool[09] ref=28 nice= 0 idle/workers= 3/ 3 cpus=00000003
pool[10] ref=28 nice= 0 idle/workers= 17/ 17 cpus=0000000c
pool[11] ref= 1 nice=-20 idle/workers= 1/ 1 cpus=0000000f
pool[12] ref= 2 nice=-20 idle/workers= 1/ 1 cpus=00000003
pool[13] ref= 2 nice=-20 idle/workers= 1/ 1 cpus=0000000c
Workqueue CPU -> pool
=====================
[ workqueue \ CPU 0 1 2 3 dfl]
events percpu 0 2 4 6
events_highpri percpu 1 3 5 7
events_long percpu 0 2 4 6
events_unbound unbound 9 9 10 10 8
events_freezable percpu 0 2 4 6
events_power_efficient percpu 0 2 4 6
events_freezable_pwr_ef percpu 0 2 4 6
rcu_gp percpu 0 2 4 6
rcu_par_gp percpu 0 2 4 6
slub_flushwq percpu 0 2 4 6
netns ordered 8 8 8 8 8
...
See the command's help message for more info.
Monitoring
==========
Use tools/workqueue/wq_monitor.py to monitor workqueue operations: ::
$ tools/workqueue/wq_monitor.py events
total infl CPUtime CPUhog CMW/RPR mayday rescued
events 18545 0 6.1 0 5 - -
events_highpri 8 0 0.0 0 0 - -
events_long 3 0 0.0 0 0 - -
events_unbound 38306 0 0.1 - 7 - -
events_freezable 0 0 0.0 0 0 - -
events_power_efficient 29598 0 0.2 0 0 - -
events_freezable_pwr_ef 10 0 0.0 0 0 - -
sock_diag_events 0 0 0.0 0 0 - -
total infl CPUtime CPUhog CMW/RPR mayday rescued
events 18548 0 6.1 0 5 - -
events_highpri 8 0 0.0 0 0 - -
events_long 3 0 0.0 0 0 - -
events_unbound 38322 0 0.1 - 7 - -
events_freezable 0 0 0.0 0 0 - -
events_power_efficient 29603 0 0.2 0 0 - -
events_freezable_pwr_ef 10 0 0.0 0 0 - -
sock_diag_events 0 0 0.0 0 0 - -
...
See the command's help message for more info.
Debugging
=========
Because the work functions are executed by generic worker threads
there are a few tricks needed to shed some light on misbehaving
workqueue users.
Worker threads show up in the process list as: ::
root 5671 0.0 0.0 0 0 ? S 12:07 0:00 [kworker/0:1]
root 5672 0.0 0.0 0 0 ? S 12:07 0:00 [kworker/1:2]
root 5673 0.0 0.0 0 0 ? S 12:12 0:00 [kworker/0:0]
root 5674 0.0 0.0 0 0 ? S 12:13 0:00 [kworker/1:0]
If kworkers are going crazy (using too much cpu), there are two types
of possible problems:
1. Something being scheduled in rapid succession
2. A single work item that consumes lots of cpu cycles
The first one can be tracked using tracing: ::
$ echo workqueue:workqueue_queue_work > /sys/kernel/tracing/set_event
$ cat /sys/kernel/tracing/trace_pipe > out.txt
(wait a few secs)
^C
If something is busy looping on work queueing, it would be dominating
the output and the offender can be determined with the work item
function.
For the second type of problems it should be possible to just check
the stack trace of the offending worker thread. ::
$ cat /proc/THE_OFFENDING_KWORKER/stack
The work item's function should be trivially visible in the stack
trace.
Non-reentrance Conditions
=========================
Workqueue guarantees that a work item cannot be re-entrant if the following
conditions hold after a work item gets queued:
1. The work function hasn't been changed.
2. No one queues the work item to another workqueue.
3. The work item hasn't been reinitiated.
In other words, if the above conditions hold, the work item is guaranteed to be
executed by at most one worker system-wide at any given time.
Note that requeuing the work item (to the same queue) in the self function
doesn't break these conditions, so it's safe to do. Otherwise, caution is
required when breaking the conditions inside a work function.
Kernel Inline Documentations Reference
======================================
.. kernel-doc:: include/linux/workqueue.h
.. kernel-doc:: kernel/workqueue.c
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
Workqueue
1-9Workqueue
날짜: 2010년 9월
저자: Tejun Heo <tj@kernel.org>
저자: Florian Mickler <florian@mickler.org>
소개
10-27소개
Asynchronous process execution context가 필요한 경우는 많으며, workqueue(wq) API가 이런 상황에서 가장 흔히 사용하는 mechanism입니다.
이러한 asynchronous execution context가 필요하면 실행할 function을 설명하는 work item을 queue에 넣습니다. 독립 thread가 asynchronous execution context 역할을 합니다. Queue를 workqueue라 하고 thread를 worker라고 합니다.
Workqueue에 work item이 있는 동안 worker는 각 work item과 연결된 function을 차례로 실행합니다. Work item이 남지 않으면 worker는 idle 상태가 되고, 새 work item이 queue에 들어오면 다시 실행을 시작합니다.
Concurrency managed workqueue가 필요한 이유
28-67Concurrency Managed Workqueue가 필요한 이유
원래 wq 구현에서 multi-threaded(MT) wq는 CPU마다 worker thread 하나를 두었고 single-threaded(ST) wq는 system 전체에 worker thread 하나를 두었습니다. MT wq 하나만으로도 CPU 수만큼 worker를 유지해야 했습니다. Kernel의 MT wq 사용자가 늘고 CPU core 수도 계속 증가하면서 일부 system은 boot만 해도 기본 32k PID space를 소진했습니다.
MT wq는 resource를 많이 낭비하면서도 제공하는 concurrency 수준은 만족스럽지 못했습니다. MT에서는 덜 심했지만 ST와 MT 모두 각 wq가 별도 worker pool을 유지했습니다. MT wq는 CPU마다 execution context 하나만, ST wq는 system 전체에 하나만 제공했습니다. Work item은 제한된 context를 두고 경쟁했고 단일 execution context 주변의 deadlock 가능성 등 여러 문제가 생겼습니다.
Concurrency 수준과 resource 사용량 사이의 긴장은 불필요한 tradeoff도 강요했습니다. 예를 들어 libata는 polling PIO에 ST wq를 사용하면서 polling PIO 둘이 동시에 진행할 수 없다는 제약을 받아들였습니다. MT wq도 concurrency가 크게 낫지 않아 async나 fscache처럼 더 높은 concurrency가 필요한 사용자는 자체 thread pool을 구현해야 했습니다.
Concurrency Managed Workqueue(cmwq)는 다음 목표에 초점을 맞춘 wq 재구현입니다.
- 기존 workqueue API와 compatibility를 유지합니다.
- 모든 wq가 공유하는 per-CPU unified worker pool로 resource를 크게 낭비하지 않으면서 필요에 따라 유연한 concurrency를 제공합니다.
- Worker pool과 concurrency 수준을 자동으로 조절하여 API 사용자가 세부 사항을 걱정하지 않게 합니다.
설계
68-156설계
Function의 asynchronous execution을 쉽게 하기 위해 work item이라는 abstraction을 도입합니다.
Work item은 비동기로 실행할 function pointer를 보유한 단순한 struct입니다. Driver나 subsystem이 function을 비동기로 실행하려면 그 function을 가리키는 work item을 설정하고 workqueue에 queue해야 합니다.
Work item은 thread 또는 BH(softirq) context에서 실행할 수 있습니다.
Threaded workqueue에서는 [k]worker라고 부르는 special-purpose thread가 queue의 function을 차례로 실행합니다. Queue된 work가 없으면 worker thread는 idle 상태가 됩니다. 이 thread들은 worker-pool에서 관리합니다.
Cmwq 설계는 subsystem과 driver가 work item을 넣는 user-facing workqueue와 worker-pool을 관리하고 queue된 item을 처리하는 backend mechanism을 구분합니다.
가능한 CPU마다 normal work item용과 high-priority item용 worker-pool 두 개가 있고, unbound workqueue에 들어온 item을 처리하는 추가 worker-pool이 있습니다. 추가 backing pool 수는 동적으로 변합니다.
BH workqueue도 같은 framework를 사용합니다. 다만 concurrent execution context가 하나뿐이므로 concurrency를 관리할 필요가 없습니다. Per-CPU BH worker pool은 BH execution context를 나타내는 pseudo worker 하나만 포함합니다. BH workqueue는 softirq의 convenience interface로 볼 수 있습니다.
Subsystem과 driver는 필요에 따라 전용 workqueue API function으로 work item을 만들고 queue할 수 있습니다. Workqueue flag로 CPU locality, concurrency limit, priority 등 실행 방식을 조정합니다. 자세한 내용은 아래 `alloc_workqueue()` API 설명을 참조하십시오.
Work item이 workqueue에 queue되면 queue parameter와 workqueue attribute에 따라 target worker-pool을 정하고 그 pool의 shared worklist에 추가합니다. 별도 override가 없다면 bound workqueue item은 issuer가 실행 중인 CPU에 연결된 normal 또는 highpri worker-pool의 worklist로 갑니다.
모든 thread pool 구현에서 active execution context 수인 concurrency level 관리는 중요합니다. Cmwq는 resource를 절약할 만큼 작으면서 system capacity를 모두 사용할 만큼 충분한 수준을 유지하려 합니다.
실제 CPU에 bound된 각 worker-pool은 scheduler에 hook하여 concurrency를 관리합니다. Active worker가 깨거나 잠들 때 notification을 받고 현재 runnable worker 수를 추적합니다. Work item은 보통 CPU를 독점해 cycle을 많이 소비하지 않는다고 가정하므로 처리가 멈추지 않을 만큼의 concurrency만 유지하는 것이 최적입니다.
CPU에 runnable worker가 하나 이상 있으면 worker-pool은 새 work 실행을 시작하지 않습니다. 마지막 running worker가 잠들면 pending work가 있는데 CPU가 idle 상태가 되지 않도록 새 worker를 즉시 schedule합니다. 이 방식은 execution bandwidth를 잃지 않으면서 worker 수를 최소화합니다.
Idle worker를 유지하는 비용은 kthread memory뿐이므로 cmwq는 일정 시간 유지한 뒤 제거합니다.
Unbound workqueue의 backing pool 수는 동적입니다. `apply_workqueue_attrs()`로 custom attribute를 지정하면 workqueue가 일치하는 backing worker pool을 자동 생성합니다. Concurrency level 조절 책임은 사용자에게 있습니다. Bound wq가 concurrency management를 무시하게 하는 flag도 있습니다.
Forward progress 보장은 execution context가 더 필요할 때 worker를 생성할 수 있어야 하며 rescue worker가 이를 보장합니다. Memory reclaim 처리 code path에서 사용할 수 있는 모든 work item은 memory pressure 아래에서 실행할 rescue worker를 예약한 wq에 queue해야 합니다. 그렇지 않으면 worker-pool이 execution context가 비기를 기다리며 deadlock될 수 있습니다.
Application Programming Interface
157-171Application Programming Interface(API)
`alloc_workqueue()`는 wq를 할당합니다. 기존 `create_*workqueue()` function은 deprecated 상태이며 제거될 예정입니다. `alloc_workqueue()`는 `@name`, `@flags`, `@max_active` 세 argument를 받습니다. `@name`은 wq 이름이며 rescuer thread가 있으면 그 이름으로도 사용합니다.
Wq는 더 이상 execution resource를 직접 관리하지 않고 forward progress guarantee, flush, work item attribute를 위한 domain 역할을 합니다. `@flags`와 `@max_active`가 work item에 execution resource를 배정하고 schedule 및 실행하는 방법을 제어합니다.
flags
172-244`flags`
`WQ_BH`: BH workqueue는 softirq의 convenience interface입니다. 항상 per-CPU이며 모든 BH work item은 queue한 CPU의 softirq context에서 queue 순서대로 실행됩니다. `max_active`는 반드시 0이어야 하며 추가로 허용되는 flag는 `WQ_HIGHPRI`뿐입니다. BH work item은 sleep할 수 없지만 delayed queueing, flushing, canceling 등 다른 기능은 지원합니다.
`WQ_PERCPU`: Per-CPU wq에 queue한 work item을 특정 CPU에 bind합니다. CPU locality가 중요할 때 적합하며 `WQ_UNBOUND`의 반대입니다.
`WQ_UNBOUND`: 특정 CPU에 bind되지 않은 worker가 있는 special worker-pool이 item을 처리합니다. Concurrency management 없는 단순 execution context provider처럼 동작하며 가능한 빨리 실행을 시작합니다. Locality를 희생하지만 다음 경우 유용합니다.
- 필요한 concurrency 수준의 변동이 크고 issuer가 CPU 사이를 이동할 때 bound wq를 사용하면 서로 다른 CPU에 거의 쓰지 않는 worker를 많이 만들 수 있는 경우
- System scheduler가 더 잘 관리할 수 있는 오래 실행되는 CPU-intensive workload
`WQ_FREEZABLE`: System suspend의 freeze phase에 참여합니다. Wq의 work item을 drain하고 thaw될 때까지 새 item 실행을 시작하지 않습니다.
`WQ_MEM_RECLAIM`: Memory reclaim path에서 사용할 수 있는 모든 wq는 반드시 이 flag를 설정해야 합니다. Memory pressure와 관계없이 최소 하나의 execution context를 보장합니다.
`WQ_HIGHPRI`: Highpri wq의 work item은 target CPU의 highpri worker-pool로 갑니다. 높은 nice level의 worker thread가 처리합니다. Normal pool과 highpri pool은 상호작용하지 않고 각각 별도 worker pool과 concurrency management를 유지합니다.
`WQ_CPU_INTENSIVE`: CPU-intensive wq item은 concurrency level 계산에 기여하지 않습니다. Runnable CPU-intensive item이 같은 worker-pool의 다른 item 실행을 막지 않습니다. CPU cycle을 독점할 것으로 예상되는 bound item의 실행을 system scheduler가 조절하게 할 때 유용합니다.
CPU-intensive item도 실행 시작은 concurrency management의 조절을 받으며 runnable non-CPU-intensive item이 실행을 지연시킬 수 있습니다. 이 flag는 unbound wq에는 의미가 없습니다.
max_active
245-271`max_active`
`@max_active`는 wq의 work item에 배정할 수 있는 CPU당 최대 execution context 수를 정합니다. 예를 들어 값이 16이면 CPU마다 해당 wq의 work item을 최대 16개 동시에 실행할 수 있습니다. Unbound workqueue에서도 항상 per-CPU attribute입니다.
최댓값은 2048이고 0을 지정했을 때 기본값은 1024입니다. Runaway case를 방지하면서 일반적으로 제한 요소가 되지 않도록 충분히 큰 값입니다.
Active work item 수는 보통 wq 사용자가 동시에 queue하는 item 수로 조절합니다. Active item 수를 throttle할 특별한 필요가 없다면 0을 지정하는 것이 좋습니다.
언제나 work item 하나만 in-flight 상태이고 queue 순서대로 처리하는 strict ordering이 필요한 경우가 있습니다. 예전에는 `@max_active` 1과 `WQ_UNBOUND` 조합으로 가능했지만 이제는 그렇지 않습니다. 대신 `alloc_ordered_workqueue()`를 사용하십시오.
실행 시나리오 예제
272-341실행 시나리오 예제
다음 예는 configuration에 따라 cmwq가 어떻게 동작하는지 보여줍니다. Work item `w0`, `w1`, `w2`를 같은 CPU의 bound wq `q0`에 넣습니다. `w0`은 CPU를 5ms 사용하고 10ms sleep한 뒤 다시 5ms 사용하고 끝납니다. `w1`과 `w2`는 CPU를 5ms 사용한 뒤 10ms sleep합니다.
다른 task, work, processing overhead를 무시하고 단순 FIFO scheduling을 가정할 때 기존 wq에서 가능한 event sequence의 매우 단순화한 예는 다음과 같습니다.
| 시간(ms) | Event |
|---|---|
| 0 | w0 starts and burns CPU |
| 5 | w0 sleeps |
| 15 | w0 wakes up and burns CPU |
| 20 | w0 finishes |
| 20 | w1 starts and burns CPU |
| 25 | w1 sleeps |
| 35 | w1 wakes up and finishes |
| 35 | w2 starts and burns CPU |
| 40 | w2 sleeps |
| 50 | w2 wakes up and finishes |
`@max_active >= 3`인 cmwq에서는 다음과 같습니다.
| 시간(ms) | Event |
|---|---|
| 0 | w0 starts and burns CPU |
| 5 | w0 sleeps |
| 5 | w1 starts and burns CPU |
| 10 | w1 sleeps |
| 10 | w2 starts and burns CPU |
| 15 | w2 sleeps |
| 15 | w0 wakes up and burns CPU |
| 20 | w0 finishes |
| 20 | w1 wakes up and finishes |
| 25 | w2 wakes up and finishes |
`@max_active == 2`이면 다음과 같습니다.
| 시간(ms) | Event |
|---|---|
| 0 | w0 starts and burns CPU |
| 5 | w0 sleeps |
| 5 | w1 starts and burns CPU |
| 10 | w1 sleeps |
| 15 | w0 wakes up and burns CPU |
| 20 | w0 finishes |
| 20 | w1 wakes up and finishes |
| 20 | w2 starts and burns CPU |
| 25 | w2 sleeps |
| 35 | w2 wakes up and finishes |
`w1`과 `w2`를 `WQ_CPU_INTENSIVE`가 설정된 다른 wq `q1`에 넣는다고 가정하면 다음과 같습니다.
| 시간(ms) | Event |
|---|---|
| 0 | w0 starts and burns CPU |
| 5 | w0 sleeps |
| 5 | w1 and w2 start and burn CPU |
| 10 | w1 sleeps |
| 15 | w2 sleeps |
| 15 | w0 wakes up and burns CPU |
| 20 | w0 finishes |
| 20 | w1 wakes up and finishes |
| 25 | w2 wakes up and finishes |
지침
342-375지침
- Memory reclaim 중 사용하는 work item을 wq가 처리할 수 있다면 `WQ_MEM_RECLAIM`을 잊지 말아야 합니다. Reclaim에서 여러 work item 사이에 dependency가 있다면 각각 `WQ_MEM_RECLAIM`을 설정한 별도 wq에 넣으십시오.
- Strict ordering이 필요하지 않다면 ST wq를 사용할 필요가 없습니다.
- 특별한 필요가 없다면 `@max_active`에 0을 사용하는 것이 좋습니다. 대부분의 use case에서 concurrency level은 기본 limit보다 훨씬 낮습니다.
- Wq는 `WQ_MEM_RECLAIM`, flush, work item attribute에 대한 forward progress guarantee domain입니다. Reclaim과 관계없고 group flush나 special attribute가 필요 없는 item은 system wq 중 하나를 사용할 수 있으며 dedicated wq와 실행 특성 차이가 없습니다.
- Producer stress test 결과 `@max_active`보다 많은 outstanding item을 만들 수 있다면 system wq를 포화시켜 deadlock을 일으킬 수 있으므로 dedicated workqueue를 사용해야 합니다.
- Work item이 CPU cycle을 매우 많이 소비할 것으로 예상되지 않는다면 wq operation과 item 실행 locality가 높은 bound wq가 보통 유리합니다.
Affinity scope
376-443Affinity scope
Unbound workqueue는 cache locality를 높이도록 affinity scope에 따라 CPU를 group화합니다. 기본 `cache` scope라면 last-level cache boundary에 따라 group을 만들고, work item은 issuer CPU와 last-level cache를 공유하는 CPU의 worker에 배정됩니다. 시작 후 worker가 scope 밖으로 이동할 수 있는지는 `affinity_strict` 설정에 달려 있습니다.
현재 지원하는 affinity scope는 다음과 같습니다.
`default`: 항상 아래 scope 중 하나로 설정되는 module parameter `workqueue.default_affinity_scope`의 scope를 사용합니다.
`cpu`: CPU를 group화하지 않습니다. 한 CPU에서 발행한 item은 같은 CPU의 worker가 처리하며, unbound workqueue가 concurrency management 없는 per-CPU workqueue처럼 동작합니다.
`smt`: SMT boundary에 따라 CPU를 group화합니다. 보통 각 physical CPU core의 logical thread를 함께 묶습니다.
`cache`: Cache boundary에 따라 CPU를 group화합니다. 구체적인 cache boundary는 architecture code가 정하며 흔히 L3를 사용합니다. 기본 affinity scope입니다.
`numa`: NUMA boundary에 따라 CPU를 group화합니다.
`system`: 모든 CPU를 같은 group에 넣습니다. Issuer CPU와 가까운 CPU에서 item을 처리하려고 시도하지 않습니다.
기본 affinity scope는 `workqueue.default_affinity_scope` module parameter로 바꾸며, 특정 workqueue scope는 `apply_workqueue_attrs()`로 바꿀 수 있습니다.
`WQ_SYSFS`를 설정하면 `/sys/devices/virtual/workqueue/WQ_NAME/` directory 아래에 affinity 관련 interface file이 생깁니다.
`affinity_scope`: 읽으면 현재 scope를 보고 쓰면 변경합니다. 현재 scope가 default이면 `default (cache)`처럼 괄호 안에 현재 effective scope도 표시합니다.
`affinity_strict`: 기본값 0은 scope가 strict하지 않음을 나타냅니다. Item 실행을 시작할 때 worker가 scope 안에 있도록 best-effort로 시도하는 repatriation을 수행하지만, 시작 후 scheduler는 필요에 따라 worker를 system 어디로든 옮길 수 있습니다. Scope locality와 다른 가용 CPU 활용을 함께 얻습니다.
1로 설정하면 scope의 모든 worker가 항상 scope 안에 있다고 보장합니다. Scope 횡단이 power consumption이나 workload isolation에 영향을 줄 때 유용합니다. Strict NUMA scope로 구형 kernel의 workqueue 동작을 맞출 수도 있습니다.
Affinity scope와 성능
444-463Affinity scope와 성능
추가 tuning 없이도 unbound workqueue가 대다수 use case에서 최적으로 동작하면 이상적이지만, 현재 kernel에서는 locality와 utilization 사이 tradeoff가 뚜렷해 workqueue를 많이 사용할 때 명시적 configuration이 필요합니다.
높은 locality는 같은 CPU cycle로 더 많은 work를 수행하는 효율을 높입니다. 그러나 issuer가 work item을 affinity scope 전체에 충분히 분산하지 않으면 전체 system utilization을 낮출 수 있습니다. 다음 dm-crypt performance test가 이 tradeoff를 보여줍니다.
Test는 4개 L3 cache에 나뉜 12-core/24-thread CPU인 AMD Ryzen 9 3900x에서 수행했으며 일관성을 위해 CPU clock boost를 껐습니다. `/dev/dm-0`은 Samsung 990 PRO NVME SSD 위에 기본 `cryptsetup` 설정으로 만든 dm-crypt device입니다.
시나리오 1: 충분한 issuer와 system 전체에 분산된 work
464-505시나리오 1: 충분한 issuer와 machine 전체에 분산된 work
사용한 command:
$ fio --filename=/dev/dm-0 --direct=1 --rw=randrw --bs=32k --ioengine=libaio \
--iodepth=64 --runtime=60 --numjobs=24 --time_based --group_reporting \
--name=iops-test-job --verify=sha512Issuer 24개가 각각 IO 64개를 동시에 발행합니다. `--verify=sha512`는 fio가 매번 content를 생성하고 다시 읽게 하므로 issuer와 `kcryptd` 사이 execution locality가 중요합니다. 다음은 다섯 번 실행해 측정한 `kcryptd` affinity scope별 read bandwidth와 CPU utilization입니다. Bandwidth 단위는 MiBps, CPU util 단위는 percent입니다.
| Affinity | Bandwidth (MiBps) | CPU util (%) |
|---|---|---|
| system | 1159.40 ±1.34 | 99.31 ±0.02 |
| cache | 1166.40 ±0.89 | 99.34 ±0.01 |
| cache (strict) | 1166.00 ±0.71 | 99.35 ±0.01 |
System 전체에 issuer가 충분히 분산되면 strict 여부와 관계없이 `cache`의 단점이 없습니다. 세 configuration 모두 machine 전체를 포화시키며 cache-affine configuration은 향상된 locality 덕분에 0.6% 높은 성능을 보입니다.
시나리오 2: 더 적은 issuer와 포화에 충분한 work
506-548시나리오 2: 더 적은 issuer와 포화에 충분한 work
사용한 command:
$ fio --filename=/dev/dm-0 --direct=1 --rw=randrw --bs=32k \
--ioengine=libaio --iodepth=64 --runtime=60 --numjobs=8 \
--time_based --group_reporting --name=iops-test-job --verify=sha512이전 시나리오와 다른 점은 `--numjobs=8`뿐입니다. Issuer는 3분의 1이지만 total work는 여전히 system을 포화시키기에 충분합니다.
| Affinity | Bandwidth (MiBps) | CPU util (%) |
|---|---|---|
| system | 1155.40 ±0.89 | 97.41 ±0.05 |
| cache | 1154.40 ±1.14 | 96.15 ±0.09 |
| cache (strict) | 1112.00 ±4.64 | 93.26 ±0.35 |
System을 포화시키기에 충분한 work입니다. `system`과 `cache` 모두 machine을 거의 포화시키지만 완전하지는 않습니다. `cache`는 CPU를 덜 사용하지만 효율이 좋아 `system`과 같은 bandwidth를 냅니다.
Issuer 8개가 L3 cache scope 4개 사이를 이동해 `cache (strict)`도 machine을 거의 포화시키지만 work conservation 손실이 영향을 주기 시작하여 bandwidth가 3.7% 감소합니다.
시나리오 3: 더 적은 issuer와 포화에 부족한 work
549-585시나리오 3: 더 적은 issuer와 포화에 부족한 work
사용한 command:
$ fio --filename=/dev/dm-0 --direct=1 --rw=randrw --bs=32k \
--ioengine=libaio --iodepth=64 --runtime=60 --numjobs=4 \
--time_based --group_reporting --name=iops-test-job --verify=sha512다시 다른 점은 `--numjobs=4`뿐입니다. Issuer가 4개로 줄어 system 전체를 포화시킬 work가 부족하고 bandwidth는 completion latency에 의존합니다.
| Affinity | Bandwidth (MiBps) | CPU util (%) |
|---|---|---|
| system | 993.60 ±1.82 | 75.49 ±0.06 |
| cache | 973.40 ±1.52 | 74.90 ±0.07 |
| cache (strict) | 828.20 ±4.49 | 66.84 ±0.29 |
이제 locality와 utilization의 tradeoff가 더 분명합니다. `cache`는 `system`보다 bandwidth가 2% 낮고, 원문의 `cache (struct)` 표기는 20%나 낮다고 설명합니다.
결론과 권장 사항
586-617결론과 권장 사항
위 실험에서 `cache` affinity scope가 `system`보다 갖는 efficiency 이점은 일관되고 눈에 띄지만 작습니다. 다만 scope 사이 거리에 따라 영향이 달라지고 topology가 복잡한 processor에서는 더 커질 수 있습니다.
일부 시나리오에서 work conservation 손실이 해롭지만 `cache (strict)`보다는 훨씬 낫고, workqueue utilization 극대화가 흔한 case일 가능성도 낮습니다. 따라서 `cache`가 unbound pool의 기본 affinity scope입니다.
- 대부분의 case에 훌륭한 단일 option이 없으므로 CPU를 많이 소비할 수 있는 workqueue 사용자는 `apply_workqueue_attrs()`로 구성하거나 `WQ_SYSFS`를 활성화하는 것이 좋습니다.
- Strict `cpu` affinity scope의 unbound workqueue는 `WQ_CPU_INTENSIVE` per-CPU workqueue와 똑같이 동작합니다. 후자에 실질적 이점이 없고 unbound workqueue가 훨씬 유연합니다.
- Affinity scope는 Linux v6.5에서 도입되었습니다. 이전 동작을 모방하려면 strict `numa` affinity scope를 사용하십시오.
- Non-strict affinity scope의 work conservation 손실은 scheduler에서 비롯된 것으로 보입니다. 이론적으로 kernel이 올바르게 동작하지 못할 이유가 없으므로 향후 scheduler 개선으로 tuning 항목 대부분이 불필요해질 수 있습니다.
Configuration 검사
618-694Configuration 검사
`tools/workqueue/wq_dump.py`로 unbound CPU affinity configuration, worker pool 및 workqueue와 pool의 mapping을 검사합니다.
$ tools/workqueue/wq_dump.py
Affinity Scopes
===============
wq_unbound_cpumask=0000000f
CPU
nr_pods 4
pod_cpus [0]=00000001 [1]=00000002 [2]=00000004 [3]=00000008
pod_node [0]=0 [1]=0 [2]=1 [3]=1
cpu_pod [0]=0 [1]=1 [2]=2 [3]=3
SMT
nr_pods 4
pod_cpus [0]=00000001 [1]=00000002 [2]=00000004 [3]=00000008
pod_node [0]=0 [1]=0 [2]=1 [3]=1
cpu_pod [0]=0 [1]=1 [2]=2 [3]=3
CACHE (default)
nr_pods 2
pod_cpus [0]=00000003 [1]=0000000c
pod_node [0]=0 [1]=1
cpu_pod [0]=0 [1]=0 [2]=1 [3]=1
NUMA
nr_pods 2
pod_cpus [0]=00000003 [1]=0000000c
pod_node [0]=0 [1]=1
cpu_pod [0]=0 [1]=0 [2]=1 [3]=1
SYSTEM
nr_pods 1
pod_cpus [0]=0000000f
pod_node [0]=-1
cpu_pod [0]=0 [1]=0 [2]=0 [3]=0
Worker Pools
============
pool[00] ref= 1 nice= 0 idle/workers= 4/ 4 cpu= 0
pool[01] ref= 1 nice=-20 idle/workers= 2/ 2 cpu= 0
pool[02] ref= 1 nice= 0 idle/workers= 4/ 4 cpu= 1
pool[03] ref= 1 nice=-20 idle/workers= 2/ 2 cpu= 1
pool[04] ref= 1 nice= 0 idle/workers= 4/ 4 cpu= 2
pool[05] ref= 1 nice=-20 idle/workers= 2/ 2 cpu= 2
pool[06] ref= 1 nice= 0 idle/workers= 3/ 3 cpu= 3
pool[07] ref= 1 nice=-20 idle/workers= 2/ 2 cpu= 3
pool[08] ref=42 nice= 0 idle/workers= 6/ 6 cpus=0000000f
pool[09] ref=28 nice= 0 idle/workers= 3/ 3 cpus=00000003
pool[10] ref=28 nice= 0 idle/workers= 17/ 17 cpus=0000000c
pool[11] ref= 1 nice=-20 idle/workers= 1/ 1 cpus=0000000f
pool[12] ref= 2 nice=-20 idle/workers= 1/ 1 cpus=00000003
pool[13] ref= 2 nice=-20 idle/workers= 1/ 1 cpus=0000000c
Workqueue CPU -> pool
=====================
[ workqueue \ CPU 0 1 2 3 dfl]
events percpu 0 2 4 6
events_highpri percpu 1 3 5 7
events_long percpu 0 2 4 6
events_unbound unbound 9 9 10 10 8
events_freezable percpu 0 2 4 6
events_power_efficient percpu 0 2 4 6
events_freezable_pwr_ef percpu 0 2 4 6
rcu_gp percpu 0 2 4 6
rcu_par_gp percpu 0 2 4 6
slub_flushwq percpu 0 2 4 6
netns ordered 8 8 8 8 8
...`wq_dump.py` 출력의 affinity pod 구성과 backing pool mapping을 구조화했습니다.
자세한 내용은 command help message를 참조하십시오.
Monitoring
695-725Monitoring
`tools/workqueue/wq_monitor.py`로 workqueue operation을 monitor합니다.
$ tools/workqueue/wq_monitor.py events
total infl CPUtime CPUhog CMW/RPR mayday rescued
events 18545 0 6.1 0 5 - -
events_highpri 8 0 0.0 0 0 - -
events_long 3 0 0.0 0 0 - -
events_unbound 38306 0 0.1 - 7 - -
events_freezable 0 0 0.0 0 0 - -
events_power_efficient 29598 0 0.2 0 0 - -
events_freezable_pwr_ef 10 0 0.0 0 0 - -
sock_diag_events 0 0 0.0 0 0 - -
total infl CPUtime CPUhog CMW/RPR mayday rescued
events 18548 0 6.1 0 5 - -
events_highpri 8 0 0.0 0 0 - -
events_long 3 0 0.0 0 0 - -
events_unbound 38322 0 0.1 - 7 - -
events_freezable 0 0 0.0 0 0 - -
events_power_efficient 29603 0 0.2 0 0 - -
events_freezable_pwr_ef 10 0 0.0 0 0 - -
sock_diag_events 0 0 0.0 0 0 - -
...자세한 내용은 command help message를 참조하십시오.
Debugging
726-765Debugging
Work function은 generic worker thread가 실행하므로 잘못 동작하는 workqueue 사용자를 파악하려면 몇 가지 방법이 필요합니다.
Process list에서 worker thread는 다음처럼 나타납니다.
root 5671 0.0 0.0 0 0 ? S 12:07 0:00 [kworker/0:1]
root 5672 0.0 0.0 0 0 ? S 12:07 0:00 [kworker/1:2]
root 5673 0.0 0.0 0 0 ? S 12:12 0:00 [kworker/0:0]
root 5674 0.0 0.0 0 0 ? S 12:13 0:00 [kworker/1:0]Kworker가 CPU를 너무 많이 사용한다면 가능한 문제는 두 종류입니다.
- 어떤 항목이 빠르게 연속 schedule되는 경우
- Work item 하나가 CPU cycle을 많이 소비하는 경우
첫 번째 문제는 tracing으로 추적할 수 있습니다.
$ echo workqueue:workqueue_queue_work > /sys/kernel/tracing/set_event
$ cat /sys/kernel/tracing/trace_pipe > out.txt
(wait a few secs)
^CWork queueing을 busy loop로 수행하는 항목은 output을 지배하므로 work item function으로 원인을 찾을 수 있습니다.
두 번째 문제는 문제가 있는 worker thread의 stack trace를 확인하면 됩니다.
$ cat /proc/THE_OFFENDING_KWORKER/stackStack trace에서 work item function이 쉽게 보여야 합니다.
Non-reentrance 조건
766-783Non-reentrance 조건
Work item이 queue된 뒤 다음 조건을 만족하면 workqueue는 그 item이 re-entrant하지 않음을 보장합니다.
- Work function이 변경되지 않았습니다.
- 누구도 work item을 다른 workqueue에 queue하지 않습니다.
- Work item을 다시 초기화하지 않았습니다.
즉 위 조건을 만족하면 어느 시점에도 system 전체에서 worker 하나만 work item을 실행한다고 보장합니다.
Self function 안에서 같은 queue로 work item을 다시 queue하는 것은 이 조건을 깨지 않으므로 안전합니다. 그 외에는 work function 안에서 조건을 깰 때 주의해야 합니다.
Kernel inline documentation reference
784-789Kernel inline documentation reference
Workqueue header와 implementation의 kernel-doc reference는 다음과 같습니다.
.. kernel-doc:: include/linux/workqueue.h.. kernel-doc:: kernel/workqueue.c
요약과 해설
workqueue.rst:1-789Cmwq는 모든 workqueue가 per-CPU worker pool을 공유하게 하여 기존 MT 및 ST workqueue의 thread 낭비와 제한된 concurrency 문제를 해결합니다. Scheduler feedback으로 필요한 만큼의 worker만 runnable 상태로 유지합니다.
`WQ_MEM_RECLAIM`은 memory pressure에서도 forward progress를 보장하고, `WQ_UNBOUND`와 affinity scope는 locality와 system utilization 사이의 균형을 조절합니다. Strict scope는 isolation을 높이지만 work conservation을 크게 잃을 수 있습니다.
일반적으로 `@max_active`는 0을 사용하고 strict ordering에는 `alloc_ordered_workqueue()`를 사용합니다. CPU를 많이 소비하는 workqueue는 `apply_workqueue_attrs()` 또는 `WQ_SYSFS`로 실제 workload에 맞춰야 합니다.
`wq_dump.py`와 `wq_monitor.py`, workqueue tracepoint 및 worker stack으로 configuration, saturation, rapid requeue, CPU-hogging item을 진단할 수 있습니다.