System Suspend and Device Interrupts

Device IRQ suspend와 resume 경계

System suspend에서는 모든 device의 `->prepare`, `->suspend`, `->suspend_late`가 끝난 late phase 뒤 `suspend_device_irqs()`가 일반 device IRQ를 disable합니다.

Late phase 뒤 suspended device가 interrupt를 발생할 정당한 이유가 없고, 아직 완전히 suspend되지 않은 device의 interrupt도 막는 편이 안전합니다. 과거 shared IRQ handler가 이미 suspend된 device의 memory address space에 접근해 예측 불가능한 동작을 일으키는 문제가 있었고 진단도 매우 어려웠습니다. `suspend_device_irqs()`와 suspend/resume의 noirq phase가 이를 실용적으로 완화했습니다.

System resume에서는 device의 early phase, 즉 `->resume_early` 시작 직전에 `resume_device_irqs()`가 IRQ를 다시 enable합니다.

====================================
System Suspend and Device Interrupts
====================================

Copyright (C) 2014 Intel Corp.
Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>


Suspending and Resuming Device IRQs
-----------------------------------

Device interrupt request lines (IRQs) are generally disabled during system
suspend after the "late" phase of suspending devices (that is, after all of the
->prepare, ->suspend and ->suspend_late callbacks have been executed for all
devices).  That is done by suspend_device_irqs().

The rationale for doing so is that after the "late" phase of device suspend
there is no legitimate reason why any interrupts from suspended devices should
trigger and if any devices have not been suspended properly yet, it is better to
block interrupts from them anyway.  Also, in the past we had problems with
interrupt handlers for shared IRQs that device drivers implementing them were
not prepared for interrupts triggering after their devices had been suspended.
In some cases they would attempt to access, for example, memory address spaces
of suspended devices and cause unpredictable behavior to ensue as a result.
Unfortunately, such problems are very difficult to debug and the introduction
of suspend_device_irqs(), along with the "noirq" phase of device suspend and
resume, was the only practical way to mitigate them.

Device IRQs are re-enabled during system resume, right before the "early" phase
of resuming devices (that is, before starting to execute ->resume_early
callbacks for devices).  The function doing that is resume_device_irqs().

IRQF_NO_SUSPEND

Timer interrupt, IPI, 일부 special-purpose interrupt는 noirq phase와 nonboot CPU offline/online을 포함한 suspend-resume 전체 cycle 동안 정상적으로 발생할 수 있습니다. `request_irq()`에서 `IRQF_NO_SUSPEND`를 사용하면 `suspend_device_irqs()`가 해당 IRQ를 enable 상태로 둡니다.

이 flag는 interrupt가 suspended system을 깨운다고 보장하지 않습니다. System wakeup에는 `enable_irq_wake()`가 필요합니다.

`IRQF_NO_SUSPEND`는 개별 handler가 아니라 IRQ 전체에 적용됩니다. Shared IRQ라면 flag를 요청하지 않은 사용자의 handler까지 suspend 뒤 평소처럼 실행되므로 `IRQF_NO_SUSPEND`와 `IRQF_SHARED`의 동시 사용은 피해야 합니다.

The IRQF_NO_SUSPEND Flag
------------------------

There are interrupts that can legitimately trigger during the entire system
suspend-resume cycle, including the "noirq" phases of suspending and resuming
devices as well as during the time when nonboot CPUs are taken offline and
brought back online.  That applies to timer interrupts in the first place,
but also to IPIs and to some other special-purpose interrupts.

The IRQF_NO_SUSPEND flag is used to indicate that to the IRQ subsystem when
requesting a special-purpose interrupt.  It causes suspend_device_irqs() to
leave the corresponding IRQ enabled so as to allow the interrupt to work as
expected during the suspend-resume cycle, but does not guarantee that the
interrupt will wake the system from a suspended state -- for such cases it is
necessary to use enable_irq_wake().

Note that the IRQF_NO_SUSPEND flag affects the entire IRQ and not just one
user of it.  Thus, if the IRQ is shared, all of the interrupt handlers installed
for it will be executed as usual after suspend_device_irqs(), even if the
IRQF_NO_SUSPEND flag was not passed to request_irq() (or equivalent) by some of
the IRQ's users.  For this reason, using IRQF_NO_SUSPEND and IRQF_SHARED at the
same time should be avoided.

System wakeup IRQ

System wakeup interrupt는 working state에서 일반 I/O interrupt로 쓰이더라도 sleep state에서는 system을 깨우도록 platform signal routing을 별도로 설정해야 할 수 있습니다. 예를 들어 SoC의 전용 wakeup interrupt controller input을 sleep 중 enable하고 wake 뒤 불필요한 interrupt를 막기 위해 다시 disable합니다.

Driver는 `enable_irq_wake()`로 해당 IRQ의 platform wakeup logic을 켜고 `disable_irq_wake()`로 끕니다.

Wakeup IRQ는 `suspend_device_irqs()` 뒤에도 enable 상태지만 첫 interrupt에서 disable되고 pending 및 suspended로 표시됩니다. PM core는 event를 통지받아 진행 중인 suspend를 pending wakeup 검사 지점에서 중단합니다. 이미 suspended라면 system을 깨웁니다. 이후 `resume_device_irqs()`가 IRQ를 다시 enable합니다.

`suspend_device_irqs()` 뒤 system wakeup IRQ의 handler는 실행하지 않습니다. 이 시점에 handler가 실행되는 것은 `IRQF_NO_SUSPEND` IRQ뿐이며, 그런 IRQ에는 `enable_irq_wake()`를 사용하면 안 됩니다.

System Wakeup Interrupts, enable_irq_wake() and disable_irq_wake()
------------------------------------------------------------------

System wakeup interrupts generally need to be configured to wake up the system
from sleep states, especially if they are used for different purposes (e.g. as
I/O interrupts) in the working state.

That may involve turning on a special signal handling logic within the platform
(such as an SoC) so that signals from a given line are routed in a different way
during system sleep so as to trigger a system wakeup when needed.  For example,
the platform may include a dedicated interrupt controller used specifically for
handling system wakeup events.  Then, if a given interrupt line is supposed to
wake up the system from sleep states, the corresponding input of that interrupt
controller needs to be enabled to receive signals from the line in question.
After wakeup, it generally is better to disable that input to prevent the
dedicated controller from triggering interrupts unnecessarily.

The IRQ subsystem provides two helper functions to be used by device drivers for
those purposes.  Namely, enable_irq_wake() turns on the platform's logic for
handling the given IRQ as a system wakeup interrupt line and disable_irq_wake()
turns that logic off.

Calling enable_irq_wake() causes suspend_device_irqs() to treat the given IRQ
in a special way.  Namely, the IRQ remains enabled, but on the first interrupt
it will be disabled, marked as pending and "suspended" so that it will be
re-enabled by resume_device_irqs() during the subsequent system resume.  Also
the PM core is notified about the event which causes the system suspend in
progress to be aborted (that doesn't have to happen immediately, but at one
of the points where the suspend thread looks for pending wakeup events).

This way every interrupt from a wakeup interrupt source will either cause the
system suspend currently in progress to be aborted or wake up the system if
already suspended.  However, after suspend_device_irqs() interrupt handlers are
not executed for system wakeup IRQs.  They are only executed for IRQF_NO_SUSPEND
IRQs at that time, but those IRQs should not be configured for system wakeup
using enable_irq_wake().

Suspend-to-idle의 interrupt

Suspend-to-idle, 즉 `freeze` state는 device suspend의 noirq phase 직후 모든 processor를 idle로 두고 interrupt를 기다립니다.

`IRQF_NO_SUSPEND` IRQ는 CPU를 idle에서 꺼내지만 IRQ subsystem의 system wakeup을 발생시키지는 않습니다. 반면 system wakeup IRQ는 full system suspend와 비슷하게 s2idle을 종료합니다.

차이는 s2idle wakeup이 평상시 working-state interrupt delivery mechanism으로 전달되므로 platform의 특별한 wakeup routing logic이 필요 없다는 점입니다.

Interrupts and Suspend-to-Idle
------------------------------

Suspend-to-idle (also known as the "freeze" sleep state) is a relatively new
system sleep state that works by idling all of the processors and waiting for
interrupts right after the "noirq" phase of suspending devices.

Of course, this means that all of the interrupts with the IRQF_NO_SUSPEND flag
set will bring CPUs out of idle while in that state, but they will not cause the
IRQ subsystem to trigger a system wakeup.

System wakeup interrupts, in turn, will trigger wakeup from suspend-to-idle in
analogy with what they do in the full system suspend case.  The only difference
is that the wakeup from suspend-to-idle is signaled using the usual working
state interrupt delivery mechanisms and doesn't require the platform to use
any special interrupt handling logic for it to work.

IRQF_NO_SUSPEND와 enable_irq_wake 조합

같은 IRQ에 `enable_irq_wake()`와 `IRQF_NO_SUSPEND`를 함께 써야 할 정당한 경우는 매우 드물며, 같은 device에 둘 다 사용하는 것은 항상 잘못입니다.

Non-shared IRQ에서는 전자가 handler를 suspend 뒤 실행하지 않는 반면 후자는 실행하므로 규칙이 직접 충돌합니다. 두 설정 모두 개별 handler가 아니라 IRQ 전체에 적용되므로 wakeup source와 NO_SUSPEND source가 IRQ를 공유하는 것도 보통 의미가 없습니다.

드물게 wakeup driver와 NO_SUSPEND 사용자가 IRQ를 공유하려면 wakeup driver가 spurious IRQ와 genuine wakeup을 구별하고 진짜 event를 `pm_system_wakeup()`으로 core에 알려야 합니다. 또한 `enable_irq_wake()`를 사용하고 `IRQF_COND_SUSPEND`로 요구 조건을 충족한다고 표시해야 합니다. 이 조건이 없으면 `IRQF_COND_SUSPEND` 사용은 유효하지 않습니다.

IRQF_NO_SUSPEND and enable_irq_wake()
-------------------------------------

There are very few valid reasons to use both enable_irq_wake() and the
IRQF_NO_SUSPEND flag on the same IRQ, and it is never valid to use both for the
same device.

First of all, if the IRQ is not shared, the rules for handling IRQF_NO_SUSPEND
interrupts (interrupt handlers are invoked after suspend_device_irqs()) are
directly at odds with the rules for handling system wakeup interrupts (interrupt
handlers are not invoked after suspend_device_irqs()).

Second, both enable_irq_wake() and IRQF_NO_SUSPEND apply to entire IRQs and not
to individual interrupt handlers, so sharing an IRQ between a system wakeup
interrupt source and an IRQF_NO_SUSPEND interrupt source does not generally
make sense.

In rare cases an IRQ can be shared between a wakeup device driver and an
IRQF_NO_SUSPEND user. In order for this to be safe, the wakeup device driver
must be able to discern spurious IRQs from genuine wakeup events (signalling
the latter to the core with pm_system_wakeup()), must use enable_irq_wake() to
ensure that the IRQ will function as a wakeup source, and must request the IRQ
with IRQF_COND_SUSPEND to tell the core that it meets these requirements. If
these requirements are not met, it is not valid to use IRQF_COND_SUSPEND.