XFRM

HA synchronization 목표

XFRM sync patch는 Krisztian과 여러 기여자의 초기 patch, Jamal의 추가 patch를 바탕으로 합니다.

최종 목표는 attribute를 삽입하고 event를 생성해 HA 환경에서 SA를 한 machine에서 다른 machine으로 안전하게 옮기는 것입니다. Takeover machine이 SA에 접근할 수 있을 때 active machine과 최대한 정확히 같은 처리를 하도록 state를 동기화합니다.

기존 SA add/delete/update event에 더해 lifetime byte counter를 정확히 맞춰 SA decay를 보존하고 inbound·outbound replay counter를 동기화해 replay attack과 failover packet loss를 줄입니다. Backup은 active member에 최대한 가까운 최신 상태를 유지합니다.

이 값은 SA가 packet을 받을 때마다 바뀌므로 event 폭증을 막는 Nagle 유사 rate-limit을 사용합니다. 예를 들어 replay sequence threshold에 도달하거나 10초가 지날 때 알려 달라는 조건을 둡니다. Threshold는 system-wide sysctl 또는 SA별 update로 설정합니다.

동기화 대상은 lifetime byte counter와 양방향 replay sequence입니다. Failover machine을 미리 안다면 packet arrival이 time countdown을 구동하지 않으므로 lifetime time limit은 중요하지 않습니다.

.. SPDX-License-Identifier: GPL-2.0

====
XFRM
====

The sync patches work is based on initial patches from
Krisztian <hidden@balabit.hu> and others and additional patches
from Jamal <hadi@cyberus.ca>.

The end goal for syncing is to be able to insert attributes + generate
events so that the SA can be safely moved from one machine to another
for HA purposes.
The idea is to synchronize the SA so that the takeover machine can do
the processing of the SA as accurate as possible if it has access to it.

We already have the ability to generate SA add/del/upd events.
These patches add ability to sync and have accurate lifetime byte (to
ensure proper decay of SAs) and replay counters to avoid replay attacks
with as minimal loss at failover time.
This way a backup stays as closely up-to-date as an active member.

Because the above items change for every packet the SA receives,
it is possible for a lot of the events to be generated.
For this reason, we also add a nagle-like algorithm to restrict
the events. i.e we are going to set thresholds to say "let me
know if the replay sequence threshold is reached or 10 secs have passed"
These thresholds are set system-wide via sysctls or can be updated
per SA.

The identified items that need to be synchronized are:
- the lifetime byte counter
note that: lifetime time limit is not important if you assume the failover
machine is known ahead of time since the decay of the time countdown
is not driven by packet arrival.
- the replay sequence for both inbound and outbound

1) Message Structure

AE netlink message 구조와 flag

Message format은 `nlmsghdr:aevent_id:optional-TLVs`입니다. Netlink message type은 `XFRM_MSG_NEWAE`와 `XFRM_MSG_GETAE`이며 GETAE에는 TLV가 없고 NEWAE에는 최소 두 TLV가 있습니다.

`struct xfrm_aevent_id`는 `xfrm_usersa_id sa_id`, source address `saddr`, `flags`, `reqid`를 담습니다. 고유 SA는 `xfrm_usersa_id`, `reqid`, `saddr` 조합으로 식별합니다.

Flag 해석은 kernel↔user 방향과 config, query, event 원인에 따라 달라집니다. Netlink pid는 방향을 구분하도록 user→kernel이면 0, kernel→userspace이면 event를 만든 process ID로 설정합니다.

Event notification을 받으려면 program이 multicast group `XFRMNLGRP_AEVENTS`를 subscribe해야 합니다.

----------------------

nlmsghdr:aevent_id:optional-TLVs.

The netlink message types are:

XFRM_MSG_NEWAE and XFRM_MSG_GETAE.

A XFRM_MSG_GETAE does not have TLVs.

A XFRM_MSG_NEWAE will have at least two TLVs (as is
discussed further below).

aevent_id structure looks like::

   struct xfrm_aevent_id {
             struct xfrm_usersa_id           sa_id;
             xfrm_address_t                  saddr;
             __u32                           flags;
             __u32                           reqid;
   };

The unique SA is identified by the combination of xfrm_usersa_id,
reqid and saddr.

flags are used to indicate different things. The possible
flags are::

        XFRM_AE_RTHR=1, /* replay threshold*/
        XFRM_AE_RVAL=2, /* replay value */
        XFRM_AE_LVAL=4, /* lifetime value */
        XFRM_AE_ETHR=8, /* expiry timer threshold */
        XFRM_AE_CR=16, /* Event cause is replay update */
        XFRM_AE_CE=32, /* Event cause is timer expiry */
        XFRM_AE_CU=64, /* Event cause is policy update */

How these flags are used is dependent on the direction of the
message (kernel<->user) as well the cause (config, query or event).
This is described below in the different messages.

The pid will be set appropriately in netlink to recognize direction
(0 to the kernel and pid = processid that created the event
when going from kernel to user space)

A program needs to subscribe to multicast group XFRMNLGRP_AEVENTS
to get notified of these events.

2) TLVS reflect the different parameters:

AE TLV parameter

`XFRMA_LTIME_VAL`은 직전 event 이후 현재까지 누적된 byte lifetime counter를 전달합니다.

`XFRMA_REPLAY_VAL`은 직전 event 이후 현재 replay sequence counter를 전달합니다.

`XFRMA_REPLAY_THRESH`는 replay sequence가 초과할 때 kernel이 event를 발생시키는 threshold입니다.

`XFRMA_ETIMER_THRESH`는 event를 rate-limit하는 Nagle 값으로 사용하는 millisecond timer입니다.

-----------------------------------------

a) byte value (XFRMA_LTIME_VAL)

This TLV carries the running/current counter for byte lifetime since
last event.

b)replay value (XFRMA_REPLAY_VAL)

This TLV carries the running/current counter for replay sequence since
last event.

c)replay threshold (XFRMA_REPLAY_THRESH)

This TLV carries the threshold being used by the kernel to trigger events
when the replay sequence is exceeded.

d) expiry timer (XFRMA_ETIMER_THRESH)

This is a timer value in milliseconds which is used as the nagle
value to rate limit the events.

3) Default configurations for the parameters:

Default threshold configuration

기본적으로 `XFRMNLGRP_AEVENTS` multicast group을 듣는 listener가 하나 이상 없으면 event를 꺼 둡니다.

SA를 설치하는 program이 두 threshold를 지정해야 하지만 racoon 같은 기존 application과의 호환성을 위해 생략 시 default 값을 제공합니다.

`/proc/sys/net/core/sysctl_xfrm_aevent_etime`은 `XFRMA_ETIMER_THRESH` 기본값을 100ms 단위로 지정하며 default 10은 1초입니다.

`/proc/sys/net/core/sysctl_xfrm_aevent_rseqth`는 `XFRMA_REPLAY_THRESH` 기본값을 packet count 단위로 지정하며 default는 2 packet입니다.

---------------------------------------------

By default these events should be turned off unless there is
at least one listener registered to listen to the multicast
group XFRMNLGRP_AEVENTS.

Programs installing SAs will need to specify the two thresholds, however,
in order to not change existing applications such as racoon
we also provide default threshold values for these different parameters
in case they are not specified.

the two sysctls/proc entries are:

a) /proc/sys/net/core/sysctl_xfrm_aevent_etime
used to provide default values for the XFRMA_ETIMER_THRESH in incremental
units of time of 100ms. The default is 10 (1 second)

b) /proc/sys/net/core/sysctl_xfrm_aevent_rseqth
used to provide default values for XFRMA_REPLAY_THRESH parameter
in incremental packet count. The default is two packets.

4) Message types

GETAE와 NEWAE 방향별 동작

Userspace가 kernel에 보내는 `XFRM_MSG_GETAE`에는 TLV가 없습니다. 응답은 query 조건에 맞춘 `XFRM_MSG_NEWAE`입니다.

응답에는 항상 `XFRMA_LTIME_VAL`과 `XFRMA_REPLAY_VAL`이 있습니다. GETAE에서 `XFRM_AE_RTHR`를 설정하면 replay threshold, `XFRM_AE_ETHR`를 설정하면 expiry timer threshold도 받습니다.

`XFRM_MSG_NEWAE`는 userspace가 특정 SA를 configure할 때, kernel이 event를 알릴 때, GETAE에 응답할 때 모두 사용합니다.

User→kernel configuration에서는 적절한 TLV를 넣어 value나 threshold를 갱신합니다. Kernel은 success/failure 응답을 보내고 성공하면 listener에도 NEWAE event를 보냅니다.

GETAE에 대한 kernel→user 응답은 항상 lifetime·replay value TLV를 포함하고, 명시적으로 요청한 threshold TLV도 포함합니다.

Userspace가 NEWAE로 SA value나 threshold를 설정해 발생한 kernel→user event에는 `XFRM_AE_CU` flag를 넣고 lifetime·replay value TLV를 항상 포함합니다.

Replay threshold 초과나 timeout으로 발생한 event는 각각 `XFRM_AE_CR` 또는 `XFRM_AE_CE`를 설정합니다. 두 flag는 상호 배타적이며 message에는 lifetime·replay value TLV가 항상 있습니다.

----------------

a) XFRM_MSG_GETAE issued by user-->kernel.
   XFRM_MSG_GETAE does not carry any TLVs.

The response is a XFRM_MSG_NEWAE which is formatted based on what
XFRM_MSG_GETAE queried for.

The response will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.
* if XFRM_AE_RTHR flag is set, then XFRMA_REPLAY_THRESH is also retrieved
* if XFRM_AE_ETHR flag is set, then XFRMA_ETIMER_THRESH is also retrieved

b) XFRM_MSG_NEWAE is issued by either user space to configure
   or kernel to announce events or respond to a XFRM_MSG_GETAE.

i) user --> kernel to configure a specific SA.

any of the values or threshold parameters can be updated by passing the
appropriate TLV.

A response is issued back to the sender in user space to indicate success
or failure.

In the case of success, additionally an event with
XFRM_MSG_NEWAE is also issued to any listeners as described in iii).

ii) kernel->user direction as a response to XFRM_MSG_GETAE

The response will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.

The threshold TLVs will be included if explicitly requested in
the XFRM_MSG_GETAE message.

iii) kernel->user to report as event if someone sets any values or
     thresholds for an SA using XFRM_MSG_NEWAE (as described in #i above).
     In such a case XFRM_AE_CU flag is set to inform the user that
     the change happened as a result of an update.
     The message will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.

iv) kernel->user to report event when replay threshold or a timeout
    is exceeded.

In such a case either XFRM_AE_CR (replay exceeded) or XFRM_AE_CE (timeout
happened) is set to inform the user what happened.
Note the two flags are mutually exclusive.
The message will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.

Exceptions to threshold settings

Timer threshold 예외

Traffic이 burst로 들어오고 packet이 없는 동안 timer threshold가 만료되는 SA에서는 특수 동작이 나타납니다. Timer expiry 뒤 첫 packet이 오면 새 timeout 기간이나 packet threshold를 기다리지 않고 즉시 timeout event를 발생시킵니다. 이는 구현 단순성과 효율을 위한 선택입니다.

--------------------------------

If you have an SA that is getting hit by traffic in bursts such that
there is a period where the timer threshold expires with no packets
seen, then an odd behavior is seen as follows:
The first packet arrival after a timer expiry will trigger a timeout
event; i.e we don't wait for a timeout period or a packet threshold
to be reached. This is done for simplicity and efficiency reasons.

-JHS