Netlink 입문

확장 가능한 ioctl 대안

Netlink는 흔히 `ioctl()`의 대안으로 설명됩니다. 고정 형식 C 구조체를 `ioctl()`에 넘기는 방식 대신, 인자를 쉽게 추가하거나 확장할 수 있는 형식을 제공하는 것이 목표입니다.

이를 위해 최소한의 고정 형식 metadata header 뒤에 TLV(type, length, value) 형식의 attribute 여러 개를 배치합니다. 다만 프로토콜이 오랜 기간 문서화되지 않은 채 유기적으로 발전했기 때문에 일관되게 설명하기 어렵습니다. 이 문서는 오늘날의 실용적인 사용법부터 시작해 뒤에서 역사적 형식을 다룹니다.

.. SPDX-License-Identifier: BSD-3-Clause

=======================
Introduction to Netlink
=======================

Netlink is often described as an ioctl() replacement.
It aims to replace fixed-format C structures as supplied
to ioctl() with a format which allows an easy way to add
or extended the arguments.

To achieve this Netlink uses a minimal fixed-format metadata header
followed by multiple attributes in the TLV (type, length, value) format.

Unfortunately the protocol has evolved over the years, in an organic
and undocumented fashion, making it hard to coherently explain.
To make the most practical sense this document starts by describing
netlink as it is used today and dives into more "historical" uses
in later sections.

socket 열기와 do·dump 흐름

Netlink 통신은 socket을 통해 이루어집니다. `socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC)`은 사용자가 모든 Netlink header를 제공하는 Generic Netlink raw socket을 엽니다.

socket은 커널 양방향 통신을 자연스럽게 지원합니다. 애플리케이션이 `send()`로 요청하는 동작 자체는 동기적으로 수행되지만 reply를 읽으려면 별도의 `recv()` 호출이 필요합니다.

dump는 특정 종류의 모든 객체를 사용자 공간에 전달합니다. `recv()` 한 번이 여러 Netlink message를 담을 수 있으므로 각 수신 buffer 안의 message를 다시 순회하고, `NLMSG_DONE`을 만날 때까지 바깥 수신 loop도 계속해야 합니다.

`socket()`의 마지막 인자는 Netlink 내부 protocol입니다. 과거에는 통신할 subsystem을 이 값으로 직접 식별했습니다.

Opening a socket
================

Netlink communication happens over sockets, a socket needs to be
opened first:

.. code-block:: c

  fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);

The use of sockets allows for a natural way of exchanging information
in both directions (to and from the kernel). The operations are still
performed synchronously when applications send() the request but
a separate recv() system call is needed to read the reply.

A very simplified flow of a Netlink "call" will therefore look
something like:

.. code-block:: c

  fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);

  /* format the request */
  send(fd, &request, sizeof(request));
  n = recv(fd, &response, RSP_BUFFER_SIZE);
  /* interpret the response */

Netlink also provides natural support for "dumping", i.e. communicating
to user space all objects of a certain type (e.g. dumping all network
interfaces).

.. code-block:: c

  fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);

  /* format the dump request */
  send(fd, &request, sizeof(request));
  while (1) {
    n = recv(fd, &buffer, RSP_BUFFER_SIZE);
    /* one recv() call can read multiple messages, hence the loop below */
    for (nl_msg in buffer) {
      if (nl_msg.nlmsg_type == NLMSG_DONE)
        goto dump_finished;
      /* process the object */
    }
  }
  dump_finished:

The first two arguments of the socket() call require little explanation -
it is opening a Netlink socket, with all headers provided by the user
(hence NETLINK, RAW). The last argument is the protocol within Netlink.
This field used to identify the subsystem with which the socket will
communicate.

Classic과 Generic Netlink

초기 Netlink는 subsystem ID를 정적으로 배정했고 지원 infrastructure가 적었습니다. 문서는 이 계열을 Classic Netlink라고 부릅니다. `include/uapi/linux/netlink.h` 상단 목록에는 일반 networking의 `NETLINK_ROUTE`, iSCSI의 `NETLINK_ISCSI`, audit의 `NETLINK_AUDIT` 등이 있습니다.

2005년에 도입된 Generic Netlink는 subsystem 동적 등록과 ID 할당, introspection을 지원하고 kernel 쪽 interface 구현을 단순화합니다. 사용 family 수가 Classic보다 한 자릿수 이상 많고 새 Classic protocol을 추가할 계획도 없으므로 이후 설명은 Generic Netlink를 중심으로 합니다.

Classic vs Generic Netlink
--------------------------

Initial implementation of Netlink depended on a static allocation
of IDs to subsystems and provided little supporting infrastructure.
Let us refer to those protocols collectively as **Classic Netlink**.
The list of them is defined on top of the ``include/uapi/linux/netlink.h``
file, they include among others - general networking (NETLINK_ROUTE),
iSCSI (NETLINK_ISCSI), and audit (NETLINK_AUDIT).

**Generic Netlink** (introduced in 2005) allows for dynamic registration of
subsystems (and subsystem ID allocation), introspection and simplifies
implementing the kernel side of the interface.

The following section describes how to use Generic Netlink, as the
number of subsystems using Generic Netlink outnumbers the older
protocols by an order of magnitude. There are also no plans for adding
more Classic Netlink protocols to the kernel.
Basic information on how communicating with core networking parts of
the Linux kernel (or another of the 20 subsystems using Classic
Netlink) differs from Generic Netlink is provided later in this document.

Generic Netlink header와 교환 종류

각 Netlink protocol은 공통 고정 metadata header에 이어 자체 protocol header를 정의합니다. Ethernet, IP, TCP header가 쌓이는 것처럼 Netlink, Generic Netlink, family header 순서로 겹칩니다. 모든 message는 `struct nlmsghdr`로 시작하고 Generic Netlink에서는 `struct genlmsghdr`가 뒤따릅니다.

Classic에서는 `nlmsg_type`이 subsystem 내부 operation을 가리켰지만, Generic은 protocol 하나에 여러 subsystem을 multiplex하므로 이 필드로 family를 고르고 `cmd`로 operation을 고릅니다. 0부터 15까지의 `nlmsg_type`은 두 방식 모두 control message용으로 예약되어 있습니다.

`nlmsg_seq`는 response에 되돌아와 request를 대응시키며 notification에서는 0입니다. 실제 영향이 작더라도 요청마다 증가시키는 것이 좋은 관례입니다. `nlmsg_pid`는 Netlink 주소에 해당하며 kernel 상대 통신에서는 0으로 둘 수 있습니다. `genlmsghdr.version`은 실제로 중요한 버전 관리에 쓰인 family가 없으므로 1이 안전한 선택입니다.

Generic Netlink
===============

In addition to the Netlink fixed metadata header each Netlink protocol
defines its own fixed metadata header. (Similarly to how network
headers stack - Ethernet > IP > TCP we have Netlink > Generic N. > Family.)

A Netlink message always starts with struct nlmsghdr, which is followed
by a protocol-specific header. In case of Generic Netlink the protocol
header is struct genlmsghdr.

The practical meaning of the fields in case of Generic Netlink is as follows:

.. code-block:: c

  struct nlmsghdr {
	__u32	nlmsg_len;	/* Length of message including headers */
	__u16	nlmsg_type;	/* Generic Netlink Family (subsystem) ID */
	__u16	nlmsg_flags;	/* Flags - request or dump */
	__u32	nlmsg_seq;	/* Sequence number */
	__u32	nlmsg_pid;	/* Port ID, set to 0 */
  };
  struct genlmsghdr {
	__u8	cmd;		/* Command, as defined by the Family */
	__u8	version;	/* Irrelevant, set to 1 */
	__u16	reserved;	/* Reserved, set to 0 */
  };
  /* TLV attributes follow... */

In Classic Netlink :c:member:`nlmsghdr.nlmsg_type` used to identify
which operation within the subsystem the message was referring to
(e.g. get information about a netdev). Generic Netlink needs to mux
multiple subsystems in a single protocol so it uses this field to
identify the subsystem, and :c:member:`genlmsghdr.cmd` identifies
the operation instead. (See :ref:`res_fam` for
information on how to find the Family ID of the subsystem of interest.)
Note that the first 16 values (0 - 15) of this field are reserved for
control messages both in Classic Netlink and Generic Netlink.
See :ref:`nl_msg_type` for more details.

There are 3 usual types of message exchanges on a Netlink socket:

 - performing a single action (``do``);
 - dumping information (``dump``);
 - getting asynchronous notifications (``multicast``).

Classic Netlink is very flexible and presumably allows other types
of exchanges to happen, but in practice those are the three that get
used.

Asynchronous notifications are sent by the kernel and received by
the user sockets which subscribed to them. ``do`` and ``dump`` requests
are initiated by the user. :c:member:`nlmsghdr.nlmsg_flags` should
be set as follows:

 - for ``do``: ``NLM_F_REQUEST | NLM_F_ACK``
 - for ``dump``: ``NLM_F_REQUEST | NLM_F_ACK | NLM_F_DUMP``

:c:member:`nlmsghdr.nlmsg_seq` should be a set to a monotonically
increasing value. The value gets echoed back in responses and doesn't
matter in practice, but setting it to an increasing value for each
message sent is considered good hygiene. The purpose of the field is
matching responses to requests. Asynchronous notifications will have
:c:member:`nlmsghdr.nlmsg_seq` of ``0``.

:c:member:`nlmsghdr.nlmsg_pid` is the Netlink equivalent of an address.
This field can be set to ``0`` when talking to the kernel.
See :ref:`nlmsg_pid` for the (uncommon) uses of the field.

The expected use for :c:member:`genlmsghdr.version` was to allow
versioning of the APIs provided by the subsystems. No subsystem to
date made significant use of this field, so setting it to ``1`` seems
like a safe bet.

control message와 ACK wire 형식

protocol별 message type은 `NLMSG_MIN_TYPE`인 `0x10`부터 시작해야 합니다. 그 아래 16개 ID는 Netlink control message용입니다.

요청에 `NLM_F_ACK`가 없으면 오류가 없을 때 `NLMSG_ERROR` 성공 응답이 오지 않습니다. 성공 여부를 별도 처리하는 예외를 피하려면 항상 ACK를 요청하는 것이 권장됩니다.

response header와 request header가 모두 들어 있으므로 sequence 대응이나 실패 요청 로깅에 쓸 수 있습니다. 성공(`error == 0`)이거나 `NETLINK_CAP_ACK` socket option을 켠 경우 request payload는 되돌아오지 않으며 `NLM_F_CAPPED`가 이를 표시합니다. 모든 요청 사본을 kernel에서 다시 읽는 낭비를 줄이므로 CAP_ACK 사용이 일반적이고 권장됩니다.

일부 구현은 `do` 요청에도 custom `NLMSG_DONE`을 보낼 수 있습니다. 이때 payload는 구현별 형식이거나 없을 수 있습니다.

.. _nl_msg_type:

Netlink message types
---------------------

As previously mentioned :c:member:`nlmsghdr.nlmsg_type` carries
protocol specific values but the first 16 identifiers are reserved
(first subsystem specific message type should be equal to
``NLMSG_MIN_TYPE`` which is ``0x10``).

There are only 4 Netlink control messages defined:

 - ``NLMSG_NOOP`` - ignore the message, not used in practice;
 - ``NLMSG_ERROR`` - carries the return code of an operation;
 - ``NLMSG_DONE`` - marks the end of a dump;
 - ``NLMSG_OVERRUN`` - socket buffer has overflown, not used to date.

``NLMSG_ERROR`` and ``NLMSG_DONE`` are of practical importance.
They carry return codes for operations. Note that unless
the ``NLM_F_ACK`` flag is set on the request Netlink will not respond
with ``NLMSG_ERROR`` if there is no error. To avoid having to special-case
this quirk it is recommended to always set ``NLM_F_ACK``.

The format of ``NLMSG_ERROR`` is described by struct nlmsgerr::

  ----------------------------------------------
  | struct nlmsghdr - response header          |
  ----------------------------------------------
  |    int error                               |
  ----------------------------------------------
  | struct nlmsghdr - original request header |
  ----------------------------------------------
  | ** optionally (1) payload of the request   |
  ----------------------------------------------
  | ** optionally (2) extended ACK             |
  ----------------------------------------------

There are two instances of struct nlmsghdr here, first of the response
and second of the request. ``NLMSG_ERROR`` carries the information about
the request which led to the error. This could be useful when trying
to match requests to responses or re-parse the request to dump it into
logs.

The payload of the request is not echoed in messages reporting success
(``error == 0``) or if ``NETLINK_CAP_ACK`` setsockopt() was set.
The latter is common
and perhaps recommended as having to read a copy of every request back
from the kernel is rather wasteful. The absence of request payload
is indicated by ``NLM_F_CAPPED`` in :c:member:`nlmsghdr.nlmsg_flags`.

The second optional element of ``NLMSG_ERROR`` are the extended ACK
attributes. See :ref:`ext_ack` for more details. The presence
of extended ACK is indicated by ``NLM_F_ACK_TLVS`` in
:c:member:`nlmsghdr.nlmsg_flags`.

``NLMSG_DONE`` is simpler, the request is never echoed but the extended
ACK attributes may be present::

  ----------------------------------------------
  | struct nlmsghdr - response header          |
  ----------------------------------------------
  |    int error                               |
  ----------------------------------------------
  | ** optionally extended ACK                 |
  ----------------------------------------------

Note that some implementations may issue custom ``NLMSG_DONE`` messages
in reply to ``do`` action requests. In that case the payload is
implementation-specific and may also be absent.

Family ID 조회 요청과 정렬

Generic Netlink 자체도 Generic Netlink API로 노출되는 subsystem입니다. 순환 의존을 피하기 위해 정적 family ID `GENL_ID_CTRL`, 즉 `NLMSG_MIN_TYPE`을 사용하며 `CTRL_CMD_GETFAMILY` command로 다른 family 정보를 조회합니다.

`CTRL_CMD_GETFAMILY`를 dump로 호출하면 kernel이 아는 모든 family 정보를 돌려줍니다. 단일 조회에서는 family 이름을 올바른 type의 attribute로 지정합니다.

예제의 `test1\0` payload를 가진 attribute는 길이가 10이고, 다음 attribute를 4바이트 경계에 맞추기 위해 `\0\0` 두 바이트를 추가합니다. 이 padding은 `nla_len`에 포함하지 않습니다.

.. _res_fam:

Resolving the Family ID
-----------------------

This section explains how to find the Family ID of a subsystem.
It also serves as an example of Generic Netlink communication.

Generic Netlink is itself a subsystem exposed via the Generic Netlink API.
To avoid a circular dependency Generic Netlink has a statically allocated
Family ID (``GENL_ID_CTRL`` which is equal to ``NLMSG_MIN_TYPE``).
The Generic Netlink family implements a command used to find out information
about other families (``CTRL_CMD_GETFAMILY``).

To get information about the Generic Netlink family named for example
``"test1"`` we need to send a message on the previously opened Generic Netlink
socket. The message should target the Generic Netlink Family (1), be a
``do`` (2) call to ``CTRL_CMD_GETFAMILY`` (3). A ``dump`` version of this
call would make the kernel respond with information about *all* the families
it knows about. Last but not least the name of the family in question has
to be specified (4) as an attribute with the appropriate type::

  struct nlmsghdr:
    __u32 nlmsg_len:	32
    __u16 nlmsg_type:	GENL_ID_CTRL               // (1)
    __u16 nlmsg_flags:	NLM_F_REQUEST | NLM_F_ACK  // (2)
    __u32 nlmsg_seq:	1
    __u32 nlmsg_pid:	0

  struct genlmsghdr:
    __u8 cmd:		CTRL_CMD_GETFAMILY         // (3)
    __u8 version:	2 /* or 1, doesn't matter */
    __u16 reserved:	0

  struct nlattr:                                   // (4)
    __u16 nla_len:	10
    __u16 nla_type:	CTRL_ATTR_FAMILY_NAME
    char data: 		test1\0

  (padding:)
    char data:		\0\0

The length fields in Netlink (:c:member:`nlmsghdr.nlmsg_len`
and :c:member:`nlattr.nla_len`) always *include* the header.
Attribute headers in netlink must be aligned to 4 bytes from the start
of the message, hence the extra ``\0\0`` after ``CTRL_ATTR_FAMILY_NAME``.
The attribute lengths *exclude* the padding.

Family 정보 response와 ACK

family를 찾으면 kernel은 정보 response와 ACK 두 message를 보냅니다. 첫 response에는 요청에서 되돌아온 sequence, 사용자 공간 프로세스의 Port ID, family 이름·ID·version과 추가 attribute가 들어갑니다. 찾던 `test1`의 예제 family ID는 123입니다.

ACK에는 `NLM_F_CAPPED`가 있어 request payload가 포함되지 않음을 나타냅니다. 두 message 모두 request의 sequence 1을 되돌려 줍니다.

attribute 순서는 보장되지 않으므로 고정 offset을 가정하지 말고 전체 `struct nlattr` 목록을 순회하며 type별로 파싱해야 합니다.

Generic Netlink socket은 한 family에 bind되지 않습니다. 각 message의 `nlmsg_type`으로 수신 family를 선택하여 같은 socket에서 여러 family와 통신할 수 있습니다.

If the family is found kernel will reply with two messages, the response
with all the information about the family::

  /* Message #1 - reply */
  struct nlmsghdr:
    __u32 nlmsg_len:	136
    __u16 nlmsg_type:	GENL_ID_CTRL
    __u16 nlmsg_flags:	0
    __u32 nlmsg_seq:	1    /* echoed from our request */
    __u32 nlmsg_pid:	5831 /* The PID of our user space process */

  struct genlmsghdr:
    __u8 cmd:		CTRL_CMD_GETFAMILY
    __u8 version:	2
    __u16 reserved:	0

  struct nlattr:
    __u16 nla_len:	10
    __u16 nla_type:	CTRL_ATTR_FAMILY_NAME
    char data: 		test1\0

  (padding:)
    data:		\0\0

  struct nlattr:
    __u16 nla_len:	6
    __u16 nla_type:	CTRL_ATTR_FAMILY_ID
    __u16: 		123  /* The Family ID we are after */

  (padding:)
    char data:		\0\0

  struct nlattr:
    __u16 nla_len:	9
    __u16 nla_type:	CTRL_ATTR_FAMILY_VERSION
    __u16: 		1

  /* ... etc, more attributes will follow. */

And the error code (success) since ``NLM_F_ACK`` had been set on the request::

  /* Message #2 - the ACK */
  struct nlmsghdr:
    __u32 nlmsg_len:	36
    __u16 nlmsg_type:	NLMSG_ERROR
    __u16 nlmsg_flags:	NLM_F_CAPPED /* There won't be a payload */
    __u32 nlmsg_seq:	1    /* echoed from our request */
    __u32 nlmsg_pid:	5831 /* The PID of our user space process */

  int error:		0

  struct nlmsghdr: /* Copy of the request header as we sent it */
    __u32 nlmsg_len:	32
    __u16 nlmsg_type:	GENL_ID_CTRL
    __u16 nlmsg_flags:	NLM_F_REQUEST | NLM_F_ACK
    __u32 nlmsg_seq:	1
    __u32 nlmsg_pid:	0

The order of attributes (struct nlattr) is not guaranteed so the user
has to walk the attributes and parse them.

Note that Generic Netlink sockets are not associated or bound to a single
family. A socket can be used to exchange messages with many different
families, selecting the recipient family on message-by-message basis using
the :c:member:`nlmsghdr.nlmsg_type` field.

Extended ACK

Extended ACK는 `NLMSG_ERROR`와 `NLMSG_DONE`에 추가 오류·경고 TLV를 싣습니다. 하위 호환성을 위해 `NETLINK_EXT_ACK` socket option을 1로 설정해 명시적으로 활성화해야 합니다.

Extended ACK는 실패뿐 아니라 성공에도 올 수 있으며 성공 시 내용은 warning으로 취급해야 합니다. Netlink 사용성을 크게 높이므로 항상 켜고, 올바르게 파싱해 사용자에게 보고하는 것이 권장됩니다.

.. _ext_ack:

Extended ACK
------------

Extended ACK controls reporting of additional error/warning TLVs
in ``NLMSG_ERROR`` and ``NLMSG_DONE`` messages. To maintain backward
compatibility this feature has to be explicitly enabled by setting
the ``NETLINK_EXT_ACK`` setsockopt() to ``1``.

Types of extended ack attributes are defined in enum nlmsgerr_attrs.
The most commonly used attributes are ``NLMSGERR_ATTR_MSG``,
``NLMSGERR_ATTR_OFFS`` and ``NLMSGERR_ATTR_MISS_*``.

``NLMSGERR_ATTR_MSG`` carries a message in English describing
the encountered problem. These messages are far more detailed
than what can be expressed thru standard UNIX error codes.

``NLMSGERR_ATTR_OFFS`` points to the attribute which caused the problem.

``NLMSGERR_ATTR_MISS_TYPE`` and ``NLMSGERR_ATTR_MISS_NEST``
inform about a missing attribute.

Extended ACKs can be reported on errors as well as in case of success.
The latter should be treated as a warning.

Extended ACKs greatly improve the usability of Netlink and should
always be enabled, appropriately parsed and reported to the user.

dump 일관성과 재시도

kernel 내부 자료구조에 따라 fast path 갱신을 방해하지 않고 모든 객체의 원자적 snapshot을 만드는 일이 어려울 수 있습니다.

dump가 중단되어 객체가 빠지는 등 일관되지 않을 수 있으면 kernel은 dump의 어떤 message에도, `NLMSG_DONE`에도 `NLM_F_DUMP_INTR`을 설정할 수 있습니다. 사용자 공간은 이 flag를 본 경우 전체 dump를 다시 시도해야 합니다.

Advanced topics
===============

Dump consistency
----------------

Some of the data structures kernel uses for storing objects make
it hard to provide an atomic snapshot of all the objects in a dump
(without impacting the fast-paths updating them).

Kernel may set the ``NLM_F_DUMP_INTR`` flag on any message in a dump
(including the ``NLMSG_DONE`` message) if the dump was interrupted and
may be inconsistent (e.g. missing objects). User space should retry
the dump if it sees the flag set.

Family와 policy introspection

Family 객체 조회는 Generic Netlink introspection의 기반입니다. 사용자 공간은 kernel이 지원하는 operation과 이해하는 attribute 정보를 확인할 수 있습니다.

특정 기능을 사용하는 요청을 보내기 전에 실행 중인 kernel이 그 기능을 지원하는지 확인해야 할 때 유용합니다.

Introspection
-------------

The basic introspection abilities are enabled by access to the Family
object as reported in :ref:`res_fam`. User can query information about
the Generic Netlink family, including which operations are supported
by the kernel and what attributes the kernel understands.
Family information includes the highest ID of an attribute kernel can parse,
a separate command (``CTRL_CMD_GETPOLICY``) provides detailed information
about supported attributes, including ranges of values the kernel accepts.

Querying family information is useful in cases when user space needs
to make sure that the kernel has support for a feature before issuing
a request.

nlmsg_pid와 Port ID

`nlmsg_pid`는 Netlink 주소에 해당하며 Port ID라고 부릅니다. 역사적으로 애플리케이션이 명시적 Port ID에 `bind()`하지 않으면 kernel이 `getpid()`의 Process ID와 같은 값을 자동 할당했기 때문에 Process ID라고도 불립니다.

TCP/IP 계열의 `bind()`처럼 0은 자동 할당을 뜻하므로 애플리케이션은 흔히 `nlmsg_pid`를 0으로 둡니다.

.. _nlmsg_pid:

nlmsg_pid
---------

:c:member:`nlmsghdr.nlmsg_pid` is the Netlink equivalent of an address.
It is referred to as Port ID, sometimes Process ID because for historical
reasons if the application does not select (bind() to) an explicit Port ID
kernel will automatically assign it the ID equal to its Process ID
(as reported by the getpid() system call).

Similarly to the bind() semantics of the TCP/IP network protocols the value
of zero means "assign automatically", hence it is common for applications
to leave the :c:member:`nlmsghdr.nlmsg_pid` field initialized to ``0``.

The field is still used today in rare cases when kernel needs to send
a unicast notification. User space application can use bind() to associate
its socket with a specific PID, it then communicates its PID to the kernel.
This way the kernel can reach the specific user space process.

This sort of communication is utilized in UMH (User Mode Helper)-like
scenarios when kernel needs to trigger user space processing or ask user
space for a policy decision.

multicast notification과 구독

event notification은 Netlink의 강점입니다. kernel에서 사용자로만 흐르는 단방향 통신이며 `NLMSG_ERROR`나 `NLMSG_DONE` 같은 control message를 수반하지 않습니다.

Generic Netlink control family는 등록 family 변화에 대한 multicast를 제공합니다. 새 family가 추가되면 구독 socket은 `CTRL_CMD_NEWFAMILY` message를 받고, 내용은 `CTRL_CMD_GETFAMILY` response와 같은 family 이름·ID·version 정보입니다.

multicast group ID도 family ID처럼 동적입니다. family 정보의 `CTRL_ATTR_MCAST_GROUPS` nest에서 `CTRL_ATTR_MCAST_GRP_NAME`과 `CTRL_ATTR_MCAST_GRP_ID` 쌍을 찾아야 합니다.

notification은 비동기라 request response 사이에 섞일 수 있습니다. message 처리를 단순하게 유지하려면 notification 수신 socket과 kernel request 전송 socket을 분리하는 것이 권장됩니다.

Multicast notifications
-----------------------

One of the strengths of Netlink is the ability to send event notifications
to user space. This is a unidirectional form of communication (kernel ->
user) and does not involve any control messages like ``NLMSG_ERROR`` or
``NLMSG_DONE``.

For example the Generic Netlink family itself defines a set of multicast
notifications about registered families. When a new family is added the
sockets subscribed to the notifications will get the following message::

  struct nlmsghdr:
    __u32 nlmsg_len:	136
    __u16 nlmsg_type:	GENL_ID_CTRL
    __u16 nlmsg_flags:	0
    __u32 nlmsg_seq:	0
    __u32 nlmsg_pid:	0

  struct genlmsghdr:
    __u8 cmd:		CTRL_CMD_NEWFAMILY
    __u8 version:	2
    __u16 reserved:	0

  struct nlattr:
    __u16 nla_len:	10
    __u16 nla_type:	CTRL_ATTR_FAMILY_NAME
    char data: 		test1\0

  (padding:)
    data:		\0\0

  struct nlattr:
    __u16 nla_len:	6
    __u16 nla_type:	CTRL_ATTR_FAMILY_ID
    __u16: 		123  /* The Family ID we are after */

  (padding:)
    char data:		\0\0

  struct nlattr:
    __u16 nla_len:	9
    __u16 nla_type:	CTRL_ATTR_FAMILY_VERSION
    __u16: 		1

  /* ... etc, more attributes will follow. */

The notification contains the same information as the response
to the ``CTRL_CMD_GETFAMILY`` request.

The Netlink headers of the notification are mostly 0 and irrelevant.
The :c:member:`nlmsghdr.nlmsg_seq` may be either zero or a monotonically
increasing notification sequence number maintained by the family.

To receive notifications the user socket must subscribe to the relevant
notification group. Much like the Family ID, the Group ID for a given
multicast group is dynamic and can be found inside the Family information.
The ``CTRL_ATTR_MCAST_GROUPS`` attribute contains nests with names
(``CTRL_ATTR_MCAST_GRP_NAME``) and IDs (``CTRL_ATTR_MCAST_GRP_ID``) of
the groups family.

Once the Group ID is known a setsockopt() call adds the socket to the group:

.. code-block:: c

  unsigned int group_id;

  /* .. find the group ID... */

  setsockopt(fd, SOL_NETLINK, NETLINK_ADD_MEMBERSHIP,
             &group_id, sizeof(group_id));

The socket will now receive notifications.

It is recommended to use separate sockets for receiving notifications
and sending requests to the kernel. The asynchronous nature of notifications
means that they may get mixed in with the responses making the message
handling much harder.

datagram buffer 크기

Netlink socket은 stream이 아니라 datagram socket이므로 message 하나를 단일 `recv()` 또는 `recvmsg()` 호출로 온전히 받아야 합니다. 사용자 buffer가 짧으면 message가 잘리고 `struct msghdr`의 `MSG_TRUNC` flag가 설정됩니다. 여기서 `msghdr`는 `recvmsg()`의 두 번째 인자이며 Netlink header가 아닙니다.

truncation이 발생하면 남은 message 부분은 폐기됩니다. 따라서 짧은 buffer로 먼저 읽은 뒤 나머지를 이어 받을 수 없습니다.

Buffer sizing
-------------

Netlink sockets are datagram sockets rather than stream sockets,
meaning that each message must be received in its entirety by a single
recv()/recvmsg() system call. If the buffer provided by the user is too
short, the message will be truncated and the ``MSG_TRUNC`` flag set
in struct msghdr (struct msghdr is the second argument
of the recvmsg() system call, *not* a Netlink header).

Upon truncation the remaining part of the message is discarded.

Netlink expects that the user buffer will be at least 8kB or a page
size of the CPU architecture, whichever is bigger. Particular Netlink
families may, however, require a larger buffer. 32kB buffer is recommended
for most efficient handling of dumps (larger buffer fits more dumped
objects and therefore fewer recvmsg() calls are needed).

Classic Netlink와 family 용어

Classic과 Generic의 핵심 차이는 subsystem ID 동적 할당과 introspection입니다. 이론상 protocol 차이는 크지 않지만 Classic은 Generic에서 버린 여러 실험적 개념을 사용했습니다. 이 구간은 Generic 사용자들이 uAPI header를 읽을 때 그런 개념을 안전하게 무시할 수 있도록 배경을 제공합니다.

예시는 Linux networking stack 설정의 큰 부분을 담당하는 `NETLINK_ROUTE`를 중심으로 합니다. 이 family의 실제 사용법은 별도 장이나 책이 필요할 만큼 방대합니다.

Generic Netlink family ID는 Classic protocol 번호와 다른 ID 공간입니다. 따라서 Classic `NETLINK_CRYPTO` protocol ID 21과 같은 숫자를 Generic Netlink가 어느 family에 할당해도 충돌하지 않습니다.

.. _classic_netlink:

Classic Netlink
===============

The main differences between Classic and Generic Netlink are the dynamic
allocation of subsystem identifiers and availability of introspection.
In theory the protocol does not differ significantly, however, in practice
Classic Netlink experimented with concepts which were abandoned in Generic
Netlink (really, they usually only found use in a small corner of a single
subsystem). This section is meant as an explainer of a few of such concepts,
with the explicit goal of giving the Generic Netlink
users the confidence to ignore them when reading the uAPI headers.

Most of the concepts and examples here refer to the ``NETLINK_ROUTE`` family,
which covers much of the configuration of the Linux networking stack.
Real documentation of that family, deserves a chapter (or a book) of its own.

Families
--------

Netlink refers to subsystems as families. This is a remnant of using
sockets and the concept of protocol families, which are part of message
demultiplexing in ``NETLINK_ROUTE``.

Sadly every layer of encapsulation likes to refer to whatever it's carrying
as "families" making the term very confusing:

 1. AF_NETLINK is a bona fide socket protocol family
 2. AF_NETLINK's documentation refers to what comes after its own
    header (struct nlmsghdr) in a message as a "Family Header"
 3. Generic Netlink is a family for AF_NETLINK (struct genlmsghdr follows
    struct nlmsghdr), yet it also calls its users "Families".

Note that the Generic Netlink Family IDs are in a different "ID space"
and overlap with Classic Netlink protocol numbers (e.g. ``NETLINK_CRYPTO``
has the Classic Netlink protocol ID of 21 which Generic Netlink will
happily allocate to one of its families as well).

strict checking과 알 수 없는 attribute

`NETLINK_GET_STRICT_CHK` socket option은 `NETLINK_ROUTE`의 엄격한 입력 검사를 켭니다. 과거 kernel은 자신이 처리하지 않는 구조체 필드를 검증하지 않았고, 애플리케이션이 그 필드를 잘못 초기화하거나 전혀 초기화하지 않았을 수 있어 나중에 필드를 사용하는 것이 회귀 위험이 되었습니다.

역사적으로 Netlink는 알 수 없는 attribute를 모두 무시했습니다. 애플리케이션이 kernel 지원 여부를 미리 탐지하지 않고 상태 변경을 요청한 뒤 실제 반영된 부분을 확인하게 하려는 발상이었습니다.

새 Generic Netlink family와 strict checking을 선택한 family에서는 더 이상 그렇지 않습니다. 수행되는 검증 종류는 `enum netlink_validation`을 참조합니다.

Strict checking
---------------

The ``NETLINK_GET_STRICT_CHK`` socket option enables strict input checking
in ``NETLINK_ROUTE``. It was needed because historically kernel did not
validate the fields of structures it didn't process. This made it impossible
to start using those fields later without risking regressions in applications
which initialized them incorrectly or not at all.

``NETLINK_GET_STRICT_CHK`` declares that the application is initializing
all fields correctly. It also opts into validating that message does not
contain trailing data and requests that kernel rejects attributes with
type higher than largest attribute type known to the kernel.

``NETLINK_GET_STRICT_CHK`` is not used outside of ``NETLINK_ROUTE``.

Unknown attributes
------------------

Historically Netlink ignored all unknown attributes. The thinking was that
it would free the application from having to probe what kernel supports.
The application could make a request to change the state and check which
parts of the request "stuck".

This is no longer the case for new Generic Netlink families and those opting
in to strict checking. See enum netlink_validation for validation types
performed.

고정 구조체와 NETLINK_ROUTE request type

Classic Netlink message는 `nlmsghdr` 뒤에 필드가 많은 고정 구조체를 자주 두었고, attribute 안에도 각 member를 별도 attribute로 나누지 않은 다중 member 구조체를 넣었습니다. 이는 검증과 확장을 어렵게 했기 때문에 새 attribute에서 binary 구조체를 사용하는 것은 적극적으로 권장되지 않습니다.

`NETLINK_ROUTE`는 request를 `NEW`, `DEL`, `GET`, `SET` 네 종류로 분류합니다. netdev, route, address, qdisc 같은 각 객체는 일부 또는 전부를 처리할 수 있습니다. request type은 message type의 가장 낮은 2비트로 정의되어 새 객체 command가 4 간격으로 할당됩니다.

다른 protocol과 Generic Netlink command 이름에도 `GET`, `SET` 같은 동사가 자주 나타나지만, 이 request type 체계 자체는 널리 채택되지 않았습니다.

Fixed metadata and structures
-----------------------------

Classic Netlink made liberal use of fixed-format structures within
the messages. Messages would commonly have a structure with
a considerable number of fields after struct nlmsghdr. It was also
common to put structures with multiple members inside attributes,
without breaking each member into an attribute of its own.

This has caused problems with validation and extensibility and
therefore using binary structures is actively discouraged for new
attributes.

Request types
-------------

``NETLINK_ROUTE`` categorized requests into 4 types ``NEW``, ``DEL``, ``GET``,
and ``SET``. Each object can handle all or some of those requests
(objects being netdevs, routes, addresses, qdiscs etc.) Request type
is defined by the 2 lowest bits of the message type, so commands for
new objects would always be allocated with a stride of 4.

Each object would also have its own fixed metadata shared by all request
types (e.g. struct ifinfomsg for netdev requests, struct ifaddrmsg for address
requests, struct tcmsg for qdisc requests).

Even though other protocols and Generic Netlink commands often use
the same verbs in their message names (``GET``, ``SET``) the concept
of request types did not find wider adoption.

notification echo와 request별 flags

`NLM_F_ECHO`는 요청 결과로 발생한 notification을 요청 socket에도 queue하도록 요구합니다. 요청이 실제로 만든 영향을 확인하는 데 유용하지만 모든 family가 구현하지는 않습니다.

Classic Netlink는 `nlmsg_flags` 상위 byte에 `GET`, `NEW`, `DEL`별 flag를 정의했습니다. request type이 일반화되지 않았기 때문에 이 flags는 드물게 쓰이며 새 family에서는 deprecated로 간주됩니다.

`NEW` flags의 의미는 family마다 다소 어긋나 명확하지 않습니다. 원문의 의도 설명에서 `NLM_F_REPLACE`는 일치 객체가 있을 때 교체하고 없으면 실패하며, `NLM_F_EXCL`은 반대 의미라고 기술합니다. 이어지는 uAPI 주석은 `CREATE|EXCL`을 ADD, `REPLACE`를 CHANGE, `CREATE|REPLACE`를 true CHANGE, `CREATE`를 append, `EXCL`을 check로 대응시킵니다. 원문 내부 설명과 실제 family별 동작의 차이는 해당 uAPI 구현을 기준으로 검토해야 합니다.

이 flags는 request type보다 먼저 생긴 것으로 보입니다. `CREATE` 없는 `REPLACE`는 처음에 `SET` 대신 쓰였고, `CREATE` 없는 `EXCL`은 `GET` 이전에 객체 존재를 검사하는 데 쓰였습니다. `NLM_F_APPEND`는 한 key에 route의 여러 next-hop처럼 여러 객체가 연결될 때 전체 목록을 교체하지 않고 새 객체를 목록 끝에 추가하도록 합니다.

Notification echo
-----------------

``NLM_F_ECHO`` requests for notifications resulting from the request
to be queued onto the requesting socket. This is useful to discover
the impact of the request.

Note that this feature is not universally implemented.

Other request-type-specific flags
---------------------------------

Classic Netlink defined various flags for its ``GET``, ``NEW``
and ``DEL`` requests in the upper byte of nlmsg_flags in struct nlmsghdr.
Since request types have not been generalized the request type specific
flags are rarely used (and considered deprecated for new families).

For ``GET`` - ``NLM_F_ROOT`` and ``NLM_F_MATCH`` are combined into
``NLM_F_DUMP``, and not used separately. ``NLM_F_ATOMIC`` is never used.

For ``DEL`` - ``NLM_F_NONREC`` is only used by nftables and ``NLM_F_BULK``
only by FDB some operations.

The flags for ``NEW`` are used most commonly in classic Netlink. Unfortunately,
the meaning is not crystal clear. The following description is based on the
best guess of the intention of the authors, and in practice all families
stray from it in one way or another. ``NLM_F_REPLACE`` asks to replace
an existing object, if no matching object exists the operation should fail.
``NLM_F_EXCL`` has the opposite semantics and only succeeds if object already
existed.
``NLM_F_CREATE`` asks for the object to be created if it does not
exist, it can be combined with ``NLM_F_REPLACE`` and ``NLM_F_EXCL``.

A comment in the main Netlink uAPI header states::

   4.4BSD ADD		NLM_F_CREATE|NLM_F_EXCL
   4.4BSD CHANGE	NLM_F_REPLACE

   True CHANGE		NLM_F_CREATE|NLM_F_REPLACE
   Append		NLM_F_CREATE
   Check		NLM_F_EXCL

which seems to indicate that those flags predate request types.
``NLM_F_REPLACE`` without ``NLM_F_CREATE`` was initially used instead
of ``SET`` commands.
``NLM_F_EXCL`` without ``NLM_F_CREATE`` was used to check if object exists
without creating it, presumably predating ``GET`` commands.

``NLM_F_APPEND`` indicates that if one key can have multiple objects associated
with it (e.g. multiple next-hop objects for a route) the new object should be
added to the list rather than replacing the entire list.

Netlink uAPI reference

마지막 구간은 `include/uapi/linux/netlink.h`의 kernel-doc을 포함해 Netlink uAPI 상수와 구조체 정의를 직접 참조합니다.

uAPI reference
==============

.. kernel-doc:: include/uapi/linux/netlink.h