DWARF module versioning

DWARF module versioning과 command 사용법

`CONFIG_MODVERSIONS`를 활성화하면 module symbol version은 보통 전처리된 source code에서 `genksyms`로 계산합니다. 하지만 Rust처럼 source code만으로 최종 ABI 정보가 충분하지 않은 언어와는 맞지 않습니다.

`CONFIG_GENDWARFKSYMS`와 `CONFIG_DEBUG_INFO`를 선택하면 필요한 최종 module ABI 세부 정보가 들어 있는 DWARF debugging information에서 `gendwarfksyms`가 symbol version을 계산합니다.

`gendwarfksyms`는 command line에서 object file 목록을 받고 standard input에서 줄마다 symbol 이름 하나씩을 받습니다. 기본 형식은 `gendwarfksyms [options] elf-object-file ... < symbol-list`입니다.

=======================
DWARF module versioning
=======================

Introduction
============

When CONFIG_MODVERSIONS is enabled, symbol versions for modules
are typically calculated from preprocessed source code using the
**genksyms** tool.  However, this is incompatible with languages such
as Rust, where the source code has insufficient information about
the resulting ABI. With CONFIG_GENDWARFKSYMS (and CONFIG_DEBUG_INFO)
selected, **gendwarfksyms** is used instead to calculate symbol versions
from the DWARF debugging information, which contains the necessary
details about the final module ABI.

Usage
-----

gendwarfksyms accepts a list of object files on the command line, and a
list of symbol names (one per line) in standard input::

        Usage: gendwarfksyms [options] elf-object-file ... < symbol-list

        Options:
          -d, --debug          Print debugging information
              --dump-dies      Dump DWARF DIE contents
              --dump-die-map   Print debugging information about die_map changes
              --dump-types     Dump type strings
              --dump-versions  Dump expanded type strings used for symbol versions
          -s, --stable         Support kABI stability features
          -T, --symtypes file  Write a symtypes file
          -h, --help           Print this message

Export type 확보와 symtypes 형식

Symbol은 보통 정의된 translation unit(TU)에서 export되지만 TU가 외부 symbol을 export하는 것도 가능합니다. 독립 assembly code의 export symbol version을 계산할 때 이런 방식을 사용합니다.

Symbol을 실제 export하는 TU에 compiler가 필요한 DWARF type information을 내도록 `gendwarfksyms`는 `EXPORT_SYMBOL()` macro에 `__GENDWARFKSYMS_EXPORT(sym)`로 export symbol pointer를 추가합니다. 이 pointer는 `static typeof(sym) *`, `__used`, `.discard.gendwarfksyms` section 속성을 가지며 `&sym`으로 초기화됩니다.

DWARF에서 symbol pointer를 찾으면 symbol 정의가 다른 곳에 있어도 그 type으로 version을 계산할 수 있습니다. Pointer 이름은 `__gendwarfksyms_ptr_`로 시작하고 뒤에 export symbol 이름이 와야 합니다.

`gendwarfksyms`는 처리한 object마다 export symbol type과 version 계산에 사용한 참조 type을 담는 symtypes file을 만들 수 있습니다. Build 사이 symbol version 변화 원인을 찾는 데 유용하며 kernel build에서 `KBUILD_SYMTYPES=1`로 생성합니다.

각 줄의 첫 column에는 type reference 또는 symbol 이름이 옵니다. Type reference는 한 글자 prefix, `#`, type 이름으로 구성됩니다.

공백이 있는 type 이름은 `s#'core::result::Result<u8, core::num::error::ParseIntError>'`처럼 single quote로 감쌉니다. 나머지 줄은 type string입니다. `genksyms`의 C-style string과 달리 `gendwarfksyms`는 `--dump-dies`가 만드는 단순 parsed DWARF 형식을 사용하되 완전히 확장한 string 대신 type reference를 사용합니다.

Type information availability
=============================

While symbols are typically exported in the same translation unit (TU)
where they're defined, it's also perfectly fine for a TU to export
external symbols. For example, this is done when calculating symbol
versions for exports in stand-alone assembly code.

To ensure the compiler emits the necessary DWARF type information in the
TU where symbols are actually exported, gendwarfksyms adds a pointer
to exported symbols in the `EXPORT_SYMBOL()` macro using the following
macro::

        #define __GENDWARFKSYMS_EXPORT(sym)                             \
                static typeof(sym) *__gendwarfksyms_ptr_##sym __used    \
                        __section(".discard.gendwarfksyms") = &sym;


When a symbol pointer is found in DWARF, gendwarfksyms can use its
type for calculating symbol versions even if the symbol is defined
elsewhere. The name of the symbol pointer is expected to start with
`__gendwarfksyms_ptr_`, followed by the name of the exported symbol.

Symtypes output format
======================

Similarly to genksyms, gendwarfksyms supports writing a symtypes
file for each processed object that contain types for exported
symbols and each referenced type that was used in calculating symbol
versions. These files can be useful when trying to determine what
exactly caused symbol versions to change between builds. To generate
symtypes files during a kernel build, set `KBUILD_SYMTYPES=1`.

Matching the existing format, the first column of each line contains
either a type reference or a symbol name. Type references have a
one-letter prefix followed by "#" and the name of the type. Four
reference types are supported::

        e#<type> = enum
        s#<type> = struct
        t#<type> = typedef
        u#<type> = union

Type names with spaces in them are wrapped in single quotes, e.g.::

        s#'core::result::Result<u8, core::num::error::ParseIntError>'

The rest of the line contains a type string. Unlike with genksyms that
produces C-style type strings, gendwarfksyms uses the same simple parsed
DWARF format produced by **--dump-dies**, but with type references
instead of fully expanded strings.

Stable kABI 기능과 rule wire format

Distribution maintainer는 LTS update나 backport 때문에 kernel data structure를 ABI 호환 방식으로 바꿔야 할 때가 많습니다. Object file을 처리하는 방식에서는 기존의 `#ifndef __GENKSYMS__`로 변경을 symbol versioning에서 숨기는 기법이 작동하지 않습니다.

`gendwarfksyms`는 ABI에 영향을 주지 않는 변경을 version 계산 입력에서 숨기는 kABI 안정성 기능을 제공합니다. 모든 기능은 `--stable` option 뒤에서만 활성화되고 mainline kernel에서는 사용하지 않습니다. Kernel build에서는 `KBUILD_GENDWARFKSYMS_STABLE=1`로 켭니다.

사용 예와 source annotation helper macro는 `scripts/gendwarfksyms/examples`에 있습니다. 이 기능은 symbol versioning 입력만 변환하므로 변경이 실제 ABI를 깨지 않는지 보장할 책임은 사용자에게 있습니다.

kABI rule은 `gendwarfksyms` 출력 일부를 조정해 symbol version 계산을 제어합니다. Object의 `.discard.gendwarfksyms.kabi_rules` section에 NUL 종료 string을 `version\0type\0target\0value\0` 순서로 저장하고 필요한 만큼 반복합니다.

Source에서 rule을 선언하는 `___KABI_RULE` macro는 `__COUNTER__`로 고유 static char array를 만들고 `__used`, byte alignment 1, `.discard.gendwarfksyms.kabi_rules` section을 지정합니다. String에는 version 1, hint, target, value를 NUL로 구분해 넣습니다. `__KABI_RULE`은 target과 value를 stringize해 이 macro를 감쌉니다.

현재는 이 문서의 rule만 지원하지만 record 형식은 필요에 따라 새 rule을 추가할 수 있도록 확장 가능합니다.

Maintaining a stable kABI
=========================

Distribution maintainers often need the ability to make ABI compatible
changes to kernel data structures due to LTS updates or backports. Using
the traditional `#ifndef __GENKSYMS__` to hide these changes from symbol
versioning won't work when processing object files. To support this
use case, gendwarfksyms provides kABI stability features designed to
hide changes that won't affect the ABI when calculating versions. These
features are all gated behind the **--stable** command line flag and are
not used in the mainline kernel. To use stable features during a kernel
build, set `KBUILD_GENDWARFKSYMS_STABLE=1`.

Examples for using these features are provided in the
**scripts/gendwarfksyms/examples** directory, including helper macros
for source code annotation. Note that as these features are only used to
transform the inputs for symbol versioning, the user is responsible for
ensuring that their changes actually won't break the ABI.

kABI rules
----------

kABI rules allow distributions to fine-tune certain parts
of gendwarfksyms output and thus control how symbol
versions are calculated. These rules are defined in the
`.discard.gendwarfksyms.kabi_rules` section of the object file and
consist of simple null-terminated strings with the following structure::

        version\0type\0target\0value\0

This string sequence is repeated as many times as needed to express all
the rules. The fields are as follows:

- `version`: Ensures backward compatibility for future changes to the
  structure. Currently expected to be "1".
- `type`: Indicates the type of rule being applied.
- `target`: Specifies the target of the rule, typically the fully
  qualified name of the DWARF Debugging Information Entry (DIE).
- `value`: Provides rule-specific data.

The following helper macros, for example, can be used to specify rules
in the source code::

        #define ___KABI_RULE(hint, target, value)                            \
                static const char __PASTE(__gendwarfksyms_rule_,             \
                                          __COUNTER__)[] __used __aligned(1) \
                        __section(".discard.gendwarfksyms.kabi_rules") =     \
                                "1\0" #hint "\0" target "\0" value

        #define __KABI_RULE(hint, target, value) \
                ___KABI_RULE(hint, #target, #value)


Currently, only the rules discussed in this section are supported, but
the format is extensible enough to allow further rules to be added as
need arises.

Definition visibility와 enum rule

추가 include가 TU에 들어오면 declaration이 full definition으로 바뀔 수 있습니다. ABI가 같아도 그 type을 참조하는 모든 symbol version이 달라집니다. Build를 깨지 않고 include를 제거할 수 없다면 `declonly` rule로 DWARF에 full definition이 있어도 declaration-only type으로 취급합니다.

Helper는 `KABI_DECLONLY(fqn) __KABI_RULE(declonly, fqn, )`입니다. 예제의 `struct s` 정의 뒤 `KABI_DECLONLY(s)`를 두면 version 계산에는 declaration처럼 보입니다.

Enum의 모든 enumerator와 값은 symbol version 계산에 포함됩니다. 나중에 enumerator를 추가하되 version을 바꾸지 않으려면 `enumerator_ignore`로 이름이 지정된 항목을 입력에서 숨깁니다.

`KABI_ENUMERATOR_IGNORE(fqn, field)`는 이 rule을 선언합니다. 예제 `enum e { A, B, C, D }`에서 B와 C를 각각 ignore합니다.

Enum에 end marker가 있고 새 값을 중간에 추가했다면 마지막 enumerator가 version 계산에서 예전 값을 유지해야 할 수 있습니다. `enumerator_value` rule은 계산에 사용할 enumerator 값을 override합니다.

`KABI_ENUMERATOR_VALUE(fqn, field, value)`로 선언합니다. 예제는 새 C를 숨기고 `LAST` 값을 2로 override하여 이전 enumerator 배열과 같은 versioning 입력을 만듭니다.

Managing definition visibility
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A declaration can change into a full definition when additional includes
are pulled into the translation unit. This changes the versions of any
symbol that references the type even if the ABI remains unchanged. As
it may not be possible to drop includes without breaking the build, the
`declonly` rule can be used to specify a type as declaration-only, even
if the debugging information contains the full definition.

The rule fields are expected to be as follows:

- `type`: "declonly"
- `target`: The fully qualified name of the target data structure
  (as shown in **--dump-dies** output).
- `value`: This field is ignored.

Using the `__KABI_RULE` macro, this rule can be defined as::

        #define KABI_DECLONLY(fqn) __KABI_RULE(declonly, fqn, )

Example usage::

        struct s {
                /* definition */
        };

        KABI_DECLONLY(s);

Adding enumerators
~~~~~~~~~~~~~~~~~~

For enums, all enumerators and their values are included in calculating
symbol versions, which becomes a problem if we later need to add more
enumerators without changing symbol versions. The `enumerator_ignore`
rule allows us to hide named enumerators from the input.

The rule fields are expected to be as follows:

- `type`: "enumerator_ignore"
- `target`: The fully qualified name of the target enum
  (as shown in **--dump-dies** output) and the name of the
  enumerator field separated by a space.
- `value`: This field is ignored.

Using the `__KABI_RULE` macro, this rule can be defined as::

        #define KABI_ENUMERATOR_IGNORE(fqn, field) \
                __KABI_RULE(enumerator_ignore, fqn field, )

Example usage::

        enum e {
                A, B, C, D,
        };

        KABI_ENUMERATOR_IGNORE(e, B);
        KABI_ENUMERATOR_IGNORE(e, C);

If the enum additionally includes an end marker and new values must
be added in the middle, we may need to use the old value for the last
enumerator when calculating versions. The `enumerator_value` rule allows
us to override the value of an enumerator for version calculation:

- `type`: "enumerator_value"
- `target`: The fully qualified name of the target enum
  (as shown in **--dump-dies** output) and the name of the
  enumerator field separated by a space.
- `value`: Integer value used for the field.

Using the `__KABI_RULE` macro, this rule can be defined as::

        #define KABI_ENUMERATOR_VALUE(fqn, field, value) \
                __KABI_RULE(enumerator_value, fqn field, value)

Example usage::

        enum e {
                A, B, C, LAST,
        };

        KABI_ENUMERATOR_IGNORE(e, C);
        KABI_ENUMERATOR_VALUE(e, LAST, 2);

Structure size와 type string override

Core kernel이 할당하고 module은 일부 member만 접근하는 data structure는 부분적으로 opaque할 수 있습니다. 원래 member layout이 바뀌지 않는다면 끝에 member를 추가해도 ABI를 깨지 않을 수 있습니다.

새 member 자체는 hiding member 기법으로 versioning에서 숨길 수 있지만 structure size 증가까지 숨길 수는 없습니다. `byte_size` rule은 version 계산에 사용할 structure size를 override합니다.

`KABI_BYTE_SIZE(fqn, value)` helper를 사용합니다. 예제는 원래 `unsigned long a`와 `void *p` 뒤에 `KABI_IGNORE(0, unsigned long n)`으로 새 member를 추가하고 `KABI_BYTE_SIZE(s, 16)`으로 versioning size를 16 byte로 유지합니다.

Distribution이 실수로 published ABI에 포함된 opaque structure를 크게 바꿔야 하는 드문 경우에는 세부 rule로 stable version을 유지하기 번거로울 수 있습니다. `type_string` rule은 type이나 symbol의 전체 type string을 override하며 kernel에서 사라진 type도 versioning 입력에 추가할 수 있습니다.

`KABI_TYPE_STRING(type, str)`은 `___KABI_RULE("type_string", type, str)`을 사용합니다. 예제는 `s#s` structure에 명시적인 `structure_type` string을 주고, `my_symbol`에는 `variable s#s`를 지정합니다.

`type_string`은 다른 방법으로 stable symbol version을 합리적으로 유지할 수 없을 때만 최후 수단으로 써야 합니다. Type의 모든 변경을 숨기므로 실제 ABI break가 탐지되지 않을 위험이 커집니다.

Managing structure size changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A data structure can be partially opaque to modules if its allocation is
handled by the core kernel, and modules only need to access some of its
members. In this situation, it's possible to append new members to the
structure without breaking the ABI, as long as the layout for the original
members remains unchanged.

To append new members, we can hide them from symbol versioning as
described in section :ref:`Hiding members <hiding_members>`, but we can't
hide the increase in structure size. The `byte_size` rule allows us to
override the structure size used for symbol versioning.

The rule fields are expected to be as follows:

- `type`: "byte_size"
- `target`: The fully qualified name of the target data structure
  (as shown in **--dump-dies** output).
- `value`: A positive decimal number indicating the structure size
  in bytes.

Using the `__KABI_RULE` macro, this rule can be defined as::

        #define KABI_BYTE_SIZE(fqn, value) \
                __KABI_RULE(byte_size, fqn, value)

Example usage::

        struct s {
                /* Unchanged original members */
                unsigned long a;
                void *p;

                /* Appended new members */
                KABI_IGNORE(0, unsigned long n);
        };

        KABI_BYTE_SIZE(s, 16);

Overriding type strings
~~~~~~~~~~~~~~~~~~~~~~~

In rare situations where distributions must make significant changes to
otherwise opaque data structures that have inadvertently been included
in the published ABI, keeping symbol versions stable using the more
targeted kABI rules can become tedious. The `type_string` rule allows us
to override the full type string for a type or a symbol, and even add
types for versioning that no longer exist in the kernel.

The rule fields are expected to be as follows:

- `type`: "type_string"
- `target`: The fully qualified name of the target data structure
  (as shown in **--dump-dies** output) or symbol.
- `value`: A valid type string (as shown in **--symtypes**) output)
  to use instead of the real type.

Using the `__KABI_RULE` macro, this rule can be defined as::

        #define KABI_TYPE_STRING(type, str) \
                ___KABI_RULE("type_string", type, str)

Example usage::

        /* Override type for a structure */
        KABI_TYPE_STRING("s#s",
                "structure_type s { "
                        "member base_type int byte_size(4) "
                                "encoding(5) n "
                        "data_member_location(0) "
                "} byte_size(8)");

        /* Override type for a symbol */
        KABI_TYPE_STRING("my_symbol", "variable s#s");

The `type_string` rule should be used only as a last resort if maintaining
a stable symbol versions cannot be reasonably achieved using other
means. Overriding a type string increases the risk of actual ABI breakages
going unnoticed as it hides all changes to the type.

Reserved space 사용과 member 숨김

Kernel data structure에 member를 추가하는 것은 흔한 ABI 호환 변경입니다. 변경이 예상되면 distribution maintainer가 미리 공간을 예약했다가 나중에 사용해 ABI break를 피할 수 있습니다. 예약 공간이 없다면 기존 alignment hole을 활용할 수도 있습니다. 이런 변경은 별도 rule보다 union이 자연스러운 경우가 많습니다.

공간은 보통 structure 끝에 integer나 array를 붙여 예약하지만 어떤 type도 사용할 수 있습니다. 각 reserved member에는 고유 이름이 필요합니다. 용도를 미리 알 수 없는 편의를 위해 `__kabi_`로 시작하는 이름은 symbol version 계산에서 제외합니다.

예제는 `struct s` 끝에 `long __kabi_reserved_0`을 둡니다. 실제 사용 시 원래 `long __kabi_reserved_0`과 새 `struct b b`를 같은 anonymous union에 넣습니다.

예약할 때 `__kabi_` naming을 썼다면 union 첫 member 이름은 `__kabi_reserved`로 시작해야 합니다. 그러면 version 계산에는 첫 member의 원래 type을 사용하지만 이름은 다시 제외하며 union의 나머지는 무시합니다.

이 naming convention을 따르지 않던 기존 member를 교체한다면 원래 이름도 보존해야 합니다. 첫 union member 이름을 `__kabi_renamed` 뒤에 원래 이름이 오도록 바꿉니다.

예제 directory의 `KABI_(RESERVE|USE|REPLACE)*` macro는 이 절차를 단순화하고 replacement member가 올바르게 align되며 예약 공간을 넘지 않도록 검사합니다.

지원 기간에 어떤 structure가 바뀔지 항상 예측할 수는 없습니다. 이때 기존 alignment hole에 새 member를 넣을 수 있습니다. 예제의 `struct s`는 `int a`와 `unsigned long b` 사이에 4-byte hole이 있습니다.

Structure size는 바뀌지 않지만 추가 member를 symbol versioning에서 숨겨야 합니다. 새 member를 union으로 감싸고 한 field 이름을 `__kabi_ignored`로 시작하게 합니다. 예제는 `char __kabi_ignored_0`과 실제 `int n`을 같은 union에 둡니다.

`--stable`에서는 member 추가 전후 구조체가 같은 symbol version을 만듭니다. 예제의 `KABI_IGNORE` macro가 이 code를 단순화합니다.

Adding structure members
------------------------

Perhaps the most common ABI compatible change is adding a member to a
kernel data structure. When changes to a structure are anticipated,
distribution maintainers can pre-emptively reserve space in the
structure and take it into use later without breaking the ABI. If
changes are needed to data structures without reserved space, existing
alignment holes can potentially be used instead. While kABI rules could
be added for these type of changes, using unions is typically a more
natural method. This section describes gendwarfksyms support for using
reserved space in data structures and hiding members that don't change
the ABI when calculating symbol versions.

Reserving space and replacing members
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Space is typically reserved for later use by appending integer types, or
arrays, to the end of the data structure, but any type can be used. Each
reserved member needs a unique name, but as the actual purpose is usually
not known at the time the space is reserved, for convenience, names that
start with `__kabi_` are left out when calculating symbol versions::

        struct s {
                long a;
                long __kabi_reserved_0; /* reserved for future use */
        };

The reserved space can be taken into use by wrapping the member in a
union, which includes the original type and the replacement member::

        struct s {
                long a;
                union {
                        long __kabi_reserved_0; /* original type */
                        struct b b; /* replaced field */
                };
        };

If the `__kabi_` naming scheme was used when reserving space, the name
of the first member of the union must start with `__kabi_reserved`. This
ensures the original type is used when calculating versions, but the name
is again left out. The rest of the union is ignored.

If we're replacing a member that doesn't follow this naming convention,
we also need to preserve the original name to avoid changing versions,
which we can do by changing the first union member's name to start with
`__kabi_renamed` followed by the original name.

The examples include `KABI_(RESERVE|USE|REPLACE)*` macros that help
simplify the process and also ensure the replacement member is correctly
aligned and its size won't exceed the reserved space.

.. _hiding_members:

Hiding members
~~~~~~~~~~~~~~

Predicting which structures will require changes during the support
timeframe isn't always possible, in which case one might have to resort
to placing new members into existing alignment holes::

        struct s {
                int a;
                /* a 4-byte alignment hole */
                unsigned long b;
        };


While this won't change the size of the data structure, one needs to
be able to hide the added members from symbol versioning. Similarly
to reserved fields, this can be accomplished by wrapping the added
member to a union where one of the fields has a name starting with
`__kabi_ignored`::

        struct s {
                int a;
                union {
                        char __kabi_ignored_0;
                        int n;
                };
                unsigned long b;
        };

With **--stable**, both versions produce the same symbol version. The
examples include a `KABI_IGNORE` macro to simplify the code.