Stream Parser (strparser)

stream 위 application message parser

strparser는 byte stream 위에서 실행되는 application layer protocol의 message 경계를 해석하는 utility입니다. kernel upper layer와 함께 application message 지원을 제공하며, KCM(Kernel Connection Multiplexor)은 BPF program으로 message를 parse할 때 사용합니다.

receive callback mode에서는 TCP socket의 `data_ready` callback에서 호출되어 socket에 도착하는 즉시 message를 parse하고 전달합니다. general mode에서는 외부 source가 skb sequence를 공급하고 처리되는 순서대로 message를 조립해 전달하므로 임의 data stream에 적용할 수 있습니다.

.. SPDX-License-Identifier: GPL-2.0

=========================
Stream Parser (strparser)
=========================

Introduction
============

The stream parser (strparser) is a utility that parses messages of an
application layer protocol running over a data stream. The stream
parser works in conjunction with an upper layer in the kernel to provide
kernel support for application layer messages. For instance, Kernel
Connection Multiplexor (KCM) uses the Stream Parser to parse messages
using a BPF program.

The strparser works in one of two modes: receive callback or general
mode.

In receive callback mode, the strparser is called from the data_ready
callback of a TCP socket. Messages are parsed and delivered as they are
received on the socket.

In general mode, a sequence of skbs are fed to strparser from an
outside source. Message are parsed and delivered as the sequence is
processed. This modes allows strparser to be applied to arbitrary
streams of data.

Context와 callback API

API는 parser context 구조체, callback 집합, utility 함수와 receive callback mode용 `data_ready` 함수로 구성됩니다. `parse_msg`는 다음 application message의 길이를 판정하고 KCM에서는 BPF parsing을 수행할 수 있습니다. message가 완성되면 `rcv_msg`가 호출됩니다.

Interface
=========

The API includes a context structure, a set of callbacks, utility
functions, and a data_ready function for receive callback mode. The
callbacks include a parse_msg function that is called to perform
parsing (e.g.  BPF parsing in case of KCM), and a rcv_msg function
that is called when a full message has been completed.

초기화, pause, stop과 process 함수

`strp_init(strp, sk, cb)`는 upper layer가 할당한 `struct strparser`를 초기화합니다. receive callback mode에서는 연결된 TCP `sk`, general mode에서는 NULL을 전달하고 callback 집합을 등록합니다.

`strp_pause()`는 parse와 새 message 전달을 일시 중지하고 `strp_unpause()`는 재개합니다. `strp_stop()`은 오류 시 내부에서 또는 upper layer 요청으로 parser 동작을 완전히 멈춥니다. 중지한 뒤 `strp_done()`을 호출해 instance가 보유한 resource를 해제해야 합니다.

general mode의 `strp_process()`는 `orig_skb`, offset과 length, 최대 message 크기, 완료 timeout을 받아 skb를 parse합니다. 처리 byte 수 또는 음수 오류를 반환하지만 원본 skb 자체는 consume하지 않습니다.

receive callback mode에서 upper layer의 socket `data_ready` callback은 `strp_data_ready()`를 호출합니다. 최대 message 크기는 TCP receive socket buffer limit이고 timeout은 socket receive timeout입니다. `strp_check_rcv()`는 새 socket message를 확인하며 보통 초기화 직후나 `strp_unpause()` 뒤 호출합니다.

Functions
=========

     ::

        strp_init(struct strparser *strp, struct sock *sk,
                const struct strp_callbacks *cb)

     Called to initialize a stream parser. strp is a struct of type
     strparser that is allocated by the upper layer. sk is the TCP
     socket associated with the stream parser for use with receive
     callback mode; in general mode this is set to NULL. Callbacks
     are called by the stream parser (the callbacks are listed below).

     ::

        void strp_pause(struct strparser *strp)

     Temporarily pause a stream parser. Message parsing is suspended
     and no new messages are delivered to the upper layer.

     ::

        void strp_unpause(struct strparser *strp)

     Unpause a paused stream parser.

     ::

        void strp_stop(struct strparser *strp);

     strp_stop is called to completely stop stream parser operations.
     This is called internally when the stream parser encounters an
     error, and it is called from the upper layer to stop parsing
     operations.

     ::

        void strp_done(struct strparser *strp);

     strp_done is called to release any resources held by the stream
     parser instance. This must be called after the stream processor
     has been stopped.

     ::

        int strp_process(struct strparser *strp, struct sk_buff *orig_skb,
                         unsigned int orig_offset, size_t orig_len,
                         size_t max_msg_size, long timeo)

    strp_process is called in general mode for a stream parser to
    parse an sk_buff. The number of bytes processed or a negative
    error number is returned. Note that strp_process does not
    consume the sk_buff. max_msg_size is maximum size the stream
    parser will parse. timeo is timeout for completing a message.

    ::

        void strp_data_ready(struct strparser *strp);

    The upper layer calls strp_tcp_data_ready when data is ready on
    the lower socket for strparser to process. This should be called
    from a data_ready callback that is set on the socket. Note that
    maximum messages size is the limit of the receive socket
    buffer and message timeout is the receive timeout for the socket.

    ::

        void strp_check_rcv(struct strparser *strp);

    strp_check_rcv is called to check for new messages on the socket.
    This is normally called at initialization of a stream parser
    instance or after strp_unpause.

Parser callback과 반환 규약

필수 `parse_msg(strp, skb)`는 skb가 담은 다음 application message header를 해석해 길이를 결정합니다. 입력 `skb->cb`는 `struct strp_msg`이고 여기서는 message 시작 offset만 의미가 있습니다. 양수는 완성 message 길이, 0은 header parse에 data가 더 필요함을 뜻합니다.

`-ESTRPIPE`는 현재 message를 kernel이 처리하지 말고 socket 제어를 userspace에 돌려 직접 읽게 합니다. 다른 음수는 동기화를 잃어 stream 복구가 불가능한 parse 오류이므로 userspace가 TCP socket을 닫을 것으로 가정합니다. receive callback mode에서 음수 오류는 TCP socket error로 설정되고 socket을 깨웁니다. 이미 현재 message 일부를 읽은 뒤 `-ESTRPIPE`가 나오면 stream을 복구할 수 없어 attached socket에는 `ENODATA`를 설정합니다.

`lock`과 `unlock` callback은 timeout 같은 asynchronous 작업 중 `strp` 구조를 보호합니다. receive callback mode 기본값은 연결 socket의 `lock_sock`과 `release_sock`입니다. general mode는 upper layer가 알맞은 구현을 설정해야 합니다.

필수 `rcv_msg()`는 full message가 완성되어 queue에 들어오면 호출되고 callee가 skb를 consume해야 합니다. callback 안에서 `strp_pause()`를 호출해 추가 message 수신을 막을 수 있습니다. `skb->cb`의 `struct strp_msg`에는 시작 `offset`과 message `full_len`이 있으며 strparser가 skb를 trim하지 않으므로 `skb->len - offset`은 `full_len`보다 클 수 있습니다.

선택적 `read_sock()`은 제공되면 `sock->ops->read_sock` 대신 사용합니다. `read_sock_done()`은 receive callback mode에서 TCP socket read loop가 끝날 때 cleanup 기회를 주며 NULL이면 기본 함수를 씁니다. `abort_parser()`는 parse 오류 시 호출됩니다. 기본 구현은 parser를 중지하고 receive callback mode이면 socket error를 설정하지만 `strp_init`에 non-NULL callback을 주어 바꿀 수 있습니다.

Callbacks
=========

There are seven callbacks:

    ::

        int (*parse_msg)(struct strparser *strp, struct sk_buff *skb);

    parse_msg is called to determine the length of the next message
    in the stream. The upper layer must implement this function. It
    should parse the sk_buff as containing the headers for the
    next application layer message in the stream.

    The skb->cb in the input skb is a struct strp_msg. Only
    the offset field is relevant in parse_msg and gives the offset
    where the message starts in the skb.

    The return values of this function are:

    =========    ===========================================================
    >0           indicates length of successfully parsed message
    0            indicates more data must be received to parse the message
    -ESTRPIPE    current message should not be processed by the
                 kernel, return control of the socket to userspace which
                 can proceed to read the messages itself
    other < 0    Error in parsing, give control back to userspace
                 assuming that synchronization is lost and the stream
                 is unrecoverable (application expected to close TCP socket)
    =========    ===========================================================

    In the case that an error is returned (return value is less than
    zero) and the parser is in receive callback mode, then it will set
    the error on TCP socket and wake it up. If parse_msg returned
    -ESTRPIPE and the stream parser had previously read some bytes for
    the current message, then the error set on the attached socket is
    ENODATA since the stream is unrecoverable in that case.

    ::

        void (*lock)(struct strparser *strp)

    The lock callback is called to lock the strp structure when
    the strparser is performing an asynchronous operation (such as
    processing a timeout). In receive callback mode the default
    function is to lock_sock for the associated socket. In general
    mode the callback must be set appropriately.

    ::

        void (*unlock)(struct strparser *strp)

    The unlock callback is called to release the lock obtained
    by the lock callback. In receive callback mode the default
    function is release_sock for the associated socket. In general
    mode the callback must be set appropriately.

    ::

        void (*rcv_msg)(struct strparser *strp, struct sk_buff *skb);

    rcv_msg is called when a full message has been received and
    is queued. The callee must consume the sk_buff; it can
    call strp_pause to prevent any further messages from being
    received in rcv_msg (see strp_pause above). This callback
    must be set.

    The skb->cb in the input skb is a struct strp_msg. This
    struct contains two fields: offset and full_len. Offset is
    where the message starts in the skb, and full_len is the
    the length of the message. skb->len - offset may be greater
    than full_len since strparser does not trim the skb.

    ::

        int (*read_sock)(struct strparser *strp, read_descriptor_t *desc,
                     sk_read_actor_t recv_actor);

    The read_sock callback is used by strparser instead of
    sock->ops->read_sock, if provided.
    ::

        int (*read_sock_done)(struct strparser *strp, int err);

     read_sock_done is called when the stream parser is done reading
     the TCP socket in receive callback mode. The stream parser may
     read multiple messages in a loop and this function allows cleanup
     to occur when exiting the loop. If the callback is not set (NULL
     in strp_init) a default function is used.

     ::

        void (*abort_parser)(struct strparser *strp, int err);

     This function is called when stream parser encounters an error
     in parsing. The default function stops the stream parser and
     sets the error in the socket if the parser is in receive callback
     mode. The default function can be changed by setting the callback
     to non-NULL in strp_init.

Instance 통계

각 stream parser instance는 `strp_stats`에 여러 counter를 유지합니다. 여러 instance를 합산할 때는 편의 구조 `strp_aggr_stats`를 사용하며 `save_strp_stats`와 `aggregate_strp_stats` helper가 저장과 집계를 수행합니다.

Statistics
==========

Various counters are kept for each stream parser instance. These are in
the strp_stats structure. strp_aggr_stats is a convenience structure for
accumulating statistics for multiple stream parser instances.
save_strp_stats and aggregate_strp_stats are helper functions to save
and aggregate statistics.

Message 조립 timeout과 크기 제한

새 message 조립을 시작하면 timer를 설정합니다. receive callback mode는 연결 TCP socket의 `rcvtime`, general mode는 `strp_process()` 인수를 timeout으로 사용합니다. 완성 전에 timer가 만료되면 parser를 abort하고 receive callback mode에서는 TCP socket에 `ETIMEDOUT`을 설정합니다.

receive callback mode의 message 길이는 연결 TCP receive buffer 크기로 제한됩니다. `parse_msg()`가 socket buffer보다 큰 길이를 반환하면 parser를 abort하고 socket에 `EMSGSIZE`를 설정합니다. 이 구조 때문에 strparser가 붙은 socket의 최대 receive skbuff 크기는 `2*sk_rcvbuf`입니다. general mode에서는 `strp_process()`에 최대 길이를 직접 전달합니다.

Message assembly limits
=======================

The stream parser provide mechanisms to limit the resources consumed by
message assembly.

A timer is set when assembly starts for a new message. In receive
callback mode the message timeout is taken from rcvtime for the
associated TCP socket. In general mode, the timeout is passed as an
argument in strp_process. If the timer fires before assembly completes
the stream parser is aborted and the ETIMEDOUT error is set on the TCP
socket if in receive callback mode.

In receive callback mode, message length is limited to the receive
buffer size of the associated TCP socket. If the length returned by
parse_msg is greater than the socket buffer size then the stream parser
is aborted with EMSGSIZE error set on the TCP socket. Note that this
makes the maximum size of receive skbuffs for a socket with a stream
parser to be 2*sk_rcvbuf of the TCP socket.

In general mode the message length limit is passed in as an argument
to strp_process.

저자

Stream Parser 문서 저자는 Tom Herbert입니다.

Author
======

Tom Herbert (tom@quantonium.net)