IPVLAN Driver HOWTO

개요, build와 설치

IPVLAN은 macvlan과 개념적으로 비슷하지만 slave 사이의 multiplexing/demultiplexing을 L3에서 수행한다는 차이가 있습니다. 그래서 master와 slave가 같은 L2, 즉 같은 MAC address를 공유합니다. 주된 사용처는 network namespace입니다.

kernel에 내장하려면 `CONFIG_IPVLAN=y`, module로 만들려면 `CONFIG_IPVLAN=m`을 선택합니다.

.. SPDX-License-Identifier: GPL-2.0

===================
IPVLAN Driver HOWTO
===================

Initial Release:
        Mahesh Bandewar <maheshb AT google.com>

1. Introduction:
================
This is conceptually very similar to the macvlan driver with one major
exception of using L3 for mux-ing /demux-ing among slaves. This property makes
the master device share the L2 with its slave devices. I have developed this
driver in conjunction with network namespaces and not sure if there is use case
outside of it.


2. Building and Installation:
=============================

In order to build the driver, please select the config item CONFIG_IPVLAN.
The driver can be built into the kernel (CONFIG_IPVLAN=y) or as a module
(CONFIG_IPVLAN=m).

iproute2 구성

별도 module parameter는 없고 `ip link add link <master> name <slave> type ipvlan [ mode MODE ] [ FLAGS ]`로 구성합니다. mode는 기본 `l3`, `l3s`, `l2` 중 하나이고 flag는 기본 `bridge`, `private`, `vepa` 중 하나입니다.

원문 예시는 `eth0` 아래 `ipvl0`를 option 없이 만들어 L3 bridge로 쓰는 방법, `mode l2 bridge`, `mode l2 private`, `mode l2 vepa`를 각각 명시하는 command를 보존합니다.

3. Configuration:
=================

There are no module parameters for this driver and it can be configured
using IProute2/ip utility.
::

    ip link add link <master> name <slave> type ipvlan [ mode MODE ] [ FLAGS ]
       where
         MODE: l3 (default) | l3s | l2
         FLAGS: bridge (default) | private | vepa

e.g.

    (a) Following will create IPvlan link with eth0 as master in
        L3 bridge mode::

          bash# ip link add link eth0 name ipvl0 type ipvlan
    (b) This command will create IPvlan link in L2 bridge mode::

          bash# ip link add link eth0 name ipvl0 type ipvlan mode l2 bridge

    (c) This command will create an IPvlan device in L2 private mode::

          bash# ip link add link eth0 name ipvlan type ipvlan mode l2 private

    (d) This command will create an IPvlan device in L2 vepa mode::

          bash# ip link add link eth0 name ipvlan type ipvlan mode l2 vepa

L2, L3와 L3S mode

한 master에 붙은 모든 slave는 같은 operating mode를 사용해야 합니다. L2 mode에서는 slave에 연결된 network stack이 TX를 처리하고 packet을 master queue로 넘깁니다. slave는 multicast와 broadcast도 송수신할 수 있습니다.

L3 mode에서는 slave stack이 L3까지 처리한 뒤 master stack이 L2 처리와 routing을 맡습니다. slave는 multicast와 broadcast를 보내거나 받을 수 없고 routing은 주로 default namespace에서 통제되므로 더 제한적입니다.

L3S는 L3와 유사하지만 iptables connection tracking이 작동하도록 symmetric path를 제공합니다. plain L3보다 성능은 조금 낮지만 conntrack이 필요할 때 선택합니다.

4. Operating modes:
===================

IPvlan has two modes of operation - L2 and L3. For a given master device,
you can select one of these two modes and all slaves on that master will
operate in the same (selected) mode. The RX mode is almost identical except
that in L3 mode the slaves won't receive any multicast / broadcast traffic.
L3 mode is more restrictive since routing is controlled from the other (mostly)
default namespace.

4.1 L2 mode:
------------

In this mode TX processing happens on the stack instance attached to the
slave device and packets are switched and queued to the master device to send
out. In this mode the slaves will RX/TX multicast and broadcast (if applicable)
as well.

4.2 L3 mode:
------------

In this mode TX processing up to L3 happens on the stack instance attached
to the slave device and packets are switched to the stack instance of the
master device for the L2 processing and routing from that instance will be
used before packets are queued on the outbound device. In this mode the slaves
will not receive nor can send multicast / broadcast traffic.

4.3 L3S mode:
-------------

This is very similar to the L3 mode except that iptables (conn-tracking)
works in this mode and hence it is L3-symmetric (L3s). This will have slightly less
performance but that shouldn't matter since you are choosing this mode over plain-L3
mode to make conn-tracking work.

Bridge, private와 VEPA flag

기본 `bridge` flag에서는 slave끼리 직접 통신하면서 master를 통한 외부 통신도 할 수 있습니다. `private`는 slave 사이 cross communication을 금지합니다. `vepa`는 IEEE 802.1Qbg처럼 switching을 외부 장치로 넘깁니다.

IPVLAN의 VEPA에는 master MAC을 공유하는 데서 오는 제한이 있습니다. 인접 slave로 보낸 packet의 source와 destination MAC이 같아질 수 있어 switch나 router가 redirect message를 보낼 수 있습니다.

5. Mode flags:
==============

At this time following mode flags are available

5.1 bridge:
-----------
This is the default option. To configure the IPvlan port in this mode,
user can choose to either add this option on the command-line or don't specify
anything. This is the traditional mode where slaves can cross-talk among
themselves apart from talking through the master device.

5.2 private:
------------
If this option is added to the command-line, the port is set in private
mode. i.e. port won't allow cross communication between slaves.

5.3 vepa:
---------
If this is added to the command-line, the port is set in VEPA mode.
i.e. port will offload switching functionality to the external entity as
described in 802.1Qbg
Note: VEPA mode in IPvlan has limitations. IPvlan uses the mac-address of the
master-device, so the packets which are emitted in this mode for the adjacent
neighbor will have source and destination mac same. This will make the switch /
router send the redirect message.

macvlan 대신 IPVLAN을 고를 때

외부 switch/router port가 MAC 하나만 허용하거나, virtual device 수가 NIC MAC table capacity를 넘어 promiscuous mode와 성능 저하를 일으키거나, 신뢰할 수 없는 namespace가 slave의 L2를 변경·악용하지 못하게 해야 한다면 IPVLAN이 적합합니다. 그 밖에는 workload와 네트워크 정책에 따라 macvlan과 비교해야 합니다.

6. What to choose (macvlan vs. ipvlan)?
=======================================

These two devices are very similar in many regards and the specific use
case could very well define which device to choose. if one of the following
situations defines your use case then you can choose to use ipvlan:


(a) The Linux host that is connected to the external switch / router has
    policy configured that allows only one mac per port.
(b) No of virtual devices created on a master exceed the mac capacity and
    puts the NIC in promiscuous mode and degraded performance is a concern.
(c) If the slave device is to be put into the hostile / untrusted network
    namespace where L2 on the slave could be changed / misused.

두 namespace 구성 예

예제는 host `host1`의 master `eth0`에 L2 mode slave `ipvl0`, `ipvl1`을 만들고 각각 `ns0`, `ns1` namespace로 이동합니다. 각 namespace에서 slave와 loopback을 올리고 IPv4 address와 default route를 설정합니다.

원문의 command 순서는 두 namespace 생성, `eth0` 아래 두 slave 생성, 각 slave를 namespace로 이동, `ip netns exec`로 shell을 열어 slave와 `lo`를 up, loopback과 `$IPADDR`를 설정하고 `$ROUTER`를 통한 default route를 추가하는 흐름입니다.

6. Example configuration:
=========================

::

  +=============================================================+
  |  Host: host1                                                |
  |                                                             |
  |   +----------------------+      +----------------------+    |
  |   |   NS:ns0             |      |  NS:ns1              |    |
  |   |                      |      |                      |    |
  |   |                      |      |                      |    |
  |   |        ipvl0         |      |         ipvl1        |    |
  |   +----------#-----------+      +-----------#----------+    |
  |              #                              #               |
  |              ################################               |
  |                              # eth0                         |
  +==============================#==============================+


(a) Create two network namespaces - ns0, ns1::

        ip netns add ns0
        ip netns add ns1

(b) Create two ipvlan slaves on eth0 (master device)::

        ip link add link eth0 ipvl0 type ipvlan mode l2
        ip link add link eth0 ipvl1 type ipvlan mode l2

(c) Assign slaves to the respective network namespaces::

        ip link set dev ipvl0 netns ns0
        ip link set dev ipvl1 netns ns1

(d) Now switch to the namespace (ns0 or ns1) to configure the slave devices

        - For ns0::

                (1) ip netns exec ns0 bash
                (2) ip link set dev ipvl0 up
                (3) ip link set dev lo up
                (4) ip -4 addr add 127.0.0.1 dev lo
                (5) ip -4 addr add $IPADDR dev ipvl0
                (6) ip -4 route add default via $ROUTER dev ipvl0

        - For ns1::

                (1) ip netns exec ns1 bash
                (2) ip link set dev ipvl1 up
                (3) ip link set dev lo up
                (4) ip -4 addr add 127.0.0.1 dev lo
                (5) ip -4 addr add $IPADDR dev ipvl1
                (6) ip -4 route add default via $ROUTER dev ipvl1