요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
====================================
HOWTO for the linux packet generator
====================================
Enable CONFIG_NET_PKTGEN to compile and build pktgen either in-kernel
or as a module. A module is preferred; modprobe pktgen if needed. Once
running, pktgen creates a thread for each CPU with affinity to that CPU.
Monitoring and controlling is done via /proc. It is easiest to select a
suitable sample script and configure that.
On a dual CPU::
ps aux | grep pkt
root 129 0.3 0.0 0 0 ? SW 2003 523:20 [kpktgend_0]
root 130 0.3 0.0 0 0 ? SW 2003 509:50 [kpktgend_1]
For monitoring and control pktgen creates::
/proc/net/pktgen/pgctrl
/proc/net/pktgen/kpktgend_X
/proc/net/pktgen/ethX
Tuning NIC for max performance
==============================
The default NIC settings are (likely) not tuned for pktgen's artificial
overload type of benchmarking, as this could hurt the normal use-case.
Specifically increasing the TX ring buffer in the NIC::
# ethtool -G ethX tx 1024
A larger TX ring can improve pktgen's performance, while it can hurt
in the general case, 1) because the TX ring buffer might get larger
than the CPU's L1/L2 cache, 2) because it allows more queueing in the
NIC HW layer (which is bad for bufferbloat).
One should hesitate to conclude that packets/descriptors in the HW
TX ring cause delay. Drivers usually delay cleaning up the
ring-buffers for various performance reasons, and packets stalling
the TX ring might just be waiting for cleanup.
This cleanup issue is specifically the case for the driver ixgbe
(Intel 82599 chip). This driver (ixgbe) combines TX+RX ring cleanups,
and the cleanup interval is affected by the ethtool --coalesce setting
of parameter "rx-usecs".
For ixgbe use e.g. "30" resulting in approx 33K interrupts/sec (1/30*10^6)::
# ethtool -C ethX rx-usecs 30
Kernel threads
==============
Pktgen creates a thread for each CPU with affinity to that CPU.
Which is controlled through procfile /proc/net/pktgen/kpktgend_X.
Example: /proc/net/pktgen/kpktgend_0::
Running:
Stopped: eth4@0
Result: OK: add_device=eth4@0
Most important are the devices assigned to the thread.
The two basic thread commands are:
* add_device DEVICE@NAME -- adds a single device
* rem_device_all -- remove all associated devices
When adding a device to a thread, a corresponding procfile is created
which is used for configuring this device. Thus, device names need to
be unique.
To support adding the same device to multiple threads, which is useful
with multi queue NICs, the device naming scheme is extended with "@":
device@something
The part after "@" can be anything, but it is custom to use the thread
number.
Viewing devices
===============
The Params section holds configured information. The Current section
holds running statistics. The Result is printed after a run or after
interruption. Example::
/proc/net/pktgen/eth4@0
Params: count 100000 min_pkt_size: 60 max_pkt_size: 60
frags: 0 delay: 0 clone_skb: 64 ifname: eth4@0
flows: 0 flowlen: 0
queue_map_min: 0 queue_map_max: 0
dst_min: 192.168.81.2 dst_max:
src_min: src_max:
src_mac: 90:e2:ba:0a:56:b4 dst_mac: 00:1b:21:3c:9d:f8
udp_src_min: 9 udp_src_max: 109 udp_dst_min: 9 udp_dst_max: 9
src_mac_count: 0 dst_mac_count: 0
Flags: UDPSRC_RND NO_TIMESTAMP QUEUE_MAP_CPU
Current:
pkts-sofar: 100000 errors: 0
started: 623913381008us stopped: 623913396439us idle: 25us
seq_num: 100001 cur_dst_mac_offset: 0 cur_src_mac_offset: 0
cur_saddr: 192.168.8.3 cur_daddr: 192.168.81.2
cur_udp_dst: 9 cur_udp_src: 42
cur_queue_map: 0
flows: 0
Result: OK: 15430(c15405+d25) usec, 100000 (60byte,0frags)
6480562pps 3110Mb/sec (3110669760bps) errors: 0
Configuring devices
===================
This is done via the /proc interface, and most easily done via pgset
as defined in the sample scripts.
You need to specify PGDEV environment variable to use functions from sample
scripts, i.e.::
export PGDEV=/proc/net/pktgen/eth4@0
source samples/pktgen/functions.sh
Examples::
pg_ctrl start starts injection.
pg_ctrl stop aborts injection. Also, ^C aborts generator.
pgset "clone_skb 1" sets the number of copies of the same packet
pgset "clone_skb 0" use single SKB for all transmits
pgset "burst 8" uses xmit_more API to queue 8 copies of the same
packet and update HW tx queue tail pointer once.
"burst 1" is the default
pgset "pkt_size 9014" sets packet size to 9014
pgset "frags 5" packet will consist of 5 fragments
pgset "count 200000" sets number of packets to send, set to zero
for continuous sends until explicitly stopped.
pgset "delay 5000" adds delay to hard_start_xmit(). nanoseconds
pgset "dst 10.0.0.1" sets IP destination address
(BEWARE! This generator is very aggressive!)
pgset "dst_min 10.0.0.1" Same as dst
pgset "dst_max 10.0.0.254" Set the maximum destination IP.
pgset "src_min 10.0.0.1" Set the minimum (or only) source IP.
pgset "src_max 10.0.0.254" Set the maximum source IP.
pgset "dst6 fec0::1" IPV6 destination address
pgset "src6 fec0::2" IPV6 source address
pgset "dstmac 00:00:00:00:00:00" sets MAC destination address
pgset "srcmac 00:00:00:00:00:00" sets MAC source address
pgset "queue_map_min 0" Sets the min value of tx queue interval
pgset "queue_map_max 7" Sets the max value of tx queue interval, for multiqueue devices
To select queue 1 of a given device,
use queue_map_min=1 and queue_map_max=1
pgset "src_mac_count 1" Sets the number of MACs we'll range through.
The 'minimum' MAC is what you set with srcmac.
pgset "dst_mac_count 1" Sets the number of MACs we'll range through.
The 'minimum' MAC is what you set with dstmac.
pgset "flag [name]" Set a flag to determine behaviour. Current flags
are: IPSRC_RND # IP source is random (between min/max)
IPDST_RND # IP destination is random
UDPSRC_RND, UDPDST_RND,
MACSRC_RND, MACDST_RND
TXSIZE_RND, IPV6,
MPLS_RND, VID_RND, SVID_RND
FLOW_SEQ,
QUEUE_MAP_RND # queue map random
QUEUE_MAP_CPU # queue map mirrors smp_processor_id()
UDPCSUM,
IPSEC # IPsec encapsulation (needs CONFIG_XFRM)
NODE_ALLOC # node specific memory allocation
NO_TIMESTAMP # disable timestamping
SHARED # enable shared SKB
pgset 'flag ![name]' Clear a flag to determine behaviour.
Note that you might need to use single quote in
interactive mode, so that your shell wouldn't expand
the specified flag as a history command.
pgset "spi [SPI_VALUE]" Set specific SA used to transform packet.
pgset "udp_src_min 9" set UDP source port min, If < udp_src_max, then
cycle through the port range.
pgset "udp_src_max 9" set UDP source port max.
pgset "udp_dst_min 9" set UDP destination port min, If < udp_dst_max, then
cycle through the port range.
pgset "udp_dst_max 9" set UDP destination port max.
pgset "mpls 0001000a,0002000a,0000000a" set MPLS labels (in this example
outer label=16,middle label=32,
inner label=0 (IPv4 NULL)) Note that
there must be no spaces between the
arguments. Leading zeros are required.
Do not set the bottom of stack bit,
that's done automatically. If you do
set the bottom of stack bit, that
indicates that you want to randomly
generate that address and the flag
MPLS_RND will be turned on. You
can have any mix of random and fixed
labels in the label stack.
pgset "mpls 0" turn off mpls (or any invalid argument works too!)
pgset "vlan_id 77" set VLAN ID 0-4095
pgset "vlan_p 3" set priority bit 0-7 (default 0)
pgset "vlan_cfi 0" set canonical format identifier 0-1 (default 0)
pgset "svlan_id 22" set SVLAN ID 0-4095
pgset "svlan_p 3" set priority bit 0-7 (default 0)
pgset "svlan_cfi 0" set canonical format identifier 0-1 (default 0)
pgset "vlan_id 9999" > 4095 remove vlan and svlan tags
pgset "svlan 9999" > 4095 remove svlan tag
pgset "tos XX" set former IPv4 TOS field (e.g. "tos 28" for AF11 no ECN, default 00)
pgset "traffic_class XX" set former IPv6 TRAFFIC CLASS (e.g. "traffic_class B8" for EF no ECN, default 00)
pgset "rate 300M" set rate to 300 Mb/s
pgset "ratep 1000000" set rate to 1Mpps
pgset "xmit_mode netif_receive" RX inject into stack netif_receive_skb()
Works with "burst" but not with "clone_skb".
Default xmit_mode is "start_xmit".
Sample scripts
==============
A collection of tutorial scripts and helpers for pktgen is in the
samples/pktgen directory. The helper parameters.sh file support easy
and consistent parameter parsing across the sample scripts.
Usage example and help::
./pktgen_sample01_simple.sh -i eth4 -m 00:1B:21:3C:9D:F8 -d 192.168.8.2
Usage:::
./pktgen_sample01_simple.sh [-vx] -i ethX
-i : ($DEV) output interface/device (required)
-s : ($PKT_SIZE) packet size
-d : ($DEST_IP) destination IP. CIDR (e.g. 198.18.0.0/15) is also allowed
-m : ($DST_MAC) destination MAC-addr
-p : ($DST_PORT) destination PORT range (e.g. 433-444) is also allowed
-t : ($THREADS) threads to start
-f : ($F_THREAD) index of first thread (zero indexed CPU number)
-c : ($SKB_CLONE) SKB clones send before alloc new SKB
-n : ($COUNT) num messages to send per thread, 0 means indefinitely
-b : ($BURST) HW level bursting of SKBs
-v : ($VERBOSE) verbose
-x : ($DEBUG) debug
-6 : ($IP6) IPv6
-w : ($DELAY) Tx Delay value (ns)
-a : ($APPEND) Script will not reset generator's state, but will append its config
The global variables being set are also listed. E.g. the required
interface/device parameter "-i" sets variable $DEV. Copy the
pktgen_sampleXX scripts and modify them to fit your own needs.
Interrupt affinity
===================
Note that when adding devices to a specific CPU it is a good idea to
also assign /proc/irq/XX/smp_affinity so that the TX interrupts are bound
to the same CPU. This reduces cache bouncing when freeing skbs.
Plus using the device flag QUEUE_MAP_CPU, which maps the SKBs TX queue
to the running threads CPU (directly from smp_processor_id()).
Enable IPsec
============
Default IPsec transformation with ESP encapsulation plus transport mode
can be enabled by simply setting::
pgset "flag IPSEC"
pgset "flows 1"
To avoid breaking existing testbed scripts for using AH type and tunnel mode,
you can use "pgset spi SPI_VALUE" to specify which transformation mode
to employ.
Disable shared SKB
==================
By default, SKBs sent by pktgen are shared (user count > 1).
To test with non-shared SKBs, remove the "SHARED" flag by simply setting::
pg_set "flag !SHARED"
However, if the "clone_skb" or "burst" parameters are configured, the skb
still needs to be held by pktgen for further access. Hence the skb must be
shared.
Current commands and configuration options
==========================================
**Pgcontrol commands**::
start
stop
reset
**Thread commands**::
add_device
rem_device_all
**Device commands**::
count
clone_skb
burst
debug
frags
delay
src_mac_count
dst_mac_count
pkt_size
min_pkt_size
max_pkt_size
queue_map_min
queue_map_max
skb_priority
tos (ipv4)
traffic_class (ipv6)
mpls
udp_src_min
udp_src_max
udp_dst_min
udp_dst_max
node
flag
IPSRC_RND
IPDST_RND
UDPSRC_RND
UDPDST_RND
MACSRC_RND
MACDST_RND
TXSIZE_RND
IPV6
MPLS_RND
VID_RND
SVID_RND
FLOW_SEQ
QUEUE_MAP_RND
QUEUE_MAP_CPU
UDPCSUM
IPSEC
NODE_ALLOC
NO_TIMESTAMP
SHARED
spi (ipsec)
dst_min
dst_max
src_min
src_max
dst_mac
src_mac
clear_counters
src6
dst6
dst6_max
dst6_min
flows
flowlen
rate
ratep
xmit_mode <start_xmit|netif_receive>
vlan_cfi
vlan_id
vlan_p
svlan_cfi
svlan_id
svlan_p
References:
- ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/
- ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/examples/
Paper from Linux-Kongress in Erlangen 2004.
- ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/pktgen_paper.pdf
Thanks to:
Grant Grundler for testing on IA-64 and parisc, Harald Welte, Lennert Buytenhek
Stephen Hemminger, Andi Kleen, Dave Miller and many others.
Good luck with the linux net-development.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
활성화와 procfs 제어 파일
1-26`CONFIG_NET_PKTGEN`을 활성화하면 pktgen을 kernel 내장 또는 module로 build할 수 있습니다. Module 방식이 권장되며 필요하면 `modprobe pktgen`으로 load합니다. 실행되면 각 CPU마다 해당 CPU에 affinity가 고정된 thread를 하나씩 만듭니다.
Monitoring과 control은 `/proc`를 통해 수행합니다. 가장 쉬운 시작 방법은 목적에 맞는 sample script를 골라 parameter를 조정하는 것입니다. Dual-CPU 예에서는 `kpktgend_0`과 `kpktgend_1` kernel thread가 각각 보입니다.
전체 generator 제어는 `/proc/net/pktgen/pgctrl`, CPU별 thread 제어는 `/proc/net/pktgen/kpktgend_X`, device별 설정과 통계는 `/proc/net/pktgen/ethX`에서 수행합니다.
제어 범위에 따라 파일이 나뉩니다.
CPU별 thread가 자신에게 배정된 device를 통해 packet을 생성합니다.
.. SPDX-License-Identifier: GPL-2.0
====================================
HOWTO for the linux packet generator
====================================
Enable CONFIG_NET_PKTGEN to compile and build pktgen either in-kernel
or as a module. A module is preferred; modprobe pktgen if needed. Once
running, pktgen creates a thread for each CPU with affinity to that CPU.
Monitoring and controlling is done via /proc. It is easiest to select a
suitable sample script and configure that.
On a dual CPU::
ps aux | grep pkt
root 129 0.3 0.0 0 0 ? SW 2003 523:20 [kpktgend_0]
root 130 0.3 0.0 0 0 ? SW 2003 509:50 [kpktgend_1]
For monitoring and control pktgen creates::
/proc/net/pktgen/pgctrl
/proc/net/pktgen/kpktgend_X
/proc/net/pktgen/ethX
최대 성능을 위한 NIC tuning
27-56NIC의 기본 설정은 정상 workload를 위한 것이므로 pktgen의 인위적인 과부하 benchmark에 최적화되어 있지 않을 가능성이 큽니다. Benchmark 설정은 일반 사용 성능을 해칠 수 있습니다.
`ethtool -G ethX tx 1024`처럼 NIC의 TX ring을 늘리면 pktgen 성능이 좋아질 수 있습니다. 하지만 ring이 CPU L1/L2 cache보다 커질 수 있고 NIC hardware layer에 더 많은 queueing을 허용하여 bufferbloat를 악화하므로 일반 환경에는 불리할 수 있습니다.
Hardware TX ring 안의 packet이나 descriptor가 곧바로 지연 원인이라고 단정해서는 안 됩니다. Driver는 성능상 이유로 ring buffer cleanup을 늦추는 경우가 많으므로, ring에 머무는 packet은 단지 cleanup을 기다리는 것일 수 있습니다.
Intel 82599용 `ixgbe`는 TX와 RX ring cleanup을 결합하며 cleanup 간격은 ethtool coalesce의 `rx-usecs` 설정에 영향을 받습니다. 예를 들어 `ethtool -C ethX rx-usecs 30`은 대략 초당 33,000회 interrupt를 만듭니다.
Benchmark throughput과 일반 latency 사이의 tradeoff입니다.
Tuning NIC for max performance
==============================
The default NIC settings are (likely) not tuned for pktgen's artificial
overload type of benchmarking, as this could hurt the normal use-case.
Specifically increasing the TX ring buffer in the NIC::
# ethtool -G ethX tx 1024
A larger TX ring can improve pktgen's performance, while it can hurt
in the general case, 1) because the TX ring buffer might get larger
than the CPU's L1/L2 cache, 2) because it allows more queueing in the
NIC HW layer (which is bad for bufferbloat).
One should hesitate to conclude that packets/descriptors in the HW
TX ring cause delay. Drivers usually delay cleaning up the
ring-buffers for various performance reasons, and packets stalling
the TX ring might just be waiting for cleanup.
This cleanup issue is specifically the case for the driver ixgbe
(Intel 82599 chip). This driver (ixgbe) combines TX+RX ring cleanups,
and the cleanup interval is affected by the ethtool --coalesce setting
of parameter "rx-usecs".
For ixgbe use e.g. "30" resulting in approx 33K interrupts/sec (1/30*10^6)::
# ethtool -C ethX rx-usecs 30
CPU thread와 device 별칭
57-85Pktgen은 CPU마다 그 CPU에 affinity가 고정된 thread를 만들고 `/proc/net/pktgen/kpktgend_X`로 제어합니다. Thread 파일은 현재 실행 중인 device, 중지된 device, 마지막 명령 결과를 보여 줍니다.
핵심 thread 명령은 `add_device DEVICE@NAME`과 `rem_device_all`입니다. 첫 명령은 device 하나를 thread에 추가하고, 둘째 명령은 연결된 모든 device를 제거합니다.
Thread에 device를 추가하면 해당 device를 설정할 procfile이 생성되므로 이름은 고유해야 합니다. Multi-queue NIC에서 같은 실제 device를 여러 thread에 추가할 수 있도록 `device@something` 형식을 사용합니다. `@` 뒤 문자열은 무엇이든 가능하지만 관례적으로 thread 번호를 씁니다.
같은 NIC를 고유 별칭으로 여러 CPU thread에 연결합니다.
Kernel threads
==============
Pktgen creates a thread for each CPU with affinity to that CPU.
Which is controlled through procfile /proc/net/pktgen/kpktgend_X.
Example: /proc/net/pktgen/kpktgend_0::
Running:
Stopped: eth4@0
Result: OK: add_device=eth4@0
Most important are the devices assigned to the thread.
The two basic thread commands are:
* add_device DEVICE@NAME -- adds a single device
* rem_device_all -- remove all associated devices
When adding a device to a thread, a corresponding procfile is created
which is used for configuring this device. Thus, device names need to
be unique.
To support adding the same device to multiple threads, which is useful
with multi queue NICs, the device naming scheme is extended with "@":
device@something
The part after "@" can be anything, but it is custom to use the thread
number.
Device 상태와 실행 결과 읽기
86-116Device procfile의 `Params` 절은 구성된 parameter를, `Current` 절은 실행 중 통계를 보관합니다. `Result`는 실행이 끝나거나 중단된 뒤 출력됩니다.
예제의 `Params`에는 packet count·size·fragment·delay·SKB clone 수, flow, TX queue 범위, source/destination IP와 MAC, UDP port 범위, 활성 flag가 기록됩니다. `Current`에는 지금까지의 packet·error 수, start/stop/idle 시간, sequence, 현재 주소와 port, queue map, flow 상태가 표시됩니다.
`Result`는 총 소요 시간과 clone·delay 내역, packet 수와 크기, packet per second, Mbit/s 및 bit/s, error 수를 요약합니다. 예제는 100,000개의 60-byte packet을 error 없이 약 6.48Mpps와 3110Mb/s로 전송한 결과입니다.
설정·실시간 상태·최종 결과를 구분합니다.
Viewing devices
===============
The Params section holds configured information. The Current section
holds running statistics. The Result is printed after a run or after
interruption. Example::
/proc/net/pktgen/eth4@0
Params: count 100000 min_pkt_size: 60 max_pkt_size: 60
frags: 0 delay: 0 clone_skb: 64 ifname: eth4@0
flows: 0 flowlen: 0
queue_map_min: 0 queue_map_max: 0
dst_min: 192.168.81.2 dst_max:
src_min: src_max:
src_mac: 90:e2:ba:0a:56:b4 dst_mac: 00:1b:21:3c:9d:f8
udp_src_min: 9 udp_src_max: 109 udp_dst_min: 9 udp_dst_max: 9
src_mac_count: 0 dst_mac_count: 0
Flags: UDPSRC_RND NO_TIMESTAMP QUEUE_MAP_CPU
Current:
pkts-sofar: 100000 errors: 0
started: 623913381008us stopped: 623913396439us idle: 25us
seq_num: 100001 cur_dst_mac_offset: 0 cur_src_mac_offset: 0
cur_saddr: 192.168.8.3 cur_daddr: 192.168.81.2
cur_udp_dst: 9 cur_udp_src: 42
cur_queue_map: 0
flows: 0
Result: OK: 15430(c15405+d25) usec, 100000 (60byte,0frags)
6480562pps 3110Mb/sec (3110669760bps) errors: 0
기본 device parameter와 주소 범위
117-166Device 설정은 `/proc` interface로 하며 sample script에 정의된 `pgset` helper를 쓰는 것이 가장 쉽습니다. `PGDEV` 환경 변수를 대상 procfile로 지정한 뒤 `samples/pktgen/functions.sh`를 source합니다.
`pg_ctrl start`는 packet injection을 시작하고 `pg_ctrl stop`이나 Ctrl-C는 중단합니다. `clone_skb 1`은 같은 packet 복사본 수를 설정하며 `clone_skb 0`은 모든 전송에 단일 SKB를 사용합니다. `burst 8`은 `xmit_more` API로 같은 packet 8개를 queue하고 hardware TX queue tail pointer를 한 번만 갱신합니다. 기본값은 `burst 1`입니다.
`pkt_size`, `frags`, `count`는 packet 크기, fragment 수, 보낼 packet 수를 정합니다. `count 0`은 명시적으로 중단할 때까지 계속 전송합니다. `delay`는 `hard_start_xmit()`에 nanosecond 단위 지연을 추가합니다.
`dst`, `dst_min`, `dst_max`, `src_min`, `src_max`로 IPv4 단일 주소나 순회 범위를 설정합니다. Generator는 매우 공격적으로 traffic을 만들므로 destination을 신중하게 선택해야 합니다. `dst6`와 `src6`는 IPv6 주소, `dstmac`과 `srcmac`은 MAC 주소를 설정합니다.
`queue_map_min`과 `queue_map_max`는 multi-queue device의 TX queue 범위를 정합니다. 특정 queue 1만 선택하려면 둘 다 1로 둡니다. `src_mac_count`와 `dst_mac_count`는 순회할 MAC 수를 정하며 시작 MAC은 각각 `srcmac`과 `dstmac`으로 설정한 값입니다.
Packet 생성량과 주소·queue를 설정합니다.
Configuring devices
===================
This is done via the /proc interface, and most easily done via pgset
as defined in the sample scripts.
You need to specify PGDEV environment variable to use functions from sample
scripts, i.e.::
export PGDEV=/proc/net/pktgen/eth4@0
source samples/pktgen/functions.sh
Examples::
pg_ctrl start starts injection.
pg_ctrl stop aborts injection. Also, ^C aborts generator.
pgset "clone_skb 1" sets the number of copies of the same packet
pgset "clone_skb 0" use single SKB for all transmits
pgset "burst 8" uses xmit_more API to queue 8 copies of the same
packet and update HW tx queue tail pointer once.
"burst 1" is the default
pgset "pkt_size 9014" sets packet size to 9014
pgset "frags 5" packet will consist of 5 fragments
pgset "count 200000" sets number of packets to send, set to zero
for continuous sends until explicitly stopped.
pgset "delay 5000" adds delay to hard_start_xmit(). nanoseconds
pgset "dst 10.0.0.1" sets IP destination address
(BEWARE! This generator is very aggressive!)
pgset "dst_min 10.0.0.1" Same as dst
pgset "dst_max 10.0.0.254" Set the maximum destination IP.
pgset "src_min 10.0.0.1" Set the minimum (or only) source IP.
pgset "src_max 10.0.0.254" Set the maximum source IP.
pgset "dst6 fec0::1" IPV6 destination address
pgset "src6 fec0::2" IPV6 source address
pgset "dstmac 00:00:00:00:00:00" sets MAC destination address
pgset "srcmac 00:00:00:00:00:00" sets MAC source address
pgset "queue_map_min 0" Sets the min value of tx queue interval
pgset "queue_map_max 7" Sets the max value of tx queue interval, for multiqueue devices
To select queue 1 of a given device,
use queue_map_min=1 and queue_map_max=1
pgset "src_mac_count 1" Sets the number of MACs we'll range through.
The 'minimum' MAC is what you set with srcmac.
pgset "dst_mac_count 1" Sets the number of MACs we'll range through.
The 'minimum' MAC is what you set with dstmac.
Random flag, MPLS, VLAN과 rate
167-234`flag name`은 동작 flag를 켜고 `flag !name`은 끕니다. Interactive shell에서는 `!`가 history expansion으로 처리되지 않도록 작은따옴표를 사용해야 할 수 있습니다.
`IPSRC_RND`, `IPDST_RND`, `UDPSRC_RND`, `UDPDST_RND`, `MACSRC_RND`, `MACDST_RND`, `TXSIZE_RND`는 각 source/destination IP·UDP port·MAC·packet size를 설정 범위에서 무작위로 고릅니다. `IPV6`는 IPv6, `MPLS_RND`는 MPLS label, `VID_RND`와 `SVID_RND`는 VLAN·service VLAN ID 무작위화를 켭니다.
`FLOW_SEQ`는 flow sequence, `QUEUE_MAP_RND`는 무작위 queue map, `QUEUE_MAP_CPU`는 `smp_processor_id()`와 같은 CPU 번호의 queue를 사용합니다. `UDPCSUM`은 UDP checksum을, `IPSEC`은 `CONFIG_XFRM`이 필요한 IPsec encapsulation을, `NODE_ALLOC`은 node-specific memory allocation을 켭니다. `NO_TIMESTAMP`는 timestamp를 끄고 `SHARED`는 shared SKB를 사용합니다.
`spi`는 packet transform에 사용할 특정 Security Association을 정합니다. UDP source와 destination의 min/max는 port 순회 범위를 정의하며 min이 max보다 작으면 범위를 반복합니다.
`mpls`는 comma로 구분한 hexadecimal label stack을 설정합니다. Argument 사이에는 공백이 없어야 하고 leading zero가 필요합니다. Bottom-of-stack bit는 자동으로 설정하므로 직접 넣지 않습니다. 직접 설정하면 해당 주소를 무작위 생성하겠다는 뜻으로 해석되어 `MPLS_RND`가 켜집니다. 고정 label과 무작위 label을 한 stack에 섞을 수 있습니다. `mpls 0` 또는 유효하지 않은 argument는 MPLS를 끕니다.
`vlan_id`, `vlan_p`, `vlan_cfi`와 `svlan_id`, `svlan_p`, `svlan_cfi`는 각각 VLAN 및 service VLAN의 ID(0~4095), priority(0~7), canonical format identifier(0~1)를 설정합니다. `vlan_id 9999`처럼 4095보다 큰 값은 VLAN과 SVLAN tag를 제거하며, `svlan 9999`는 SVLAN tag를 제거합니다.
`tos`는 기존 IPv4 TOS field, `traffic_class`는 기존 IPv6 traffic class를 설정합니다. `rate 300M`은 300Mb/s, `ratep 1000000`은 1Mpps로 rate를 제한합니다.
기본 `xmit_mode`는 `start_xmit`입니다. `xmit_mode netif_receive`는 `netif_receive_skb()`를 통해 RX packet을 stack에 주입합니다. 이 mode는 `burst`와 함께 동작하지만 `clone_skb`와는 함께 쓸 수 없습니다.
무작위화와 전송 동작 flag를 묶어 봅니다.
Bottom-of-stack bit 여부가 고정·무작위 label을 구분합니다.
pgset "flag [name]" Set a flag to determine behaviour. Current flags
are: IPSRC_RND # IP source is random (between min/max)
IPDST_RND # IP destination is random
UDPSRC_RND, UDPDST_RND,
MACSRC_RND, MACDST_RND
TXSIZE_RND, IPV6,
MPLS_RND, VID_RND, SVID_RND
FLOW_SEQ,
QUEUE_MAP_RND # queue map random
QUEUE_MAP_CPU # queue map mirrors smp_processor_id()
UDPCSUM,
IPSEC # IPsec encapsulation (needs CONFIG_XFRM)
NODE_ALLOC # node specific memory allocation
NO_TIMESTAMP # disable timestamping
SHARED # enable shared SKB
pgset 'flag ![name]' Clear a flag to determine behaviour.
Note that you might need to use single quote in
interactive mode, so that your shell wouldn't expand
the specified flag as a history command.
pgset "spi [SPI_VALUE]" Set specific SA used to transform packet.
pgset "udp_src_min 9" set UDP source port min, If < udp_src_max, then
cycle through the port range.
pgset "udp_src_max 9" set UDP source port max.
pgset "udp_dst_min 9" set UDP destination port min, If < udp_dst_max, then
cycle through the port range.
pgset "udp_dst_max 9" set UDP destination port max.
pgset "mpls 0001000a,0002000a,0000000a" set MPLS labels (in this example
outer label=16,middle label=32,
inner label=0 (IPv4 NULL)) Note that
there must be no spaces between the
arguments. Leading zeros are required.
Do not set the bottom of stack bit,
that's done automatically. If you do
set the bottom of stack bit, that
indicates that you want to randomly
generate that address and the flag
MPLS_RND will be turned on. You
can have any mix of random and fixed
labels in the label stack.
pgset "mpls 0" turn off mpls (or any invalid argument works too!)
pgset "vlan_id 77" set VLAN ID 0-4095
pgset "vlan_p 3" set priority bit 0-7 (default 0)
pgset "vlan_cfi 0" set canonical format identifier 0-1 (default 0)
pgset "svlan_id 22" set SVLAN ID 0-4095
pgset "svlan_p 3" set priority bit 0-7 (default 0)
pgset "svlan_cfi 0" set canonical format identifier 0-1 (default 0)
pgset "vlan_id 9999" > 4095 remove vlan and svlan tags
pgset "svlan 9999" > 4095 remove svlan tag
pgset "tos XX" set former IPv4 TOS field (e.g. "tos 28" for AF11 no ECN, default 00)
pgset "traffic_class XX" set former IPv6 TRAFFIC CLASS (e.g. "traffic_class B8" for EF no ECN, default 00)
pgset "rate 300M" set rate to 300 Mb/s
pgset "ratep 1000000" set rate to 1Mpps
pgset "xmit_mode netif_receive" RX inject into stack netif_receive_skb()
Works with "burst" but not with "clone_skb".
Default xmit_mode is "start_xmit".
Sample script 사용법
235-270`samples/pktgen`에는 tutorial script와 helper가 모여 있습니다. `parameters.sh`는 여러 sample script가 parameter를 쉽고 일관되게 parse하도록 지원합니다.
예제는 `pktgen_sample01_simple.sh`에 output interface, destination MAC, destination IP를 전달합니다. `-i` interface는 필수이고 `-s` packet size, `-d` destination IP 또는 CIDR, `-m` destination MAC, `-p` destination port 또는 범위, `-t` thread 수, `-f` 시작 thread index를 지정합니다.
`-c`는 새 SKB를 할당하기 전에 보낼 clone 수, `-n`은 thread당 message 수이며 0은 무한 실행, `-b`는 hardware-level burst입니다. `-v`는 verbose, `-x`는 debug, `-6`은 IPv6, `-w`는 TX delay(ns), `-a`는 기존 generator 상태를 reset하지 않고 설정을 덧붙입니다.
각 option이 설정하는 global variable도 help에 표시됩니다. 예를 들어 필수 `-i`가 `$DEV`를 설정합니다. 자신의 요구에 맞게 `pktgen_sampleXX` script를 복사하여 수정하면 됩니다.
자주 쓰는 command-line option입니다.
Sample scripts
==============
A collection of tutorial scripts and helpers for pktgen is in the
samples/pktgen directory. The helper parameters.sh file support easy
and consistent parameter parsing across the sample scripts.
Usage example and help::
./pktgen_sample01_simple.sh -i eth4 -m 00:1B:21:3C:9D:F8 -d 192.168.8.2
Usage:::
./pktgen_sample01_simple.sh [-vx] -i ethX
-i : ($DEV) output interface/device (required)
-s : ($PKT_SIZE) packet size
-d : ($DEST_IP) destination IP. CIDR (e.g. 198.18.0.0/15) is also allowed
-m : ($DST_MAC) destination MAC-addr
-p : ($DST_PORT) destination PORT range (e.g. 433-444) is also allowed
-t : ($THREADS) threads to start
-f : ($F_THREAD) index of first thread (zero indexed CPU number)
-c : ($SKB_CLONE) SKB clones send before alloc new SKB
-n : ($COUNT) num messages to send per thread, 0 means indefinitely
-b : ($BURST) HW level bursting of SKBs
-v : ($VERBOSE) verbose
-x : ($DEBUG) debug
-6 : ($IP6) IPv6
-w : ($DELAY) Tx Delay value (ns)
-a : ($APPEND) Script will not reset generator's state, but will append its config
The global variables being set are also listed. E.g. the required
interface/device parameter "-i" sets variable $DEV. Copy the
pktgen_sampleXX scripts and modify them to fit your own needs.
전체 command와 configuration option
303-407Pgcontrol command는 `start`, `stop`, `reset`입니다. Thread command는 `add_device`와 `rem_device_all`입니다.
Device 기본 command에는 `count`, `clone_skb`, `burst`, `debug`, `frags`, `delay`, source/destination MAC count, packet size 및 min/max packet size가 있습니다. Queue 관련 command는 `queue_map_min`, `queue_map_max`, `skb_priority`입니다.
Protocol field에는 IPv4 `tos`, IPv6 `traffic_class`, `mpls`, UDP source/destination min/max, NUMA `node`가 있습니다. `flag` 아래에는 모든 random·IPv6·MPLS·VLAN·flow·queue·checksum·IPsec·allocation·timestamp·shared option이 나열됩니다.
IPsec `spi`, IPv4 destination/source min/max, destination/source MAC, `clear_counters`, IPv6 source/destination 및 destination min/max, `flows`, `flowlen`, bit rate `rate`, packet rate `ratep`도 지원합니다.
`xmit_mode`는 `start_xmit` 또는 `netif_receive`를 선택합니다. VLAN과 service VLAN에는 각각 CFI, ID, priority command가 있습니다. 이 목록은 앞 절의 사용 예를 한곳에 모은 reference입니다.
Procfile 범위별 명령을 정리합니다.
Current commands and configuration options
==========================================
**Pgcontrol commands**::
start
stop
reset
**Thread commands**::
add_device
rem_device_all
**Device commands**::
count
clone_skb
burst
debug
frags
delay
src_mac_count
dst_mac_count
pkt_size
min_pkt_size
max_pkt_size
queue_map_min
queue_map_max
skb_priority
tos (ipv4)
traffic_class (ipv6)
mpls
udp_src_min
udp_src_max
udp_dst_min
udp_dst_max
node
flag
IPSRC_RND
IPDST_RND
UDPSRC_RND
UDPDST_RND
MACSRC_RND
MACDST_RND
TXSIZE_RND
IPV6
MPLS_RND
VID_RND
SVID_RND
FLOW_SEQ
QUEUE_MAP_RND
QUEUE_MAP_CPU
UDPCSUM
IPSEC
NODE_ALLOC
NO_TIMESTAMP
SHARED
spi (ipsec)
dst_min
dst_max
src_min
src_max
dst_mac
src_mac
clear_counters
src6
dst6
dst6_max
dst6_min
flows
flowlen
rate
ratep
xmit_mode <start_xmit|netif_receive>
vlan_cfi
vlan_id
vlan_p
svlan_cfi
svlan_id
svlan_p
참고 자료와 기여자
408-422원문은 SLU의 pktgen testing FTP archive와 examples archive를 참고 자료로 제공합니다. 2004년 Erlangen Linux-Kongress에서 발표된 pktgen paper PDF도 같은 archive에 있습니다. URL은 원문에 그대로 보존됩니다.
IA-64와 parisc에서 시험한 Grant Grundler를 비롯해 Harald Welte, Lennert Buytenhek, Stephen Hemminger, Andi Kleen, Dave Miller 및 많은 기여자에게 감사를 표합니다. 문서는 Linux network development의 성공을 기원하며 끝납니다.
References:
- ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/
- ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/examples/
Paper from Linux-Kongress in Erlangen 2004.
- ftp://robur.slu.se/pub/Linux/net-development/pktgen-testing/pktgen_paper.pdf
Thanks to:
Grant Grundler for testing on IA-64 and parisc, Harald Welte, Lennert Buytenhek
Stephen Hemminger, Andi Kleen, Dave Miller and many others.
Good luck with the linux net-development.
요약·해설
pktgen.rst:1-422Pktgen은 CPU별 kernel thread가 procfs 설정에 따라 고속 packet을 생성하는 network benchmark 도구입니다. NIC ring과 interrupt affinity를 함께 조정하고, device별 packet·주소·queue·protocol parameter를 설정하며, CPU locality와 일반 traffic에 미치는 영향을 구분해 측정해야 합니다.
Module load부터 결과 확인까지의 기본 절차입니다.