EHCI driver

제목과 작성일

EHCI driver 문서이며 기록된 날짜는 2002년 12월 27일입니다.

===========
EHCI driver
===========

27-Dec-2002

USB 2.0과 세 가지 전송 속도

EHCI driver는 USB 2.0을 지원하는 host controller hardware를 통해 high-speed USB 2.0 device와 통신합니다. USB 2.0 standard는 USB 1.1과 호환됩니다.

USB 1.1은 full speed와 low speed만 다뤘습니다. High-speed device도 USB 1.1 system에서 사용할 수 있지만 USB 1.1 속도로 낮아집니다.

The EHCI driver is used to talk to high speed USB 2.0 devices using
USB 2.0-capable host controller hardware.  The USB 2.0 standard is
compatible with the USB 1.1 standard. It defines three transfer speeds:

    - "High Speed" 480 Mbit/sec (60 MByte/sec)
    - "Full Speed" 12 Mbit/sec (1.5 MByte/sec)
    - "Low Speed" 1.5 Mbit/sec

USB 1.1 only addressed full speed and low speed.  High speed devices
can be used on USB 1.1 systems, but they slow down to USB 1.1 speeds.

USB 1.1 companion controller와 TT

USB 1.1 device도 USB 2.0 system에서 사용할 수 있습니다. EHCI controller에 직접 연결하면 OHCI 또는 UHCI인 USB 1.1 companion controller로 넘겨집니다.

USB 2.0 hub에 연결된 USB 1.1 device는 hub의 Transaction Translator(TT)를 통해 EHCI와 상호 작용합니다. TT는 low/full-speed transaction을 high-speed split transaction으로 바꿔 bandwidth 낭비를 줄입니다.

USB 1.1 devices may also be used on USB 2.0 systems.  When plugged
into an EHCI controller, they are given to a USB 1.1 "companion"
controller, which is a OHCI or UHCI controller as normally used with
such devices.  When USB 1.1 devices plug into USB 2.0 hubs, they
interact with the EHCI controller through a "Transaction Translator"
(TT) in the hub, which turns low or full speed transactions into
high speed "split transactions" that don't waste transfer bandwidth.

구현체, 장치 보급, Linux USB core

작성 당시 Intel, NEC, Philips, VIA의 EHCI 구현에서 driver 동작이 확인됐고 다른 vendor 구현에서도 동작할 것으로 기대했습니다.

`usb-storage` device는 2001년 중반부터 있었고 hub는 2001년 말부터 보급됐습니다. USB 2.0 내장 system은 2002년 초 등장해 그해 하반기에 더 일반화됐습니다.

USB 2.0 지원에는 EHCI만이 아니라 hub driver를 포함한 Linux-USB core API 변경도 필요했습니다. 다만 USB device driver에 노출되는 기본 `usbcore` API는 크게 바꿀 필요가 없었습니다.

작성자는 David Brownell `<dbrownell@users.sourceforge.net>`입니다.

At this writing, this driver has been seen to work with implementations
of EHCI from (in alphabetical order):  Intel, NEC, Philips, and VIA.
Other EHCI implementations are becoming available from other vendors;
you should expect this driver to work with them too.

While usb-storage devices have been available since mid-2001 (working
quite speedily on the 2.4 version of this driver), hubs have only
been available since late 2001, and other kinds of high speed devices
appear to be on hold until more systems come with USB 2.0 built-in.
Such new systems have been available since early 2002, and became much
more typical in the second half of 2002.

Note that USB 2.0 support involves more than just EHCI.  It requires
other changes to the Linux-USB core APIs, including the hub driver,
but those changes haven't needed to really change the basic "usbcore"
APIs exposed to USB device drivers.

- David Brownell
  <dbrownell@users.sourceforge.net>

Platform과 DMA 동작

이 driver는 x86 hardware에서 정기적으로 시험됐고 PPC에서도 사용됐으므로 big/little endian 문제는 해결된 것으로 봅니다.

PCI 처리도 갖춰져 있어 복잡한 DMA mapping 문제가 있는 system에서도 I/O가 동작한다고 설명합니다.

Functionality
=============

This driver is regularly tested on x86 hardware, and has also been
used on PPC hardware so big/little endianness issues should be gone.
It's believed to do all the right PCI magic so that I/O works even on
systems with interesting DMA mapping issues.

Transfer type 지원 범위

Control, bulk, interrupt transfer와 USB 2.0 hub TT를 통과하는 USB 1.1 request를 처리합니다. 다만 bug가 있을 수 있습니다.

High-speed isochronous(ISO) transfer는 동작하지만 작성 당시 이를 쓰는 Linux driver는 없었습니다.

TT를 통한 full-speed ISO transfer는 아직 지원하지 않았습니다. EHCI가 이를 high-speed ISO와 다른 data structure로 표현해 code 공유가 어렵기 때문이며, 그래서 대부분의 USB audio/video device를 high-speed bus에 연결할 수 없었습니다.

Transfer Types
--------------

At this writing the driver should comfortably handle all control, bulk,
and interrupt transfers, including requests to USB 1.1 devices through
transaction translators (TTs) in USB 2.0 hubs.  But you may find bugs.

High Speed Isochronous (ISO) transfer support is also functional, but
at this writing no Linux drivers have been using that support.

Full Speed Isochronous transfer support, through transaction translators,
is not yet available.  Note that split transaction support for ISO
transfers can't share much code with the code for high speed ISO transfers,
since EHCI represents these with a different data structure.  So for now,
most USB audio and video devices can't be connected to high speed buses.

Queueing과 root hub handoff

모든 transfer type을 queue할 수 있습니다. 따라서 한 interface 또는 `usbfs`의 control transfer가 다른 driver의 transfer를 방해하지 않고, interrupt 처리 비용 때문에 data를 잃지 않으면서 한 frame 주기의 interrupt transfer를 사용할 수 있습니다.

EHCI root hub code는 USB 1.1 device를 companion controller로 넘깁니다. EHCI driver는 companion driver 내부를 알 필요가 없으며 기존 OHCI/UHCI driver도 EHCI가 함께 있다는 이유로 바꿀 필요가 없습니다.

Driver Behavior
---------------

Transfers of all types can be queued.  This means that control transfers
from a driver on one interface (or through usbfs) won't interfere with
ones from another driver, and that interrupt transfers can use periods
of one frame without risking data loss due to interrupt processing costs.

The EHCI root hub code hands off USB 1.1 devices to its companion
controller.  This driver doesn't need to know anything about those
drivers; a OHCI or UHCI driver that works already doesn't need to change
just because the EHCI driver is also present.

Power management와 periodic scheduling 제한

작성 당시 power management에는 문제가 있어 suspend/resume이 완전히 올바르게 동작하지 않았습니다.

Interrupt와 isochronous 같은 periodic transaction scheduling에도 단축 구현이 사용됐습니다. 이 때문에 scheduling 가능한 periodic transaction 수가 제한되고 한 frame보다 짧은 polling interval을 사용할 수 없었습니다.

There are some issues with power management; suspend/resume doesn't
behave quite right at the moment.

Also, some shortcuts have been taken with the scheduling periodic
transactions (interrupt and isochronous transfers).  These place some
limits on the number of periodic transactions that can be scheduled,
and prevent use of polling intervals of less than one frame.

Module load와 companion fallback

PCI card 또는 motherboard에 EHCI controller가 있고 driver를 module로 빌드했다면 `modprobe ehci-hcd`로 load하고 `rmmod ehci-hcd`로 제거합니다.

`ohci-hcd` 또는 `uhci-hcd` 같은 companion controller driver도 필요합니다. EHCI에 문제가 있으면 EHCI module을 제거해 companion controller가 기존 device를 더 낮은 속도로 인계하도록 할 수 있습니다.

Use by
======

Assuming you have an EHCI controller (on a PCI card or motherboard)
and have compiled this driver as a module, load this like::

    # modprobe ehci-hcd

and remove it by::

    # rmmod ehci-hcd

You should also have a driver for a "companion controller", such as
"ohci-hcd"  or "uhci-hcd".  In case of any trouble with the EHCI driver,
remove its module and then the driver for that companion controller will
take over (at lower speed) all the devices that were previously handled
by the EHCI driver.

log2_irq_thresh module parameter

`modprobe`에 전달하는 `log2_irq_thresh`는 기본 interrupt delay의 log2 값을 microframe 단위로 정합니다. 기본값 0은 1 microframe, 즉 125 usec이고 최대값 6은 64 microframe입니다.

이 값은 EHCI controller가 interrupt를 발생시킬 수 있는 빈도를 제어합니다.

Module parameters (pass to "modprobe") include:

    log2_irq_thresh (default 0):
	Log2 of default interrupt delay, in microframes.  The default
	value is 0, indicating 1 microframe (125 usec).  Maximum value
	is 6, indicating 2^6 = 64 microframes.  This controls how often
	the EHCI controller can issue interrupts.

EHCI sysfs 진단 파일

2.5 kernel에서 USB debugging을 켜면 각 EHCI controller의 sysfs directory에 `async`, `periodic`, `registers` file이 나타납니다. 이 내용은 driver 문제 식별에 도움이 됩니다.

If you're using this driver on a 2.5 kernel, and you've enabled USB
debugging support, you'll see three files in the "sysfs" directory for
any EHCI controller:

	"async"
		dumps the asynchronous schedule, used for control
		and bulk transfers.  Shows each active qh and the qtds
		pending, usually one qtd per urb.  (Look at it with
		usb-storage doing disk I/O; watch the request queues!)
	"periodic"
		dumps the periodic schedule, used for interrupt
		and isochronous transfers.  Doesn't show qtds.
	"registers"
		show controller register state, and

The contents of those files can help identify driver problems.

Device driver의 high-speed 차이와 hub 시험

Device driver는 EHCI 위에서 실행되는지 보통 신경 쓸 필요가 없지만 `usb_device->speed == USB_SPEED_HIGH`를 확인할 수 있습니다.

High-speed device는 high-bandwidth periodic interrupt/ISO transfer 같은 기능을 쓸 수 있고, periodic polling interval 등 descriptor 값의 encoding도 high speed에서 달라집니다.

USB 2.0 hub와 TT를 통한 시험이 중요합니다. TT 사용 시 disconnect 같은 failure가 OHCI/UHCI와 다르게 보고될 수 있으며 일부 driver는 이런 다른 fault를 잘못 처리했습니다.

Device drivers shouldn't care whether they're running over EHCI or not,
but they may want to check for "usb_device->speed == USB_SPEED_HIGH".
High speed devices can do things that full speed (or low speed) ones
can't, such as "high bandwidth" periodic (interrupt or ISO) transfers.
Also, some values in device descriptors (such as polling intervals for
periodic transfers) use different encodings when operating at high speed.

However, do make a point of testing device drivers through USB 2.0 hubs.
Those hubs report some failures, such as disconnections, differently when
transaction translators are in use; some drivers have been seen to behave
badly when they see different faults than OHCI or UHCI report.

Throughput와 microframe 계산

USB 2.0 throughput은 host controller의 request 처리 속도와 device 응답 속도에 좌우됩니다. 모든 device가 480 Mbit/sec raw rate를 따르지만 packet 사이 지연, driver 지능, 전체 system load가 aggregate throughput에 영향을 주며 latency도 중요합니다.

Bulk transfer는 항상 512-byte packet을 사용하고 한 USB 2.0 microframe에 최대 13개가 들어갑니다. USB 1.1 frame 하나에는 USB 2.0 microframe 8개가 있으며 microframe은 125 usec입니다.

Hardware와 driver가 허용하면 bulk에 50 MByte/sec 이상을 쓸 수 있습니다. ISO와 interrupt periodic mode는 더 큰 packet을 허용해 명시된 480 Mbit/sec에 더 가까워질 수 있습니다.

Performance
===========

USB 2.0 throughput is gated by two main factors:  how fast the host
controller can process requests, and how fast devices can respond to
them.  The 480 Mbit/sec "raw transfer rate" is obeyed by all devices,
but aggregate throughput is also affected by issues like delays between
individual high speed packets, driver intelligence, and of course the
overall system load.  Latency is also a performance concern.

Bulk transfers are most often used where throughput is an issue.  It's
good to keep in mind that bulk transfers are always in 512 byte packets,
and at most 13 of those fit into one USB 2.0 microframe.  Eight USB 2.0
microframes fit in a USB 1.1 frame; a microframe is 1 msec/8 = 125 usec.

So more than 50 MByte/sec is available for bulk transfers, when both
hardware and device driver software allow it.  Periodic transfer modes
(isochronous and interrupt) allow the larger packet sizes which let you
approach the quoted 480 MBit/sec transfer rate.

Hardware 병목

작성 당시 개별 USB 2.0 device는 대체로 약 20 MByte/sec에서 한계에 도달했으며 device에 따라 더 빠르거나 느렸습니다.

첫 NEC EHCI 구현은 aggregate 약 28 MByte/sec의 hardware bottleneck이 있었습니다. USB와 PCI access를 동시에 수행하지 못해 microframe당 13회 대신 6~7회 transaction만 시도하는 것으로 보였습니다.

이후 구현은 silicon 자원을 더 투입해 60 MByte/sec 목표에 가까워질 것으로 기대했으며 NEC의 갱신 구현과 다른 vendor silicon도 포함됐습니다.

Hardware Performance
--------------------

At this writing, individual USB 2.0 devices tend to max out at around
20 MByte/sec transfer rates.  This is of course subject to change;
and some devices now go faster, while others go slower.

The first NEC implementation of EHCI seems to have a hardware bottleneck
at around 28 MByte/sec aggregate transfer rate.  While this is clearly
enough for a single device at 20 MByte/sec, putting three such devices
onto one bus does not get you 60 MByte/sec.  The issue appears to be
that the controller hardware won't do concurrent USB and PCI access,
so that it's only trying six (or maybe seven) USB transactions each
microframe rather than thirteen.  (Seems like a reasonable trade off
for a product that beat all the others to market by over a year!)

It's expected that newer implementations will better this, throwing
more silicon real estate at the problem so that new motherboard chip
sets will get closer to that 60 MByte/sec target.  That includes an
updated implementation from NEC, as well as other vendors' silicon.

Interrupt latency

EHCI controller가 request 완료 interrupt를 host에 알리는 최소 latency는 1 microframe, 즉 125 usec입니다. Module option으로 조정할 수 있습니다.

`ehci-hcd` 기본값은 최소 latency이며, transfer 크기에 따라 control 또는 bulk request 완료를 흔히 250 usec 안에 알 수 있습니다.

There's a minimum latency of one microframe (125 usec) for the host
to receive interrupts from the EHCI controller indicating completion
of requests.  That latency is tunable; there's a module option.  By
default ehci-hcd driver uses the minimum latency, which means that if
you issue a control or bulk request you can often expect to learn that
it completed in less than 250 usec (depending on transfer size).

Queue를 채우는 software 전략

20 MByte/sec를 내려면 Linux-USB device driver가 EHCI queue를 계속 채워야 합니다. 큰 request를 발행하거나 작은 request가 연속될 때 bulk queueing을 사용해야 합니다.

`usb_bulk_msg()` loop가 4 KB chunk를 쓰면 USB 2.0 bandwidth의 절반 이상을 낭비할 수 있습니다. I/O 완료 후 다음 request를 내는 지연이 I/O 자체보다 길기 때문입니다. 16 KB는 더 낫고 128 KB 연속 chunk는 낭비가 훨씬 적습니다.

큰 synchronous buffer에만 의존하기보다 여러 bulk request를 HC에 queue한 뒤 모두 완료되거나 error로 cancel될 때까지 기다리는 편이 좋습니다. 이런 URB queueing은 USB 1.1 HC driver에서도 동작해야 합니다.

Software Performance
--------------------

To get even 20 MByte/sec transfer rates, Linux-USB device drivers will
need to keep the EHCI queue full.  That means issuing large requests,
or using bulk queuing if a series of small requests needs to be issued.
When drivers don't do that, their performance results will show it.

In typical situations, a usb_bulk_msg() loop writing out 4 KB chunks is
going to waste more than half the USB 2.0 bandwidth.  Delays between the
I/O completion and the driver issuing the next request will take longer
than the I/O.  If that same loop used 16 KB chunks, it'd be better; a
sequence of 128 KB chunks would waste a lot less.

But rather than depending on such large I/O buffers to make synchronous
I/O be efficient, it's better to just queue up several (bulk) requests
to the HC, and wait for them all to complete (or be canceled on error).
Such URB queuing should work with all the USB 1.1 HC drivers too.

usb_sg_* API와 DMA

Linux 2.5 kernel에는 scatterlist의 모든 buffer를 queue하는 새 `usb_sg_*()` API가 정의됐습니다.

이 API는 IOMMU가 적용될 수 있는 scatterlist DMA mapping과 IRQ reduction도 사용해 high-speed transfer 성능을 높입니다.

In the Linux 2.5 kernels, new usb_sg_*() api calls have been defined; they
queue all the buffers from a scatterlist.  They also use scatterlist DMA
mapping (which might apply an IOMMU) and IRQ reduction, all of which will
help make high speed transfers run as fast as they can.

남은 문서화 과제

Interrupt와 ISO transfer 성능 문제는 추가 설명이 필요합니다. Periodic transfer는 완전히 scheduling되므로 핵심은 high-bandwidth mode를 시작하는 방법일 가능성이 큽니다.

표준 80%보다 큰 periodic bandwidth allocation은 sysfs `uframe_periodic_max` parameter로 가능하며 이에 대한 설명도 남은 과제입니다.

TBD:
   Interrupt and ISO transfer performance issues.  Those periodic
   transfers are fully scheduled, so the main issue is likely to be how
   to trigger "high bandwidth" modes.

TBD:
   More than standard 80% periodic bandwidth allocation is possible
   through sysfs uframe_periodic_max parameter. Describe that.