MEN Chameleon Bus

MEN Chameleon Bus 문서 구성

이 문서는 MEN Chameleon Bus, 즉 MCB의 architecture, resource handling과 driver 작성 방법을 설명합니다.

목차는 소개와 구현 한계, bus·carrier·parser architecture, memory·IRQ resource, driver structure·probe·초기화·DMA로 구성됩니다.

=================
MEN Chameleon Bus
=================

.. Table of Contents
   =================
   1 Introduction
       1.1 Scope of this Document
       1.2 Limitations of the current implementation
   2 Architecture
       2.1 MEN Chameleon Bus
       2.2 Carrier Devices
       2.3 Parser
   3 Resource handling
       3.1 Memory Resources
       3.2 IRQs
   4 Writing an MCB driver
       4.1 The driver structure
       4.2 Probing and attaching
       4.3 Initializing the driver
       4.4 Using DMA

MCB 범위와 현재 한계

문서는 MEN Chameleon Bus의 현재 architecture와 구현을 짧게 소개하며 MCB 기반 장치의 모든 가능성을 설명하지는 않습니다.

현재 구현은 memory resource 하나만 사용하고 PCI legacy IRQ를 공유하는 PCI·PCIe carrier device로 제한됩니다.

VME controller나 M-Module carrier 같은 multi-resource MCB device, DMA controller descriptor용 SRAM이나 video memory처럼 다른 MCB device에 의존하는 장치는 구현되지 않았습니다.

MSI·MSI-X를 지원해 MCB device마다 하나 이상의 IRQ를 제공하는 PCIe carrier용 per-carrier IRQ domain도 아직 없습니다.

Introduction
============

This document describes the architecture and implementation of the MEN
Chameleon Bus (called MCB throughout this document).

Scope of this Document
----------------------

This document is intended to be a short overview of the current
implementation and does by no means describe the complete possibilities of MCB
based devices.

Limitations of the current implementation
-----------------------------------------

The current implementation is limited to PCI and PCIe based carrier devices
that only use a single memory resource and share the PCI legacy IRQ.  Not
implemented are:

- Multi-resource MCB devices like the VME Controller or M-Module carrier.
- MCB devices that need another MCB device, like SRAM for a DMA Controller's
  buffer descriptors or a video controller's video memory.
- A per-carrier IRQ domain for carrier devices that have one (or more) IRQs
  per MCB device like PCIe based carriers with MSI or MSI-X support.

MCB architecture와 FPGA table

MCB는 MEN Chameleon Bus 자체, MCB carrier device driver, Chameleon table parser의 세 기능 block으로 나뉩니다.

MEN Mikro Elektronik GmbH의 일부 hardware에는 Chameleon FPGA가 있으며 MCB는 이 FPGA에 붙는 인공 bus system입니다.

장치는 하나의 FPGA에 여러 기능을 구현한 multi-function device이며 보통 PCI 또는 PCIe link로 연결됩니다.

각 FPGA header는 device ID, PCI BAR, BAR 시작점 기준 offset, FPGA 내부 크기, interrupt 번호와 현재 MCB 구현이 처리하지 않는 기타 속성을 나열합니다.

Architecture
============

MCB is divided into 3 functional blocks:

- The MEN Chameleon Bus itself,
- drivers for MCB Carrier Devices and
- the parser for the Chameleon table.

MEN Chameleon Bus
-----------------

The MEN Chameleon Bus is an artificial bus system that attaches to a so
called Chameleon FPGA device found on some hardware produced my MEN Mikro
Elektronik GmbH. These devices are multi-function devices implemented in a
single FPGA and usually attached via some sort of PCI or PCIe link. Each
FPGA contains a header section describing the content of the FPGA. The
header lists the device id, PCI BAR, offset from the beginning of the PCI
BAR, size in the FPGA, interrupt number and some other properties currently
not handled by the MCB implementation.

Carrier abstraction과 parser

Carrier device는 Chameleon FPGA가 연결된 실제 physical bus의 abstraction입니다. IP core driver가 PCI IRQ 같은 carrier 속성을 조회해야 할 수 있습니다.

MCB carrier는 callback으로 MCB function call을 실제 hardware bus query로 변환합니다. 예를 들어 `get_irq()`는 장치가 사용할 IRQ 번호를 조회합니다.

Parser는 Chameleon device의 첫 512 byte에서 table을 읽습니다. 현재 Chameleon v2만 지원하지만 이전 또는 미래 variant를 지원하도록 확장하기 쉽습니다.

Parser는 table entry마다 새 MCB device를 할당하고 table의 resource assignment대로 resource를 지정합니다. 완료 후 MCB bus와 Linux driver core에 device를 등록합니다.

Carrier Devices
---------------

A carrier device is just an abstraction for the real world physical bus the
Chameleon FPGA is attached to. Some IP Core drivers may need to interact with
properties of the carrier device (like querying the IRQ number of a PCI
device). To provide abstraction from the real hardware bus, an MCB carrier
device provides callback methods to translate the driver's MCB function calls
to hardware related function calls. For example a carrier device may
implement the get_irq() method which can be translated into a hardware bus
query for the IRQ number the device should use.

Parser
------

The parser reads the first 512 bytes of a Chameleon device and parses the
Chameleon table. Currently the parser only supports the Chameleon v2 variant
of the Chameleon table but can easily be adopted to support an older or
possible future variant. While parsing the table's entries new MCB devices
are allocated and their resources are assigned according to the resource
assignment in the Chameleon table. After resource assignment is finished, the
MCB devices are registered at the MCB and thus at the driver core of the
Linux kernel.

Memory와 IRQ resource

현재 각 MCB device에는 정확히 하나의 memory resource와 하나의 IRQ resource를 할당하지만 미래에는 바뀔 수 있습니다.

Memory resource는 carrier 안에서 MCB device의 physical address이며 `ioremap()` 계열 함수에 전달합니다. Kernel에는 이미 `request_mem_region()`으로 요청된 상태입니다.

IRQ resource는 MCB bus에서 요청합니다. Carrier driver가 `get_irq()` callback을 구현하면 carrier가 지정한 번호를 반환하고, 아니면 Chameleon table의 IRQ 번호를 반환합니다. 이 값은 `request_irq()`에 사용할 수 있습니다.

Resource handling
=================

The current implementation assigns exactly one memory and one IRQ resource
per MCB device. But this is likely going to change in the future.

Memory Resources
----------------

Each MCB device has exactly one memory resource, which can be requested from
the MCB bus. This memory resource is the physical address of the MCB device
inside the carrier and is intended to be passed to ioremap() and friends. It
is already requested from the kernel by calling request_mem_region().

IRQs
----

Each MCB device has exactly one IRQ resource, which can be requested from the
MCB bus. If a carrier device driver implements the ->get_irq() callback
method, the IRQ number assigned by the carrier device will be returned,
otherwise the IRQ number inside the Chameleon table will be returned. This
number is suitable to be passed to request_irq().

MCB driver 구조

각 MCB driver는 device driver 정보와 FPGA 내부 IP core를 식별하는 device ID table을 가집니다.

`mcb_device_id` table은 지원 device ID와 빈 sentinel entry를 포함하며 `MODULE_DEVICE_TABLE(mcb, ...)`로 공개합니다.

`mcb_driver`에는 일반 driver 이름·owner, `probe`, `remove`, ID table을 지정합니다.

Writing an MCB driver
=====================

The driver structure
--------------------

Each MCB driver has a structure to identify the device driver as well as
device ids which identify the IP Core inside the FPGA. The driver structure
also contains callback methods which get executed on driver probe and
removal from the system::

        static const struct mcb_device_id foo_ids[] = {
                { .device = 0x123 },
                { }
        };
        MODULE_DEVICE_TABLE(mcb, foo_ids);

        static struct mcb_driver foo_driver = {
        driver = {
                .name = "foo-bar",
                .owner = THIS_MODULE,
        },
                .probe = foo_probe,
                .remove = foo_remove,
                .id_table = foo_ids,
        };

Probe와 module 초기화

Driver가 load되고 지원 MCB device가 발견되면 MCB core가 `probe` callback을 호출합니다. Driver를 제거할 때는 `remove` callback을 호출합니다.

Boot 또는 module insert 시 보통 `mcb_register_driver(&foo_driver)`로 MCB core에 driver를 등록하면 충분합니다.

Module exit에서는 `mcb_unregister_driver()`로 등록을 해제합니다.

이 init·exit boilerplate는 `module_mcb_driver(foo_driver)` macro로 줄일 수 있습니다.

Probing and attaching
---------------------

When a driver is loaded and the MCB devices it services are found, the MCB
core will call the driver's probe callback method. When the driver is removed
from the system, the MCB core will call the driver's remove callback method::

        static init foo_probe(struct mcb_device *mdev, const struct mcb_device_id *id);
        static void foo_remove(struct mcb_device *mdev);

Initializing the driver
-----------------------

When the kernel is booted or your foo driver module is inserted, you have to
perform driver initialization. Usually it is enough to register your driver
module at the MCB core::

        static int __init foo_init(void)
        {
                return mcb_register_driver(&foo_driver);
        }
        module_init(foo_init);

        static void __exit foo_exit(void)
        {
                mcb_unregister_driver(&foo_driver);
        }
        module_exit(foo_exit);

The module_mcb_driver() macro can be used to reduce the above code::

        module_mcb_driver(foo_driver);

Carrier device를 이용한 DMA

Kernel DMA API를 사용하려면 carrier device의 `struct device`가 필요합니다.

`mcb_device`는 DMA용 carrier device pointer를 `dma_dev`에 포함하므로 이를 `dma_set_mask_and_coherent()` 같은 API에 전달합니다.

예제는 요구 DMA bit mask를 설정하고 실패 시 오류를 처리합니다.

Using DMA
---------

To make use of the kernel's DMA-API's function, you will need to use the
carrier device's 'struct device'. Fortunately 'struct mcb_device' embeds a
pointer (->dma_dev) to the carrier's device for DMA purposes::

        ret = dma_set_mask_and_coherent(&mdev->dma_dev, DMA_BIT_MASK(dma_bits));
        if (rc)
                /* Handle errors */