Direct Access for files

DAX 동기와 ext2·erofs 사용

일반 file read/write와 `mmap` userspace mapping에는 page cache가 buffer page를 제공합니다. 그러나 memory-like block device에서는 page cache page가 원본 storage의 불필요한 copy가 됩니다.

DAX(Direct Access)는 storage device를 직접 read/write해 추가 copy를 없애고 file mapping에서는 device를 userspace에 직접 map합니다.

DAX block device에 filesystem을 평소처럼 만들 수 있지만 현재 DAX file의 block size는 kernel `PAGE_SIZE`와 같아야 하므로 filesystem 생성 시 block size를 지정해야 할 수 있습니다.

지원 filesystem은 ext2, ext4, xfs, virtiofs, erofs 다섯 가지입니다. ext2와 erofs는 mount에 `-o dax` 또는 `/etc/fstab`의 `dax` option을 주면 전체 file에 DAX가 활성화되며 아래의 `-o dax=always`와 같습니다.

=======================
Direct Access for files
=======================

Motivation
----------

The page cache is usually used to buffer reads and writes to files.
It is also used to provide the pages which are mapped into userspace
by a call to mmap.

For block devices that are memory-like, the page cache pages would be
unnecessary copies of the original storage.  The `DAX` code removes the
extra copy by performing reads and writes directly to the storage device.
For file mappings, the storage device is mapped directly into userspace.


Usage
-----

If you have a block device which supports `DAX`, you can make a filesystem
on it as usual.  The `DAX` code currently only supports files with a block
size equal to your kernel's `PAGE_SIZE`, so you may need to specify a block
size when creating the filesystem.

Currently 5 filesystems support `DAX`: ext2, ext4, xfs, virtiofs and erofs.
Enabling `DAX` on them is different.

Enabling DAX on ext2 and erofs
------------------------------

When mounting the filesystem, use the ``-o dax`` option on the command line or
add 'dax' to the options in ``/etc/fstab``.  This works to enable `DAX` on all files
within the filesystem.  It is equivalent to the ``-o dax=always`` behavior below.

xfs·ext4의 persistent·active DAX 정책

kernel의 active file access mode flag `S_DAX`는 `statx`의 `STATX_ATTR_DAX`에 대응합니다. persistent advisory flag `FS_XFLAG_DAX`는 regular file과 directory에 설정·해제할 수 있지만 변경이 즉시 `S_DAX`에 반영되지는 않습니다.

directory의 `FS_XFLAG_DAX`는 이후 생성되는 regular file과 subdirectory가 상속합니다. flag 변경 당시 이미 존재하는 child에는 소급 적용되지 않습니다.

storage가 DAX를 지원할 때 `-o dax=inode`는 `FS_XFLAG_DAX`를 따르며 기본값입니다. `dax=never`는 persistent flag를 무시하고 `S_DAX`를 절대 설정하지 않으며, `dax=always`는 persistent flag를 무시하고 항상 설정합니다. legacy `-o dax`는 `dax=always` alias이고 향후 제거될 수 있어 명시적 form이 권장됩니다.

mount override 중에도 `FS_XFLAG_DAX` 수정과 상속은 그대로 일어나지만 in-core `S_DAX`는 override됩니다. 다시 `dax=inode`로 remount하고 inode가 kernel memory에서 evict되어야 persistent policy가 active state에 반영됩니다.

정책은 file 생성 전 parent flag 설정, 적절한 mount option, existing file·directory의 persistent flag 변경으로 제어합니다. 기존 regular file의 `FS_XFLAG_DAX` toggle은 모든 process가 file을 close한 뒤에야 `S_DAX` 변경으로 반영됩니다.

Enabling DAX on xfs and ext4
----------------------------

Summary
-------

 1. There exists an in-kernel file access mode flag `S_DAX` that corresponds to
    the statx flag `STATX_ATTR_DAX`.  See the manpage for statx(2) for details
    about this access mode.

 2. There exists a persistent flag `FS_XFLAG_DAX` that can be applied to regular
    files and directories. This advisory flag can be set or cleared at any
    time, but doing so does not immediately affect the `S_DAX` state.

 3. If the persistent `FS_XFLAG_DAX` flag is set on a directory, this flag will
    be inherited by all regular files and subdirectories that are subsequently
    created in this directory. Files and subdirectories that exist at the time
    this flag is set or cleared on the parent directory are not modified by
    this modification of the parent directory.

 4. There exist dax mount options which can override `FS_XFLAG_DAX` in the
    setting of the `S_DAX` flag.  Given underlying storage which supports `DAX` the
    following hold:

    ``-o dax=inode``  means "follow `FS_XFLAG_DAX`" and is the default.

    ``-o dax=never``  means "never set `S_DAX`, ignore `FS_XFLAG_DAX`."

    ``-o dax=always`` means "always set `S_DAX` ignore `FS_XFLAG_DAX`."

    ``-o dax``      is a legacy option which is an alias for ``dax=always``.

    .. warning::

      The option ``-o dax`` may be removed in the future so ``-o dax=always`` is
      the preferred method for specifying this behavior.

    .. note::

      Modifications to and the inheritance behavior of `FS_XFLAG_DAX` remain
      the same even when the filesystem is mounted with a dax option.  However,
      in-core inode state (`S_DAX`) will be overridden until the filesystem is
      remounted with dax=inode and the inode is evicted from kernel memory.

 5. The `S_DAX` policy can be changed via:

    a) Setting the parent directory `FS_XFLAG_DAX` as needed before files are
       created

    b) Setting the appropriate dax="foo" mount option

    c) Changing the `FS_XFLAG_DAX` flag on existing regular files and
       directories.  This has runtime constraints and limitations that are
       described in 6) below.

 6. When changing the `S_DAX` policy via toggling the persistent `FS_XFLAG_DAX`
    flag, the change to existing regular files won't take effect until the
    files are closed by all processes.

FS_XFLAG_DAX 상속 예제와 S_DAX 결정

file마다 persistent configuration인 `FS_XFLAG_DAX`와 현재 feature 활성 상태를 나타내는 volatile `S_DAX` 두 flag가 있습니다. persistent flag는 filesystem에 보존되고 `FS_IOC_FSGETXATTR`·`FS_IOC_FSSETXATTR` ioctl 또는 `xfs_io`로 조회·변경합니다.

새 file과 directory는 생성 순간 parent directory의 `FS_XFLAG_DAX`를 상속하므로 subtree의 default behavior를 정하려면 child 생성 전에 parent flag를 설정합니다.

Example A는 `a/b/c`를 먼저 만든 뒤 `a`에 `chattr +x`를 하고 `a/b/c/d`, `a/e`를 만듭니다. 기존 `b`, `c`는 바뀌지 않아 그 아래 새 `d`도 no-DAX이고, flag 설정 뒤 `a` 바로 아래 만든 `e`만 상속합니다. 결과는 DAX `a,e`, no-DAX `b,c,d`입니다.

Example B는 `a`를 만들고 먼저 flag를 설정한 뒤 `a/b/c/d`를 생성하므로 모든 child가 연쇄 상속합니다. 결과는 DAX `a,b,c,d`이고 no-DAX는 없습니다.

Example C는 `a/b/c`를 만든 뒤 `c`에만 flag를 설정하고 `c/d`를 만듭니다. 결과는 DAX `c,d`, no-DAX `a,b`입니다.

`S_DAX`는 kernel이 file inode를 memory에 instantiate할 때 underlying media support, `FS_XFLAG_DAX`, filesystem mount option을 조합해 설정합니다. `statx`로 조회할 수 있으며 regular file에만 설정되므로 directory의 `statx`는 `S_DAX`를 보고하지 않습니다.

underlying media가 DAX를 지원하지 않거나 mount option이 override해도 persistent `FS_XFLAG_DAX` 자체의 설정·상속은 일어납니다.

Details
-------

There are 2 per-file dax flags.  One is a persistent inode setting (`FS_XFLAG_DAX`)
and the other is a volatile flag indicating the active state of the feature
(`S_DAX`).

`FS_XFLAG_DAX` is preserved within the filesystem.  This persistent config
setting can be set, cleared and/or queried using the `FS_IOC_FS`[`GS`]`ETXATTR` ioctl
(see ioctl_xfs_fsgetxattr(2)) or an utility such as 'xfs_io'.

New files and directories automatically inherit `FS_XFLAG_DAX` from
their parent directory **when created**.  Therefore, setting `FS_XFLAG_DAX` at
directory creation time can be used to set a default behavior for an entire
sub-tree.

To clarify inheritance, here are 3 examples:

Example A:

.. code-block:: shell

  mkdir -p a/b/c
  xfs_io -c 'chattr +x' a
  mkdir a/b/c/d
  mkdir a/e

  ------[outcome]------

  dax: a,e
  no dax: b,c,d

Example B:

.. code-block:: shell

  mkdir a
  xfs_io -c 'chattr +x' a
  mkdir -p a/b/c/d

  ------[outcome]------

  dax: a,b,c,d
  no dax:

Example C:

.. code-block:: shell

  mkdir -p a/b/c
  xfs_io -c 'chattr +x' c
  mkdir a/b/c/d

  ------[outcome]------

  dax: c,d
  no dax: a,b

The current enabled state (`S_DAX`) is set when a file inode is instantiated in
memory by the kernel.  It is set based on the underlying media support, the
value of `FS_XFLAG_DAX` and the filesystem's dax mount option.

statx can be used to query `S_DAX`.

.. note::

  That only regular files will ever have `S_DAX` set and therefore statx
  will never indicate that `S_DAX` is set on directories.

Setting the `FS_XFLAG_DAX` flag (specifically or through inheritance) occurs even
if the underlying media does not support dax and/or the filesystem is
overridden with a mount option.

virtiofs의 server 결정 DAX hint

virtiofs의 DAX 의미는 ext4·xfs와 대체로 같지만 `-o dax=inode`일 때 client가 persistent `FS_XFLAG_DAX`를 직접 따르지 않고 FUSE protocol을 통해 server에서 DAX enable hint를 받습니다.

최종 결정은 virtiofs server가 전적으로 내리며 server는 host의 persistent flag 등 다양한 algorithm을 사용할 수 있습니다.

guest 안에서 `FS_XFLAG_DAX`를 set·clear하는 것은 지원하지만 해당 file의 실제 DAX 상태가 그대로 바뀐다고 보장되지 않습니다. guest userspace는 `statx(2)`의 `STATX_ATTR_DAX`를 확인해야 합니다.

Enabling DAX on virtiofs
----------------------------
The semantic of DAX on virtiofs is basically equal to that on ext4 and xfs,
except that when '-o dax=inode' is specified, virtiofs client derives the hint
whether DAX shall be enabled or not from virtiofs server through FUSE protocol,
rather than the persistent `FS_XFLAG_DAX` flag. That is, whether DAX shall be
enabled or not is completely determined by virtiofs server, while virtiofs
server itself may deploy various algorithm making this decision, e.g. depending
on the persistent `FS_XFLAG_DAX` flag on the host.

It is still supported to set or clear persistent `FS_XFLAG_DAX` flag inside
guest, but it is not guaranteed that DAX will be enabled or disabled for
corresponding file then. Users inside guest still need to call statx(2) and
check the statx flag `STATX_ATTR_DAX` to see if DAX is enabled for this file.

Block driver 구현 조건

block driver가 DAX를 지원하려면 `direct_access` block device operation을 구현합니다. 512-byte sector 단위 sector number를 physical memory page의 page frame number(pfn)로 변환하고 접근용 kernel virtual address도 반환합니다.

`size` parameter는 요청 byte 수입니다. method는 해당 offset에서 연속 접근 가능한 byte 수를 반환하고 오류 시 negative errno를 반환할 수 있습니다.

storage는 CPU가 항상 byte-addressable해야 합니다. 작은 window로 큰 memory를 paging해 노출하는 device나 CPU를 장시간 stall할 수 있는 device는 `direct_access`를 구현해서는 안 됩니다.

참고 구현은 NVDIMM persistent memory driver인 `pmem`입니다.

Implementation Tips for Block Driver Writers
--------------------------------------------

To support `DAX` in your block driver, implement the 'direct_access'
block device operation.  It is used to translate the sector number
(expressed in units of 512-byte sectors) to a page frame number (pfn)
that identifies the physical page for the memory.  It also returns a
kernel virtual address that can be used to access the memory.

The direct_access method takes a 'size' parameter that indicates the
number of bytes being requested.  The function should return the number
of bytes that can be contiguously accessed at that offset.  It may also
return a negative errno if an error occurs.

In order to support this method, the storage must be byte-accessible by
the CPU at all times.  If your device uses paging techniques to expose
a large amount of memory through a smaller window, then you cannot
implement direct_access.  Equally, if your device can occasionally
stall the CPU for an extended period, you should also not attempt to
implement direct_access.

These block devices may be used for inspiration:
- pmem: NVDIMM persistent memory driver

Filesystem 구현 요구사항

filesystem은 DAX inode의 `i_flags`에 `S_DAX`를 설정할 수 있어야 합니다. `->read_iter`와 `->write_iter`는 flag가 설정된 inode에서 `dax_iomap_rw()`를 사용합니다.

DAX file의 mmap operation은 VMA에 `VM_MIXEDMAP`과 `VM_HUGEPAGE`를 설정하고 `fault`, `pmd_fault`, `page_mkwrite`, `pfn_mkwrite` handler를 포함하는 `vm_ops`를 설치해야 합니다. handler는 알맞은 fault size와 iomap operations를 `dax_iomap_fault()`에 전달하는 방식이 권장됩니다.

DAX file truncation에는 `block_truncate_page()` 대신 적절한 iomap operation과 `iomap_zero_range()`를 사용하고 read, write, truncate, page fault 사이에 충분한 locking을 보장해야 합니다.

block allocation iomap handler는 mmap을 통해 uninitialized data가 노출되지 않도록 allocated block을 zero하고 written extent로 변환한 뒤 반환해야 합니다.

참고 filesystem은 `Documentation/filesystems/ext2.rst`, `Documentation/admin-guide/xfs.rst`, `Documentation/filesystems/ext4/`입니다.

Implementation Tips for Filesystem Writers
------------------------------------------

Filesystem support consists of:

* Adding support to mark inodes as being `DAX` by setting the `S_DAX` flag in
  i_flags
* Implementing ->read_iter and ->write_iter operations which use
  :c:func:`dax_iomap_rw()` when inode has `S_DAX` flag set
* Implementing an mmap file operation for `DAX` files which sets the
  `VM_MIXEDMAP` and `VM_HUGEPAGE` flags on the `VMA`, and setting the vm_ops to
  include handlers for fault, pmd_fault, page_mkwrite, pfn_mkwrite. These
  handlers should probably call :c:func:`dax_iomap_fault()` passing the
  appropriate fault size and iomap operations.
* Calling :c:func:`iomap_zero_range()` passing appropriate iomap operations
  instead of :c:func:`block_truncate_page()` for `DAX` files
* Ensuring that there is sufficient locking between reads, writes,
  truncates and page faults

The iomap handlers for allocating blocks must make sure that allocated blocks
are zeroed out and converted to written extents before being returned to avoid
exposure of uninitialized data through mmap.

These filesystems may be used for inspiration:

.. seealso::

  ext2: see Documentation/filesystems/ext2.rst

.. seealso::

  xfs:  see Documentation/admin-guide/xfs.rst

.. seealso::

  ext4: see Documentation/filesystems/ext4/

Media error 처리

libnvdimm은 pmem block device별 known media error location을 `gendisk->badblocks`에 기록합니다. 알려진 위치나 아직 발견되지 않은 latent error에서 fault가 나면 application은 `SIGBUS`를 받습니다.

underlying NVDIMM이 ACPI의 `clear_poison` DSM을 지원하면 pmem driver를 통해 영향을 받은 sector에 write하여 error를 clear할 수 있습니다.

DAX I/O는 보통 `driver/bio` path를 거치지 않으므로 backup이나 내장 redundancy에서 data를 복원하는 기본 경로가 두 가지입니다. 첫째 affected file을 삭제하고 backup에서 복원하면 filesystem block이 free되고 다음 allocation에서 driver를 통해 zero되어 bad sector가 clear됩니다.

둘째 bad block을 포함한 file 부분을 truncate하거나 hole-punch합니다. 최소한 정렬된 sector 전체를 punch해야 하지만 filesystem block 전체일 필요는 없습니다.

향후 DM block layer 또는 filesystem 수준 redundancy·mirroring으로 더 강한 recovery를 만들 수 있지만, driver를 통과하는 I/O 또는 driver를 통한 zeroing으로 error를 clear한다는 두 원칙에 의존해야 합니다.

Handling Media Errors
---------------------

The libnvdimm subsystem stores a record of known media error locations for
each pmem block device (in gendisk->badblocks). If we fault at such location,
or one with a latent error not yet discovered, the application can expect
to receive a `SIGBUS`. Libnvdimm also allows clearing of these errors by simply
writing the affected sectors (through the pmem driver, and if the underlying
NVDIMM supports the clear_poison DSM defined by ACPI).

Since `DAX` IO normally doesn't go through the ``driver/bio`` path, applications or
sysadmins have an option to restore the lost data from a prior ``backup/inbuilt``
redundancy in the following ways:

1. Delete the affected file, and restore from a backup (sysadmin route):
   This will free the filesystem blocks that were being used by the file,
   and the next time they're allocated, they will be zeroed first, which
   happens through the driver, and will clear bad sectors.

2. Truncate or hole-punch the part of the file that has a bad-block (at least
   an entire aligned sector has to be hole-punched, but not necessarily an
   entire filesystem block).

These are the two basic paths that allow `DAX` filesystems to continue operating
in the presence of media errors. More robust error recovery mechanisms can be
built on top of this in the future, for example, involving redundancy/mirroring
provided at the block layer through DM, or additionally, at the filesystem
level. These would have to rely on the above two tenets, that error clearing
can happen either by sending an IO through the driver, or zeroing (also through
the driver).

DAX 제약과 struct page 부재

kernel이나 module이 DAX-capable filesystem·device에 저장되어도 여전히 RAM으로 copy됩니다.

DAX code는 ARM, MIPS, SPARC처럼 virtually mapped cache를 사용하는 architecture에서 올바르게 동작하지 않습니다.

DAX mmap range에 대응하는 `struct page`가 없으면 그 userspace memory에 `get_user_pages()`가 실패합니다. 일부 device driver는 optional struct page support를 추가해 해결하며 `drivers/nvdimm`의 `CONFIG_NVDIMM_PFN`이 예입니다.

struct page가 없는 DAX-mapped memory를 non-DAX file의 `O_DIRECT` read/write target으로 쓰면 실패합니다. 반대로 DAX file 자체의 `O_DIRECT` I/O는 동작하며 핵심은 접근 대상 memory의 page representation입니다.

non-struct-page 경우에는 RDMA, `sendfile()`, `splice()`도 동작하지 않습니다.

Shortcomings
------------

Even if the kernel or its modules are stored on a filesystem that supports
`DAX` on a block device that supports `DAX`, they will still be copied into RAM.

The DAX code does not work correctly on architectures which have virtually
mapped caches such as ARM, MIPS and SPARC.

Calling :c:func:`get_user_pages()` on a range of user memory that has been
mmapped from a `DAX` file will fail when there are no 'struct page' to describe
those pages.  This problem has been addressed in some device drivers
by adding optional struct page support for pages under the control of
the driver (see `CONFIG_NVDIMM_PFN` in ``drivers/nvdimm`` for an example of
how to do this). In the non struct page cases `O_DIRECT` reads/writes to
those memory ranges from a non-`DAX` file will fail 


.. note::

  `O_DIRECT` reads/writes _of a `DAX` file do work, it is the memory that
  is being accessed that is key here).  Other things that will not work in
  the non struct page case include RDMA, :c:func:`sendfile()` and
  :c:func:`splice()`.