리눅스 커널 NVMe 드라이버 코드 분석

1. Function Level Analysis of
Linux NVMe Driver
Comparative analysis on legacy and MQ-based NVMe driver
Ingu Kang
Embedded Systems Lab.
Kookmin University
2016-05-09

2. General
Architecture of
Linux I/O Stack
2

3. Linux NVMe
Function Call Flow
● The NVMe driver of Linux ~3.18
bypasses the block layer
routines, and directly takes bio
structure instances.
● On Linux 3.19+, bios go through
blk-mq and are converted into
“request” structure.
● blk-mq queues requests into the
NVMe driver by calling
nvme_queue_rq().
● On Linux 4.4+, some refactoring
has been done, including
optimization and bug-fixes.
3

4. 1. nvme_make_request() takes a
bio from the upper layer,
converting it into iod (I/O
descriptor). iod is allocated
independently for each bio.
2. nvme_submit_iod() takes an
iod and gets a struct
nvme_cmd_info as well as its
index on the space pre-allocated
in current CPU's nvmeq(struct)
for it.
3. nvme_submit_iod() builds up
a command in a submission
queue (SQ) based on the iod. It
also sets up the nvme_cmd_info
(pre-allocated) that saves the
reference to an iod and a
callback. Those will be used in
completion queue entry
processing. Finally, it rings the
doorbell of SQ.
Note: we save pointers to iod and
callback into cmd_info, as
SQ/CQ entries can contain
command_id but not iod and
Linux ~3.18
4

5. 1. blk_mq_make_request() takes a
bio and builds up a request
instance from it. The instance is
picked from the instance pool of a
hctx, which was pre-allocated on
device initialization. request->tag
keeps the index of request instance
itself in pool array. 'tag' will be used
as command_id in
nvme_submit_iod() later on. The
request is then added to
plug->mq_list.
2. blk_mq_flush_plug_list() and
blk_mq_insert_requests() flush
requests from the plug->mq_list to
the ctx->rq_list (note: ctx->rq_list
is a per-CPU SW queue).
3. flush_busy_ctxs() and
__blk_mq_insert_request() flush
requests from the ctx->rq_list to
the locally defined rq_list that acts
as 1-to-1 HW dispatch queue, and
process them with
nvme_queue_rq().
Linux 3.19+ (1)
5

6. 4. nvme_queue_rq() allocates an
iod and converts a request into the
allocated iod.
5. nvme_submit_iod() converts iod
into command. request->tag is
reused as command_id. Callback
and iod information is set into
nvme_cmd_info for later use in the
completion routine. It then rings
the SQ doorbell.
Note 1: The address of
pre-allocated nvme_cmd_info is
obtained by calling
blk_mq_rq_to_pdu(), which
calculates address to PDU data
area by adding sizeof(struct
request) to address of request
instance.
Note 2: nvme_cmd_info and
nvme_iod are merged in the most
recent version of NVMe driver. All
the iods are pre-allocated on device
initialization, but the DMA memory
segments information array
iod->sg is NOT.
Linux 3.19+ (2)
6

7. Thanks!
7