Commit bf2c777a authored by Pankaj Darak's avatar Pankaj Darak

QDMA DPDK 2019.2 reference driver release

parent e316ccf3
Release: 2019.1 RELEASE: 2019.2
=============== ===============
This release is based on DPDK v18.11 containing QDMA poll mode driver and This release is based on DPDK v18.11 and contains QDMA poll mode driver and
QDMA test application. This release is validated on QDMA 2019.1 design. QDMA test application. This release is validated on QDMA 2019.2 patch based
example design and QDMA 2019.1 based example design.
This release includes a patch file for dpdk-pktgen v3.6.1 that extends This release includes a patch file for dpdk-pktgen v3.6.1 that extends
dpdk-pktgen application to handle packets with packet sizes more than 1518 bytes dpdk-pktgen application to handle packets with packet sizes more than 1518 bytes
...@@ -48,13 +49,22 @@ SUPPORTED FEATURES: ...@@ -48,13 +49,22 @@ SUPPORTED FEATURES:
- Driver restructuring to partition QDMA access code such that it can be used across different drivers - Driver restructuring to partition QDMA access code such that it can be used across different drivers
- Device configuration through additional driver APIs - Device configuration through additional driver APIs
KNOWN BUGS: 2019.2 Features
=========== ---------------
- Support reset of a VF device (via rte_eth_dev_reset() API)
- Support PF device removal when its VF device is active
- Split user logic related data path handling code to qdma_user.c and qdma_user.h
- Bug Fixes
- Correct the PF and VF function Ids used in the driver in accordance with HW
- Fix dma_from_device command in qdma_testapp application that requests more data from user logic than specified
- Fix memory offset calculation in dma_from_device and dma_to_device command in qdma_testapp application for MM mode
KNOWN ISSUES:
=============
- Sometimes completions are not received when C2H PIDX updates are held for 64 descriptors - Sometimes completions are not received when C2H PIDX updates are held for 64 descriptors
- dma_from_device command in qdma_testapp application requests more data from user logic than specified - Function Level Reset(FLR) of PF device when VFs are attached to this PF results in mailbox communication failure
LIMITATIONS: DRIVER LIMITATIONS:
=========== ===================
- Big endian systems are not supported - Big endian systems are not supported
- All VFs created by a PF should be shut down before terminating the parent PF. This is needed as all the QDMA VF configuration is through this parent PF.
- For optimal QDMA streaming performance, packet buffers of the descriptor ring should be aligned to at least 256 bytes. - For optimal QDMA streaming performance, packet buffers of the descriptor ring should be aligned to at least 256 bytes.
\ No newline at end of file
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SOURCEDIR = source
BUILDDIR = build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
down:
wget -O kernel-doc "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/plain/scripts/kernel-doc?h=v4.14.52" && chmod +x kernel-doc
genrst:
mkdir -p $(BUILDDIR)
./kernel-doc -rst source/pmd_callbacks.h > build/pmd_callbacks.rst
./kernel-doc -rst source/rte_pmd_qdma.h > build/rte_pmd_qdma.rst
html: down genrst
$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
clean:
rm -rf $(BUILDDIR)
rm -rf kernel-doc
###############################################################################
Xilinx QDMA DPDK Driver Documentation Generation
###############################################################################
1. Installation:
Xilinx QDMA DPDK driver documenation is designed based on Sphinx.
In oprder to generate the documentation, make sure to install the
Sphinx software. Details of required packages are available at
http://www.sphinx-doc.org/en/master/usage/installation.html
After installing the required packages, follow the steps below to generate the documentation.
Go to dpdk/docs/git_doc and run 'make html'
[xilinx@]# make html
'build' directory is created. Open 'build/html/index.html' for QDMA DPDK driver documentation
To remove the generated documentation, run 'make clean'
[xilinx@]# make clean
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: f5075114aa9834768550066f6676a617
tags: 645f666f9bcd5a90fca523b33c5a78b7
.. _build_dpdk:
Building QDMA DPDK Software
===========================
DPDK requires certain packages to be installed on host system.
For a full list, refer to the official DPDK documentation:
https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html.
**Note**: If the NUMA library is missing, it should be installed.
For example:
For Ubuntu: ``sudo apt-get install libnuma-dev``
For Red Hat: ``sudo yum install numactl-devel``
Modifying the driver for PCIe device ID
---------------------------------------
During the PCIe DMA IP customization in Vivado, user can specify a PCIe Device ID.
This Device ID must be added to the driver to identify the PCIe QDMA device.
The current driver is designed to recognize the PCIe Device IDs
that get generated with the PCIe example design when this value has not been modified.
If the PCIe Device ID is modified during IP customization,
one needs to modify QDMA PMD to recognize this new ID.
User can also remove PCIe Device IDs that will not be used by the end solution.
To modify the PCIe Device ID in the driver,
For PF devices,
update ``struct rte_pci_id qdma_pci_id_tbl[]`` inside ``drivers/net/qdma/qdma_ethdev.c``
For VF devices,
update ``struct rte_pci_id qdma_vf_pci_id_tbl[]`` inside ``drivers/net/qdma/qdma_vf_ethdev.c``
Also add the device IDs in ``usertools/dpdk-devbind.py`` in ``xilinx_qdma_pf`` for PF device
and ``xilinx_qdma_vf`` for VF device as specified in later section.
Once modified, the driver and application must be recompiled.
Xilinx QDMA DPDK Software database structure
--------------------------------------------
Below Table describes the DPDK software database structure and its contents
on the Xilinx GitHub https://github.com/Xilinx/dma_ip_drivers, subdirectory QDMA/DPDK.
+--------------------------+-------------------------------------------------------------+
| Directory | Description |
+==========================+=============================================================+
| drivers/net/qdma | Xilinx QDMA DPDK poll mode driver |
+--------------------------+-------------------------------------------------------------+
| examples/qdma_testapp | Xilinx CLI based test application for QDMA |
+--------------------------+-------------------------------------------------------------+
| tools/0001-PKTGEN-3.6.1- | This is dpdk-pktgen patch based on dpdk-pktgen v3.6.1. |
| Patch-to-add-Jumbo-packet| This patch extends dpdk-pktgen application to handle packets|
| -support.patch | with packet sizes more than 1518 bytes and it disables the |
| | packet size classification logic to remove application |
| | overhead in performance measurement. This patch is used for |
| | performance testing with dpdk-pktgen application. |
+--------------------------+-------------------------------------------------------------+
| RELEASE.txt | Release Notes |
+--------------------------+-------------------------------------------------------------+
Setup: Download and modifications
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The reference driver code requires DPDK version 18.11.
Follow the steps below to download the proper version of DPDK and apply
driver code and test application supplied in the GitHub.
Extract the DPDK driver software database from the Xilinx GitHub to the server where VCU1525
is installed. Henceforth, this area is referred as <dpdk_sw_database>.
Create a directory for the DPDK download on the server where the VCU1525
is installed and move to this directory.
::
mkdir <server_dir>/<dpdk_test_area>
cd <server_dir>/<dpdk_test_area>
git clone http://dpdk.org/git/dpdk-stable
cd dpdk-stable
git checkout v18.11
cp -r <dpdk_sw_database>/drivers/net/qdma ./drivers/net/
cp -r <dpdk_sw_database>/examples/qdma_testapp ./examples/
Additionally, make below changes to the DPDK 18.11 tree to build QDMA driver,
support 2K queues and populate Xilinx devices for binding.
1. To build QDMA driver
a. Add below lines to ``./config/common_base`` in DPDK 18.11 tree
::
#
#Complie Xilinx QDMA PMD driver
#
CONFIG_RTE_LIBRTE_QDMA_PMD=y
CONFIG_RTE_LIBRTE_QDMA_DEBUG_DRIVER=n
To enable driver debug logs, set
CONFIG_RTE_LIBRTE_QDMA_DEBUG_DRIVER=y
b. Add below lines to ``drivers/net/Makefile``, where PMDs are added
::
DIRS-$(CONFIG_RTE_LIBRTE_QDMA_PMD) += qdma
c. Add below lines to ``mk/rte.app.mk``, where PMDs are added
::
_LDLIBS-$(CONFIG_RTE_LIBRTE_QDMA_PMD) += -lrte_pmd_qdma
2. To add Xilinx devices for device binding, add below lines to ``./usertools/dpdk-devbind.py`` after cavium_pkx class, where PCI base class for devices are listed.
::
xilinx_qdma_pf = {'Class': '05', 'Vendor': '10ee', 'Device': '9011,9111,9211,9311,9014,9114,9214,9314,9018,9118,9218,9318,901f,911f,921f,931f,9021,9121,9221,9321,9024,9124,9224,9324,9028,9128,9228,9328,902f,912f,922f,932f,9031,9131,9231,9331,9034,9134,9234,9334,9038,9138,9238,9338,903f,913f,923f,933f,9041,9141,9241,9341,9044,9144,9244,9344,9048,9148,9248,9348',
'SVendor': None, 'SDevice': None}
xilinx_qdma_vf = {'Class': '05', 'Vendor': '10ee', 'Device': 'a011,a111,a211,a311,a014,a114,a214,a314,a018,a118,a218,a318,a01f,a11f,a21f,a31f,a021,a121,a221,a321,a024,a124,a224,a324,a028,a128,a228,a328,a02f,a12f,a22f,a32f,a031,a131,a231,a331,a034,a134,a234,a334,a038,a138,a238,a338,a03f,a13f,a23f,a33f,a041,a141,a241,a341,a044,a144,a244,a344,a048,a148,a248,a348',
'SVendor': None, 'SDevice': None}
Update entries in network devices class in ``./usertools/dpdk-devbind.py`` to add Xilinx devices
::
network_devices = [network_class, cavium_pkx, xilinx_qdma_pf, xilinx_qdma_vf]
3. To support 2K queues and 256 PCIe functions, update below configurations in ``./config/common_base``
::
CONFIG_RTE_MAX_MEMZONE=7680
CONFIG_RTE_MAX_ETHPORTS=256
CONFIG_RTE_MAX_QUEUES_PER_PORT=2048
Setup: Host system
^^^^^^^^^^^^^^^^^^
DPDK requires that hugepages are setup on the server.
The following modifications must be made to the ``/boot/grub/grub.cfg`` on the host system
- Add hugepages for DPDK
Add following parameter to ``/etc/default/grub file``
::
GRUB_CMDLINE_LINUX="default_hugepagesz=1GB hugepagesz=1G hugepages=20"
| This example adds 20 1GB hugepages, which are required to support 2048 queues, with descriptor ring of 1024 entries and each descriptor buffer length of 4KB.
| The number of hugepages required should be changed if the above configuration (queues, ring size, buffer size) changes.
- Enable IOMMU for VM testing
Update ``/etc/default/grub`` file as below.
::
GRUB_CMDLINE_LINUX="default_hugepagesz=1GB hugepagesz=1G hugepages=20 iommu=pt intel_iommu=on"
Execute the following command to modify the ``/boot/grub/grub.cfg`` with the configuration set in the above steps and permanently add them to the kernel command line.
::
update-grub
Reboot host system after making the above modifications.
Setup: Make Commands
^^^^^^^^^^^^^^^^^^^^
* Compile DPDK & QDMA driver
Execute the following to compile the driver.
::
cd <server_dir>/<dpdk_test_area>/dpdk-stable
make config T=x86_64-native-linuxapp-gcc install
- In the make output, verify that the QDMA files are being built.
Below figure shows the QDMA files that are built as part of make.
.. image:: images/make_output.png
The following should appear when make completes
::
Build complete [x86_64-native-linuxapp-gcc]
- Verify that ``librte_pmd_qdma.a`` is installed in ``./x86_64-native-linuxapp-gcc/lib`` directory.
Additionally, for memory mapped mode, BRAM size can be configured with ``make`` command.
Default BRAM size is set to 512KB in the driver makefile.
::
make config T=x86_64-native-linuxapp-gcc BRAM_SIZE=<BRAM size in bytes in decimal> install
* Compile Test application
Change to root user and compile the application
::
sudo su
cd examples/qdma_testapp
make RTE_SDK=`pwd`/../.. RTE_TARGET=x86_64-native-linuxapp-gcc
The following should appear when make completes
::
INSTALL-MAP qdma_testapp.map
Additionally, for memory mapped mode, BRAM size can be configured with make command.
Default BRAM size is set to 512KB in the driver makefile.
::
make BRAM_SIZE=<BRAM size in bytes in decimal> RTE_SDK=`pwd`/../.. RTE_TARGET=x86_64-native-linuxapp-gcc
If any of above steps are missed or require code modifications,
perform ``make clean`` before required modifications and re-building.
For driver related modifications, perform ``make clean``
from inside ``x86_64-native-linuxapp-gcc`` directory.
****************
Developers Guide
****************
.. toctree::
:maxdepth: 1
driver-design.rst
qdma_usecases.rst
******************
DPDK Callback APIs
******************
.. include:: ../build/pmd_callbacks.rst
Driver Design
*************
QDMA DPDK driver is implemented as an Ethernet poll mode driver in DPDK v18.11.
It supports both streaming (network) and memory (compute) interface to QDMA.
Below diagram gives a high level system overview of the QDMA DPDK driver.
.. image:: /images/dpdk_overview.png
:align: center
Driver interfaces and callbacks
===============================
The Driver registers two driver interfaces (``net_qdma`` for PFs and ``net_qdma_vf`` for VFs) with DPDK RTE.
Below ``eth_dev_ops`` callbacks are implemented by both driver interfaces of the DPDK driver.
::
static struct eth_dev_ops qdma_eth_dev_ops = {
.dev_configure = qdma_dev_configure,
.dev_infos_get = qdma_dev_infos_get,
.dev_start = qdma_dev_start,
.dev_stop = qdma_dev_stop,
.dev_close = qdma_dev_close,
.link_update = qdma_dev_link_update,
.rx_queue_setup = qdma_dev_rx_queue_setup,
.tx_queue_setup = qdma_dev_tx_queue_setup,
.rx_queue_release = qdma_dev_rx_queue_release,
.tx_queue_release = qdma_dev_tx_queue_release,
.rx_queue_start = qdma_dev_rx_queue_start,
.rx_queue_stop = qdma_dev_rx_queue_stop,
.tx_queue_start = qdma_dev_tx_queue_start,
.tx_queue_stop = qdma_dev_tx_queue_stop,
.stats_get = qdma_dev_stats_get,
};
Below sequence diagram depicts the application call flow to these driver callbacks and their high level operations.
.. image:: images/dpdk_call_flow.png
Implementation details of the callback APIs is described further in
.. toctree::
:maxdepth: 1
dpdk-callbacks.rst
Additional interfaces are exported by the driver for QDMA configuration. These new interfaces are described in
.. toctree::
:maxdepth: 1
qdma_pmd_export.rst
Resource Management
===================
.. toctree::
:maxdepth: 1
qdma_resmgmt.rst
Mailbox Communication
=====================
.. toctree::
:maxdepth: 1
qdma_mailbox.rst
QDMA Hardware Error Monitoring
==============================
There are various causes for errors in QDMA IP e.g. descriptor errors, DMA engine errors, etc.
If the errors are enabled by SW, QDMA logs the status of error in the respective global status registers.
DPDK driver executes a poll thread ``qdma_check_errors()`` every 1 second to check the error status and log the type of HW error occured on the console.
DPDK Driver debug
=================
DPDK driver provides API ``rte_pmd_qdma_xdebug()`` to dump debug information. Below debug information can be dumped using this API
- Dump QDMA registers
- Dump queue contexts for the given queue id
- Dump software information for the given queue id
- Dump the specified descriptor ring of the given queue id
User application can call this API to dump the required debug information.
QDMA Features
=============
QDMA DPDK Driver supports the following list of features
Hardware Features
-----------------
* SRIOV with 4 Physical Functions and 252 Virtual Functions
* Memory Mapped (MM) and Stream (ST) interfaces per queue
* 2048 queues sets
- 2048 H2C (host-to-card) Descriptor rings
- 2048 C2H (card-to-host) Descriptor rings
- 2048 Completion rings
* Mailbox communication between PF and VF driver
* Interrupt support for Mailbox events
* HW Error reporting
* Zero byte transfers
* Immediate data transfers
* Descriptor bypass (8, 16, 32, 64 descriptor sizes) support
* Descriptor Prefetching
* Streaming C2H completion entry coalescing
* Disabling overflow check in completion ring
* Streaming H2C to C2H and C2H to H2C loopback support
* Dynamic queue configuration
* Completion ring descriptors of 8, 16, 32, 64 bytes sizes
* Flexible BAR mapping for QDMA configuration register space
* ECC support
* Completions in Memory mapped mode
For details on Hardware Features refer to QDMA_Product_Guide_.
.. _QDMA_Product_Guide: https://www.xilinx.com/support/documentation/ip_documentation/qdma/v3_0/pg302-qdma.pdf
Software Features
-----------------
* Support DPDK v18.11 LTS
* Support driver binding to igb_uio and vfio-pci
* Support device arguments (module parameters) for device level configuration
* Allows only Privileged Physical Functions to program the contexts and configuration registers
* Device configuration through additional driver APIs
* Interoperability between Linux driver (as PF/VF) and DPDK driver (as PF/VF)
* SW versioning
#################################
Xilinx QDMA DPDK Poll Mode Driver
#################################
The Xilinx PCI Express Multi Queue DMA (QDMA) IP provides high-performance
direct memory access (DMA) via PCI Express.
Xilinx provides a DPDK poll mode driver based on DPDK v18.11 that runs on a
PCI Express root port host PC to interact with the QDMA endpoint IP via PCI Express.
.. toctree::
:maxdepth: 1
:caption: Table of Contents
features.rst
system-requirements.rst
build.rst
userguide.rst
dev-guide.rst
perf.rst
QDMA DPDK Performance
---------------------
Refer to QDMA_Performance_Answer_Record_ for detailed performance results and test environment.
.. _QDMA_Performance_Answer_Record: https://www.xilinx.com/support/answers/71453.html
*************
QDMA Mailbox
*************
The QDMA Subsystem for PCIe provides an optional feature to support the Single Root I/O
Virtualization (SR-IOV). SR-IOV classifies the functions as:
- Physical Functions (PF) :
Full featured PCIe functions which include SR-IOV capabilities among others.
- Virtual Functions (VF):
PCIe functions featuring configuration space with Base Address
Registers (BARs) but lacking the full configuration resources and are controlled by the PF
configuration. The main role of the VF is data transfer.
VF can communicate to a PF through mailbox. Each function implements one 128B inbox and 128B outbox message buffer.
These mailboxes are accessible to the driver via PCIe BAR of its own function.
HW also provides ability to interrupt the driver for an incoming mailbox message to a PCIe function.
For further details on the mailbox internals and mailbox registers, refer to QDMA_Product_Guide_.
.. _QDMA_Product_Guide: https://www.xilinx.com/support/documentation/ip_documentation/qdma/v3_0/pg302-qdma.pdf
Physical function (PF) is privileged with full access to QDMA registers and resources, but VFs updates only data handling registers and interrupts.
VF drivers must communicate with the driver attached to the PF through the mailbox for configuration and resource allocation.
The PF and VF drivers define the message formatting to be exchanged between the driver.
The driver enables mailbox interrupt and registers a Rx handler for notification of incoming mailbox message.
It also creates a Tx thread to send a message and poll for its response from the peer driver.
Following are the list of messages supported between PF and VF driver
::
enum mbox_msg_op {
/** @MBOX_OP_BYE: vf offline */
MBOX_OP_BYE,
/** @MBOX_OP_HELLO: vf online */
MBOX_OP_HELLO,
/** @: FMAP programming request */
MBOX_OP_FMAP,
/** @MBOX_OP_CSR: global CSR registers request */
MBOX_OP_CSR,
/** @MBOX_OP_QREQ: request queues */
MBOX_OP_QREQ,
/** @MBOX_OP_QNOTIFY_ADD: notify of queue addition */
MBOX_OP_QNOTIFY_ADD,
/** @MBOX_OP_QNOTIFY_DEL: notify of queue deletion */
MBOX_OP_QNOTIFY_DEL,
/** @MBOX_OP_QCTXT_WRT: queue context write */
MBOX_OP_QCTXT_WRT,
/** @MBOX_OP_QCTXT_RD: queue context read */
MBOX_OP_QCTXT_RD,
/** @MBOX_OP_QCTXT_CLR: queue context clear */
MBOX_OP_QCTXT_CLR,
/** @MBOX_OP_QCTXT_INV: queue context invalidate */
MBOX_OP_QCTXT_INV,
/** @MBOX_OP_INTR_CTXT_WRT: interrupt context write */
MBOX_OP_INTR_CTXT_WRT,
/** @MBOX_OP_INTR_CTXT_RD: interrupt context read */
MBOX_OP_INTR_CTXT_RD,
/** @MBOX_OP_INTR_CTXT_CLR: interrupt context clear */
MBOX_OP_INTR_CTXT_CLR,
/** @MBOX_OP_INTR_CTXT_INV: interrupt context invalidate */
MBOX_OP_INTR_CTXT_INV,
/** @MBOX_OP_HELLO_RESP: response to @MBOX_OP_HELLO */
MBOX_OP_HELLO_RESP = 0x81,
/** @MBOX_OP_FMAP_RESP: response to @MBOX_OP_FMAP */
MBOX_OP_FMAP_RESP,
/** @MBOX_OP_CSR_RESP: response to @MBOX_OP_CSR */
MBOX_OP_CSR_RESP,
/** @MBOX_OP_QREQ_RESP: response to @MBOX_OP_QREQ */
MBOX_OP_QREQ_RESP,
/** @MBOX_OP_QADD: notify of queue addition */
MBOX_OP_QNOTIFY_ADD_RESP,
/** @MBOX_OP_QNOTIFY_DEL: notify of queue deletion */
MBOX_OP_QNOTIFY_DEL_RESP,
/** @MBOX_OP_QCTXT_WRT_RESP: response to @MBOX_OP_QCTXT_WRT */
MBOX_OP_QCTXT_WRT_RESP,
/** @MBOX_OP_QCTXT_RD_RESP: response to @MBOX_OP_QCTXT_RD */
MBOX_OP_QCTXT_RD_RESP,
/** @MBOX_OP_QCTXT_CLR_RESP: response to @MBOX_OP_QCTXT_CLR */
MBOX_OP_QCTXT_CLR_RESP,
/** @MBOX_OP_QCTXT_INV_RESP: response to @MBOX_OP_QCTXT_INV */
MBOX_OP_QCTXT_INV_RESP,
/** @MBOX_OP_INTR_CTXT_WRT_RESP: response to @MBOX_OP_INTR_CTXT_WRT */
MBOX_OP_INTR_CTXT_WRT_RESP,
/** @MBOX_OP_INTR_CTXT_RD_RESP: response to @MBOX_OP_INTR_CTXT_RD */
MBOX_OP_INTR_CTXT_RD_RESP,
/** @MBOX_OP_INTR_CTXT_CLR_RESP: response to @MBOX_OP_INTR_CTXT_CLR */
MBOX_OP_INTR_CTXT_CLR_RESP,
/** @MBOX_OP_INTR_CTXT_INV_RESP: response to @MBOX_OP_INTR_CTXT_INV */
MBOX_OP_INTR_CTXT_INV_RESP,
/** @MBOX_OP_MAX: total mbox opcodes*/
MBOX_OP_MAX
};
The following sequence diagrams shows the mailbox communication between PF and VF for VF configuration.
.. image:: /images/mailbox_communication.png
:align: center
\ No newline at end of file
***************************
QDMA DPDK PMD Exported APIs
***************************
.. include:: ../build/rte_pmd_qdma.rst
******************************
QDMA Queue Resource Management
******************************
QDMA IP supports 2K queues. QDMA Resource Manager defines the strategy to allocate the queues across the available PFs and VFs.
Resource Manager maintains a global resource linked list in the driver. It creates a linked list of nodes for each PCIe device (PCIe bus) it manages.
Each device (bus) node in the Resource Manager list is initialized with queue base and number of queues to manage on that device.
Given the request for a number of queues for a PCIe function on a PCIe device, Resource Manager finds the best fit available
queue base with contiguous requested number of queues on that PCIe device.
Resource Manager manages queue base allocation for all the PCIe devices and PCIe functions registered with it by the driver.
Each device node in the global resource linked list maintains a "free list" with all the queues initially assigned to it, and a "device list" initialized to NULL.
When a PCIe function is initialized and requests the required queues, Resource Manager finds the best fit available queue base
with contiguous requested number of queues from the "free list" of the requested device node. If found, it creates a PCIe function node in the "device list"
and assigns the queue base and requested queues to this PCIe function node and removes these queues from the "free list".
Driver can request a specific queue base for a PCIe function and if the requested queue base is available,
Resource Manager honours it, else, allocates the queues at a base determined by the best fit algorithm.
Host system can have drivers in user space or kernel space to manage the devices connected.
E.g. Linux QDMA Driver is a kernel mode driver and DPDK PMD is a User Space driver.
Each driver maintains its own resouce manager with queue base and total queues to manage specified for each device it manages.
Below, we demonstrate few examples on how Resource Manager is used in different combinations of drivers managing QDMA devices
Single device managed by Single driver
======================================
.. image:: /images/resmgmt_sdev_sdri.PNG
:align: center
In this scenario, one PCIe QDMA device is connected to the Host System and at any given time either the Linux kernel driver or the user space DPDK driver is
managing the device. There is a single Resource Manager managing all the queues of the device and assigning the queues to functions based
on the request from each function.
Single device managed by multiple drivers
==========================================
.. image:: /images/resmgmt_sdev_mdri.PNG
:align: center
In this scenario, one device is connected to the Host System. Linux kernel driver and user space DPDK driver are loaded to manage different PCIe functions of the device.
Each driver will create its own Resource Manager using a queue base and total queues to manage the functions binded to them.
Each Resource Manager has its own pool of queues and the PCIe functions get their queues resourced from this pool.
Multiple devices on single Host managed by Single driver
========================================================
.. image:: /images/resmgmt_mdev_sdri.PNG
:align: center
In this scenario, more than one devices are connected to the Host System.
Either user space DPDK driver or Linux kernel driver is loaded to manage these devices.
Driver will instantiate Resource Manager and create one node per PCIe device conected,
resourcing each PCIe device (node) with the queue base and total number of queues to manage.
PCIe functions corresponding to each PCIe device are allocated queues from the respective device nodes.
.. _sys_req:
System Requirements
===================
Xilinx Accelerator Card
-----------------------
1. VCU1525
2. U200
Host System
-----------
x86_64 host system with at least one Gen 3 x16 PCIe slot and minimum 32GB RAM
on same CPU node for 2K queues.
For VM testing, host system must support virtualization and it must be enabled in the BIOS.
Host System Configuration
-------------------------
QDMA DPDK driver latest release is verified on following host system configuration for PF and VF functionality
+--------------------------+-------------------------------------------------------------+
| Host System | Configuration Details |
+==========================+=============================================================+
| Operating System | Ubuntu 16.04.3 LTS, Ubuntu 18.04.1 LTS |
+--------------------------+-------------------------------------------------------------+
| Linux Kernel | 4.4.0-93-generic |
+--------------------------+-------------------------------------------------------------+
| CPU Cores | 32 |
+--------------------------+-------------------------------------------------------------+
| RAM | 64GB on local NUMA node |
+--------------------------+-------------------------------------------------------------+
| Hypervisor | KVM |
+--------------------------+-------------------------------------------------------------+
| Qemu Version | QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.15)|
+--------------------------+-------------------------------------------------------------+
| Vivado Version | 2019.1 |
+--------------------------+-------------------------------------------------------------+
.. _guest_system_cfg:
Guest System Configuration
--------------------------
QDMA DPDK driver latest release is verified on the following guest system configuration for VF functionality
========================= ==================================
Guest System(VM) Configuration Details
========================= ==================================
Operating System Ubuntu 18.04 LTS
Linux Kernel 4.15.1-20-generic
RAM 4GB
Cores 4
========================= ==================================
This diff is collapsed.
This diff is collapsed.
.fa:before{-webkit-font-smoothing:antialiased}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;content:""}.clearfix:after{clear:both}@font-face{font-family:FontAwesome;font-weight:normal;font-style:normal;src:url("../font/fontawesome_webfont.eot");src:url("../font/fontawesome_webfont.eot?#iefix") format("embedded-opentype"),url("../font/fontawesome_webfont.woff") format("woff"),url("../font/fontawesome_webfont.ttf") format("truetype"),url("../font/fontawesome_webfont.svg#FontAwesome") format("svg")}.fa:before{display:inline-block;font-family:FontAwesome;font-style:normal;font-weight:normal;line-height:1;text-decoration:inherit}a .fa{display:inline-block;text-decoration:inherit}li .fa{display:inline-block}li .fa-large:before,li .fa-large:before{width:1.875em}ul.fas{list-style-type:none;margin-left:2em;text-indent:-0.8em}ul.fas li .fa{width:0.8em}ul.fas li .fa-large:before,ul.fas li .fa-large:before{vertical-align:baseline}.fa-book:before{content:""}.icon-book:before{content:""}.fa-caret-down:before{content:""}.icon-caret-down:before{content:""}.fa-caret-up:before{content:""}.icon-caret-up:before{content:""}.fa-caret-left:before{content:""}.icon-caret-left:before{content:""}.fa-caret-right:before{content:""}.icon-caret-right:before{content:""}.rst-versions{position:fixed;bottom:0;left:0;width:300px;color:#fcfcfc;background:#1f1d1d;border-top:solid 10px #343131;font-family:"Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;z-index:400}.rst-versions a{color:#2980B9;text-decoration:none}.rst-versions .rst-badge-small{display:none}.rst-versions .rst-current-version{padding:12px;background-color:#272525;display:block;text-align:right;font-size:90%;cursor:pointer;color:#27AE60;*zoom:1}.rst-versions .rst-current-version:before,.rst-versions .rst-current-version:after{display:table;content:""}.rst-versions .rst-current-version:after{clear:both}.rst-versions .rst-current-version .fa{color:#fcfcfc}.rst-versions .rst-current-version .fa-book{float:left}.rst-versions .rst-current-version .icon-book{float:left}.rst-versions .rst-current-version.rst-out-of-date{background-color:#E74C3C;color:#fff}.rst-versions .rst-current-version.rst-active-old-version{background-color:#F1C40F;color:#000}.rst-versions.shift-up .rst-other-versions{display:block}.rst-versions .rst-other-versions{font-size:90%;padding:12px;color:gray;display:none}.rst-versions .rst-other-versions hr{display:block;height:1px;border:0;margin:20px 0;padding:0;border-top:solid 1px #413d3d}.rst-versions .rst-other-versions dd{display:inline-block;margin:0}.rst-versions .rst-other-versions dd a{display:inline-block;padding:6px;color:#fcfcfc}.rst-versions.rst-badge{width:auto;bottom:20px;right:20px;left:auto;border:none;max-width:300px}.rst-versions.rst-badge .icon-book{float:none}.rst-versions.rst-badge .fa-book{float:none}.rst-versions.rst-badge.shift-up .rst-current-version{text-align:right}.rst-versions.rst-badge.shift-up .rst-current-version .fa-book{float:left}.rst-versions.rst-badge.shift-up .rst-current-version .icon-book{float:left}.rst-versions.rst-badge .rst-current-version{width:auto;height:30px;line-height:30px;padding:0 6px;display:block;text-align:center}@media screen and (max-width: 768px){.rst-versions{width:85%;display:none}.rst-versions.shift{display:block}img{width:100%;height:auto}}
/*# sourceMappingURL=badge_only.css.map */
This source diff could not be displayed because it is too large. You can view the blob instead.
/*
* doctools.js
* ~~~~~~~~~~~
*
* Sphinx JavaScript utilities for all documentation.
*
* :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/
/**
* select a different prefix for underscore
*/
$u = _.noConflict();
/**
* make the code below compatible with browsers without
* an installed firebug like debugger
if (!window.console || !console.firebug) {
var names = ["log", "debug", "info", "warn", "error", "assert", "dir",
"dirxml", "group", "groupEnd", "time", "timeEnd", "count", "trace",
"profile", "profileEnd"];
window.console = {};
for (var i = 0; i < names.length; ++i)
window.console[names[i]] = function() {};
}
*/
/**
* small helper function to urldecode strings
*/
jQuery.urldecode = function(x) {
return decodeURIComponent(x).replace(/\+/g, ' ');
};
/**
* small helper function to urlencode strings
*/
jQuery.urlencode = encodeURIComponent;
/**
* This function returns the parsed url parameters of the
* current request. Multiple values per key are supported,
* it will always return arrays of strings for the value parts.
*/
jQuery.getQueryParameters = function(s) {
if (typeof s === 'undefined')
s = document.location.search;
var parts = s.substr(s.indexOf('?') + 1).split('&');
var result = {};
for (var i = 0; i < parts.length; i++) {
var tmp = parts[i].split('=', 2);
var key = jQuery.urldecode(tmp[0]);
var value = jQuery.urldecode(tmp[1]);
if (key in result)
result[key].push(value);
else
result[key] = [value];
}
return result;
};
/**
* highlight a given string on a jquery object by wrapping it in
* span elements with the given class name.
*/
jQuery.fn.highlightText = function(text, className) {
function highlight(node, addItems) {
if (node.nodeType === 3) {
var val = node.nodeValue;
var pos = val.toLowerCase().indexOf(text);
if (pos >= 0 &&
!jQuery(node.parentNode).hasClass(className) &&
!jQuery(node.parentNode).hasClass("nohighlight")) {
var span;
var isInSVG = jQuery(node).closest("body, svg, foreignObject").is("svg");
if (isInSVG) {
span = document.createElementNS("http://www.w3.org/2000/svg", "tspan");
} else {
span = document.createElement("span");
span.className = className;
}
span.appendChild(document.createTextNode(val.substr(pos, text.length)));
node.parentNode.insertBefore(span, node.parentNode.insertBefore(
document.createTextNode(val.substr(pos + text.length)),
node.nextSibling));
node.nodeValue = val.substr(0, pos);
if (isInSVG) {
var bbox = span.getBBox();
var rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
rect.x.baseVal.value = bbox.x;
rect.y.baseVal.value = bbox.y;
rect.width.baseVal.value = bbox.width;
rect.height.baseVal.value = bbox.height;
rect.setAttribute('class', className);
var parentOfText = node.parentNode.parentNode;
addItems.push({
"parent": node.parentNode,
"target": rect});
}
}
}
else if (!jQuery(node).is("button, select, textarea")) {
jQuery.each(node.childNodes, function() {
highlight(this, addItems);
});
}
}
var addItems = [];
var result = this.each(function() {
highlight(this, addItems);
});
for (var i = 0; i < addItems.length; ++i) {
jQuery(addItems[i].parent).before(addItems[i].target);
}
return result;
};
/*
* backward compatibility for jQuery.browser
* This will be supported until firefox bug is fixed.
*/
if (!jQuery.browser) {
jQuery.uaMatch = function(ua) {
ua = ua.toLowerCase();
var match = /(chrome)[ \/]([\w.]+)/.exec(ua) ||
/(webkit)[ \/]([\w.]+)/.exec(ua) ||
/(opera)(?:.*version|)[ \/]([\w.]+)/.exec(ua) ||
/(msie) ([\w.]+)/.exec(ua) ||
ua.indexOf("compatible") < 0 && /(mozilla)(?:.*? rv:([\w.]+)|)/.exec(ua) ||
[];
return {
browser: match[ 1 ] || "",
version: match[ 2 ] || "0"
};
};
jQuery.browser = {};
jQuery.browser[jQuery.uaMatch(navigator.userAgent).browser] = true;
}
/**
* Small JavaScript module for the documentation.
*/
var Documentation = {
init : function() {
this.fixFirefoxAnchorBug();
this.highlightSearchWords();
this.initIndexTable();
if (DOCUMENTATION_OPTIONS.NAVIGATION_WITH_KEYS) {
this.initOnKeyListeners();
}
},
/**
* i18n support
*/
TRANSLATIONS : {},
PLURAL_EXPR : function(n) { return n === 1 ? 0 : 1; },
LOCALE : 'unknown',
// gettext and ngettext don't access this so that the functions
// can safely bound to a different name (_ = Documentation.gettext)
gettext : function(string) {
var translated = Documentation.TRANSLATIONS[string];
if (typeof translated === 'undefined')
return string;
return (typeof translated === 'string') ? translated : translated[0];
},
ngettext : function(singular, plural, n) {
var translated = Documentation.TRANSLATIONS[singular];
if (typeof translated === 'undefined')
return (n == 1) ? singular : plural;
return translated[Documentation.PLURALEXPR(n)];
},
addTranslations : function(catalog) {
for (var key in catalog.messages)
this.TRANSLATIONS[key] = catalog.messages[key];
this.PLURAL_EXPR = new Function('n', 'return +(' + catalog.plural_expr + ')');
this.LOCALE = catalog.locale;
},
/**
* add context elements like header anchor links
*/
addContextElements : function() {
$('div[id] > :header:first').each(function() {
$('<a class="headerlink">\u00B6</a>').
attr('href', '#' + this.id).
attr('title', _('Permalink to this headline')).
appendTo(this);
});
$('dt[id]').each(function() {
$('<a class="headerlink">\u00B6</a>').
attr('href', '#' + this.id).
attr('title', _('Permalink to this definition')).
appendTo(this);
});
},
/**
* workaround a firefox stupidity
* see: https://bugzilla.mozilla.org/show_bug.cgi?id=645075
*/
fixFirefoxAnchorBug : function() {
if (document.location.hash && $.browser.mozilla)
window.setTimeout(function() {
document.location.href += '';
}, 10);
},
/**
* highlight the search words provided in the url in the text
*/
highlightSearchWords : function() {
var params = $.getQueryParameters();
var terms = (params.highlight) ? params.highlight[0].split(/\s+/) : [];
if (terms.length) {
var body = $('div.body');
if (!body.length) {
body = $('body');
}
window.setTimeout(function() {
$.each(terms, function() {
body.highlightText(this.toLowerCase(), 'highlighted');
});
}, 10);
$('<p class="highlight-link"><a href="javascript:Documentation.' +
'hideSearchWords()">' + _('Hide Search Matches') + '</a></p>')
.appendTo($('#searchbox'));
}
},
/**
* init the domain index toggle buttons
*/
initIndexTable : function() {
var togglers = $('img.toggler').click(function() {
var src = $(this).attr('src');
var idnum = $(this).attr('id').substr(7);
$('tr.cg-' + idnum).toggle();
if (src.substr(-9) === 'minus.png')
$(this).attr('src', src.substr(0, src.length-9) + 'plus.png');
else
$(this).attr('src', src.substr(0, src.length-8) + 'minus.png');
}).css('display', '');
if (DOCUMENTATION_OPTIONS.COLLAPSE_INDEX) {
togglers.click();
}
},
/**
* helper function to hide the search marks again
*/
hideSearchWords : function() {
$('#searchbox .highlight-link').fadeOut(300);
$('span.highlighted').removeClass('highlighted');
},
/**
* make the url absolute
*/
makeURL : function(relativeURL) {
return DOCUMENTATION_OPTIONS.URL_ROOT + '/' + relativeURL;
},
/**
* get the current relative url
*/
getCurrentURL : function() {
var path = document.location.pathname;
var parts = path.split(/\//);
$.each(DOCUMENTATION_OPTIONS.URL_ROOT.split(/\//), function() {
if (this === '..')
parts.pop();
});
var url = parts.join('/');
return path.substring(url.lastIndexOf('/') + 1, path.length - 1);
},
initOnKeyListeners: function() {
$(document).keyup(function(event) {
var activeElementType = document.activeElement.tagName;
// don't navigate when in search box or textarea
if (activeElementType !== 'TEXTAREA' && activeElementType !== 'INPUT' && activeElementType !== 'SELECT') {
switch (event.keyCode) {
case 37: // left
var prevHref = $('link[rel="prev"]').prop('href');
if (prevHref) {
window.location.href = prevHref;
return false;
}
case 39: // right
var nextHref = $('link[rel="next"]').prop('href');
if (nextHref) {
window.location.href = nextHref;
return false;
}
}
}
});
}
};
// quick alias for translations
_ = Documentation.gettext;
$(document).ready(function() {
Documentation.init();
});
This source diff could not be displayed because it is too large. You can view the blob instead.
/*
* graphviz.css
* ~~~~~~~~~~~~
*
* Sphinx stylesheet -- graphviz extension.
*
* :copyright: Copyright 2007-2018 by the Sphinx team, see AUTHORS.
* :license: BSD, see LICENSE for details.
*
*/
img.graphviz {
border: 0;
max-width: 100%;
}
object.graphviz {
max-width: 100%;
}
This diff is collapsed.
This diff is collapsed.
require=(function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=typeof require=="function"&&require;if(!u&&a)return a(o,!0);if(i)return i(o,!0);var f=new Error("Cannot find module '"+o+"'");throw f.code="MODULE_NOT_FOUND",f}var l=n[o]={exports:{}};t[o][0].call(l.exports,function(e){var n=t[o][1][e];return s(n?n:e)},l,l.exports,e,t,n,r)}return n[o].exports}var i=typeof require=="function"&&require;for(var o=0;o<r.length;o++)s(r[o]);return s})({"sphinx-rtd-theme":[function(require,module,exports){
var jQuery = (typeof(window) != 'undefined') ? window.jQuery : require('jquery');
// Sphinx theme nav state
function ThemeNav () {
var nav = {
navBar: null,
win: null,
winScroll: false,
winResize: false,
linkScroll: false,
winPosition: 0,
winHeight: null,
docHeight: null,
isRunning: null
};
nav.enable = function () {
var self = this;
jQuery(function ($) {
self.init($);
self.reset();
self.win.on('hashchange', self.reset);
// Set scroll monitor
self.win.on('scroll', function () {
if (!self.linkScroll) {
self.winScroll = true;
}
});
setInterval(function () { if (self.winScroll) self.onScroll(); }, 25);
// Set resize monitor
self.win.on('resize', function () {
self.winResize = true;
});
setInterval(function () { if (self.winResize) self.onResize(); }, 25);
self.onResize();
});
};
nav.init = function ($) {
var doc = $(document),
self = this;
this.navBar = $('div.wy-side-scroll:first');
this.win = $(window);
// Set up javascript UX bits
$(document)
// Shift nav in mobile when clicking the menu.
.on('click', "[data-toggle='wy-nav-top']", function() {
$("[data-toggle='wy-nav-shift']").toggleClass("shift");
$("[data-toggle='rst-versions']").toggleClass("shift");
})
// Nav menu link click operations
.on('click', ".wy-menu-vertical .current ul li a", function() {
var target = $(this);
// Close menu when you click a link.
$("[data-toggle='wy-nav-shift']").removeClass("shift");
$("[data-toggle='rst-versions']").toggleClass("shift");
// Handle dynamic display of l3 and l4 nav lists
self.toggleCurrent(target);
self.hashChange();
})
.on('click', "[data-toggle='rst-current-version']", function() {
$("[data-toggle='rst-versions']").toggleClass("shift-up");
})
// Make tables responsive
$("table.docutils:not(.field-list)")
.wrap("<div class='wy-table-responsive'></div>");
// Add expand links to all parents of nested ul
$('.wy-menu-vertical ul').not('.simple').siblings('a').each(function () {
var link = $(this);
expand = $('<span class="toctree-expand"></span>');
expand.on('click', function (ev) {
self.toggleCurrent(link);
ev.stopPropagation();
return false;
});
link.prepend(expand);
});
};
nav.reset = function () {
// Get anchor from URL and open up nested nav
var anchor = encodeURI(window.location.hash);
if (anchor) {
try {
var link = $('.wy-menu-vertical')
.find('[href="' + anchor + '"]');
$('.wy-menu-vertical li.toctree-l1 li.current')
.removeClass('current');
link.closest('li.toctree-l2').addClass('current');
link.closest('li.toctree-l3').addClass('current');
link.closest('li.toctree-l4').addClass('current');
}
catch (err) {
console.log("Error expanding nav for anchor", err);
}
}
};
nav.onScroll = function () {
this.winScroll = false;
var newWinPosition = this.win.scrollTop(),
winBottom = newWinPosition + this.winHeight,
navPosition = this.navBar.scrollTop(),
newNavPosition = navPosition + (newWinPosition - this.winPosition);
if (newWinPosition < 0 || winBottom > this.docHeight) {
return;
}
this.navBar.scrollTop(newNavPosition);
this.winPosition = newWinPosition;
};
nav.onResize = function () {
this.winResize = false;
this.winHeight = this.win.height();
this.docHeight = $(document).height();
};
nav.hashChange = function () {
this.linkScroll = true;
this.win.one('hashchange', function () {
this.linkScroll = false;
});
};
nav.toggleCurrent = function (elem) {
var parent_li = elem.closest('li');
parent_li.siblings('li.current').removeClass('current');
parent_li.siblings().find('li.current').removeClass('current');
parent_li.find('> ul li.current').removeClass('current');
parent_li.toggleClass('current');
}
return nav;
};
module.exports.ThemeNav = ThemeNav();
if (typeof(window) != 'undefined') {
window.SphinxRtdTheme = { StickyNav: module.exports.ThemeNav };
}
},{"jquery":"jquery"}]},{},["sphinx-rtd-theme"]);
.highlight .hll { background-color: #ffffcc }
.highlight { background: #eeffcc; }
.highlight .c { color: #408090; font-style: italic } /* Comment */
.highlight .err { border: 1px solid #FF0000 } /* Error */
.highlight .k { color: #007020; font-weight: bold } /* Keyword */
.highlight .o { color: #666666 } /* Operator */
.highlight .ch { color: #408090; font-style: italic } /* Comment.Hashbang */
.highlight .cm { color: #408090; font-style: italic } /* Comment.Multiline */
.highlight .cp { color: #007020 } /* Comment.Preproc */
.highlight .cpf { color: #408090; font-style: italic } /* Comment.PreprocFile */
.highlight .c1 { color: #408090; font-style: italic } /* Comment.Single */
.highlight .cs { color: #408090; background-color: #fff0f0 } /* Comment.Special */
.highlight .gd { color: #A00000 } /* Generic.Deleted */
.highlight .ge { font-style: italic } /* Generic.Emph */
.highlight .gr { color: #FF0000 } /* Generic.Error */
.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
.highlight .gi { color: #00A000 } /* Generic.Inserted */
.highlight .go { color: #333333 } /* Generic.Output */
.highlight .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */
.highlight .gs { font-weight: bold } /* Generic.Strong */
.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
.highlight .gt { color: #0044DD } /* Generic.Traceback */
.highlight .kc { color: #007020; font-weight: bold } /* Keyword.Constant */
.highlight .kd { color: #007020; font-weight: bold } /* Keyword.Declaration */
.highlight .kn { color: #007020; font-weight: bold } /* Keyword.Namespace */
.highlight .kp { color: #007020 } /* Keyword.Pseudo */
.highlight .kr { color: #007020; font-weight: bold } /* Keyword.Reserved */
.highlight .kt { color: #902000 } /* Keyword.Type */
.highlight .m { color: #208050 } /* Literal.Number */
.highlight .s { color: #4070a0 } /* Literal.String */
.highlight .na { color: #4070a0 } /* Name.Attribute */
.highlight .nb { color: #007020 } /* Name.Builtin */
.highlight .nc { color: #0e84b5; font-weight: bold } /* Name.Class */
.highlight .no { color: #60add5 } /* Name.Constant */
.highlight .nd { color: #555555; font-weight: bold } /* Name.Decorator */
.highlight .ni { color: #d55537; font-weight: bold } /* Name.Entity */
.highlight .ne { color: #007020 } /* Name.Exception */
.highlight .nf { color: #06287e } /* Name.Function */
.highlight .nl { color: #002070; font-weight: bold } /* Name.Label */
.highlight .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */
.highlight .nt { color: #062873; font-weight: bold } /* Name.Tag */
.highlight .nv { color: #bb60d5 } /* Name.Variable */
.highlight .ow { color: #007020; font-weight: bold } /* Operator.Word */
.highlight .w { color: #bbbbbb } /* Text.Whitespace */
.highlight .mb { color: #208050 } /* Literal.Number.Bin */
.highlight .mf { color: #208050 } /* Literal.Number.Float */
.highlight .mh { color: #208050 } /* Literal.Number.Hex */
.highlight .mi { color: #208050 } /* Literal.Number.Integer */
.highlight .mo { color: #208050 } /* Literal.Number.Oct */
.highlight .sb { color: #4070a0 } /* Literal.String.Backtick */
.highlight .sc { color: #4070a0 } /* Literal.String.Char */
.highlight .sd { color: #4070a0; font-style: italic } /* Literal.String.Doc */
.highlight .s2 { color: #4070a0 } /* Literal.String.Double */
.highlight .se { color: #4070a0; font-weight: bold } /* Literal.String.Escape */
.highlight .sh { color: #4070a0 } /* Literal.String.Heredoc */
.highlight .si { color: #70a0d0; font-style: italic } /* Literal.String.Interpol */
.highlight .sx { color: #c65d09 } /* Literal.String.Other */
.highlight .sr { color: #235388 } /* Literal.String.Regex */
.highlight .s1 { color: #4070a0 } /* Literal.String.Single */
.highlight .ss { color: #517918 } /* Literal.String.Symbol */
.highlight .bp { color: #007020 } /* Name.Builtin.Pseudo */
.highlight .vc { color: #bb60d5 } /* Name.Variable.Class */
.highlight .vg { color: #bb60d5 } /* Name.Variable.Global */
.highlight .vi { color: #bb60d5 } /* Name.Variable.Instance */
.highlight .il { color: #208050 } /* Literal.Number.Integer.Long */
\ No newline at end of file
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Developers Guide &mdash; QDMA DPDK Driver 2019.1 documentation</title>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/graphviz.css" type="text/css" />
<link rel="index" title="Index"
href="genindex.html"/>
<link rel="search" title="Search" href="search.html"/>
<link rel="top" title="QDMA DPDK Driver 2019.1 documentation" href="index.html"/>
<link rel="next" title="Driver Design" href="driver-design.html"/>
<link rel="prev" title="User Guide" href="userguide.html"/>
<script src="_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search">
<a href="index.html" class="icon icon-home"> QDMA DPDK Driver
</a>
<div class="version">
2019.1
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<p class="caption"><span class="caption-text">Table of Contents</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="features.html">QDMA Features</a></li>
<li class="toctree-l1"><a class="reference internal" href="system-requirements.html">System Requirements</a></li>
<li class="toctree-l1"><a class="reference internal" href="build.html">Building QDMA DPDK Software</a></li>
<li class="toctree-l1"><a class="reference internal" href="userguide.html">User Guide</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Developers Guide</a><ul>
<li class="toctree-l2"><a class="reference internal" href="driver-design.html">Driver Design</a></li>
<li class="toctree-l2"><a class="reference internal" href="qdma_usecases.html">QDMA DPDK Driver UseCases</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="perf.html">QDMA DPDK Performance</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">QDMA DPDK Driver</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html">Docs</a> &raquo;</li>
<li>Developers Guide</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/dev-guide.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="developers-guide">
<h1>Developers Guide<a class="headerlink" href="#developers-guide" title="Permalink to this headline"></a></h1>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="driver-design.html">Driver Design</a></li>
<li class="toctree-l1"><a class="reference internal" href="qdma_usecases.html">QDMA DPDK Driver UseCases</a></li>
</ul>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="driver-design.html" class="btn btn-neutral float-right" title="Driver Design" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="userguide.html" class="btn btn-neutral" title="User Guide" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
&copy; Copyright 2019, Xilinx, Inc.
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'2019.1',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.StickyNav.enable();
});
</script>
</body>
</html>
\ No newline at end of file
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
****************
Developers Guide
****************
.. toctree::
:maxdepth: 1
driver-design.rst
qdma_usecases.rst
******************
DPDK Callback APIs
******************
.. include:: ../build/pmd_callbacks.rst
Driver Design
*************
QDMA DPDK driver is implemented as an Ethernet poll mode driver in DPDK v18.11.
It supports both streaming (network) and memory (compute) interface to QDMA.
Below diagram gives a high level system overview of the QDMA DPDK driver.
.. image:: /images/dpdk_overview.png
:align: center
Driver interfaces and callbacks
===============================
The Driver registers two driver interfaces (``net_qdma`` for PFs and ``net_qdma_vf`` for VFs) with DPDK RTE.
Below ``eth_dev_ops`` callbacks are implemented by both driver interfaces of the DPDK driver.
::
static struct eth_dev_ops qdma_eth_dev_ops = {
.dev_configure = qdma_dev_configure,
.dev_infos_get = qdma_dev_infos_get,
.dev_start = qdma_dev_start,
.dev_stop = qdma_dev_stop,
.dev_close = qdma_dev_close,
.link_update = qdma_dev_link_update,
.rx_queue_setup = qdma_dev_rx_queue_setup,
.tx_queue_setup = qdma_dev_tx_queue_setup,
.rx_queue_release = qdma_dev_rx_queue_release,
.tx_queue_release = qdma_dev_tx_queue_release,
.rx_queue_start = qdma_dev_rx_queue_start,
.rx_queue_stop = qdma_dev_rx_queue_stop,
.tx_queue_start = qdma_dev_tx_queue_start,
.tx_queue_stop = qdma_dev_tx_queue_stop,
.stats_get = qdma_dev_stats_get,
};
Below sequence diagram depicts the application call flow to these driver callbacks and their high level operations.
.. image:: images/dpdk_call_flow.png
Implementation details of the callback APIs is described further in
.. toctree::
:maxdepth: 1
dpdk-callbacks.rst
Additional interfaces are exported by the driver for QDMA configuration. These new interfaces are described in
.. toctree::
:maxdepth: 1
qdma_pmd_export.rst
Resource Management
===================
.. toctree::
:maxdepth: 1
qdma_resmgmt.rst
Mailbox Communication
=====================
.. toctree::
:maxdepth: 1
qdma_mailbox.rst
QDMA Hardware Error Monitoring
==============================
There are various causes for errors in QDMA IP e.g. descriptor errors, DMA engine errors, etc.
If the errors are enabled by SW, QDMA logs the status of error in the respective global status registers.
DPDK driver executes a poll thread ``qdma_check_errors()`` every 1 second to check the error status and log the type of HW error occured on the console.
DPDK Driver debug
=================
DPDK driver provides API ``rte_pmd_qdma_xdebug()`` to dump debug information. Below debug information can be dumped using this API
- Dump QDMA registers
- Dump queue contexts for the given queue id
- Dump software information for the given queue id
- Dump the specified descriptor ring of the given queue id
User application can call this API to dump the required debug information.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment