Open Fabrics Enterprise Distribution (OFED) mthca in OFED 1.4 Release Notes December 2008 =============================================================================== Table of Contents =============================================================================== 1. Overview 2. Fixed Bugs 3. Known Issues =============================================================================== 1. Overview =============================================================================== mthca is the low level driver implementation for the following Mellanox Techno- logies HCAs: InfiniHost, InfiniHost III Ex and InfiniHost III Lx. mthca Available Parameters -------------------------- In order to set mthca parameters, add the following line to /etc/modpobe.conf: options ib_mthca parameter= mthca parameters: tune_pci - increase PCI burst from the default set by BIOS if nonzero msi - attempt to use MSI if nonzero msi_x - attempt to use MSI-X if nonzero fw_cmd_doorbell - post firmware commands through doorbell page if non- zero (and supported by firmware) catas_reset_disable - disable device reset on a catastrophic event if non- zero debug_level - Enable debug tracing if > 0 (int) num_qp - maximum number of QPs per HCA (int) rdb_per_qp - number of RDB buffers per QP (int) num_cq - maximum number of CQs per HCA (int) num_mcg - maximum number of multicast groups per HCA (int) num_mpt - maximum number of memory protection table entries per HCA (int) num_mtt - maximum number of memory translation table segments per HCA (int) num_udav - maximum number of UD address vectors per HCA (int) fmr_reserved_mtts - number of memory translation table segments reserved for FMR (int) =============================================================================== 2. Fixed Bugs =============================================================================== - IB_EVENT_LID_CHANGE is generated more appropriately. - Improved MTT buddy allocator (free count per order). - Fix check of max_send_sge for special QPs. - RESET->ERR QP state transition no longer supported (IB Spec 1.2.1). - Clear ICM pages before handing to FW. - Fixed race condition between create QP and destroy QP (bugzilla 1389) =============================================================================== 3. Known Issues =============================================================================== 1. A UAR size other than 8MB prevents mthca driver loading. The default UAR size is 8MB. If the size is changed, the following error message will be logged to /var/log/messages upon attempting to load the mthca driver: ib_mthca 0000:04:00.0: Missing UAR, aborting. 2. If a user level application using multicast receives a control signal in the process of detaching from a multicast group, its QP may remain a member of the multicast group (in HCA). Workaround: Destroy the multicast group after detaching the QP from it. 3. In mem-free devices, RC QPs can be created with a maximum of (max_sge - 1) entries only; UD QPs can be created with a maximum of (max_sge - 3) entries. 4. Performance can be degraded due to a wrong BIOS configuration: The PCI Express specification requires the BIOS to set the MaxReadReq register for each HCA card for maximum performance and stability. If you experience bandwidth performance degradation, try forcing the card to behave not according to the PCI Express specification by setting the tune_pci=1 module parameter. This tune_pci=1 assignment was the default setting in OFED 1.0; therefore, it may have masked performance degradation on some systems. If tune_pci=1 improves bandwidth, please report the issue to your BIOS vendor. Please note that Mellanox Technologies does not recommend using tune_pci=1 in production systems: working with tune_pci=1 set is untested and is known to trigger instability issues on some platforms.