Switch Infiniband and Ethernet in DGX-1

Answer ID 4536   |    Updated 09/04/2017 05:52 PM

How to switch between Infiniband and Ethernet on DGX-1?

In the case of switching Infiniband and Ethernet.

Check current setting:
    lspci | grep -i mellanox

Prerequisite: 
    1. Download http://www.mellanox.com/downloads/MFT/mft-4.6.0-48-x86_64-deb.tgz
    2. Untar the downloaded package.
    3. Run ‘install.sh’ from mft-4.6.0-48-x86_64-deb/
    4. Start the mst driver: $ sudo mst start 

Case 1: Switch Ethernet to Infiniband
    1. $ sudo mlxconfig -d /dev/mst/mt4115_pciconf0 set LINK_TYPE_P1=1
    2. $ sudo mlxconfig -d /dev/mst/mt4115_pciconf1 set LINK_TYPE_P1=1
    3. $ sudo mlxconfig -d /dev/mst/mt4115_pciconf2 set LINK_TYPE_P1=1
    4. $ sudo mlxconfig -d /dev/mst/mt4115_pciconf3 set LINK_TYPE_P1=1
    5. Reboot machine to load new configurations

Case 2: Switch Infiniband to Ethernet
    1. $ sudo mlxconfig -d /dev/mst/mt4115_pciconf0 set LINK_TYPE_P1=2
    2. $ sudo mlxconfig -d /dev/mst/mt4115_pciconf1 set LINK_TYPE_P1=2
    3. $ sudo mlxconfig -d /dev/mst/mt4115_pciconf2 set LINK_TYPE_P1=2
    4. $ sudo mlxconfig -d /dev/mst/mt4115_pciconf3 set LINK_TYPE_P1=2
    5. Reboot machine to load new configurations
Was this answer helpful?
 
Your rating has been submitted, please tell us how we can make this answer more useful.

Print