Checking InfiniBand

If one of your machines has an InfiniBand device installed and you want to know what state the device is in, you can use the “ibstat” command. The output of “ibstat” shows a lot of information, but the two main lines you should look at are:

State: Active
Physical state: LinkUp

The “State” line can have several statuses:

• “Down”  (there is no physical connection detected)
• “Initializing” (the physical connection was detected but no subnet manager was found)
• “Active” (both a physical connection and subnet manager were detected)

The “Physical state” line can show a status of:

• “Polling” (there is no physical connection detected between this device and another)
• “LinkUp” (a connection between this device and another was detected)

Knowing the differences between each state can make diagnosis of InfiniBand issues much easier, whether the issue is related to a bad cable, a cable that needs to be reseated, or a subnet manager service that needs to be restarted.