Nvidia DGX H100 Service Manual

Chapter 12. Updating the ConnectX-7
Firmware
After replacing or installing the ConnectX-7 cards, make sure the rmware on the cards is up to date.
1. Download the v28.36.2024 rmware from https://network.nvidia.com/support/firmware/
connectx7ib/.
Download the rmware for both OPN options.
2. Transfer the rmware ZIP le to the DGX system and extract the archive.
3. Update the rmware on the cards that are used for cluster communication:
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:5e:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX750500B-0D00_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:dc:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX750500B-0D00_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:c0:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX750500B-0D00_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:18:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX750500B-0D00_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:40:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX750500B-0D00_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:4f:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX750500B-0D00_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:ce:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX750500B-0D00_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:9a:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX750500B-0D00_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
4. Update the rmware on the cards that are used for storage communication:
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:aa:00.0∕config -i fw-ConnectX7-rel-28_
,→36_2024-MCX755206AS-NEA_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
sudo mstflint -d ∕sys∕bus∕pci∕devices∕0000:29:00.0∕config -i fw-ConnectX7-rel-28_
,
→
36_2024-MCX755206AS-NEA_Ax-UEFI-14.29.14-FlexBoot-3.6.901.signed.bin b
5. Reboot the system for the rmware update to take eect:
sudo reboot
6. After the system starts, log in and conrm the rmware versions are all the same:
$ cat ∕sys∕class∕infiniband∕mlx5_*∕fw_ver
73

Nvidia DGX H100 Specifications

General

Brand	Nvidia
Model	DGX H100
Category	Computer Hardware
Language	English