Implementing FCoE in the Linux Operating System This white paper explains using Broadcom-based network adapters and Dell Force10 and Cisco Nexus switches. Authors: Alex Aguila, Product Test Group Major Contributor: Tony Molina, Product Test Group...
Implementing FCoE in the Linux Operating System Contents Executive summary ....................5 Introduction ......................5 Described configuration ..................... 5 Setting up your network infrastructure for FCoE ..............7 Setting up FCoE for SLES 11 SP 2 .................. 12 Installing / verifying FCoE support ................12 Configure the FCoE client VLAN ..................
Page 4
Implementing FCoE in the Linux Operating System Figure 18. Failure to detect installation media..............21 Figure 19. Entering Software Repositories Menu..............22 Figure 20. Deleting Old SLES Software Repository.............. 23 Figure 21. Adding new SLES Install Media Software Repository..........23 Figure 22.
Implementing FCoE in the Linux Operating System Executive summary This document provides instructions on setting up Fibre Channel over Ethernet (FCoE) using a Dell Force10 switch, a Cisco Nexus switch, Broadcom-based network adapters, and servers running supported Linux operating systems. It is intended for intermediate Network and System Administrators with 2-5 years of experience, and assumes a moderate level of network switch administration expertise and Linux system administration expertise.
Page 6
57810S Dual-Port 10GbE KR Blade Converged Mezzanine Card firmware version 7.2.14 was used. 4. Install the appropriate Linux Operating System and make sure it is functioning on the Dell PowerEdge server. In addition, the Ethernet interface over which you want FCoE traffic to flow must be configured and online.
Implementing FCoE in the Linux Operating System Setting up your network infrastructure for FCoE Perform the following steps to set up your network infrastructure for FCoE. While performing these steps, refer to the network diagram in Figure 1. Figure 1. Network infrastructure diagram.
Page 8
Implementing FCoE in the Linux Operating System 2. Enable the FCoE feature on the Dell Force10 MXL 10/40GbE edge switch by running the following commands from the Global Configuration Mode of the edge switch command line: service-class dynamic dot1p cam-acl l2acl 4 ipv4acl 2 ipv6acl 0 ipv4qos 2 l2qos 1 l2pt 0 ipmacacl 0 vman-qos 0 ecfmacl 0...
Page 9
(VFC 50 being placed into VSAN 255) VSAN 255 interface vfc 50 11. On the Dell Force10 MXL 10/40GbE (edge) switch, verify that an FCoE connection was established. To do this, run the following command on the edge switch command line while in EXEC mode;...
Implementing FCoE in the Linux Operating System Figure 2. Results of show fip-snooping enode command. 12. Verify that the Fibre Channel forwarder port is enabled and working correctly on the edge switch. Run the following command from the edge switch command line while in EXEC mode (see Figure 3 for results): show fip-snooping fcf Figure 3.
Implementing FCoE in the Linux Operating System Setting up FCoE for SLES 11 SP 2 Installing / verifying FCoE support If FCoE support has not been installed in SLES 11 SP2, it must be installed. To verify if FCoE support was installed in SLES 11 SP2, perform the following steps: 1.
Implementing FCoE in the Linux Operating System 2. To install FCoE and the appropriate support packages, insert Disc 1 of the SLES 11 SP2 installation media into your CD/DVD drive. Next, from YaST2, select Software from the left- side menu. Then select Software Management in the right-side menu, and press Enter, as in Figure 6.
Implementing FCoE in the Linux Operating System Figure 7. Selecting FCoE packages for installation. You may be prompted to accept any Automatic Changes to resolve dependency issues; if this happens, select OK to allow the installation to proceed. After the packages install, you are returned to the YaST2 Control Center.
Implementing FCoE in the Linux Operating System Figure 8. Network Services menu. 2. You are brought to the Fibre Channel over Ethernet Configuration screen. Use the Tab key to select the Interfaces sub-menu, as in Figure 9. Figure 9. FCoE client configuration screen with interfaces sub menu. Note that the supported Broadcom network adapters have yes listed in the FCoE column, while unsupported Broadcom network adapters will not have any listing in the FCoE column.
Implementing FCoE in the Linux Operating System Figure 10. Create FCoE VLAN interface. 4. You are prompted to confirm whether or not you really want to create a new FCoE VLAN interface, as in Figure 11; select Yes . Figure 11. Confirmation of new FCoE VLAN interface.
Implementing FCoE in the Linux Operating System Figure 12. New FCoE VLAN created. 5. Select OK , and then Quit to return to the command prompt. Run the command fcoeadm -i to verify that the FCoE VLAN has been successfully brought online, as in Figure 13. Figure 13.
Implementing FCoE in the Linux Operating System 6. Finally, to determine the device names for your FCoE LUNs, run the cat /proc/partitions command, as in Figure 14. In this example, the FCoE LUNs are listed as /dev/sdb and /dev/sdc . Figure 14.
Implementing FCoE in the Linux Operating System Figure 15. Select the FCoE Client Configuration menu. 2. Configure FCoE to start when the system boots. Select the Services menu. Then set both FCoE Services Start and Lldpad Services Start to When Booting , as in Figure 16.
Implementing FCoE in the Linux Operating System Figure 16. Configure FCoE to start at system boot. 3. Click OK and then Quit to return to the command prompt. The FCoE client should now start as part of SLES 11 SP2s boot process, and the FCoE LUNs should be available after the system boots.
Implementing FCoE in the Linux Operating System Figure 17. Restarting the rclldpad and rcfcoe daemons. In some SLES 11 SP2 installations, YaST2 fails to properly detect the SLES 11 SP2 installation media, as in Figure 18. Figure 18. Failure to detect installation media. The failure to detect installation media is most commonly due to the SLES 11 SP2s configuration files pointing to the incorrect location of the installation media software repository.
Implementing FCoE in the Linux Operating System Figure 19. Entering Software Repositories Menu. 2. Delete the existing installation media software repository, and then add the SLES 11 SP2 installation media as a new software repository, as in Figures 20 and 21.
Implementing FCoE in the Linux Operating System Figure 20. Deleting Old SLES Software Repository. Figure 21. Adding new SLES Install Media Software Repository. 3. Select the appropriate installation media, and then select Next , as in Figure 22. 4. Accept the License Agreement , select Next , and then select OK , and you will be returned to the YaST2 Control Center, as in Figure 23.
Implementing FCoE in the Linux Operating System Setting-up FCoE for RHEL 6.2 Installing / verifying FCoE support If FCoE support was not installed in RHEL 6.2, install it. To verify if FCoE support was installed in RHEL 6.2, perform the following steps: 1.
Implementing FCoE in the Linux Operating System 4. After package installation is complete, you can then change back to the home directory and unmount your installation media by running the cd ~ and umount /media command, as in Figure 27. Figure 27.
Implementing FCoE in the Linux Operating System Figure 30. Checking the SUPPORTED_DRIVERS variable. 4. Then, start the lldpad and fcoe services running the service lldpad start and service fcoe start commands. Then running the fcoeadm -i command to verify that the FCoE VLAN is properly configured, as in Figure 31.
Implementing FCoE in the Linux Operating System Figure 32. Verifying LUNs availability. Partition and mount your FCoE LUNs as any other hard disk device. Configure the FCoE client to start at boot If you want the FCoE client to start and the FCoE LUNs to automatically be available after every reboot, configure the appropriate services using chkconfig .
Implementing FCoE in the Linux Operating System Figure 35. Restarting the lldpad and fcoe daemons. Conclusion While it is not possible to cover every conceivable combination of FCoE hardware in a single document, most modern FCoE implementations are likely be fairly similar on the various available FCoE compliant devices.