Storsimple & Linux : iSCSI - MPIO 구성

[준비 사항]

1. 2개 이상의 NIC

2. iscsi-initiator-utils, device-mapper-multipath 패키지 설치

https://docs.microsoft.com/en-gb/azure/storsimple/storsimple-configure-mpio-on-linux

Step 1: Configure multipathing for automatic discovery

The multipath-supported devices can be automatically discovered and configured.

1. Initialize /etc/multipath.conf file. Type:

   mpathconf --enable

   The above command will create a sample/etc/multipath.conf file.

2. Start multipath service. Type:

   service multipathd start

   You will see the following output:

   Starting multipathd daemon:

3. Enable automatic discovery of multipaths. Type:

   mpathconf --find_multipaths y

   This will modify the defaults section of your multipath.conf as shown below:

    defaults {
    find_multipaths yes
    user_friendly_names yes
    path_grouping_policy multibus
    }
   

Step 2: Configure multipathing for StorSimple volumes

By default, all devices are black listed in the multipath.conf file and will be bypassed. You will need to create blacklist exceptions to allow multipathing for volumes from StorSimple devices.

1. Edit the /etc/mulitpath.conf file. Type:

   vi /etc/multipath.conf

2. Locate the blacklist_exceptions section in the multipath.conf file. Your StorSimple device needs to be listed as a blacklist exception in this section. You can uncomment relevant lines in this file to modify it as shown below (use only the specific model of the device you are using):

    blacklist_exceptions {
        device {
                   vendor  "MSFT"
                   product "STORSIMPLE 8100*"
        }
        device {
                   vendor  "MSFT"
                   product "STORSIMPLE 8600*"
        }
       }
   

Step 3: Configure round-robin multipathing

This load-balancing algorithm uses all the available multipaths to the active controller in a balanced, round-robin fashion.

1. Edit the /etc/multipath.conf file. Type:

   vi /etc/multipath.conf

2. Under the defaults section, set the path_grouping_policy to multibus. The path_grouping_policy specifies the default path grouping policy to apply to unspecified multipaths. The defaults section will look as shown below.

    defaults {
            user_friendly_names yes
            path_grouping_policy multibus
    }
   

Note

The most common values of path_grouping_policy include:

• failover = 1 path per priority group
• multibus = all valid paths in 1 priority group

Step 4: Enable multipathing

1. Restart the multipathd daemon. Type:

   service multipathd restart

2. The output will be as shown below:

    [root@centosSS ~]# service multipathd start
    Starting multipathd daemon:  [OK]
   

Step 5: Verify multipathing

1. First make sure that iSCSI connection is established with the StorSimple device as follows:

   a. Discover your StorSimple device. Type:

   iscsiadm -m discovery -t sendtargets -p  <IP address of network interface on the device>:<iSCSI port on StorSimple device>
   

   The output when IP address for DATA0 is 10.126.162.25 and port 3260 is opened on the StorSimple device for outbound iSCSI traffic is as shown below:

   10.126.162.25:3260,1 iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target
   10.126.162.26:3260,1 iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target
   

   Copy the IQN of your StorSimple device, iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, from the preceding output.

   b. Connect to the device using target IQN. The StorSimple device is the iSCSI target here. Type:

   iscsiadm -m node --login -T <IQN of iSCSI target>
   

   The following example shows output with a target IQN of iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target. The output indicates that you have successfully connected to the two iSCSI-enabled network interfaces on your device.

   Logging in to [iface: eth0, target: iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, portal: 10.126.162.25,3260] (multiple)
   Logging in to [iface: eth1, target: iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, portal: 10.126.162.25,3260] (multiple)
   Logging in to [iface: eth0, target: iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, portal: 10.126.162.26,3260] (multiple)
   Logging in to [iface: eth1, target: iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, portal: 10.126.162.26,3260] (multiple)
   Login to [iface: eth0, target: iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, portal: 10.126.162.25,3260] successful.
   Login to [iface: eth1, target: iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, portal: 10.126.162.25,3260] successful.
   Login to [iface: eth0, target: iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, portal: 10.126.162.26,3260] successful.
   Login to [iface: eth1, target: iqn.1991-05.com.microsoft:storsimple8100-shx0991003g00dv-target, portal: 10.126.162.26,3260] successful.
   

   If you see only one host interface and two paths here, then you need to enable both the interfaces on host for iSCSI. You can follow the detailed instructions in Linux documentation.

2. A volume is exposed to the CentOS server from the StorSimple device. For more information, see Step 6: Create a volume via the Azure portal on your StorSimple device.

3. Verify the available paths. Type:

   multipath –l
   

   The following example shows the output for two network interfaces on a StorSimple device connected to a single host network interface with two available paths.

    ```
    mpathb (36486fd20cc081f8dcd3fccb992d45a68) dm-3 MSFT,STORSIMPLE 8100
    size=100G features='0' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=0 status=active
    |- 7:0:0:1 sdc 8:32 active undef running
    `- 6:0:0:1 sdd 8:48 active undef running
    ```
   
    The following example shows the output for two network interfaces on a StorSimple device connected to two host network interfaces with four available paths.
   
    ```
    mpathb (36486fd27a23feba1b096226f11420f6b) dm-2 MSFT,STORSIMPLE 8100
    size=100G features='0' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=0 status=active
    |- 17:0:0:0 sdb 8:16 active undef running
    |- 15:0:0:0 sdd 8:48 active undef running
    |- 14:0:0:0 sdc 8:32 active undef running
    `- 16:0:0:0 sde 8:64 active undef running
    ```
   
    After the paths are configured, refer to the specific instructions on your host operating system (Centos 6.6) to mount and format this volume.
   

Troubleshoot multipathing

This section provides some helpful tips if you run into any issues during multipathing configuration.

Q. I do not see the changes in multipath.conf file taking effect.

A. If you have made any changes to the multipath.conf file, you will need to restart the multipathing service. Type the following command:

service multipathd restart

Q. I have enabled two network interfaces on the StorSimple device and two network interfaces on the host. When I list the available paths, I see only two paths. I expected to see four available paths.

A. Make sure that the two paths are on the same subnet and routable. If the network interfaces are on different vLANs and not routable, you will see only two paths. One way to verify this is to make sure that you can reach both the host interfaces from a network interface on the StorSimple device. You will need to contact Microsoft Support as this verification can only be done via a support session.

Q. When I list available paths, I do not see any output.

A. Typically, not seeing any multipathed paths suggests a problem with the multipathing daemon, and it’s most likely that any problem here lies in the multipath.conf file.

It would also be worth checking that you can actually see some disks after connecting to the target, as no response from the multipath listings could also mean you don’t have any disks.

• Use the following command to rescan the SCSI bus:

  $ rescan-scsi-bus.sh(part of sg3_utils package)

• Type the following commands:

  $ dmesg | grep sd*

  Or

  $ fdisk –l

  These will return details of recently added disks.

• To determine whether it is a StorSimple disk, use the following commands:

  cat /sys/block/<DISK>/device/model

  This will return a string, which will determine if it’s a StorSimple disk.

A less likely but possible cause could also be stale iscsid pid. Use the following command to log off from the iSCSI sessions:

iscsiadm -m node --logout -p <Target_IP>

Repeat this command for all the connected network interfaces on the iSCSI target, which is your StorSimple device. Once you have logged off from all the iSCSI sessions, use the iSCSI target IQN to reestablish the iSCSI session. Type the following command:

iscsiadm -m node --login -T <TARGET_IQN>

Q. I am not sure if my device is whitelisted.

A. To verify whether your device is whitelisted, use the following troubleshooting interactive command:

multipathd –k
multipathd> show devices
available block devices:
ram0 devnode blacklisted, unmonitored
ram1 devnode blacklisted, unmonitored
ram2 devnode blacklisted, unmonitored
ram3 devnode blacklisted, unmonitored
ram4 devnode blacklisted, unmonitored
ram5 devnode blacklisted, unmonitored
ram6 devnode blacklisted, unmonitored
ram7 devnode blacklisted, unmonitored
ram8 devnode blacklisted, unmonitored
ram9 devnode blacklisted, unmonitored
ram10 devnode blacklisted, unmonitored
ram11 devnode blacklisted, unmonitored
ram12 devnode blacklisted, unmonitored
ram13 devnode blacklisted, unmonitored
ram14 devnode blacklisted, unmonitored
ram15 devnode blacklisted, unmonitored
loop0 devnode blacklisted, unmonitored
loop1 devnode blacklisted, unmonitored
loop2 devnode blacklisted, unmonitored
loop3 devnode blacklisted, unmonitored
loop4 devnode blacklisted, unmonitored
loop5 devnode blacklisted, unmonitored
loop6 devnode blacklisted, unmonitored
loop7 devnode blacklisted, unmonitored
sr0 devnode blacklisted, unmonitored
sda devnode whitelisted, monitored
dm-0 devnode blacklisted, unmonitored
dm-1 devnode blacklisted, unmonitored
dm-2 devnode blacklisted, unmonitored
sdb devnode whitelisted, monitored
sdc devnode whitelisted, monitored
dm-3 devnode blacklisted, unmonitored

For more information, go to use troubleshooting interactive command for multipathing.

List of useful commands

Type	Command	Description
iSCSI	service iscsid start	Start iSCSI service
	service iscsid stop	Stop iSCSI service
	service iscsid restart	Restart iSCSI service
	iscsiadm -m discovery -t sendtargets -p <TARGET_IP>	Discover available targets on the specified address
	iscsiadm -m node --login -T <TARGET_IQN>	Log in to the iSCSI target
	iscsiadm -m node --logout -p <Target_IP>	Log out from the iSCSI target
	cat /etc/iscsi/initiatorname.iscsi	Print iSCSI initiator name
	iscsiadm –m session –s <sessionid> -P 3	Check the state of the iSCSI session and volume discovered on the host
	iscsi –m session	Shows all the iSCSI sessions established between the host and the StorSimple device
Multipathing	service multipathd start	Start multipath daemon
	service multipathd stop	Stop multipath daemon
	service multipathd restart	Restart multipath daemon
	chkconfig multipathd on  OR  mpathconf –with_chkconfig y	Enable multipath daemon to start at boot time
	multipathd –k	Start the interactive console for troubleshooting
	multipath –l	List multipath connections and devices
	mpathconf --enable	Create a sample mulitpath.conf file in /etc/mulitpath.conf