원문 : http://www.ischo.net -- 조인상 //시스템 엔지니어

+++++++++++++++++++++++++++++++++++++++++++++++++++++++

Solaris 환경에서 SAN Multipath IO 를 구현하기 위하여 MPxIO(Multiplexed I/O) 를 사용한다.

 - 설정파일 : /kernel/drv/fp.conf, /kernel/drv/scsi_vhci.conf

 - 관련 command : stmsboot

1. MPxIO 기본 설정치로 구성하기

- MPxIO Disable

# stmsboot -d  

- MPxIO Enable

# stmsboot -e

# reboot                 // enable 후 reboot 해야 적용된다.

* stmsboot -e 로 enable 했을 경우 mpt(MPT Controller)와 fp(Fibre Channel) Controller 가 모두 적용되며

                   -D 옵션으로 특정 controller만 적용 시킬수 있다.

   MPT/SAS controller 까지 MPIO가 적용되면 internal disk까지 path 변경이 되기 때문에 SAN영역만 적용시키고자 할 경우 아래와 같은 명령어를 수행한다. 

  # stmsboot -D fp -e        //  Fibre Channel Controller만 enable

2. 특정 Port에만  MPxIO 설정하기

# luxadm -e port

Found path to 3 HBA ports

/devices/ssm@0,0/pci@1d,700000/fibre-channel@2/fp@0,0:devctl       CONNECTED
/devices/ssm@0,0/pci@1d,700000/fibre-channel@3/fp@0,0:devctl       CONNECTED
/devices/ssm@0,0/pci@1d,600000/fibre-channel@1/fp@0,0:devctl       CONNECTED

c2                             fc-fabric    connected    unconfigured unknown
 c2::500104f0008c6ee0           tape         connected    unconfigured unknown
 c3                             fc-fabric    connected    configured   unknown
 c3::210000e08b105616           unknown      connected    unconfigured unknown
 c3::50060e8000c45ad2           disk         connected    configured   unknown
 c4                             fc-fabric    connected    configured   unknown
 c4::210100e08b305616           unknown      connected    unconfigured unknown
 c4::50060e8000c45ad6           disk         connected    configured   unknown

# vi /kernel/drv/fp.conf

name="fp" parent="/pci@1d,700000/fibre-channel@2" port=0 mpxio-disable="no";
name="fp" parent="/pci@1d,700000/fibre-channel@3" port=0 mpxio-disable="no";

   -> /pci@6,2000/SUNW,qlc@2  와 /pci@13,2000/pci@2/SUNW,qlc@5 포트만 Multipath 적용

3.  MPxIO Failback 및 Load Balancing 설정

# vi /kernel/drv/scsi_vhci.conf

load-balance="round-robin";   // Enable Load Balancing with round-robin mode
load-balance="none";              // Diable Load Balancing

auto-failback="enable";           // Enable Failback
auto-failback="disable”;          // Disable Failback

--------------------------------------------------------------------------

4. SAN disk LUN 내역 확인방법

# luxadm disp /dev/rdsk/c6t50060E80000000000000A9A800000400d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c6t50060E80000000000000A9A800000400d0s2
  Vendor:               HITACHI
  Product ID:           OPEN-V      -SUN
  Revision:             2114
  Serial Num:           0450A9A80400
  Unformatted capacity: 22725.000 MBytes
  Write Cache:          Enabled
  Read Cache:           Enabled
    Minimum prefetch:   0x0
    Maximum prefetch:   0x0
  Device Type:          Disk device
  Path(s):

  /dev/rdsk/c6t50060E80000000000000A9A800000400d0s2
  /devices/scsi_vhci/ssd@g50060e80000000000000a9a800000400:c,raw
   Controller           /devices/pci@3d,700000/SUNW,qlc@1/fp@0,0
    Device Address              50060e8003a9a811,51
    Host controller port WWN    210000e08b08237d
    Class                       primary
    State                       ONLINE
   Controller           /devices/pci@7d,600000/SUNW,qlc@1/fp@0,0
    Device Address              50060e8003a9a821,51
    Host controller port WWN    210000e08b0cba13
    Class                       primary
    State                       ONLINE
   Controller           /devices/pci@7d,700000/SUNW,qlc@1/fp@0,0
    Device Address              50060e8003a9a831,51
    Host controller port WWN    21000000bea956c1
    Class                       primary
    State                       ONLINE
   Controller           /devices/pci@3d,600000/SUNW,qlc@1/fp@0,0
    Device Address              50060e8003a9a801,51
    Host controller port WWN    21000000bea956c1
    Class                       primary
    State                       ONLINE

- HBA port 당 LUN 표시

 # cfgadm -al -o show_SCSI_LUN 

- LUN 정보 및 LUN당 채널 정보

 # mpathadm list lun

- 추가된 LUN 정보 갱신

 # cfgadm -al -o show_FCP_dev

- 현재 HBA 포트 리스트 및 연결 되어 있는 정보 보기 

 # luxadm -e port list
 /devices/pci@3d,700000/SUNW,qlc@1/fp@0,0:devctl                    CONNECTED
 /devices/pci@3d,600000/SUNW,qlc@1/fp@0,0:devctl                    CONNECTED
 /devices/pci@7d,700000/SUNW,qlc@1/fp@0,0:devctl                    CONNECTED
 /devices/pci@7d,600000/SUNW,qlc@1/fp@0,0:devctl                    CONNECTED

# ls -l /dev/fc | grep /devices/pci@3d,700000/SUNW,qlc@1/fp@0,0:devctl

   lrwxrwxrwx 1 root root 52 May 15 12:25 fp0 -> ../../devices/pci@3d,700000/SUNW,qlc@1/fp@0,0:devctl

# # ls -l /dev/fc | grep /devices/pci@3d,600000/SUNW,qlc@1/fp@0,0:devctl
    lrwxrwxrwx 1 root root 52 May 15 12:25 fp1 -> ../../devices/pci@3d,600000/SUNW,qlc@1/fp@0,0:devctl

# luxadm -e dump_map /devices/pci@7d,600000/SUNW,qlc@1/fp@0,0:devctl
    Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
    0    11500   0         50060e8003a9a821 50060e8003a9a821 0x0  (Disk device)
    1    10c00   0         210000e08b0cba13 200000e08b0cba13 0x1f (Unknown Type,Host Bus Adapter)

 # mpathadm list initiator-port
 Initiator Port:  210000e08b08237d
 Initiator Port:  210000e08b0bd80f
 Initiator Port:  210000e08b0bdc0f
 Initiator Port:  210000e08b0cba13
 Initiator Port:  iqn.1986-03.com.sun:01:0000bea956c1.482bae18,4000002a00ff

# mpathadm show initiator-port 210000e08b08237d
 Initiator Port:  210000e08b08237d
         Transport Type:  Fibre Channel
         OS Device File:  /devices/pci@3d,700000/SUNW,qlc@1/fp@0,0

# fcinfo hba-port -l | grep "HBA Port WWN"
 HBA Port WWN: 210000e08b08237d
 HBA Port WWN: 210000e08b0bd80f
 HBA Port WWN: 210000e08b0bdc0f
 HBA Port WWN: 210000e08b0cba13

# fcinfo remote-port -p 210000e08b0cba13
 Remote Port WWN: 50060e8003a9a821
         Active FC4 Types: SCSI
         SCSI Target: yes
         Node WWN: 50060e8003a9a821

# fcinfo remote-port -p 210000e08b0cba13 -s
 Remote Port WWN: 50060e8003a9a821
         Active FC4 Types: SCSI
         SCSI Target: yes
         Node WWN: 50060e8003a9a821
         LUN: 0
           Vendor: HITACHI
           Product: OPEN-V      -SUN
           OS Device Name: /dev/rdsk/c6t50060E80000000000000A9A800000100d0s2
         LUN: 1
           Vendor: HITACHI
           Product: OPEN-V      -SUN
           OS Device Name: /dev/rdsk/c6t50060E80000000000000A9A800000101d0s2
         LUN: 2
           Vendor: HITACHI
           Product: OPEN-V      -SUN
           OS Device Name: /dev/rdsk/c6t50060E80000000000000A9A800000102d0s2