During the day I work as a Unix sysadmin for a large company in the banking sector. Recently we've had a nightmare run of issues from our SAN and the systems attached (all from the same vendor). Given the chaos we've endured, I ordered some fibre channel HBA's and a string of fibre to learn more about a technology that underpins my work on a daily basis.
Unlike work, my available hosts are UltraSPARC and Opteron based but I figure the fundamentals will be the same. Much of the work here is comparable to the iSCSI COMSTAR tutorials written by the likes of Joerg Moellenkamp and Ben Rockwood, but just omitting the iSCSI element.
My adventures doing this would have been easier if I'd bugged Egg to write this excellent summary of fibre channel disk handling first.
The first step is to install your software and move your HBA into target mode. The software is installed with one command, pkg install storage-server.
For Emulex cards this is nice and simple. Edit /kernel/drv/emlxs.conf and change the following settings:-
target-mode=1;
ddi-forceattach=1
Save the file and reboot. Once restarted you can query the HBA with fcinfo.
root@ramjet:~# fcinfo hba-port
HBA Port WWN: 10000000c95eea8f
Port Mode: Target
Port ID: 0
OS Device Name: Not Applicable
Manufacturer: Emulex
Model: LP10000
Firmware Version: 1.91a1 (T2D1.91A1)
FCode/BIOS Version: 1.50a4
Serial Number: VM64819498
Driver Name: emlxs
Driver Version: 2.60f (2010.08.31.16.55)
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb
Current Speed: not established
Node WWN: 20000000c95eea8f
For Qlogic cards its a bit more involved.
First get the PCI device id's
# prtconf -D | grep qlc
pci1077,138, instance #0 (driver name: qlc)
pci1077,138, instance #1 (driver name: qlc)
Note I'm using a none Oracle card here, and one from another vendor to boot (Dell or HP i think). In my case I needed to do this for both pci1077,138 and pci1077,2432. Replace 'pci' with 'pciex' if your cards are PCI Express - I recently found out from re-applying this to a new card setup that it doesn't carry over.
update_drv -d -i 'pci1077,138' qlc
update_drv -a -i 'pci1077,138' qlt
If you are using genuine stock Oracle or QLogic cards you'll only need to do the below.
update_drv -d -i 'pci1077,2432' qlc
update_drv -a -i 'pci1077,2432' qlt
Save the file and reboot. Once restarted you can query the HBA with fcinfo.
HBA Port WWN: 21000024ff2ec367
Port Mode: Target
Port ID: 0
OS Device Name: Not Applicable
Manufacturer: QLogic Corp.
Model: QLE2462
Firmware Version: 0.0.0
FCode/BIOS Version: N/A
Serial Number: not available
Driver Name: COMSTAR QLT
Driver Version: 20100505-1.05
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 20000024ff2ec367
HBA Port WWN: 21000024ff2ec366
Port Mode: Target
Port ID: 0
OS Device Name: Not Applicable
Manufacturer: QLogic Corp.
Model: QLE2462
Firmware Version: 0.0.0
FCode/BIOS Version: N/A
Serial Number: not available
Driver Name: COMSTAR QLT
Driver Version: 20100505-1.05
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 20000024ff2ec366
Ensure the stmf service is running, if not, start it with svcadm enable stmf:default
root@ramjet:~# svcs stmf
STATE STIME FMRI
online 17:31:06 svc:/system/stmf:default
You can now start to configure your volumes. Check stmf has found your HBA:-
root@ramjet:~# stmfadm list-target
Target: wwn.10000000C95EEA8F
Create some backing ZFS volumes
root@ramjet:~# zfs create rpool/vmware_disk
root@ramjet:~# zfs create -V 10g rpool/vmware_disk/vol_01
root@ramjet:~# sbdadm create-lu /dev/zvol/rdsk/rpool/vmware_disk/vol_01
Created the following LU:
GUID DATA SIZE SOURCE
-------------------------------- ------------------- ----------------
600144f00003ba4040274d405b8a0001 10737418240 /dev/zvol/rdsk/rpool/vmware_disk/vol_01
root@ramjet:~# zfs create -V 10g rpool/vmware_disk/vol_02
root@ramjet:~# sbdadm create-lu /dev/zvol/rdsk/rpool/vmware_disk/vol_02
Created the following LU:
GUID DATA SIZE SOURCE
-------------------------------- ------------------- ----------------
600144f00003ba4040274d405ba20002 10737418240 /dev/zvol/rdsk/rpool/vmware_disk/vol_02
Sit back and admire your handiwork for a moment...
root@ramjet:~# stmfadm list-lu -v
LU Name: 600144F00003BA4040274D405B8A0001
Operational Status: Online
Provider Name : sbd
Alias : /dev/zvol/rdsk/rpool/vmware_disk/vol_01
View Entry Count : 0
Data File : /dev/zvol/rdsk/rpool/vmware_disk/vol_01
Meta File : not set
Size : 10737418240
Block Size : 512
Management URL : not set
Vendor ID : SUN
Product ID : COMSTAR
Serial Num : not set
Write Protect : Disabled
Writeback Cache : Enabled
Access State : Active
LU Name: 600144F00003BA4040274D405BA20002
Operational Status: Online
Provider Name : sbd
Alias : /dev/zvol/rdsk/rpool/vmware_disk/vol_02
View Entry Count : 0
Data File : /dev/zvol/rdsk/rpool/vmware_disk/vol_02
Meta File : not set
Size : 10737418240
Block Size : 512
Management URL : not set
Vendor ID : SUN
Product ID : COMSTAR
Serial Num : not set
Write Protect : Disabled
Writeback Cache : Enabled
Access State : Active
Of course, such output would be unwieldy in a big environment, this is neater:-
root@ramjet:~# stmfadm list-lu
LU Name: 600144F00003BA4040274D405B8A0001
LU Name: 600144F00003BA4040274D405BA20002
Add a 'view' to allow the target device to let initiators see storage - NOTE: this allows access from any host. There are controls to do this but I've not got that far and have no need of it at present. Update: see my later post about Target and host groups.
root@ramjet:~# stmfadm add-view 600144F00003BA4040274D405B8A0001
root@ramjet:~# stmfadm add-view 600144F00003BA4040274D405BA20002
So, power on a host with a fibre hba…
Jan 26 17:43:35 ramjet fct: NOTICE: emlxs0 LINK UP, portid 2, topology Private Loop,speed 2G
You can now allocate your LUN's as required!
No comments:
Post a Comment