A special need..
Those of you who have a need for exposing iscsi targets to other machines for discovery, but who do not want to invest in an external USB hard disk OR who do not want to have a USB external hard disk connected to a desktop 24x7, there is a better way of creating and exposing iscsi target logical volumes.
This method does not need installing openfiler either as noted in my earlier article Combining Openfiler and Virtualbox (Ubuntu guest OS on windows host).
A simple solution..
The method is simple: create logical volumes based on SCSI devices on a Ubuntu Hardy Heron 8.04 installation using virtualbox.
The advantage of using Ubuntu is that it creates the hard disk devices as scsi devices (with the naming convention /dev/sd*, instead of /dev/hd*). I have seen that in other unix OS like SuSE linux, the local disks are listed as /dev/hd* (IDE). Ubuntu seems to have better disk drivers.
If you use Hardy Heron 8.04 Ubuntu, it is *really* easy to get iscsitarget package working. If you try to make the iscsi-target package work with Gutsy Gibbon 7.x release of Ubuntu, there is a very good chance that you will run into lot of compilation issues. I went down this path myself and later realized that a lot of bugs existed for Gutsy Gibbon release. (e.g. https://bugs.launchpad.net/ubuntu/gutsy/+source/iscsitarget/+bug/160104, http://ubuntuforums.org/showthread.php?t=692651) and eventually found https://bugs.launchpad.net/ubuntu/+source/iscsitarget/+bug/145539 which said that the module was fixed in Hardy Heron 8.04.
So instead of breaking my head over making iscsi-target package work in Gutsy Gibbon 2.6.22, I decided to give Hardy Heron 8.04 (still beta) a try.
Thankfully, with a little effort, I was able to make it work. In this article, I present a simplistic scenario in which we create three logical volumes that can be easily discovered by another virtualbox iscsi initiator machine(s) using open-iscsi package. This is the beauty of the entire approach.
Here is a block diagram of the end configuration:
Click here for a bigger view
Iscsi support in Hardy Heron 8.04 Ubuntu..
The first thing to understand is that as per the published features of Hardy Heron 8.04 Ubuntu at https://wiki.ubuntu.com/HardyHeron/Beta#head-da07b62e1e43afd0bef06ab8b60d2502c734a0f9, the iscsi support is enabled out of the box if we add iscsi=true in the boot options during the installation.
Click here for a bigger view
While installing Hardy 8.04 OS using virtualbox or any other virtualization software being used, remember to add iscsi=true in the boot options (after pressing F6)
Click here for a bigger view
Configuring iscsi targets..
I would like to give credit to a really awesome link I found through google: http://www.linuxconfig.org/Linux_lvm_-_Logical_Volume_Manager, which I rate as a one of the best article s for doing this!
Some other relevant links are:
https://help.ubuntu.com/community/SettingUpLVM-WithoutACleanInstall
http://t3flyers.wordpress.com/2007/04/24/logical-volume-manager-on-ubuntu-feisty-704/
http://www.howtoforge.com/linux_lvm
After Ubuntu installation, the output of uname -a looks like this:
HARDY# uname -a Linux gverma-laptop 2.6.24-16-generic #1 SMP Thu Apr 10 13:23:42 UTC 2008 i686 GNU/Linux
And the output of fdisk -l looks like this:
HARDY# fdisk -l Disk /dev/sda: 16.5 GB, 16592666624 bytes 255 heads, 63 sectors/track, 2017 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x0004ccc2 Device Boot Start End Blocks Id System /dev/sda1 * 1 1927 15478596 83 Linux /dev/sda2 1928 2017 722925 5 Extended /dev/sda5 1928 2017 722893+ 82 Linux swap / Solaris
At this point, I added two more hard disks to the virtualbox virtual machine. Virtualbox 1.5.6 allows upto three hard disk devices to be attached to a machine.
Click here for a bigger view
After booting up HARDY again, /dev/sdb and /dev/sdc were also visible in the output of fdisk -l. Using fdisk, I created a single partition in both devices that spanned the entire device, resulting in /dev/sdb1 and /dev/sdc1:
HARDY# fdisk -l ... Disk /dev/sdb: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xd310045f Device Boot Start End Blocks Id System /dev/sdb1 1 2610 20964793+ 83 Linux Disk /dev/sdc: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x93d4c692 Device Boot Start End Blocks Id System /dev/sdc1 1 2610 20964793+ 83 Linux
Also make sure that you configure host only networking so that other virtual machines (scsi initiators) are able to ssh/telnet into HARDY. Please look at article to understand how host only networking can be setup: Virtualbox Case Study: Making host only networking work between two Ubuntu Guest OS (virtual machine) on Windows Vista host
Getting the right packages installed..
Now, we need to create the target physical volume, virtual group, and logical volume, in that order. These logical volumes will serve as SCSI targets that can be discovered by other iscsi initiator machines or virtual machines as per your configuration.
Make sure you reload the package definitions from the configured repositories (get the latest packages that are published from the ubuntu repositories). You can do this from the Synaptic Package Manager:
Click here for a bigger view
Now, you should install the lvm2 package in Synaptic Package Manager:
Click here for a bigger view
Create the physical volumes:
HARDY# lvm lvm> pvcreate /dev/sdb1 Physical volume "/dev/sdb1" successfully created lvm> pvcreate /dev/sdc1 Physical volume "/dev/sdc1" successfully created lvm> pvdisplay --- NEW Physical volume --- PV Name /dev/sdb1 VG Name PV Size 19.99 GB Allocatable NO PE Size (KByte) 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID OHKhOq-AMG7-XtYz-7CrP-q2VH-2b53-tW0yMO --- NEW Physical volume --- PV Name /dev/sdc1 VG Name PV Size 19.99 GB Allocatable NO PE Size (KByte) 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID M9qXXB-GVxC-FZjI-9Zb7-IhcH-rbhb-vQ3aek
Create the virtual group:
lvm> vgcreate vg /dev/sdb1 /dev/sdc1 Volume group "vg" successfully created lvm> vgdisplay --- Volume group --- VG Name vg System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 2 Act PV 2 VG Size 39.98 GB PE Size 4.00 MB Total PE 10236 Alloc PE / Size 0 / 0 Free PE / Size 10236 / 39.98 GB VG UUID GkUMNq-3atR-qKTK-lG0b-gM0n-budV-Uc4lH8 lvm>
Create logial volumes:
lvm> lvcreate -L 1G -n ocr vg Logical volume "ocr" created lvm> lvcreate -L 1G -n vote vg Logical volume "vote" created lvm> lvcreate -L 35G -n asm vg Logical volume "asm" created
If you get an error message saying:
/proc/misc: No entry for device-mapper
you can get around it by issuing sudo modprobe dm-mod
lvm> lvdisplay --- Logical volume --- LV Name /dev/vg/ocr VG Name vg LV UUID a2ARIQ-11dn-kdoB-cHCd-gtoR-aokO-HGqoo4 LV Write Access read/write LV Status available # open 0 LV Size 1.00 GB Current LE 256 Segments 1 Allocation inherit Read ahead sectors 0 Block device 254:0 --- Logical volume --- LV Name /dev/vg/vote VG Name vg LV UUID 1DqYL0-Ptsx-9kmE-UHmn-PmE9-ebVm-bfr0r0 LV Write Access read/write LV Status available # open 0 LV Size 1.00 GB Current LE 256 Segments 1 Allocation inherit Read ahead sectors 0 Block device 254:1 --- Logical volume --- LV Name /dev/vg/asm VG Name vg LV UUID UKWcPz-aNGl-VdPm-51wa-Ewvm-ahvD-Y7N57v LV Write Access read/write LV Status available # open 0 LV Size 35.00 GB Current LE 8960 Segments 2 Allocation inherit Read ahead sectors 0 Block device 254:2 lvm>
Now, you can check the logical volume devices like this:
HARDY# ls -l /dev/vg total 0 lrwxrwxrwx 1 root root 18 2008-05-03 02:42 asm -> /dev/mapper/vg-asm lrwxrwxrwx 1 root root 18 2008-05-03 02:41 ocr -> /dev/mapper/vg-ocr lrwxrwxrwx 1 root root 19 2008-05-03 02:41 vote -> /dev/mapper/vg-vote
You can also check the newly created devices in the /dev/disk/* directories on HARDY:
HARDY# ls -l /dev/disk/* /dev/disk/by-id: .. lrwxrwxrwx 1 root root 19 2008-05-03 02:42 dm-name-vg-asm -> ../../mapper/vg-asm lrwxrwxrwx 1 root root 19 2008-05-03 02:41 dm-name-vg-ocr -> ../../mapper/vg-ocr lrwxrwxrwx 1 root root 20 2008-05-03 02:41 dm-name-vg-vote -> ../../mapper/vg-vote .. /dev/disk/by-path: total 0 lrwxrwxrwx 1 root root 9 2008-05-02 22:30 pci-0000:00:01.1-scsi-0:0:1:0 -> ../../sdb lrwxrwxrwx 1 root root 10 2008-05-03 02:34 pci-0000:00:01.1-scsi-0:0:1:0-part1 -> ../../sdb1 lrwxrwxrwx 1 root root 9 2008-05-02 22:30 pci-0000:00:01.1-scsi-1:0:1:0 -> ../../sdc lrwxrwxrwx 1 root root 10 2008-05-03 02:35 pci-0000:00:01.1-scsi-1:0:1:0-part1 -> ../../sdc1
You need to install the iscsitarget package using Synaptic Package Manager:
Click here for a bigger view
Now, you need to configure the /etc/ietd.conf file with the Target names and entries for each logical volume. The target names just need to unique within the network and are totally upto your imagination.
HARDY# more /etc/ietd.conf Target iqn.2001-04.com.ubuntu:scsi.disk.vg.vote Lun 0 Path=/dev/vg/vote,Type=fileio Target iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr Lun 0 Path=/dev/vg/ocr,Type=fileio Target iqn.2001-04.com.ubuntu:scsi.disk.vg.asm Lun 0 Path=/dev/vg/asm,Type=fileio
To enforce these entries, we need to restart the iscsitarget service manually now.
Keep in mind that there is no dash (-) between iscsi and target in the name of the service, as in openfiler 2.2 (openfiler's IET implementation service has the name iscsi-target)
HARDY# /etc/init.d/iscsitarget restart Removing iSCSI enterprise target devices: succeeded. Stopping iSCSI enterprise target service: succeeded. Removing iSCSI enterprise target modules: succeeded. Starting iSCSI enterprise target service: succeeded.
To double check if the logical volumes have been discovered and published, you can check content of /proc/net/iet/volume on HARDY:
:::::::::::::: /proc/net/iet/volume :::::::::::::: tid:3 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.asm lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/asm tid:2 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/ocr tid:1 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.vote lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/vote
If you do a more of /proc/net/iet/session, you can clearly see that no iscsi-initiator machine have connected to HARDY yet (there are no session sub-entries under the volume name):
HARDY# more /proc/net/iet/session tid:3 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.asm tid:2 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr tid:1 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.vote
Here is a quick check of the physical volume, volume group, and logical volume :
HARDY# lvm
lvm> lvscan
ACTIVE '/dev/vg/ocr' [1.00 GB] inherit
ACTIVE '/dev/vg/vote' [1.00 GB] inherit
ACTIVE '/dev/vg/asm' [35.00 GB] inherit
lvm> pvscan
PV /dev/sdb1 VG vg lvm2 [19.99 GB / 0 free]
PV /dev/sdc1 VG vg lvm2 [19.99 GB / 2.98 GB free]
Total: 2 [39.98 GB] / in use: 2 [39.98 GB] / in no VG: 0 [0 ]
lvm> vgscan
Reading all physical volumes. This may take a while...
Found volume group "vg" using metadata type lvm2
lvm>
Discovering the iscsi targets from another iscsi initiator machine..
We will assume that the iscsi initiator machine name is GUTSY and it has the open-iscsi package installed (this also installs iscsiadm utility along with it). We will also assume that the IP of HARDY (iscsi target) is 192.168.0.7.
You can leave the majority default values in /etc/iscsi.conf. Make sure you disable the CHAP authentication parameter values since we have not configured them in the scsi target.
Discover the new iscsi targets in HARDY:
GUTSY# iscsiadm -m discovery -t st -p 192.168.0.7 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.asm 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.vote
Verify the newly discovered target machine:
GUTSY# sudo iscsiadm -m discovery
192.168.0.6:3260 via sendtargets
192.168.0.7:3260 via sendtargets --> the new iscsi target got added to the local databaseCheck the combination of discovered iscsi-target and logical volumes (the new ones are in BLUE color):
GUTSY# sudo iscsiadm -m node
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.ocr
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.vote
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.asm
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.voteBut, at this point, we have no active sessions connected to these targets:
GUTSY# iscsiadm -m session iscsiadm: No active sessions.
Establish connection to the scsi targets from the initiator:
GUTSY# iscsiadm -m node \ -T iqn.2001-04.com.ubuntu:scsi.disk.vg.asm \ -p 192.168.0.7 -l Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.asm, portal: 192.168.0.7,3260] GUTSY# iscsiadm -m node \ -T iqn.2001-04.com.ubuntu:scsi.disk.vg.vote \ -p 192.168.0.7 -l Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.vote, portal: 192.168.0.7,3260] GUTSY# iscsiadm -m node \ -T iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr \ -p 192.168.0.7 -l Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr, portal: 192.168.0.7,3260]
Verify the newly formed connections:
GUTSY# iscsiadm -m session tcp: [1] 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.asm tcp: [2] 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.vote tcp: [3] 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
It would be useful to enable automatic startup/discovery of these volumes on server and client. For this, you need to do the following:
- On server(where the logical volumes are created):
HARDY# sudo gedit /etc/iscsi/iscsid.conf
Change node.startup=manual to: node.startup = Automatic
GUTSY# sudo /etc/init.d/open-iscsi restart
- On client
GUTSY# sudo iscsiadm -m discovery -t st -p <ipaddress> GUTSY# sudo iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg.asm -p <ipaddress> -o update -n node.conn[0].startup -v automatic GUTSY# sudo /etc/init.d/open-iscsi restart
- or reboot both the systems if required (make sure server boots first). To check if the scsi devices are automatically setup and list all your scsi iqns:
GUTSY # sudo iscsiadm -m session
Now assumed that the TCP scsi connections have been formed, let us check if the scsi devices are visible from fdisk -l:
GUTSY# fdisk -l .. Disk /dev/sdb: 37.5 GB, 37580963840 bytes 64 heads, 32 sectors/track, 35840 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Disk identifier: 0x00000000 Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 1073 MB, 1073741824 bytes 34 heads, 61 sectors/track, 1011 cylinders Units = cylinders of 2074 * 512 = 1061888 bytes Disk identifier: 0x00000000 Disk /dev/sdc doesn't contain a valid partition table Disk /dev/sdd: 1073 MB, 1073741824 bytes 34 heads, 61 sectors/track, 1011 cylinders Units = cylinders of 2074 * 512 = 1061888 bytes Disk identifier: 0x00000000 Disk /dev/sdd doesn't contain a valid partition table GUTSY#
The newly discovered devices can also be checked under the contents of /dev/disk/*:
GUTSY# ls -l /dev/disk/* /dev/disk/by-id: total 0 .. lrwxrwxrwx 1 root root 9 2008-05-02 23:05 scsi-149455400000000000000000001000000795600000e000000 -> ../../sdc lrwxrwxrwx 1 root root 9 2008-05-02 23:05 scsi-1494554000000000000000000020000004f5600000e000000 -> ../../sdd lrwxrwxrwx 1 root root 9 2008-05-02 23:05 scsi-149455400000000000000000003000000a45600000e000000 -> ../../sdb .. /dev/disk/by-path: total 0 lrwxrwxrwx 1 root root 9 2008-05-02 23:05 ip-192.168.0.7:3260-iscsi-iqn.2001-04.com.ubuntu:scsi.disk.vg.asm-lun-0 -> ../../sdb lrwxrwxrwx 1 root root 9 2008-05-02 23:05 ip-192.168.0.7:3260-iscsi-iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr-lun-0 -> ../../sdd lrwxrwxrwx 1 root root 9 2008-05-02 23:05 ip-192.168.0.7:3260-iscsi-iqn.2001-04.com.ubuntu:scsi.disk.vg.vote-lun-0 -> ../../sdc ..
Relevant messages in the system log of scsi initiator (GUTSY)
# dmesg ... [ 1156.290868] scsi2 : iSCSI Initiator over TCP/IP [ 1156.607026] scsi 2:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4 [ 1156.607026] sd 2:0:0:0: [sdb] 73400320 512-byte hardware sectors (37581 MB) [ 1156.607026] sd 2:0:0:0: [sdb] Write Protect is off [ 1156.607026] sd 2:0:0:0: [sdb] Mode Sense: 77 00 00 08 [ 1156.607026] sd 2:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 1156.607111] sd 2:0:0:0: [sdb] 73400320 512-byte hardware sectors (37581 MB) [ 1156.608275] sd 2:0:0:0: [sdb] Write Protect is off [ 1156.608293] sd 2:0:0:0: [sdb] Mode Sense: 77 00 00 08 [ 1156.610635] sd 2:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 1156.610665] sdb: unknown partition table [ 1156.619320] sd 2:0:0:0: [sdb] Attached SCSI disk [ 1156.619398] sd 2:0:0:0: Attached scsi generic sg2 type 0 [ 1163.865083] scsi3 : iSCSI Initiator over TCP/IP [ 1164.132972] scsi 3:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4 [ 1164.135287] sd 3:0:0:0: [sdc] 2097152 512-byte hardware sectors (1074 MB) [ 1164.136424] sd 3:0:0:0: [sdc] Write Protect is off [ 1164.136438] sd 3:0:0:0: [sdc] Mode Sense: 77 00 00 08 [ 1164.138002] sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 1164.140341] sd 3:0:0:0: [sdc] 2097152 512-byte hardware sectors (1074 MB) [ 1164.141443] sd 3:0:0:0: [sdc] Write Protect is off [ 1164.141460] sd 3:0:0:0: [sdc] Mode Sense: 77 00 00 08 [ 1164.144746] sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 1164.144809] sdc: unknown partition table [ 1164.154241] sd 3:0:0:0: [sdc] Attached SCSI disk [ 1164.154290] sd 3:0:0:0: Attached scsi generic sg3 type 0 [ 1170.533158] scsi4 : iSCSI Initiator over TCP/IP [ 1170.798343] scsi 4:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4 [ 1170.798343] sd 4:0:0:0: [sdd] 2097152 512-byte hardware sectors (1074 MB) [ 1170.801229] sd 4:0:0:0: [sdd] Write Protect is off [ 1170.801263] sd 4:0:0:0: [sdd] Mode Sense: 77 00 00 08 [ 1170.806093] sd 4:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 1170.809875] sd 4:0:0:0: [sdd] 2097152 512-byte hardware sectors (1074 MB) [ 1170.812347] sd 4:0:0:0: [sdd] Write Protect is off [ 1170.812380] sd 4:0:0:0: [sdd] Mode Sense: 77 00 00 08 [ 1170.816757] sd 4:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA [ 1170.816757] sdd: unknown partition table [ 1170.828375] sd 4:0:0:0: [sdd] Attached SCSI disk [ 1170.828446] sd 4:0:0:0: Attached scsi generic sg4 type 0
Cross checking incoming sessions at scsi target (HARDY)..
In the meanwhile, in HARDY, the incoming sessions can be checked by:
HARDY# more /proc/net/iet/* :::::::::::::: /proc/net/iet/session :::::::::::::: tid:3 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.asm sid:281474997486080 initiator:iqn.1993-08.org.debian:01:950a218cdd1 cid:0 ip:192.168.0.6 state:active hd:none dd:none tid:2 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr sid:844424984461824 initiator:iqn.1993-08.org.debian:01:950a218cdd1 cid:0 ip:192.168.0.6 state:active hd:none dd:none tid:1 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.vote sid:562949990973952 initiator:iqn.1993-08.org.debian:01:950a218cdd1 cid:0 ip:192.168.0.6 state:active hd:none dd:none :::::::::::::: /proc/net/iet/volume :::::::::::::: tid:3 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.asm lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/asm tid:2 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/ocr tid:1 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.vote lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/vote root@gverma-laptop:/home/gverma#
Conclusion..
Congratulations! You have been successful in creating a poor man's scsi target and discovered them using open source softwares in an easy manner. Now GUTSY can read/write to /dev/vg/asm etc logical volumes. For more machines to do the same thing, similar setup is needed on them and they should also be access these volumes in a clustered fashion. This kind of setup is most beneficial for creating your own Oracle 10g RAC sandbox environment or for doing installs over shared disk.
In this case, we did not format the discovered logical volumes in ext3 or OCFS2 file system, because it is assumed that these volumes will be used as raw devices only.
Appendix
Content contributed by Sundar Mahadevan, who tenaciously resolved weird scsi target issues and graciously published this for the benefit of similar seekers):
The following helps newbies:
For iscsi to be implemented, you need to have iscsitarget installed onthe machine from where the logical volumes are going to be created and you need to have open-iscsi (initiator) installed on the machine that has to access the already created logical volumes.
When I tried this setup, I was getting some really weird errors. In this appendix, I outline my experience and offer the solution that ultimately worked for me. The intent of this sharing is to prevent anxiety for other people who may be in similar situations (I remember it was quite frustrating for me).
System1 : ubuntu 8.10
created /dev/vg1/asm, /dev/vg1/ocr and /dev/vg1/vote on the secondary
hard drive with lvm2
has iscsitarget and iscsitarget-source installed
System2: ubuntu 8.10
has open-iscsi installed
Connection: static (crossover cable between the 2 systems)
Error: Only one logical volume of the entire size of /dev/vg1 was discovered by the initiator! I should have got three disks like /dev/sda, sdb, sdc , but instead I got /dev/sdd or /dev/sdc.
Then i tried to make my connection as dhcp (with router connecting the 2 systems)
Error:
Apr 17 11:40:42 sunny1 iscsid: session [iqn.2009-09.com.ubuntu:asm,192.168.2.3,3260] already running. Apr 17 11:40:44 sunny1 kernel: [ 5661.360412] sd 2:0:0:0: [sdb] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK Apr 17 11:40:44 sunny1 kernel: [ 5661.360436] end_request: I/O error, dev sdb, sector 0 Apr 17 11:40:44 sunny1 kernel: [ 5661.360449] __ratelimit: 2 callbacks suppressed Apr 17 11:40:44 sunny1 kernel: [ 5661.360457] Buffer I/O error on device sdb, logical block 0 Apr 17 11:40:44 sunny1 kernel: [ 5661.360471] Buffer I/O error on device sdb, logical block 1 Apr 17 11:40:44 sunny1 kernel: [ 5661.360478] Buffer I/O error on device sdb, logical block 2 Apr 17 11:40:44 sunny1 kernel: [ 5661.360486] Buffer I/O error on device sdb, logical block 3 Apr 17 11:40:44 sunny1 kernel: [ 5661.370342] sd 2:0:0:0: [sdb] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK Apr 17 11:40:44 sunny1 kernel: [ 5661.370359] end_request: I/O error, dev sdb, sector 0 Apr 17 11:40:44 sunny1 kernel: [ 5661.370369] Buffer I/O error on device sdb, logical block 0 Apr 17 11:40:44 sunny1 kernel: [ 5661.375106] sd 2:0:0:0: [sdb] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK Apr 17 11:40:44 sunny1 kernel: [ 5661.375125] end_request: I/O error, dev sdb, sector 33554424 Apr 17 11:40:44 sunny1 kernel: [ 5661.375135] Buffer I/O error on device sdb, logical block 4194303 Apr 17 11:40:44 sunny1 kernel: [ 5661.379983] sd 2:0:0:0: [sdb]
Also i have been getting the error below from the beginning when i enter iscsiadm -m session ( from the client/initiator side)
iscsiadm: could not get host for sid 1. iscsiadm: could not get host_no for session 6. iscsiadm: could not find session info for session1 iscsiadm: can not get list of active sessions (6)
The were few bugs reported for the above error and the solution was to upgrade to 2.0.870 while my version was 2.0.865 I then did a fresh install on the client/initator side (ubuntu 9.04) and then installed open-iscsi 2.0.870 and just executed these commands:
iscsiadm -m discovery t st -p 192.168.2.3 iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg.asm -p 192.168.2.3 -l
and that was it!!...all logical partitions were detected from the client/initiator.
Great article thanks! Do you know if the Ubuntu iSCSI targets will work with Windows Server 2008 R2 Clustering? I have used OpenFiler for 2003 Clustering but it does not seem to work with 08.
ReplyDelete