Enterprise Storage Stack
Linux Installation
Enterprise Storage Stack - Linux Installation
v 4.0
This version is distributed by email. If you need a copy of this version, please contact sales@easyco.com.
The provided file is a zipped tar file that contains the /usr/local/fbd directory.
Extracting the tar file
The default location for ESS is /usr/local/ess
If you have decrypted the download in the tmp directory, you can extract the tar file with:
cd /usr/local mkdir fbd cd fbd tar xvzf /tmp/ess-4.00.1.tar.gz
Extracted directory layout
Once the tar file is extracted, you should see a file layout similar to:
[root@sys3 fbd]# ls -d `find .` drwxr-xr-x 4 root root 4096 2014-02-27 16:49 ./ -rwxr-xr-x 1 root root 516647 2014-02-27 16:49 ./bm-flash* -rwxr-xr-x 1 root root 516676 2014-02-27 16:49 ./fbd-maint* drwxr-xr-x 2 root root 4096 2014-02-27 16:50 ./init.d/ -rwxr-xr-x 1 root root 3370 2014-02-27 16:50 ./init.d/fbd1* -rw-r--r-- 1 root root 559 2014-02-27 16:46 ./LICENSE drwxr-xr-x 2 root root 4096 2014-02-27 17:36 ./modules/ -rw-r--r-- 1 root root 4813323 2014-02-27 17:22 ./modules/dm-fbd.ko-cent63_2.6.32-279.el6.x86_64 -rw-r--r-- 1 root root 4911171 2014-02-27 17:22 ./modules/dm-fbd.ko-cent63_2.6.32-358.11.1.el6.x86_64 -rw-r--r-- 1 root root 4919816 2014-02-27 17:22 ./modules/dm-fbd.ko-debian-73_3.2.0-4-amd64 -rw-r--r-- 1 root root 5444480 2014-02-27 17:22 ./modules/dm-fbd.ko-ubuntu_2.6.35.14 -rw-r--r-- 1 root root 5826532 2014-02-27 17:21 ./modules/dm-fbd.ko-ubuntu_2.6.38-8-server -rw-r--r-- 1 root root 5980693 2014-02-27 17:19 ./modules/dm-fbd.ko-ubuntu_3.2.0-23-generic -rw-r--r-- 1 root root 5984333 2014-02-27 17:19 ./modules/dm-fbd.ko-ubuntu_3.2.0-37-generic -rw-r--r-- 1 root root 5976893 2014-02-27 17:20 ./modules/dm-fbd.ko-ubuntu_3.2.0-39-generic -rw-r--r-- 1 root root 5976221 2014-02-27 17:20 ./modules/dm-fbd.ko-ubuntu_3.2.0-48-generic -rw-r--r-- 1 root root 5582406 2014-02-27 17:20 ./modules/dm-fbd.ko-ubuntu_3.4.58 -rw-r--r-- 1 root root 6055282 2014-02-27 17:21 ./modules/dm-fbd.ko-ubuntu_3.5.0-23-generic -rw-r--r-- 1 root root 6054514 2014-02-27 17:21 ./modules/dm-fbd.ko-ubuntu_3.5.0-34-generic -rw-r--r-- 1 root root 6051792 2014-02-27 17:20 ./modules/dm-fbd.ko-ubuntu_3.7.7 -rw-r--r-- 1 root root 5545281 2014-02-27 17:21 ./modules/dm-fbd.ko-ubuntu_3.9.2-1-server -rw-r--r-- 1 root root 304 2014-02-27 16:49 ./README
Installing the kernel module
Select the kernel module that matches your system. Copy this file into the "extra" directory of your running kernel:
mkdir -p /lib/modules/3.5.0-34-generic/extra cp /usr/local/ess/modules/dm-fbd.ko-ubuntu-3.5.0-34-generic /lib/modules/3.5.0-34-generic/extra/dm-fbd.ko depmod -a
v 4.1
Version 4.1 is installed from an RPM file for your target platform. This RPM will automatically install the correct kernel module to match your kernel. This installation uses DKMS and requires gcc and kernel headers on your system.
Download the dkms rpm file and the ess rpm file into a temporary directory (/tmp is fine).
rpm -ivh /tmp/dkms...rpm rpm -ivh /tmp/ess...rpm
Some licensees have special needs. Contact support@easyco.com if you have any questions.
Authorization Codes
Many licensees have the option of modifying the authorization proceedure for ESS, so this section may not apply to all users.
ESS uses a system that keys execution to several "system identifiers". These are things like drive serial numbers, network MAC addresses, etc. The system is designed to allow some changes to a system without needing a new code.
In order to request a code, you need a serial number, and a capacity. Codes may change with future releases, so major version upgrades may require new codes in the future. You will know this when you try to load a new release. We are also setting up an automated system so that upgrade will get new codes over the internet without requiring user intervention.
Use the following command to generate a request:
./fbd-maint authorize serial_number capacity E
The serial number is a string assigned to you by EasyCo
The capacity is the maximum size, in MB, of the backing array
This command will generate a string. Email this string to support@EasyCo and we will send you back a second string that you use with the command:
./fbd-maint activate AUTH_STRING
Generic Startup / Shutdown Process
This example will involve a single RAID array of SSDs managed with Linux software RAID. At a higher level, the resulting storage is configured with Linux Volume Manager (LVM2) which creates logical volumes that are then exported as block devices over iSCSI, formatted with a file system, or otherwise used by local and remote applications.
1) Setting up the Hardware SSDs
ESS uses standard "disk drives". With Linux, these usually appear as scsi block devices denoted as '/dev/sda', '/dev/sdb', etc.
"Linux Standard Practices" suggest that SSDs should always be partitioned and that these partitions should be combined into a single RAID device. So you should create a partition on each SSD as type "linux raid". The partition should start at a "natural boundary" of the SSD flash. If you use partitions, make sure that the partition starts on a 4096 sector boundary.
Most users opt to use the entire disk for the array. This works fine and is often the easiest to setup.
2) Setting up the Linux Software RAID device
Assuming you are using RAID-5, you do this with:
mdadm --create /dev/md0 --level=5 --raid-devices=23 --spare-devices=1 --chunk=64 --meta=1.0 /dev/sd[b-z]1
This will create /dev/md0
3) Format the array for use with ESS
You will need to decide which Operating Mode you wish to run in before you can format your logical device.
This is done once to setup geometry to ESS operation.
Linear mode:
/usr/local/fbd/fbd-maint format /dev/md0 --wtblksz=$((22*2048)) -q
Compressed mode:
/usr/local/fbd/fbd-maint format /dev/md0 --wtblksz=$((22*2048)) --comp=200 -q
4) Setup the start/stop script
The script
/usr/local/fbd/init.d/fbd1
has a number of parameters that need to set for ESS to operate. These are located near the top of the script and are documented there. Additional documentation for other start/stop parameters are under development.
5) Mount the ESS block device
Use the script
/usr/local/fbd/init.d/fbd1 start
6) Setup LVM2
First create the volume group
pvcreate /dev/mapper/fbd1 vgcreate ess /dev/mapper/fbd1
Then create logical volumes
lvcreate -L 50G -n vol1 ess
7) Format File Systems etc.
Once volume manager is running you have standard logical volumes that you can use. You can format and export file systems, setup block exports with iSCSI, etc.
Setup on Reboot
When the system is rebooted, you need to:
a) Start the /dev/md0 RAID array
b) Mount the ess device with the /usr/local/fbd1/init.d/fbd1... start script
c) Start LVM with 'vgchange -a yes'
d) Mount and/or export logical volumes
Many users choose to customize /usr/local/fbd1/init.d/fbd1 adding the array, LVM, and export commands to the "start)" and "stop)" portions of the script. They can then add this script as a standard sys5 init script with:
ln -s /etc/init.d/fbd1 /usr/local/fbd1/init.d/fbd1
chkconfig --add fbd1
Setup for Shutdown
On shutdown, you need to do these steps backwards:
a) Unmount and stop exporting logical volumes
b) Stop LVM with 'vgchange ess -a no'
c) Unmount the ess device with the /usr/local/ess/init.d/fbd1... stop script
d) Stop the /dev/md0 RAID array
Again, most users opt to setup these steps in the "stop)" part of the fbd1 script.