Drives

Each server is provisioned with at least two hard drives in a RAID1 array with an installed operating system.

Additional hard drives ordered by you will not be configured automatically - this has to be done by configuration tools.

Installation of RAID configuration tools

Depending on the server type there are differences in hard drives and controllers, as to why different tools are required:

RAID rebuild after defective hard drive

After our technicians changed the hard drive, you will have to check the status of the RAID array and potentially start the rebuild.

ssacli

Rebuild starts automatically. Status and progress can be checked via ssacli ctrl slot=0 show config.

nvme-cli and mdadm

First the mdadm array and its hard drives have to be identified:

mdadm --detail --scan
ARRAY /dev/md0 metadata=1.0 name=any:0 UUID=08bcece4:9410c68c:24e5305b:5d49dd00

mdadm --detail /dev/md0
(...)
    Number   Major   Minor   RaidDevice State
       0     259        0        0      removed
       1     259        1        1      active sync   /dev/nvme0n1

As is visible in above example, one hard drive is in state “removed”. The new hard drive has not yet been added to the array.

Please check beforehand which device name the new hard drive has, e.g. via fdisk -l, and make sure that it is not used anywhere.

Having the device name of the hard drive, it can be added to the array:

mdadm --zero-superblock /dev/nvme1n1
mdadm --manage /dev/md0 --add /dev/nvme1n1

Now the hard drive is being synced. The progress can be followed by command mdadm --detail /dev/md0.

Creating RAID

For example, to combine additional hard drives into a RAID array.

ssacli

Firstly, the identifiers of the unassigned hard drives have to be determined. This is done by using the command ssacli ctrl slot=0 pd all show which output may for example look like follows:

ssacli ctrl slot=0 pd all show
(...)
   Unassigned

      physicaldrive 1I:2:1 (port 1I:box 2:bay 1, SATA SSD, 480 GB, OK)
      physicaldrive 1I:2:2 (port 1I:box 2:bay 2, SATA SSD, 480 GB, OK)

For the next step we need the ID, shown after “physicaldrive” - in this case 1I:2:1 and 1I:2:2. With these IDs a new RAID1 array can be created:

ssacli ctrl slot=0 create type=ld drives=1I:2:1,1I:2:2 raid=1

If you have four or more additional hard drives and want to create a RAID10 array, you will have to supply more IDs. The RAID level is named 1+0 instead of colloquially 10 - so for example:

ssacli ctrl slot=0 create type=ld drives=1I:2:1,1I:2:2,1I:2:3,1I:2:4 raid=1+0

Following the creation command you can show all information on configured logical drives via ssacli ctrl slot=0 ld all show detail - this also displays the path to the blockdevice which can then be partitioned and formatted as usual.

nvme-cli and mdadm

Firstly, all unassigned NVMes have to be identified. For this use commands nvme list and mdadm --detail --scan with mdadm --detail /dev/mdX, for example like this:

nvme list
(...)
Node                  Generic               SN                   Model                                    Namespace Usage                      Format           FW Rev  
--------------------- --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme3n1          /dev/ng3n1            XXXXXXXXXXXX         VO000960KYDZH                            1           0.00   B / 960.20  GB    512   B +  0 B   HPK1    
/dev/nvme2n1          /dev/ng2n1            XXXXXXXXXXXX         VO000960KYDZH                            1           0.00   B / 960.20  GB    512   B +  0 B   HPK1    
/dev/nvme1n1          /dev/ng1n1            XXXXXXXXXXXX         VO000960KYDZH                            1           0.00   B / 960.20  GB    512   B +  0 B   HPK1    
/dev/nvme0n1          /dev/ng0n1            XXXXXXXXXXXX         VO000960KYDZH                            1           0.00   B / 960.20  GB    512   B +  0 B   HPK1

mdadm --detail --scan
ARRAY /dev/md0 metadata=1.0 name=any:0 UUID=08bcece4:9410c68c:24e5305b:5d49dd00

mdadm --detail /dev/md0
(...)
    Number   Major   Minor   RaidDevice State
       0     259        0        0      active sync   /dev/nvme1n1
       1     259        1        1      active sync   /dev/nvme0n1

In this example /dev/nvme2n1 and /dev/nvme3n1 are still unassigned. With these a new RAID1 array can be created - as /dev/md0 is already in use, we will use /dev/md1.

--metadata=1.0 leads to the superblock being at the end of the array, which is required if the EFI partition is stored on the array (default in our installations). The EFI bootloader only looks at the start of hard drives, as to why there cannot be the superblock of the mdadm array. At the time the EFI bootloader is being executed, Software-RAIDs are not assembled yet.

--size constraints mdadm to a safe maximum size to keep enough space (1 MiB) for its superblock at the end of the array. Formula: ((Lower rounded Gigabyte size of the smallest hard drive * 1,000,000,000) / 1024) - 1024 So in this example: ((960 * 1,000,000,000) / 1024) - 1024

mdadm --create /dev/md1 --homehost=any --level 1 --raid-devices=2 --size=937498976K --metadata=1.0 --run /dev/nvme2n1 /dev/nvme3n1
mdadm: array /dev/md0 started.

See manpage of mdadm for further options, e.g. spares and other RAID levels.

Destroy RAID

ssacli

The logical drive that shall be destroyed has to be identified first:

ssacli ctrl slot=0 show config 
(...)
   Array A (Solid State SATA, Unused Space: 0  MB)

      logicaldrive 1 (447.10 GB, RAID 1, OK)

      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SATA SSD, 480 GB, OK)
      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SATA SSD, 480 GB, OK)

With that, the logical drive can be destroyed - in our case it has index 1:

ssacli ctrl slot=0 logicaldrive 1 delete

Now the hard drives can be used in another logical drive or directly as blockdevices.

If the hard drives shall be wiped completely, they can be erased, which is usually completed in about a hour:

ssacli ctrl slot=0 pd 1I:1:1 modify erase erasepattern=block unrestricted=off
ssacli ctrl slot=0 pd 1I:1:2 modify erase erasepattern=block unrestricted=off

Alternatively, if above erasepattern is not supported (significantly slower, takes multiple hours):

ssacli ctrl slot=0 pd 1I:1:1 modify erase erasepattern=random_random_zero
ssacli ctrl slot=0 pd 1I:1:2 modify erase erasepattern=random_random_zero

Wiping progress can be inspected via ssacli ctrl slot=0 pd all show and ssacli ctrl slot=0 pd all show detail. If wiping takes too long and shall be aborted before completion, the command ssacli ctrl slot=0 pd XYZ modify stoperase can be used for each hard drive (swap XYZ with the ID of the hard drive).

nvme-cli and mdadm

The mdadm array that shall be destroyed and its hard drives have to be identified first:

mdadm --detail --scan
ARRAY /dev/md0 metadata=1.0 name=any:0 UUID=08bcece4:9410c68c:24e5305b:5d49dd00

mdadm --detail /dev/md0
(...)
    Number   Major   Minor   RaidDevice State
       0     259        0        0      active sync   /dev/nvme1n1
       1     259        1        1      active sync   /dev/nvme0n1

With that, the array can be stopped and its superblocks destroyed:

mdadm --stop /dev/md0 --zero-superblock /dev/nvme0n1 /dev/nvme1n1
mdadm: stopped /dev/md0

Now the hard drives can be used in another array or directly as blockdevices.

If the hard drives shall be wiped completely, they can be sanitized, which is usually completed in a minute:

nvme sanitize /dev/nvme0n1 -a 0x02
nvme sanitize /dev/nvme1n1 -a 0x02

Wipe progress can be inspected for each NVMe via nvme sanitize-log XYZ (swap XYZ with the ID of the hard drive).