ESS is a software layer that enhances the capabilities of a block storage device. ESS works by applying several techniques.
Block Mapping
ESS re-maps all writes as 4K pages. Thus any 4K page can be stored anywhere on the target block device and ESS knows immediately where that 4K page is located.
Linear Writes
ESS writes data in controlled linear segments that are optimized for maximum performance with both Parity RAID arrays and Flash block devices. Flash SSDs, PCI-e cards, nVME drives, and other flash solutions all have NAND flash internally. By writing linearly, ESS off-loads much of the block management from the drive to the host. This creates and environment where the drive sees only "perfect writes" that can run at full speed and without the need to ever reorganize blocks. This block reorganization, often called garbage collection, simply never happens with an ESS controlled device because ESS writes with patterns that simply never leave any garbage behind.
Linear writes also eliminates overhead created in Parity RAID arrays. Normally, random writes to RAID-5 or RAID-6 will result in 2x or 3x write amplification. Because ESS writes perfectly aligned full stripes, RAID-5 and RAID-6 amplification is lowered to the theoretical n/(n-1) or n/(n-2).
Meta Storage
ESS records blocks and the control information that goes with those blocks as "atomic writes". Being part of an atomic write means that the data and control information that goes with the data (this is called meta data) is always written together and can never get out of sync in the case of a crash. This unique, and patented, aspect of ESS is largly responsible for the incredible speed that ESS delivers with random write workloads.
Sparse Storage
ESS is, by it's nature, a sparse storage engine. When writing, specific patterns are "recorded" but not actually written. For example, if you write a 4K blocks that is all zeros, ESS will write the meta data for that block, but will not write an actual block. This is similar to discard or trim operations common with SSDs, except that writing zeros work even across a wan or when erasing a file.
Compression
ESS can be setup to compress individual 4K blocks to save space. Compression can be also be used to increase the performance of low bandwidth arrays and to reduce flash wear by actually lowering the amount of data writting. More information is available on the compression page.
Deduplication
Deduplication support is not yet available in ESS production releases. If you have an application that can exploit deduplication, please contact EasyCo for pre-release information.
ESS can be setup to perform real-time block-level deduplication. Wtih deduplication, blocks are analyzed so that blocks with identical contents are only stored once. De-dupe is ideal for environments with many duplicate blocks such as hosting many virtual servers all with the same OS files. VDI is a prime example of this. Deduplication can have signifigant memory and CPU requirements. More information is available on the deduplication page.