Intel® Optane™ DC SSDs offer incomparable performance, but their capacities limit the available markets – until now! All-flash tiering can be used to combine the performance of Optane SSDs with the capacity of QLC drives, resulting in all-flash volumes that provide the best in performance/capacity storage applications.

Based on our testing, Optane SSD’s are hands down the fastest drives on the market, especially when you focus on latency (see Figure 1). The latency of the Optane drives was measured at just over 9 microseconds in our configuration, which was about 1/9th the latency of a NAND based NVMe drive. At queue depths of 32 and greater this advantage disappears, but the reality is that most applications operate at lower queue depths. In highly transactional applications, latency is the key metric as it is the ultimate measure of how fast things get done. Let’s be honest, everybody wants to get things done right away.

Figure 1: Latency comparison of Optane SSD and NVMe NAND, QLC and SATA SSD various cross queue depth. Optane has about 9X lower latency of standard NVMe NAND drives especially across the queue depths that most applications run in. Tests were run using 4K Random Reads.

So, if Optane SSDs are so fast, what are their shortfalls? The answer is that the capacities they are available in are not large enough to meet the needs of many enterprise applications. While their cost per GB is high, enterprises are willing to pay for performance. As the adage says, time is money.

This is where tiering value becomes apparent. Many people felt that as the industry transitioned away from hard drives to SSDs, tiering would be buried along with HDDs. In fact, the opposite is true. All-flash storage not only elevates the value of tiering but also removes the objections or negatives perpetrated against tiering when HDDs were utilized as the slow tier.

Let’s take a closer look at the value of all-flash tiering. We live in an all-flash world, but not all flash is created equal. At the high end of the spectrum, we have Optane SSDs, (and even Intel’s storage-class memory, called Intel® Optane™ DC Persistent Memory, if we want to take it one magnitude of performance greater), and at the other end of the spectrum, we have QLC memory. QLC, or Quad Level Cell NAND technology, like the name implies, allows 4 bits of data to be stored in the memory cell. This allows for greater densities than TLC NAND (Triple Level Cells), resulting in the ability to scale to larger capacities while delivering performance as well as reducing operational costs.

Optane SSDs offer high performance but the cost is high and the capacity is low. Intel D5-P4320 QLC SSDs offer great capacity at a low cost but the performance is low - compared to Optane SSDs - but still comparable to SATA SSDs and even holds its own against NVMe in read applications. Bottom line, QLC performance is magnitudes better than hard drives. QLC has lower endurance, which makes it better suited for either read applications or infrequently accessed data. Add in real-time tiering and enterprise users can now take advantage of the performance of the Optane drives while satisfying their capacity requirements with the bonus that their total cost per gigabyte approaches the cost of the capacity QLC drives. Figure 2 compares the performance of a raw Optane SSD to a hybrid volume that is made up of 2 Optane SSDs and 2 Intel QLC drives. The performance of the hybrid volume is nearly identical to the raw Optane SSD.

Figure 2: The performance of a hybrid volume comprised of 2 Optane plus 2 QLC devices is nearly that of a single pure Optane SSD device. (4K Random Reads

When we last heard from hybrid storage, all-flash pundits rightfully pointed out that tiering was great right until that time when you had to access the slow tier and your latency went from microseconds to milliseconds. Their argument was this was unacceptable for performance-sensitive applications such as databases. With all-flash tiering this argument now becomes invalid. Even the slowest flash still provides excellent performance for all but the most niche of applications. If you have one of those niche applications or an unlimited budget, by all means, buy half a petabyte of high-end NVMe storage, but for the rest of us (aka most of us), a properly balanced all-flash hybrid solution will provide the same performance but at a much better cost of ownership. Figure 3 compares the cost and capacity of pure Optane SSD to a hybrid Optane/QLC volume. In this case, the pure Optane SSD device is 375GB and the hybrid volume is 16TB in size. Referring back to Figure 2, remember that the performance of both volumes is nearly identical, but the hybrid volume costs 9X less per gigabyte and provides significantly greater capacity.

Figure 3: This chart compares the cost and capacity of an Optane SSD device to a hybrid Optane/QLC volume. Note: the VSSD configuration mirrored the Optane and QLC devices to provide a redundant configuration (Pricing from CDW on 9.19.19)s

All-flash solutions come in two generic options: All Flash Server-Based Storage or an external All-Flash Array. A server-based solution has performance advantages since it is not subject to the Network Tax – The performance penalty of moving your data across the network. A server-based solution eliminates the network tax by moving the application closer to the processor. The performance advantages are significant. Figure 4 shows latency numbers for a server-based all-flash hybrid volume compared to an industry-leading AFA. By eliminating the network tax, you can achieve 22X lower latency. This means your queries are significantly faster.

Figure 4: By running an application in a server, which moves the application closer to the CPU, you can achieve much greater performance gains than what you can attain in an All-Flash Array (AFA) due to the network tax. AFA numbers based on vendors published best-case latency numbers.

Tiering makes good sense. All-flash hybrid volumes make sense. By tiering high-performance Intel Optane SSD devices with cost-effective Intel QLC SSDs, enterprises can easily take advantage of the performance of Optane and both the capacity and cost of QLC devices. Creating these hybrid volumes in server-based storage or cloud-based servers eliminates the performance penalty associated with network latency. You finally can have your cake and eat it too.

For more information, contact Enmotus at [email protected] or visit our website at www.enmotus.com

Note: This article was initially published on Intel's Datacenter Builders Blog

Testing Configuration: 

• 2 X Pre Production xx-core [FM1] 2nd Generation Intel ® Xeon® Scalable processors (formerly code-named “Cascade Lake”)
• 2 X Intel ® SSD DC P4510 Series 4TB NVMe Drives
• 4 X Intel ® Optane ™ SSD DC P4800X Series 375GB NVMe Drives
• 1 X Intel ® SSD D3 - S4510 Series 3.8TB SATA Drive
• 2 X Seagate HDD 1TB SATA Drives
• 1 X Intel ® SSD DC S3520 Series 800GB SATA Drive (Boot Drive)
• 2 X Intel® SSD D5 - P4320 Series 7.68TB NAND QLC Drives
• 1 X Intel ® Ethernet Connection Network Adapter X772-10GBASE-T Dual Port (10GbE)
• 1 X Intel ® Ethernet Connection Network Adapter XL710-QDA2 Dual Port (40GbE)
• 1 X Intel® Ethernet Network Connection OCP X527-DA2 Dual Port (10GbE)
• 384GB RAM