Prelude

This blog was not sponsored or suggested to me by any 3rd party, the thoughts and opinions are mine, based upon my experience using and testing these technologies with a variety of environments and applications.

What prompted me to write this blog is that I have seen several press releases and articles questioning the viability of a technology that I have tested extensively and believe has significant benefits, although may be suffering from some market confusion.  My hope here is to explain the use cases and potential benefits of persistent memory generally, and Intel’s Optane technology in particular.

Several of the reports I read were focused primarily upon the financial aspects.  My focus is that of an IT practitioner and the practical uses of this technology, leaving aside financial analysis of foundry costs, product margins and other financial discussions.  However, clearly every product must have both compelling use cases and provide profitability for its manufacture, sales and distribution in order to remain viable.

Note: Intel and Micron co-developed 3D Xpoint persistent memory technology and announced it in 2015.  Production began in 2016 and was marketed by Intel using the “Optane” brand name and by Micron using the “QuantX” name.  

Overview

Over the past five years, a new persistent memory technology has emerged, known by several names including 3D-Xpoint and Optane.  The “3D Xpoint” term is a marketing label used by its developers, Intel and Micron to refer to the technology generally.  3D Xpoint is actually a type of phase change memory.

Recently I have seen articles asking, “Is the end near for Intel’s Optane?” Additionally, a press release from Micron (co-developer of 3D Xpoint, the persistent memory technology behind Optane) explaining their shift to focus on products away from 3D Xpoint interface has further provoked discussions, particularly on the future of Intel’s Optane.

In short, my belief is that persistent memory, (aka NVRAM) will become increasingly important for many workloads, whether or not the underlying technology utilizes 3D Xpoint or some other new technology.  For the near term, 3D Xpoint occupies a unique position for the following reasons:

• Currently, no alternative technologies in production have 3D Xpoint’s cost, performance and longevity capabilities.
• Existing NVRAM technologies with higher performance and longevity have significantly higher costs per GB (MRAM, ReRAM, etc.)
• NAND technology has lower cost but with much lower performance and longevity.
• Thus, there will be a role for 3D Xpoint products until another technology can provide a better set of features at a better price point.

Solid-state media is being used increasingly as the primary method for storing, or “persisting” data.  There are many types of solid-state media, but the most prevalent technology used in enterprise computing today is NAND flash. Alternatives to NAND flash have been investigated for many years, but very few of these alternatives have viable use cases that can leverage their capacity, performance and longevity characteristics cost effectively.

A Brief History of Persistent Memory

So called “flash media” was initially developed in the early 1980’s but only emerged around the year 2000 as a viable storage media for enterprise workloads.  This persistent memory technology has been used primarily for its persistent properties, as a way to store data.  Since then, costs have fallen dramatically while capacities have increased by several orders of magnitude.  However, the longevity of the underlying NAND flash media has not changed substantially.

The term “Optane” is used by Intel to refer to products that utilize 3D-Xpoint, persistent memory technology.  Somewhat confusingly, this includes two different form factors of products, with quite different use cases.  This is where much of the confusion arises, which we hope to help clarify.  The two Optane product categories are Optane Persistent Memory modules (aka Optane Pmem) along with Optane NVMe storage media.

Optane as Persistent Memory

Optane as persistent memory (Optane PMem), is designed to be utilized in one of two modes:

1.     Memory Mode: which works in conjunction with DRAM to provide memory, (DRAM acts as a cache for the PMem which provides logical memory capacity)

2.     App Direct Mode: which creates a new tier of memory class storage.  Optane PMem is seen as distinct from DRAM and NVMe storage within a system.

To further complicate things, in order to utilize PMem in “App Direct Mode” requires special modifications to either an operating system / hypervisor, or application.  That is, applications (including OS / hypervisor) access PMem directly, hence the term “App Direct.”

Using Optane Persistent Memory through “App Direct Mode” is where the greatest potential lies for using this type of media.  Unfortunately, to date very few OS’s, Hypervisors or applications have made the necessary modifications to use PMem in this manner.  For proof, see our test results noted below.

Optane as NVMe Storage 

As I have previously described, NVMe storage products with the Optane brand are different from Optane Persistent Memory.  These devices are intended to be used as solid-state disk storage.  The form factor for Optane NVMe storage devices is typically a 2.5” small form factor disk, with a U.2 interface for NVMe storage over PCIe.  These are physically interchangeable with all other U.2, NVMe storage media.

My Experience with PMem and Optane NVMe Storage

Evaluator Group performs extensive hands-on testing of many technologies in our Evaluator Group Lab.  Our environment includes more than 10 servers, many running VMware vSphere or Hyper-V, and recently bare-metal container environments. The servers use storage systems from many of the large vendors, and several software defined and hyperconverged storage offerings.  Additionally, our lab has 100, 40, 25 and 10 Gb Ethernet for IP and storage connectivity, along with 32, 16 and 8 Gb FC networks for SAN storage.

We have tested a wide variety of storage media as a part of storage systems and software defined and HCI storage, including many different flash storage media.  This includes SATA / SAS flash storage, NVMe storage, and Optane NVMe devices.  Separately, we have also tested persistent memory, specifically Optane Pmem.  Again, the use cases for these two types of Optane media are significantly different, and the way these technologies are utilized by operating systems and applications is also significantly different.

Optane Persistent Memory Testing

Evaluator Group has tested “Optane” persistent memory in both Memory Mode, and in App-Direct Mode.  Each mode has specific use cases that are more optimal.  Memory Mode is quite easy to enable and utilize, since it requires no OS, Hypervisor or application awareness.  However, providing more memory capacity, with slower performance at a somewhat lower cost point is not especially compelling.  There are some scenarios, such as virtual desktops (VDI), where it can make sense to utilize Memory Mode PMem, since memory performance is not critical, but low-cost memory capacity can often be a limiting factor.  Other similar applications may also be a good fit for Memory Mode.

For App-Direct mode use, Evaluator Group tested and published our results of the NetApp MAX Data product that combines persistent memory performance with data protection features. The performance was up to 40X better than typical all-flash storage systems, with latencies under 30 microseconds for both reads and writes.  For reference see:  Lab Insight: Enterprise Applications with NetApp MAX Data Performance

Our testing of Optane PMem as App-Direct persistent memory showed that it is possible to provide data persistence at latencies well under 50 micro-seconds to applications, which in most cases is at least 10X faster than any leading all-flash storage system.  

Optane NVMe Storage Testing

In addition to using Optane as PMem, we have tested Optane as an NVMe attached storage device in many different configurations.  The primary use case has been to utilize Optane NVMe devices for storage applications that can utilize a small, high-performance storage media.  Use as a meta-data device for Ceph storage clusters is one scenario, along with using Optane NVMe as the write-cache for software defined storage, such as VMware vSAN.

Optane NVMe storage devices are ideally suited to provide low-latency for write intensive workloads, and sustain this for many years without wearing out, unlike NAND flash-based storage media.  Currently, no other media can provide similar performance and longevity at competitive price points.

During testing, we have seen Optane NVMe improve the overall storage system performance, particularly for latency sensitive, or highly write intensive workloads.  Additionally, with recent Optane NVMe devices now certified for 100 drive writes per day (100 DWPD), these devices are unlikely to wear out over their useful lifetimes.  By comparison, most NAND Flash devices are rated at between 1 and 5 DWPD, meaning even the longest lasting NAND devices will wear out 20 times faster than Optane.

Final Thoughts

Persistent memory could become a key technology for many applications due to the very high-performance levels that are un-achievable with other technologies at competitive price points.  This is not to say that Optane branded technology deployed as persistent memory or as NVMe storage media will necessarily be successful.  However, the unique characteristics of this persistent media are useful for a number of scenarios and will become increasingly useful as more applications, operating systems and hypervisors discover ways to utilize this technology.

In my opinion, some of the issues have to do with confusion over what Optane is and where it may be utilized.  A large part of this confusion exists because it the term Optane is used to describe two different form factors (NVMe SFF disk and PMem DIMM) and three different use cases (Storage, Memory Mode and App-Direct Mode).  As a result, IT consumers and some vendors are confused about what technology to use where, and what benefits it can provide.

Hopefully, I have helped reduce some of the confusion, by clarifying the three different uses for Optane media.  Clearly, persistent storage media will be utilized in an increasing number of workloads and applications in the future.  Currently there are no competitors to Optane providing similar performance and durability at similar price points today.  As always, it will be incumbent on IT users to decide what technology provides the most effective price – performance levels for each particular application.