Overview: NVMe protocol and NVMe-over-fabrics are becoming standard for all flash data storage solutions. There are several methods for implementing NVMe-oF including Fibre Channel, RDMA based approaches such as RoCE and iWARP, and NVMe over TCP. In 2021, Evaluator Group (now known as Futurum Labs) published a Technical Insight making the case for NVMe/TCP as a standard NVMe-oF option moving forward. This blog takes a look at where things stand today. Read Technical Insight: The Case for NVMe/TCP here.
Checking In On NVMe/TCP
Analyst Take: About a year and a half ago I wrote a paper discussing the various approaches for NVMe-oF, and more specifically making a case for NVMe/TCP. My argument for NVMe/TCP was essentially this: dramatically improved performance when compared to iSCSI, while offering less cost and complexity than other NVMe-oF approaches. Yes, NVMe/FC or RoCE might have slightly greater performance, but in most cases the performance of NVMe/TCP is plenty without requiring costly or complex infrastructure changes.
Now, some time has passed and NVMe-oF continues to gain adoption. So, it warrants looking back on my case for NVMe/TCP and evaluating what is being implemented and what vendors are offering. Overall, the argument for NVMe/TCP remains the same – a large performance increase, while utilizing already inplace Ethernet infrastructure. From speaking with IT organizations, NVMe-oF implementations are largely split between NVMe/FC and NVMe/TCP. Some RoCE or iWARP implementations certainly exist, but in many cases they were done so before NVMe/TCP was an available option.
Thinking about this, it makes sense. There is a long history of existing Fibre Channel SAN, and in those cases NVMe/FC is the clear choice. For those without an existing Fibre Channel infrastructure, NVMe/TCP appears to be the route forward, as I had previously suggested. Interestingly, many organizations are also still utilizing iSCSI under an assumption that it will be simpler than NVMe/TCP. This indicates that more education around the benefits and the relative simplicity of NVMe/TCP may be needed. It also shows that while gaining adoption, NVMe-oF has not yet completely taken over and that changes such as this don’t simply happen over night.
Of course organizations can only adopt the technology if it is supported by the products they are using. Looking at storage vendor support for NVMe/TCP, a large portion have in fact supported the technology.
Notable vendors and products that have added support for NVMe/TCP include:
- Dell – PowerStore, PowerMax, and PowerFlex
- VMware – vSphere
- NetApp – ONTAP systems including AFF, FAS, and ASA
- Huawei – OceanStor Dorado
- Lightbits – LightOS
- InfiniDat – InfiniBox
- Pure Storage – FlashArray
- Lenovo – ThinkSystem DM (ONTAP)
- HPE – Alletra 9000 and Alletra Storage MP
This list is quite similar to when I had written previously, however expanded with a few more vendors and products, and a notable addition is Pure Storage. While the company originally favored support of NVMe/FC and NVMe/RoCE, earlier this year Pure added NVMe/TCP support to the FlashArray product line with the Purity 6.4.2 update.
The last major holdout on adopting NVMe/TCP remains IBM. Currently, FlashSystem supports NVMe with Fibre Channel or RDMA, but TCP support has not yet been added. I would suspect however, that IBM won’t fight against the trend for much longer, as the signs point towards NVMe/TCP being a key protocol moving forward.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other insights from The Futurum Group:
Pure Storage Adds New File Services to FlashArray
HPE Expands Alletra Storage Portfolio to Transform File, Block, and Data Protection Services
Author Information
Mitch comes to The Futurum Group through the acquisition of the Evaluator Group and is focused on the fast-paced and rapidly evolving areas of cloud computing and data storage. Mitch joined Evaluator Group in 2019 as a Research Associate covering numerous storage technologies and emerging IT trends.
With a passion for all things tech, Mitch brings deep technical knowledge and insight to The Futurum Group’s research by highlighting the latest in data center and information management solutions. Mitch’s coverage has spanned topics including primary and secondary storage, private and public clouds, networking fabrics, and more. With ever changing data technologies and rapidly emerging trends in today’s digital world, Mitch provides valuable insights into the IT landscape for enterprises, IT professionals, and technology enthusiasts alike.
