Analyst(s): Dr. Bob Sutor
Publication Date: August 29, 2024
Key Points:
- In July, the US Defense Advanced Research Projects Agency (DARPA) started the Quantum Benchmarking Initiative (QBI).
- On September 3, DARPA will host a meeting for US and international experts who believe they and their partners can build such sufficiently powerful systems. DARPA will explain its goals and engagement rules and answer participants’ questions.
- However, is there any point in having quantum benchmarks right now? Are the systems too small to extrapolate their performance to the powerful machines we will eventually need?
Overview:
Quantum systems today vary in almost every way you can imagine, in the number of qubits, the fidelity of operations, qubit modality, cost per qubit, energy usage, cooling required, physical footprint, and manufacturability. We’re not even sure which quantum computing applications will show the most value or which specific algorithms we will use at that time.
The DARPA QBI
Rather than only developing tests to prove that this quantum processing unit (QPU) is faster than that on algorithms or small applications, the QBI asks a profound question:
“[Is it] possible to actually build an industrially useful quantum
computer much faster than conventional predictions[?]”
Indeed, they want to know if it will be possible to build such a system in the first place. Why are we bothering with all the other quantum software and hardware development if we can’t?
Benchmarks
The computing industry has long used benchmarks to measure performance. Vendors use them to show their new chips and systems are better than their old ones and often issue press releases when their results significantly outperform those of competitors. We have benchmarks for PC performance and classical algorithms, and researchers are developing new ones for machine learning. Will AI become a part of benchmarks?
Quantum Benchmarks
Today’s quantum benchmarks can be challenging to understand if you do not have a technical knowledge of qubits, modalities, quantum gates and operations, and connection topologies. While we have the so-called quantum application benchmarks, they are not yet the use cases that companies or consumers will recognize or deploy. Doing better on a benchmark can be as much a case of improving your software and development tool chain as it is of switching hardware.
Should We Bother Now?
Suppose you are in 1903 and looking at the airplane designed by the Wright brothers. Someone suggests you develop benchmarks for a plane that can fly from New York to London at supersonic speed. Might that have been a tad early? Could you meaningfully extrapolate from the day’s technology to the eventual requirements? The QBI is asking the same kind of question for quantum computing. There must be some value in starting now.
The full report is available via subscription to the Quantum IQ service from Futurum Intelligence—click here for inquiry and access.
Futurum clients can read more about it in the Futurum Intelligence Platform. Nonclients can learn more here: Futurum Intelligence.
About the Futurum Quantum Computing Practice
The Futurum Quantum Computing Practice provides actionable, objective insights for market leaders and their teams so they can respond to emerging opportunities and innovate. Public access to our coverage can be seen here. Follow news and updates from the Futurum Practice on LinkedIn and X. Visit the Futurum Newsroom for more information and insights.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other insights from The Futurum Group:
Quantum in Context: A Qubit Primer
Enterprising Insights: Episode 33 – Salesforce Announces an LLM Benchmark for CRM
Quantum in Context: The Case for On-Premises Quantum Computers
Author Information
Dr. Bob Sutor is a Consulting Analyst for Futurum and an expert in quantum technologies with 40+ years of experience. He is an accomplished author of the quantum computing book Dancing with Qubits, Second Edition. Bob is dedicated to evolving quantum to help solve society's critical computational problems. For Futurum, he helps clients understand sophisticated technologies and how to make the best use of them for success in their organizations and industries.
He’s the author of a book about quantum computing called Dancing with Qubits, which was published in 2019, with the Second Edition released in March 2024. He is also the author of the 2021 book Dancing with Python, an introduction to Python coding for classical and quantum computing. Areas in which he’s worked: quantum computing, AI, blockchain, mathematics and mathematical software, Linux, open source, standards management, product management and marketing, computer algebra, and web standards.
