Using Retrieval-Augmented Generation With Access Control Technique for Managing LLMs Can Be Applied Enterprise-Wide

Using-Retrieval-Augmented-Generation-With-Access-Control-Technique-for-Managing-LLMs-Can-Be-Applied-Enterprise-Wide

The News: With the rollout of LLM-driven generative AI tools, many organizations are concerned about the ability of these tools to return correct, complete, and context-appropriate information in response to queries or inputs. The use of retrieval-augmented generation (RAG) can be used to allow LLMs to serve as an enterprise-wide front-end to a data source that contains information that evolves or changes over time.

AI software platform Coveo announced Relevance Generative Answering, a new capability within the Coveo Relevance Cloud AI platform, that takes this technique and enhances it for use with a range of use cases, including self-service, customer service, commerce, website, and workplace search, by combining the power of generative AI to surface information efficiently while ensuring that content is restricted by enterprise-governed levels of access control.

You can read the Press Release here describing the capability here.

Using Retrieval-Augmented Generation With Access Control

Analyst Take: Off-the-shelf, trained large language models (LLMs), such as ChatGPT, Bloom, and LlaMA, are emerging as potentially powerful tools for surfacing insights, generating responses, and improving access to information across various enterprise-wide applications. That said, most existing LLMs are pre-trained on a static set of data, which can limit their usefulness, in terms of relevance and timeliness. However, retraining LLMs every time a piece of data changes can be too time-consuming and expensive.

For example, an organization may wish to provide a customer-facing LLM query tool to let users find information about their products or services. However, a regular LLM likely has not been specifically trained on the most appropriate data sources to reliably provide the correct and complete answers to queries, which could include product documentation, the company knowledge base, and the company-sponsored user community site. Many of these sources will change over time, and open-source LLMs would not be able to capture any changes to the content without specifically being retrained using this data.

Using RAG to Provide Utility for LLMs

One way to handle this issue is by utilizing retrieval-augmented generation, which is a technique that allows LLMs to utilize specific datasets, even ones where the content is evolving or changing, while bypassing retraining. This allows LLMs to utilize the most recent and relevant information for generating reliable outputs. Using RAG, it is possible to control an LLM’s source of information simply by swapping out the documents it uses for knowledge retrieval, thereby limiting the possibility of the model hallucinating, or returning information that is out of date. This way, an LLM front-end could be deployed across enterprise-wide applications, but with different source documents based on each department, use case, or other segment.

However, simply using RAG techniques does not address the need to segment information, based on the user’s needs, nor the company’s desire to restrict information by type or level of user.

Incorporating RAG Across Enterprise Use Cases

Coveo takes a use-case approach to utilizing RAG, which incorporates not only relevance, but the ability to control the information returned by user access level. The Coveo platform already creates a unified index of all enterprise-wide customer service and knowledge documents across all data stores, and allows them to be updated as needed. When a user searches for an answer, the query will search this unified index while adhering to the user’s specific level of access to data, and will then return the most relevant content for the context of this user. Coveo Relevance Generative Answering will then use an LLM to summarize that relevant content, and create a personalized answer. This approach of combining data access control with generative summarization can be used to ensure that the content returned fits into the context that is appropriate for each use case and user.

As an example, consider an employee seeking information on compensation bands at their company. RAG could be used to make it easier to find compensation band data at the employee’s current level, and up to 2 levels above, but restrict access to and summarization of any other information. This ensures that workers can get relevant information, while preventing access to information that does not directly pertain to them.

Similarly, enterprises could use RAG and access control techniques to ensure that the information that is only pertinent to a specific group of customers is restricted to them, yet easy to surface. This is especially important for organizations that have tiered offerings, such as those with loyalty programs, tiered service contracts, or different classes of products, such as the multi-tiered automobile packages.

Driving Confidence by Linking to Source Data

Perhaps most importantly, Coveo’s generative AI capability will provide citations to the original documents to provide the source of truth. This helps to increase the confidence of both enterprises and their users that the information being retrieved is accurate, while also allowing any hallucinations to be quickly checked and addressed.

The enterprise-wide adoption of AI, both generative and general, requires the right controls to be implemented to ensure only the right users have easy access to data. Coveo’s approach of combining generative AI with access control is a key step in driving real-world utility for AI, which is based on making specific processes easier or more efficient, while limiting the potential for misuse.

Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.

Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.

Other insights from The Futurum Group:

Generative AI Capabilities Coming to Pega Infinity in Q3 

IBM watsonx.data is Now Generally Available as Part of Major watsonx Announcement

IBM’s Opportunity for Generative AI in the Enterprise

Image Credit: Coveo

Author Information

Keith Kirkpatrick is VP & Research Director, Enterprise Software & Digital Workflows for The Futurum Group. Keith has over 25 years of experience in research, marketing, and consulting-based fields.

He has authored in-depth reports and market forecast studies covering artificial intelligence, biometrics, data analytics, robotics, high performance computing, and quantum computing, with a specific focus on the use of these technologies within large enterprise organizations and SMBs. He has also established strong working relationships with the international technology vendor community and is a frequent speaker at industry conferences and events.

In his career as a financial and technology journalist he has written for national and trade publications, including BusinessWeek, CNBC.com, Investment Dealers’ Digest, The Red Herring, The Communications of the ACM, and Mobile Computing & Communications, among others.

He is a member of the Association of Independent Information Professionals (AIIP).

Keith holds dual Bachelor of Arts degrees in Magazine Journalism and Sociology from Syracuse University.

Related Insights
Qodo Builds the Governance Layer Agentic SDLC Has Been Missing
November 18, 2026

Qodo Builds the Governance Layer Agentic SDLC Has Been Missing

Qodo addresses governance gaps in AI development with Approval Workflows and Agent Session Review, enabling teams to manage AI reliability and maintain human oversight as 75% of organizations face AI-related...
RSM Bets $1B on Outcome-First Delivery for the Middle Market
November 18, 2026

RSM Bets $1B on Outcome-First Delivery for the Middle Market

RSM US LLP introduced the RSM Accelerator, a technology-driven platform delivering outcome-first solutions to approximately 125,000 U.S. middle market companies. Grounded in RSM's $1 billion AI and digital transformation commitment,...
Dropbox Bets on AI Infrastructure With Tigris Data Acquisition
November 18, 2026

Dropbox Bets on AI Infrastructure With Tigris Data Acquisition

Dropbox's acquisition of Tigris Data strengthens its agentic AI capabilities through S3-compatible object storage, addressing enterprise demand for AI infrastructure as 73% of software decision-makers prioritize agentic AI technology....
Kaseya Bets on Agentic IT to Close MSPs' AI Revenue Gap
November 18, 2026

Kaseya Bets on Agentic IT to Close MSPs’ AI Revenue Gap

Kaseya announced expansion of its agentic AI management platform with first-party MCP servers and 17 pre-built skills, addressing a critical gap where 48% of MSPs see AI as clients' top...
H2O.ai's Zero-Shot Forecasting Models Signal a New Era for Predictive AI
November 18, 2026

H2O.ai’s Zero-Shot Forecasting Models Signal a New Era for Predictive AI

H2O.ai launched TimeH2O and TabH2O, zero-shot forecasting models that predict on unseen data without per-dataset builds. TimeH2O ranked #1 on fev-bench with 91.9% win rate, surpassing Google's TimesFM-3 and Amazon's...
November 17, 2026

Agentic Orchestration for Enterprise

Futurum Signal: Agentic Orchestration for Enterprise evaluates 11 vendors on their ability to coordinate, govern, and audit multi-agent work in production. Microsoft, Salesforce, and ServiceNow lead the Elite Zone as...

Book a Demo

Welcome

The vision behind everything in Futurum’s Custom Research practice is this: research should show you what is happening, what comes next, and what to do about it. It should be personal to each audience, easy for people to grasp, and structured so LLMs can reason over it accurately. And it should be fast and turnkey; you want answers now, not another project to carry for quarters.

Whether you are defining business, channel, or go-to-market strategy; evaluating vendors or justifying ROI; or commissioning research to fill an emerging market need, we have your back, with a program that answers your questions with the objectivity and credibility to drive real decisions.

To do it, we bring unmatched data to bear: Futurum research, surveys, and market projections; validated market feeds; ETR’s 15 years of insight from 10,000 technology decision-makers; G2’s buyer and user data; and what our analysts hear every day. Add leading primary collection, from AI-moderated voice interviews to surveys and analyst-led interviews, all turnkey, and every project comes out credible, nuanced, and actionable.

And we don’t just drop the results in your lap. For internal work, we provide analyst-led sessions, interactive dashboards, and a range of formats. For market-facing work, Futurum delivers turnkey activation and amplification that actually gets seen, by people and by LLMs, through our media and share of voice. This is research that moves decisions and markets.

We will meet you wherever you are, from a fast-turn brief to a multi-year program, and shape the work to your goals, timeline, and budget. The right program for your moment.

If any of this is useful, I would love to talk.

Benjamin Brown, VP Custom Research, Futurum Research

Benjamin Brown

VP, Custom Research · The Futurum Group

Newsletter Sign-up Form

Get important insights straight to your inbox, receive first looks at eBooks, exclusive event invitations, custom content, and more. We promise not to spam you or sell your name to anyone. You can always unsubscribe at any time.

All fields are required






Thank you, we received your request, a member of our team will be in contact with you.