The News: Announced at Data + AI Summit 2024, Anomalo, a data quality platform provider, has introduced the expansion of its capabilities to monitor the quality of unstructured text data. This new feature, currently in private beta, enables enterprises to manage, curate, and leverage high volumes of text data while mitigating the risks associated with low-quality data. This capability is particularly critical for the burgeoning field of Generative AI, where data quality directly impacts model performance and compliance. Read more here.
Anomalo Expands Data Quality Platform to Include Unstructured Text Monitoring
Analyst Take: In the modern data landscape, unstructured data constitutes approximately 90% of all enterprise data. Unlike structured data, unstructured data lacks a standardized format, making it challenging to organize, store, search, retrieve, and analyze. This type of data often contains inconsistencies, errors, and duplicates, and can also include sensitive information such as personal identifiable information (PII) and abusive language. These issues present substantial privacy, security, and performance risks, especially when integrating unstructured data into Generative AI models and applications.
Organizations are increasingly implementing Generative AI and ingesting unstructured text to train, fine-tune, and augment models with Retrieval Augmented Generation (RAG). The volume and velocity of this data ingestion require robust mechanisms to ensure data quality and mitigate risks before incorporating it into AI models.
Anomalo’s New Capabilities
Anomalo’s newly introduced unstructured text monitoring capability allows enterprises to evaluate and curate text documents based on various quality metrics. These metrics include document length, duplicates, topics, tone, language, and abusive language.
Additionally, the monitoring capability assesses documents for PII and sentiment. This comprehensive evaluation ensures that enterprises can maintain high-quality text data, crucial for accurate analysis and reliable AI model training.
These features enable users to swiftly assess the quality of document collections and pinpoint issues within individual documents, significantly reducing the time needed for data curation and profiling. This enhancement is particularly beneficial for ensuring the reliability of unstructured text data used in Generative AI applications.
Leadership Insights
Elliot Shmukler, co-founder and CEO of Anomalo, emphasized the importance of data quality in developing high-performing data products, including Generative AI models. He noted, “It’s been well known that higher quality data leads to better data products, including traditional dashboards and machine learning models. The same is true in the world of Generative AI, where the quality of the text used to fine-tune or prompt the model via RAG could be the difference between a high performing application and one that is at best underwhelming and at worst a privacy and compliance risk. We’re supporting data teams in using high-quality data for all of their critical initiatives and with our new unstructured text monitoring capability, to support their Generative AI efforts as well.”
Industry Impact
Anomalo’s expansion into unstructured text monitoring represents a significant advancement for enterprises dealing with large volumes of unstructured data. By enabling automatic detection of data issues and understanding their root causes, Anomalo empowers data teams to preemptively address data quality problems, thereby ensuring better decision-making, smoother operations, and more reliable AI models.
Sid Stephens, data governance business lead for a top three quick service restaurant company, highlighted the practical benefits of Anomalo’s platform: “Finding the data quality problem is just the first step, you’ve got to solve the issue. Anomalo helps our enterprise data teams find the hard to predict data quality issues and reduce time to resolution. Anomalo’s monitoring on unstructured data capability is just another step to help our teams resolve issues on data-critical projects.”
Forward Looking
Anomalo’s enhanced platform, now including unstructured text data monitoring, positions the company as a player in the data quality space. As organizations continue to leverage Generative AI and integrate unstructured data at unprecedented scales, the ability to ensure data quality will be a key differentiator in achieving operational excellence and maintaining compliance. Anomalo’s proactive approach to data quality management will undoubtedly support enterprises in harnessing the full potential of their data assets while mitigating associated risks.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other Insights from The Futurum Group:
Anomalo Surges Forward with $33 Million Series B Funding
Databricks Acquires Tabular to Boost AI Data Management
Databricks New DBRX Model- Enterprise Worthy?
Author Information
With over 25 years of experience, Paul has a proven track record in implementing effective go-to-market strategies, including the identification of new market channels, the growth and cultivation of partner ecosystems, and the successful execution of strategic plans resulting in positive business outcomes for his clients.
