Analyst Insight: One of the most astounding use cases for generative AI is the ability to generate stunning images from a text prompt. Although several techniques have been used to enable this capability, the basic approach starts with using hundreds of millions of models of different images that are each paired with a caption that describes it in words. Each image is broken down into pixels, and then the process is inverted so that the model can then reassemble those pixels into images based on text prompts.
A number of organizations have released tools that are designed to provide them with this functionality, and, not surprisingly, with each subsequent release, the quality and fidelity of the images gets better and better. One only needs to peruse the web to see incredible, generative-AI images created through Adobe Firefly, Open AI’s DALL-E 2, and Microsoft’s Copilot, among many others.
However, beyond the gee-whiz factor of text-to-image generation lies several complex challenges with which business users must contend: ensuring accuracy and preventing deceptive content, mitigating bias, and ensuring transparency around digital content. Recent events have put the spotlight on text-to-image generators, and as such, have highlighted the need to focus on the training and tuning of the models used to provide these services.
Maintaining Accuracy and Preventing Deceptive Content with Generated Images
The ability to create images from text accurately is based on the model correctly linking the semantic meaning of the text prompt to the properties of the images used to train the model. But the larger challenges faced by many companies that have launched text-to-image tools is ensuring that content returned from prompts is free from bias, accurate, and reliable.
Bias-free image generation relies on ensuring that the model does not absorb conscious or unconscious human biases during the training stage, which will then be reflected in the output. For example, if a model is only fed images of Caucasian women dressed in scrubs or uniforms, and these images are the only ones labeled as “nurse,” it’s not hard to see how a text-to-image generators would only spit out renderings of nurses that mirrored the images it was trained upon. This is an over simplistic example, but the adage holds true: models will learn and incorporate labeling or classification biases if there aren’t proper guardrails and checks in place.
Conversely, text-to-image generators can run into other problems around bias, particularly when models are trained to incorporate other data or viewpoints that may skew the generated results, and create images that are historically or factually inaccurate, misleading, or deceptive. Furthermore, the model must be continuously checked to ensure it hasn’t drifted from its initial and intended training, as skewed results can creep in via user interaction with the model.
Enterprise-Grade Generative AI Image Generation Requires Action
Thankfully, major enterprise vendors are focused on ameliorating the problem. At the Munich Security Conference (MSC) in February 2024, a number of technology companies pledged to help prevent deceptive AI content from interfering with this year’s global elections in which more than four billion people in over 40 countries will vote.
The “Tech Accord to Combat Deceptive Use of AI in 2024 Elections” is a set of commitments to deploy technology countering harmful AI-generated content meant to deceive voters. As of this writing, there are 20 signatories including Adobe, Amazon, Anthropic, ARM, Eleven Labs, Google, IBM, Inflection AI, LinkedIn, McAfee, Meta, Microsoft, Nota, OpenAI, Snap, Stability AI, TikTok, TrendMicro, TruePic, and X.
The Accord’s signatories have pledged to work collaboratively on tools to detect and address online distribution of such AI content, drive educational campaigns, and provide transparency, among other concrete steps. The Accord includes a broad set of principles, including the importance of tracking the origin of deceptive election-related content and the need to raise public awareness about the problem.
Another consortium, The Coalition for Content Provenance (C2PA), includes many of the same companies, including Adobe, BBC, Intel, Microsoft, Publicis Group, Sony, and TruePic, which are focused on bringing more transparency to digital content. Recently, Google announced that it has joined as a steering committee member, which will further help to drive the adoption of Content Credentials, the C2PA’s technical standard for tamper-resistant metadata that can be attached to digital content, showing how and when the content was created or modified.
Companies Need to Focus on Internal Controls
Although large, multi-firm consortiums are necessary to align all participants around an agreed-upon set of principles, individual companies that seek to provide products and services to large enterprises need to continue their ongoing work around generative AI.
One example is Adobe, whose Product Equity team—along with the Ethical Innovation, Trust & Safety, Legal, and Firefly teams—was at the center of the company’s efforts to reduce harm and bias in Firefly. The team prioritized minimizing exposure to harmful and offensive content and ensuring diverse representation of people, cultures, and identities in Firefly’s core features: text-to-image and Text Effects generation. Together with other teams, the Product Equity team has assessed over 3,700 pieces of feedback, over 25,000 prompts, and over 50,000 images, with a goal of reducing bias or other harmful content.
The Futurum Group has recognized Adobe’s strength as a responsible AI innovator, which was driven by a desire to not only improve the technical capabilities of its product, but ensure that it was robust and trusted to deliver enterprise-grade content. As Adobe’s Firefly product is coming up on its one-year anniversary on March 21 without any major issues around the generation of inappropriate, biased, or misleading imagery, it illustrates the value in doing the hard work around image assessment and training can help to deliver a product that can be trusted for use in commercial situations, including marketing and e-commerce workflows. Further, by incorporating Firefly technology across its portfolio of products, Adobe is standing behind the technology as a trusted, enterprise-grade service.
Disclosure: The Futurum Group is a research and advisory firm that engages or has engaged in research, analysis, and advisory services with many technology companies, including those mentioned in this article. The author does not hold any equity positions with any company mentioned in this article.
Analysis and opinions expressed herein are specific to the analyst individually and data and other information that might have been provided for validation, not those of The Futurum Group as a whole.
Other Insights from The Futurum Group:
Generative AI Spurs Adobe’s Investment In Digital Content Transparency
Firefly Announcement Details Adobe Enterprise Generative AI Approach
Adobe Firefly: Blazing a Generative AI Application Trail
Author Information
Keith Kirkpatrick is VP & Research Director, Enterprise Software & Digital Workflows for The Futurum Group. Keith has over 25 years of experience in research, marketing, and consulting-based fields.
He has authored in-depth reports and market forecast studies covering artificial intelligence, biometrics, data analytics, robotics, high performance computing, and quantum computing, with a specific focus on the use of these technologies within large enterprise organizations and SMBs. He has also established strong working relationships with the international technology vendor community and is a frequent speaker at industry conferences and events.
In his career as a financial and technology journalist he has written for national and trade publications, including BusinessWeek, CNBC.com, Investment Dealers’ Digest, The Red Herring, The Communications of the ACM, and Mobile Computing & Communications, among others.
He is a member of the Association of Independent Information Professionals (AIIP).
Keith holds dual Bachelor of Arts degrees in Magazine Journalism and Sociology from Syracuse University.
