As soon as I heard Adobe was planning on including its Firefly generative AI engine directly in Photoshop – and touted this development as a “major new release” of their iconic pixel toolbox – my excitement ran high. So much so, in fact, that my working title for this article became New Photoshop is the biggest version since 1991! Imagine having professional photo-editing software which could allow me to freely manipulate synthetic images as part of my regular image workflow: this concept could revolutionize our entire industry!
So I gave it a go, only to discover my dreams were broken by it – as Stevie Wonder sang. Unfortunately, this method does not seem like magic yet.
Adobe released Photoshop beta 24.6 with Generative Fill, their newly unveiled AI tool. Current Creative Cloud subscribers have early access to this feature ahead of its wider release later in 2018, giving them AI to do things such as extend borders of an image, automatically fill blank space with images generated via prompts like Midjourney or DALL-E do.
The tool offers some convenient tricks for image editing, but also has some notable drawbacks. The update appears to be an early attempt at integrating an AI work-in-progress into professional photo-editing workflow – although one that cannot yet compete with more sophisticated image generation software currently on the market.
As someone who’s been hooked on raster art since Deluxe Paint II for Amiga and then Photoshop 1.0 for Mac, I was really looking forward to finding an innovative program like Photoshop which could allow me to manipulate images like never before. Furthermore, I hoped to work alongside AI tools I already use and replace them all with this groundbreaking generative AI program. But unfortunately that wasn’t meant to be.
Adobe Firefly’s generative AI is useful for certain tasks; indeed, Adobe’s star program already had several useful AI features (neural image resizing and facial manipulation for instance) for some time; but now, for the first time ever, powerful image generation capabilities could actually help professionals offload some of the tedious menial work in their daily design business.
The Good, the Bad and the VERY Ugly
Photoshop beta is similar to its older version in terms of features; however, there is one notable distinction: an innovative new contextual bar featuring a button called Generative Fill which appears whenever you make selections on screen. When clicked upon, this button opens a field asking what you would like Photoshop to create based on context from images you edit or simply leave blank for interpretation of image information provided.
DALL-E and other AI programs refer to this ability as “inpainting.” When you select an existing image and enter “Godzilla Monster” into the context bar, Adobe Photoshop will inpaint an enormous lizard-beast into it that matches both its colors and lighting using their cloud-based AI engine, Firefly.
Once again, selecting an unwanted person in an image of a landscape but failing to provide any prompt when clicking Generative Fill will have Photoshop remove them by inpainting more AI-generated background that’s designed to seamlessly erase them. If you want to expand a landscape’s boundaries further still, select one of its blank areas on canvas before clicking Generative Fill – it will outpaint this region of canvas with more intricately designed AI landscape details!
Instead of Photoshop’s previous content-aware fills that would randomly repeat patterns and require further manual patching work, the Firefly-enabled beta excels at outpainting landscapes and eliminating objects – saving time for many who would otherwise spend it doing such tedious work manually. With its AI technology powering this incredible feat of timesaving.
Photoshop’s generative AI falls short in fulfilling all its promises, specifically object generation quality which falls below professional-level work. Older versions of Stable Diffusion or DALL-E had better object realism representation while DALL-E has much worse. Poor representation of anatomical details lead to deformed hands; other objects become unusable due to deformation; while Firefly often misinterpreted prompts.
To assess its capabilities, I conducted a quick test: taking a photo of my son and me and doctoring it in ways that often pose difficulties for generative AI systems. AI successfully removed my board game from my image, while my child’s hand – with eyes I intentionally pixelated – lost part of its fingers when I asked PS to “change apple to toy dinosaur.” I tried adding “Converse-like sneakers” to his foot, but they did not look real or have the correct perspective. Additionally, his glasses weren’t styled after Ray Ban Wayfarers nor did they sit properly on his face; my hand got deformed while trying to add “hand in leather glove,” and there wasn’t anything close to a cowboy hat that I asked the prompt for over and over. So eventually gave up.
Compare Photoshop Firefly’s poor performance against AI tools like Flying Dog based on Stable Diffusion: it becomes clear.
At present, Photoshop’s AI is only suitable for outpainting landscapes or eliminating objects–making it highly portable! Unfortunately, however, its production-ready capabilities fall far short of other apps like Midjourney or Stable Diffusion XL due to their advanced coherence technology which gives accurate hands, faces, and objects–increasing realism dramatically over time. Here is where Adobe falls short in comparison.
Photoshop’s AI was not as helpful as advertised. When using AI to create something new, a “Generative AI” layer is created. While this doesn’t destroy the original image; rather, multiple layers are composed on top. Unfortunately, when trying to move these layers around they come with a background halo making it impossible to reposition my Godzilla-esque monster as seen below.
As you moved your AI-made object around, you might expect it to regenerate automatically as it merges with layers below it. Unfortunately, instead, it kept its halo background, rendering the generated image unusable anywhere other than its original spot. To reuse Godzilla–such as in the center of the road–you must extract and eliminate its background before repositioning it; but this may look unrealistic because lighting and color won’t match its new location in the original photo.
Cost of Cloud-Based Generative AI
While Stable Diffusion can be downloaded and installed onto a computer for free to run locally, other generative AIs like DALL-E and Midjourney require payment in order to utilize their functions due to powerful processors in the cloud. Adobe Firefly features also don’t run locally but through Adobe’s cloud.
Adobe has informed me via email that while its AI image generation will initially be free for all PS users, future costs have yet to be discussed: “Generative Fill is currently only available in the Photoshop beta app for all PS users at no extra cost and will become fully available by second half 2023; additional details will be shared closer to its release from beta.” In other words: There’s no guarantee it will remain free in its final form when it arrives – given its immense processing costs it wouldn’t be crazy for there to be additional fees associated with its introduction (though I hope I’m wrong!).
Operating on the cloud also comes with other significant downsides. Anything created using Photoshop’s Firefly capability will be sent directly to Adobe computers, raising potential privacy concerns for users. Furthermore, this beta revealed aggressive censorship measures: Our art director attempted to draw “a diagram of a human breast, anatomical self-exam” prompt but this automatically led to an immediately blocked content flag being issued against their work.
Adobe claims this process ensures images are used responsibly; any contributions to Adobe Stock or Firefly-generated images are pre-processed for content that violates our terms of service; violative content is either removed from prompts, or the prompt is blocked altogether; examples could range from nudes, third-party brands, and copyrighted cartoon characters – something many professional users find problematic when trying to complete work that is completely ethical but doesn’t fit within Adobe’s definition of what is or is not appropriate work.
Adobe has taken great strides to protect its creative community with an ethical approach that has proven highly successful in training Firefly’s AI model with Adobe Stock photos and public domain images, according to Adobe. Unfortunately, however, they still have not announced a plan to compensate the authors of these images used to train Firefly; “When this gets out of beta we plan to share details on how we plan to compensate creators.”
Content Credentials, introduced with this version of Photoshop, serve as a free, open-source digital ‘nutrition label’ that helps increase trust in online content through Adobe’s Content Authenticity Initiative (CAI). Credentials will be automatically appended to images modified with Firefly’s generative AI.
“AN UNKNOWN FUTURE
My disappointment with Firefly and Photoshop may stem from mismanaged expectations, but the reality is that, while its early reviews were positive, Firefly doesn’t stand up to professional usage beyond outpainting and object removal – essential tools but none that come close to realizing its full potential when compared with experimental tools such as DragGAN created by researchers at Max Planck Institute which can change any photo with extraordinary realism and accuracy.
Perhaps Adobe will hit AI out of the park one day, and I may fall deeply in love with their program a la the 90s. Meanwhile, they should accelerate adoption of new technologies by being truly courageous or else risk becoming irrelevant as researchers and startups emerge to compete with them.