How AI is powering groundbreaking Photoshop and Premiere plugins

Over the past five years or so, we’ve heard a lot from the likes of Adobe about how AI – or, more accurately, machine learning – will power the next-generation of creative tools. And now we’re starting to see that become a reality.

At the Siggraph animation conference in Vancouver this week, Nvidia showed off a series of plugins for Photoshop and Premiere that tap into the machine-learning capabilities of its new Turing GPU architecture and Quadro RTX hardware to make the most mundane of tasks run much more smoothly – and with less manual retouch needed afterwards.

Machine learning uses the principles of the current generation of what’s called artificial intelligence to train computers and networks to recognise and manipulate data, using huge datasets and many thousands of processor hours to, for example, allow Lightroom to make increasingly accurate guesses at what’s in your photos or Photoshop to adjust facial expressions.

So far, we’ve seen many examples from Adobe of how machine learning-based tech can be used in ways that genuinely wow you – colourising black-and-white photos or turning quick sketches into 3D models – but most of these have stayed as tech demos only glimpsed at its Sneaks R&D show-off sessions on stage at its conferences.

The tech we saw this week at Siggraph was less visually impressive, but more practically useful on a day-to-day basis. Nvidia has developed a series of Premiere Pro and Photoshop plugins that extend or enhance capabilities those apps already have, and we saw demos of these on Nvidia’s stand.

The first was called Super Rez, which was able to add impressive levels of detail to both hair and foliage in a photo of wolves on a mountain-side that was up-rezzed from 4K to 8K. It’s not quick – there was no progress bar or other indication of time given, but an Nvidia representative told us that it would take around 10 minutes to complete. However, the results were good enough to be used professionally – for example in a large-format poster print.

A similar plugin called Super Zoom had the same effect, but with a crop to effectively zoom in on a particular area.

Next up was InPaint, an AI-enhanced (and brush-based) version of Photoshop’s Content-Aware Fill. Again the results were slower than you’d expect in Photoshop – even running on Nvidia’s $6,300 Quadro RTX 6000 graphics card – but the results were very good indeed. The Nvidia demo artist was able to seamlessly paint out penguins on a cliff, with a roughness to the rocks the replace them that looked realistic and not obviously replicated from a nearby area.

The final plugin we saw was a slow-motion tool for Premiere Pro, which again produced results that were far better on the footage we were shown than Premiere’s built-in tools.

Sadly, Nvidia says that it has no plans to release these plugins – the company says that they’re designed to show what’s possible with the new graphics cards and Nvidia’s NGX SDK. And plugins that only work on graphics cards that cost $2,300 to $10,000 will have limited appeal, though we expect to see similar tech on Nvidia’s much less expensive Turing-based cards for gamers, which are expected to be announced at the Gamescom conference next week.

However, seeing this kind of technology demoed in front of you instead of just up on stage is a step forward of sorts. What’s also notable is that from conversations I had with other companies including Foundry and Allegorthimic at the show, there's more of a focus across the industry – from texture creation to noise reduction – on using machine learning to speed up drudge work.

Again, actual products you can buy or download are rare – but we’ve seen a couple of standalone releases recently from Photoshop plugin developer Topaz Labs that we’d be surprised if they weren’t turned into plugins in the future.

AI Clear was released in June and runs within the company’s own Topaz Studio application, using machine learning to reduce noise in photos. Topaz Studio is free, with the AI Clear ‘Pro Adjustment’ (ie plugin) costing US$59.99 (around $47). This week the company launched the $99.99 ($79) AI Gigapixel, a standalone application for batch processing image resizing. This is similar to NVidia’s Super Rez, though without the requirement of a multi-thousand-pound graphics card. You can see it in action in Topaz’s video below.

Comments

Elsewhere on IDG sites

Read Next...