Nvidia GauGAN will let people create their own synthetic scenery

Nvidia’s GauGAN software model produces realistic landscape images from a rough sketch or segmentation map. The new generative adversarial network model will initially be a tool in the hands of urban planners, game designers, and architects to quickly create synthetic images. The so-called “smart paintbrush” will fill in the details on the sketch with real objects it knows from the millions of images (including 41,000 from Flickr) it was trained on.

"It's like a coloring book picture that describes where a tree is, where the sun is, where the sky is," Nvidia vice president of applied deep learning research Bryan Catanzaro said. "And then the neural network is able to fill in all of the detail and texture, and the reflections, shadows, and colors, based on what it has learned about real images."

The model wouldn’t just randomly render images in real-time, but can also “switch the styling between different seasons, and how water reflected and interacted with the landscape” – reported a news website after having seen the demonstration of GauGAN at the GTC Conference on Monday. The machine that was used for the task, however, contained Nvidia’s recently-released ‘world’s most powerful’ Titan RTX desktop GPU. Nevertheless, Catanzaro also assured that the application can also run on a CPU only “if the rendering of the image was limited to once every few seconds or created on-demand.”

Catanzaro and the researchers also said that “GauGAN has a better vocabulary, and required fewer parameters than the other algorithms” competing in the same space. Nvidia also has a history of creating artificial brain MRI imagery using generative adversarial networks.