About GauGAN2 by NVIDIA

GauGAN2 is the latest version of NVIDIA Research’s AI painting demo that allows anyone to create photorealistic art using just a few words or phrases. This deep learning model is based on generative adversarial networks and can generate scenes in real time based on the input provided.

Here are four key features of GauGAN2

  1. Text-to-Image Generation: Users can simply type a phrase like “sunset at a beach,” and the AI generates the scene in real time. Adding an adjective or changing the phrase instantly modifies the picture.
  2. Segmentation Map Generation: Users can generate a segmentation map, a high-level outline that shows the location of objects in the scene. They can then switch to drawing, tweaking the scene with rough sketches using labels like sky, tree, rock, and river.
  3. Combination of Modalities: GauGAN2 is one of the first demos to combine multiple modalities — text, semantic segmentation, sketch, and style — within a single GAN framework. This makes it faster and easier to turn an artist’s vision into a high-quality AI-generated image.
  4. Customization: The AI allows users to create and customize scenes more quickly and with finer control. Users can enter a brief phrase to quickly generate the key features and theme of an image, such as a snow-capped mountain range. This starting point can then be customized with sketches to make a specific mountain taller or add a couple of trees in the foreground, or clouds in the sky.