About DreamFusion

DreamFusion is an innovative technology developed by Google Research and UC Berkeley that leverages a pretrained 2D text-to-image diffusion model to perform text-to-3D synthesis. This technology is designed to overcome the limitations of 3D synthesis, such as the need for large-scale datasets of labeled 3D assets and efficient architectures for denoising 3D data.

Here are four key features of DreamFusion

  1. Text-to-3D Synthesis: DreamFusion can convert text into 3D models. It uses a loss based on probability density distillation that enables the use of a 2D diffusion model as a prior for optimization of a parametric image generator.
  2. High-Fidelity 3D Objects: Given a caption, DreamFusion generates relightable 3D objects with high-fidelity appearance, depth, and normals. These objects are represented as a Neural Radiance Field and leverage a pretrained text-to-image diffusion prior.
  3. No 3D Training Data Required: DreamFusion requires no 3D training data and no modifications to the image diffusion model, demonstrating the effectiveness of pretrained image diffusion models as priors.
  4. Mesh Exports: The generated 3D models can be exported to meshes using the marching cubes algorithm for easy integration into 3D renderers or modeling software.