Differentiable rendering: A survey

H Kato, D Beker, M Morariu, T Ando… - arxiv preprint arxiv …, 2020 - arxiv.org
Deep neural networks (DNNs) have shown remarkable performance improvements on
vision-related tasks such as object detection or image segmentation. Despite their success …

Extracting triangular 3d models, materials, and lighting from images

J Munkberg, J Hasselgren, T Shen… - Proceedings of the …, 2022 - openaccess.thecvf.com
We present an efficient method for joint optimization of topology, materials and lighting from
multi-view image observations. Unlike recent multi-view reconstruction approaches, which …

Shape, light, and material decomposition from images using monte carlo rendering and denoising

J Hasselgren, N Hofmann… - Advances in Neural …, 2022 - proceedings.neurips.cc
Recent advances in differentiable rendering have enabled high-quality reconstruction of 3D
scenes from multi-view images. Most methods rely on simple rendering algorithms: pre …

Differentiable signed distance function rendering

D Vicini, S Speierer, W Jakob - ACM Transactions on Graphics (TOG), 2022 - dl.acm.org
Physically-based differentiable rendering has recently emerged as an attractive new
technique for solving inverse problems that recover complete 3D scene representations from …

Iron: Inverse rendering by optimizing neural sdfs and materials from photometric images

K Zhang, F Luan, Z Li… - Proceedings of the IEEE …, 2022 - openaccess.thecvf.com
We propose a neural inverse rendering pipeline called IRON that operates on photometric
images and outputs high-quality 3D content in the format of triangle meshes and material …

Large steps in inverse rendering of geometry

B Nicolet, A Jacobson, W Jakob - ACM Transactions on Graphics (TOG), 2021 - dl.acm.org
Inverse reconstruction from images is a central problem in many scientific and engineering
disciplines. Recent progress on differentiable rendering has led to methods that can …

Dr. jit: A just-in-time compiler for differentiable rendering

W Jakob, S Speierer, N Roussel, D Vicini - ACM Transactions on …, 2022 - dl.acm.org
DR. JIT is a new just-in-time compiler for physically based rendering and its derivative. DR.
JIT expedites research on these topics in two ways: first, it traces high-level simulation code …

Recursive control variates for inverse rendering

B Nicolet, F Rousselle, J Novak, A Keller… - ACM Transactions on …, 2023 - dl.acm.org
We present a method for reducing errors---variance and bias---in physically based
differentiable rendering (PBDR). Typical applications of PBDR repeatedly render a scene as …

Path replay backpropagation: Differentiating light paths using constant memory and linear time

D Vicini, S Speierer, W Jakob - ACM Transactions on Graphics (TOG), 2021 - dl.acm.org
Differentiable physically-based rendering has become an indispensable tool for solving
inverse problems involving light. Most applications in this area jointly optimize a large set of …

DIB-R++: learning to predict lighting and material with a hybrid differentiable renderer

W Chen, J Litalien, J Gao, Z Wang… - Advances in …, 2021 - proceedings.neurips.cc
We consider the challenging problem of predicting intrinsic object properties from a single
image by exploiting differentiable renderers. Many previous learning-based approaches for …