Incorporating physics into data-driven computer vision

A Kadambi, C de Melo, CJ Hsieh… - Nature Machine …, 2023 - nature.com
Many computer vision techniques infer properties of our physical world from images.
Although images are formed through the physics of light and mechanics, computer vision …

Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion

V Voleti, CH Yao, M Boss, A Letts, D Pankratz… - … on Computer Vision, 2024 - Springer
Abstract We present Stable Video 3D (SV3D)—a latent video diffusion model for high-
resolution, image-to-multi-view generation of orbital videos around a 3D object. Recent …

Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering

A Guédon, V Lepetit - … of the IEEE/CVF Conference on …, 2024 - openaccess.thecvf.com
We propose a method to allow precise and extremely fast mesh extraction from 3D Gaussian
Splatting. Gaussian Splatting has recently become very popular as it yields realistic …

Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing

J Gao, C Gu, Y Lin, Z Li, H Zhu, X Cao, L Zhang… - … on Computer Vision, 2024 - Springer
In this paper, we present a novel differentiable point-based rendering framework to achieve
photo-realistic relighting. To make the reconstructed scene relightable, we enhance vanilla …

F2-nerf: Fast neural radiance field training with free camera trajectories

P Wang, Y Liu, Z Chen, L Liu, Z Liu… - Proceedings of the …, 2023 - openaccess.thecvf.com
This paper presents a novel grid-based NeRF called F^ 2-NeRF (Fast-Free-NeRF) for novel
view synthesis, which enables arbitrary input camera trajectories and only costs a few …

Bakedsdf: Meshing neural sdfs for real-time view synthesis

L Yariv, P Hedman, C Reiser, D Verbin… - ACM SIGGRAPH 2023 …, 2023 - dl.acm.org
We present a method for reconstructing high-quality meshes of large unbounded real-world
scenes suitable for photorealistic novel view synthesis. We first optimize a hybrid neural …

Ref-nerf: Structured view-dependent appearance for neural radiance fields

D Verbin, P Hedman, B Mildenhall… - 2022 IEEE/CVF …, 2022 - ieeexplore.ieee.org
Neural Radiance Fields (NeRF) is a popular view synthesis technique that represents a
scene as a continuous volumetric function, parameterized by multilayer perceptrons that …

Delicate textured mesh recovery from nerf via adaptive surface refinement

J Tang, H Zhou, X Chen, T Hu, E Ding… - Proceedings of the …, 2023 - openaccess.thecvf.com
Abstract Neural Radiance Fields (NeRF) have constituted a remarkable breakthrough in
image-based 3D reconstruction. However, their implicit volumetric representations differ …

Extracting triangular 3d models, materials, and lighting from images

J Munkberg, J Hasselgren, T Shen… - Proceedings of the …, 2022 - openaccess.thecvf.com
We present an efficient method for joint optimization of topology, materials and lighting from
multi-view image observations. Unlike recent multi-view reconstruction approaches, which …

Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction

P Wang, L Liu, Y Liu, C Theobalt, T Komura… - arxiv preprint arxiv …, 2021 - arxiv.org
We present a novel neural surface reconstruction method, called NeuS, for reconstructing
objects and scenes with high fidelity from 2D image inputs. Existing neural surface …