Incorporating physics into data-driven computer vision

A Kadambi, C de Melo, CJ Hsieh… - Nature Machine …, 2023 - nature.com
Many computer vision techniques infer properties of our physical world from images.
Although images are formed through the physics of light and mechanics, computer vision …

Intrinsicnerf: Learning intrinsic neural radiance fields for editable novel view synthesis

W Ye, S Chen, C Bao, H Bao… - Proceedings of the …, 2023 - openaccess.thecvf.com
Existing inverse rendering combined with neural rendering methods can only perform
editable novel view synthesis on object-specific scenes, while we present intrinsic neural …

A survey on intrinsic images: Delving deep into lambert and beyond

E Garces, C Rodriguez-Pardo, D Casas… - International Journal of …, 2022 - Springer
Intrinsic imaging or intrinsic image decomposition has traditionally been described as the
problem of decomposing an image into two layers: a reflectance, the albedo invariant color …

Differentiable rendering of neural sdfs through reparameterization

SP Bangaru, M Gharbi, F Luan, TM Li… - SIGGRAPH Asia 2022 …, 2022 - dl.acm.org
We present a method to automatically compute correct gradients with respect to geometric
scene parameters in neural SDF renderers. Recent physically-based differentiable …

Diff-lfd: Contact-aware model-based learning from visual demonstration for robotic manipulation via differentiable physics-based simulation and rendering

X Zhu, JH Ke, Z Xu, Z Sun, B Bai, J Lv… - … on Robot Learning, 2023 - proceedings.mlr.press
Abstract Learning from Demonstration (LfD) is an efficient technique for robots to acquire
new skills through expert observation, significantly mitigating the need for laborious manual …

Sam-rl: Sensing-aware model-based reinforcement learning via differentiable physics-based simulation and rendering

J Lv, Y Feng, C Zhang, S Zhao… - … International Journal of …, 2023 - journals.sagepub.com
Model-based reinforcement learning is recognized with the potential to be significantly more
sample efficient than model-free reinforcement learning. How an accurate model can be …

Coupling conduction, convection and radiative transfer in a single path-space: Application to infrared rendering

M Bati, S Blanco, C Coustet, V Eymet, V Forest… - ACM Transactions on …, 2023 - dl.acm.org
In the past decades, Monte Carlo methods have shown their ability to solve PDEs,
independently of the dimensionality of the integration domain and for different use-cases (eg …

Dr. bokeh: differentiable occlusion-aware bokeh rendering

Y Sheng, Z Yu, L Ling, Z Cao… - Proceedings of the …, 2024 - openaccess.thecvf.com
Bokeh is widely used in photography to draw attention to the subject while effectively
isolating distractions in the background. Computational methods can simulate bokeh effects …

Rasterized Edge Gradients: Handling Discontinuities Differentiably

S Pidhorskyi, T Simon, G Schwartz, H Wen… - … on Computer Vision, 2024 - Springer
Computing the gradients of a rendering process is paramount for diverse applications in
computer vision and graphics. However, accurate computation of these gradients is …

End-to-end procedural material capture with proxy-free mixed-integer optimization

B Li, L Shi, W Matusik - ACM Transactions on Graphics (TOG), 2023 - dl.acm.org
Node-graph-based procedural materials are vital to 3D content creation within the computer
graphics industry. Leveraging the expressive representation of procedural materials, artists …