diff --git a/topics/02_passive/images/The_matrix_rootfop_bullet_dodge.mp4 b/topics/02_passive/images/The_matrix_rootfop_bullet_dodge.mp4 new file mode 100644 index 0000000000000000000000000000000000000000..6450a0cc173ba2cff04f625acf581abc686c978d Binary files /dev/null and b/topics/02_passive/images/The_matrix_rootfop_bullet_dodge.mp4 differ diff --git a/topics/02_passive/index.md b/topics/02_passive/index.md index 6d4b4485157b8a3090371077bf32e5e0177690f8..2e3cbb0eb42c29a70a9f5c2daf08c9dbb39ef323 100644 --- a/topics/02_passive/index.md +++ b/topics/02_passive/index.md @@ -107,20 +107,32 @@ iPhone and Android apps for photogrammetry and now LiDAR scanning have multiplie _plenoptic_: Of or relating to all the light, travelling in every direction, in a given space. -Light fields represent an advanced form of passive sensing, aiming to capture full plenoptic content: all possible light rays emanating from a scene in any given direction. This results in a four-dimensional function, as it involves selecting a ray's position and angle. If the ideal plenoptic function was known, any novel viewpoint could be synthesized by placing a virtual camera in this space, and selecting the relevant light rays. +<small>Light fields represent an advanced form of passive sensing, aiming to capture full plenoptic content: all possible light rays emanating from a scene in any given direction. This results in a four-dimensional function, as it involves selecting a ray's position and angle. If the ideal plenoptic function was known, any novel viewpoint could be synthesized by placing a virtual camera in this space, and selecting the relevant light rays.</small> + + +"We begin by asking what can potentially be seen" - Edward H. Adelson & James R. Bergen, Media Lab, Vision & Modeling Group: +[The Plenoptic Function and the Elements of Early Vision](https://persci.mit.edu/pub_pdfs/elements91.pdf) -Adelson et al. Why do we want all of the light? + Image-Based Rendering (IBR) for view synthesis is a long-standing problem in the field of computer vision and graphics. Applications in robot navigation, film, and AR/VR. -This is such an intensive calculation, that it prompts researchers to seek simulation shortcuts to reach this result: +Basically, the bullet time effect from the matrix. +<video width="320" height="240" controls> + <source src="images/The_matrix_rootfop_bullet_dodge.mp4" type="video/mp4"> +</video> + +<iframe width="560" height="315" src="https://www.youtube.com/embed/9XM5-CJzrU0?si=eWG2ipj0fqA1Ng13&start=73" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> + + + -[Using thousands of virtual cameras] (https://openaccess.thecvf.com/content/ACCV2022/papers/Li_Neural_Plenoptic_Sampling_Learning_Light-field_from_Thousands_of_Imaginary_Eyes_ACCV_2022_paper.pdf) -Papers proposing the use of thousands of virtual cameras and neural networks to capture a complete dense plenoptic function +This is such an intensive calculation, that it prompts researchers to seek simulation shortcuts to reach this result, such as this paper [using thousands of virtual cameras](https://openaccess.thecvf.com/content/ACCV2022/papers/Li_Neural_Plenoptic_Sampling_Learning_Light-field_fro +m_Thousands_of_Imaginary_Eyes_ACCV_2022_paper.pdf) and neural networks to capture a complete dense plenoptic function. In practice, we can only sample light rays in discrete locations. There are two popular optical architectures for this: @@ -132,7 +144,7 @@ Simply shoot the scene from several locations using an array of camera (or a sin ### Lenslets Lenslets: a single CMOS sensor with an array of lenses in front. -In the lenslet approach, each pixel behind a lenslet provides a unique light ray direction. The collection for all lenses is called a <strong>sub aperture image</strong>, and roughly corresponds to what a shifted camera would capture. The resolution of these images is simply the total number of lenslets, and the number of sub-aperture images available is given by the number of pixels behind a lenslet. For reference, the [Lytro Illum](https://en.wikipedia.org/wiki/Lytro)> provides 15x15 sub-aperture images of 541x434 pixels each, which is a total of ~53 Megapixels. +In the lenslet approach, each pixel behind a lenslet provides a unique light ray direction. The collection for all lenses is called a <strong>sub aperture image</strong>, and roughly corresponds to what a shifted camera would capture. The resolution of these images is simply the total number of lenslets, and the number of sub-aperture images available is given by the number of pixels behind a lenslet. For reference, the [Lytro Illum](https://en.wikipedia.org/wiki/Lytro) provides 15x15 sub-aperture images of 541x434 pixels each, which is a total of ~53 Megapixels. <img src="images/viewpoint7_png+img+margin.gif" alt="LF sub aperture images">