75 lines
No EOL
7.3 KiB
Markdown
75 lines
No EOL
7.3 KiB
Markdown
---
|
|
title: Radiance Fields
|
|
---
|
|
NeRF, MIP-NeRF, 3D Gaussian Splatting, and more
|
|
|
|
<svg style="width:60%; margin-top:2rem;" viewBox="2.9938 7.971 92.0062 32.0289" xmlns="http://www.w3.org/2000/svg">
|
|
<path style="fill:#FFD600;" d="M7,10.971l-3,8c0,0,1.633,0.124,3.233,0.575c2.077,0.982,1.97,3.073,1.249,5.039 	C8.33,24.712,8.175,24.839,8,24.971c-2.813,2.125-4.688,9-4.688,9L11,38.252c0,0-0.834-9.281,3-9.281h0.011 	c1.39,1.159,2.989,2,2.989,2c0.09,0,0.342,0,1,0c2.001,0,3.739-0.781,4.823-2h15.735l0.545,1.633 	c0.272,0.817,1.036,1.367,1.897,1.367c1.105,0,2-0.895,2-2v-19H7z M14,25.971c0-1.657,1.791-3,4-3c2.209,0,4,1.343,4,3s-1.791,3-4,3 	C15.791,28.971,14,27.628,14,25.971z" transform="matrix(1, 0, 0, 1, 7.105427357601002e-15, 0)"/>
|
|
<path style="fill:#455A64;" d="M40,24.971H25v-10h15V24.971z M10,8.971v-1H8v3h13v-2H10z M9.836,39.883 	C10.342,40.2,11,39.835,11,39.238v-0.985l-7.688-4.281c0,0-0.123,0.467-0.282,1.128c-0.131,0.546,0.101,1.117,0.577,1.415 	L9.836,39.883z" transform="matrix(1, 0, 0, 1, 7.105427357601002e-15, 0)"/>
|
|
<path d="M40,18.971H28v-2h12V18.971z M40,20.971H28v2h12V20.971z" transform="matrix(1, 0, 0, 1, 7.105427357601002e-15, 0)"/>
|
|
<path style="fill:#FF6D00;" d="M43,19.971h2v-5h-2V19.971z M40,24.971h-2v-10h2V24.971z M18,25.971c0,1.103,0.508,2.096,1.321,2.819 	c-0.415,0.11-0.855,0.181-1.321,0.181c-2.209,0-4-1.343-4-3s1.791-3,4-3c0.465,0,0.905,0.071,1.32,0.181 	C18.508,23.875,18,24.868,18,25.971z" transform="matrix(1, 0, 0, 1, 7.105427357601002e-15, 0)"/>
|
|
<path style="fill:#90A4AE;" d="M22,10.971l-2,6H4.75l0.658-1.755l-1.041-0.347C3.551,14.596,3,13.832,3,12.971c0-1.105,0.895-2,2-2 	H22z M14,28.971c0,0-1.416-1.25-4-2c-3.371,1.908-4.225,6.279-4.547,8.192L11,38.252C11,38.252,10.583,28.971,14,28.971z" transform="matrix(1, 0, 0, 1, 7.105427357601002e-15, 0)"/>
|
|
<g transform="matrix(1, 0, 0, 1, 13.000000000000007, 0)">
|
|
<rect x="65" y="16" width="15" height="3" style="stroke: rgb(0, 0, 0); stroke-opacity: 0; fill: rgb(30, 66, 154);"/>
|
|
<path style="stroke: rgb(0, 0, 0); stroke-opacity: 0; fill: rgb(255, 109, 0);" d="M 80 16 H 81 A 1 1 0 0 1 82 17 V 18 A 1 1 0 0 1 81 19 H 80 V 16 Z"/>
|
|
</g>
|
|
<rect x="71.762" y="16.199" width="5" height="0.5" style="stroke: rgb(0, 0, 0); stroke-opacity: 0; fill: rgba(255, 255, 255, 0.23);" transform="matrix(1, 0, 0, 1, 7.105427357601002e-15, 0)"/>
|
|
<rect x="72.587" y="18.292" width="5" height="0.5" style="stroke: rgb(0, 0, 0); stroke-opacity: 0; fill: rgba(255, 255, 255, 0.23);" transform="matrix(1, 0, 0, 1, 7.105427357601002e-15, 0)"/>
|
|
<rect x="65.294" y="17.261" width="10.001" height="0.5" style="stroke: rgb(0, 0, 0); stroke-opacity: 0; fill: rgba(255, 255, 255, 0.23);" transform="matrix(1, 0, 0, 1, 7.105427357601002e-15, 0)"/>
|
|
</svg>
|
|
|
|
## Intro
|
|
|
|
The concept of using 3D scanning as an alternative means for people to interact with their surroundings is intriguing. Museums could 3D scan their entire collections, capturing high-quality scans of artifacts and exhibits that can be shared with a global audience. This technology also enables the preservation of monuments at risk of being lost forever, allowing people to explore and learn from these cultural features remotely. Moreover, education can be enhanced by introducing more active learning methods based on working in 3D, which differing methods of constructing [radiance fields](https://www.reedbeta.com/blog/the-radiance-field/) make simpler.
|
|
|
|
Techniques such as NeRF, [MIP-NeRF](https://arxiv.org/abs/2103.13415), [3D Gaussian Splatting](https://arxiv.org/abs/2308.04079), and others demonstrate a fast and simpler method of 3D scanning compared to traditional laser or projection scanning. These methods are not considered "production-ready." These models are made up of very dense "point clouds" that cannot easily be translated into CAD. Supplementary software is getting better, but laser and projection scanning are still the best for high-accuracy work.
|
|
|
|
3D guassian spatting has been a new focus of research. You can use an entirely software-defined approach to create very light (the below model is 6 MB) and high-quality 3D models. With [polygon-based](https://en.wikipedia.org/wiki/Polygonal_modeling) models, you would have never been able to get this amount of accuracy in a 3D model that can run on a phone.
|
|
|
|
## Showcase
|
|
|
|
### 3D Gaussian Splatting
|
|
|
|
Below you can see an example of a 3D guassian "splat" using the method Splatfacto, created by the engineers working on the [Nerf Studio](https://docs.nerf.studio/) project inspired by the SIGGRAPH paper "[3D Gaussian Splatting for Real-Time Rendering of Radiance Fields](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/)".
|
|
|
|
<head>
|
|
<link rel="stylesheet" href="styles.css">
|
|
</head>
|
|
<br>
|
|
<div class="hextra-card hx-group hx-flex hx-flex-col hx-justify-start hx-overflow-hidden hx-rounded-lg hx-border hx-border-gray-200 hx-text-current hx-no-underline dark:hx-shadow-none hover:hx-shadow-gray-100 dark:hover:hx-shadow-none hx-shadow-gray-100 active:hx-shadow-sm active:hx-shadow-gray-200 hx-transition-all hx-duration-200 hover:hx-border-gray-300 hx-bg-gray-100 hx-shadow dark:hx-border-neutral-700 dark:hx-bg-neutral-800 dark:hx-text-gray-50">
|
|
<div id="container">
|
|
<div id="progress-container">
|
|
<dialog open id="progress-dialog">
|
|
<p>
|
|
<label for="progress-indicator">Waiting</label>
|
|
</p>
|
|
<progress max="100" id="progress-indicator"></progress>
|
|
</dialog>
|
|
</div>
|
|
<canvas id="canvas"></canvas>
|
|
<script type="module" src="script.js"></script>
|
|
</div>
|
|
<button style="margin:0.5rem; padding:0.25rem; padding-left: 1rem; padding-right: 1rem;" class="not-prose hx-font-medium hx-cursor-pointer hx-px-6 hx-py-3 hx-rounded-lg hx-text-center hx-text-white hx-inline-block hx-bg-primary-600 hover:hx-bg-primary-700 focus:hx-outline-none focus:hx-ring-4 focus:hx-ring-primary-300 dark:hx-bg-primary-600 dark:hover:hx-bg-primary-700 dark:focus:hx-ring-primary-800 hx-transition-all hx-ease-in hx-duration-200" id="load-button">Load Scene</button>
|
|
<div style="display: flex;justify-content: space-between; padding: 0.5rem; padding-top: 0;">
|
|
<a href="https://github.com/huggingface/gsplat.js/">gsplat.js</a>
|
|
<p style="margin:0px;text-decoration: none;"> left click <strong>rotate</strong>, right click <strong>pan</strong>
|
|
</p>
|
|
</div>
|
|
</div>
|
|
|
|
### NeRF
|
|
|
|
Below are some pre-rendered photos based on the [Nerfacto](https://docs.nerf.studio/nerfology/methods/Nerfacto.html) method also developed by nerfstudio. Though I have found that in some context the "splatting" method produces a smaller file and is more efficient to run, NeRF still provides pretty excellent quality.
|
|
|
|
{{< cards cols="2">}}
|
|
{{< card title="Generated" subtitle="Nerfacto" link="4000-001.min.webp" image="4000-001.min.webp" >}}
|
|
{{< card title="Generated" subtitle="Nerfacto" link="4000-002.min.webp" image="4000-002.min.webp" >}}
|
|
{{< card title="Reference" subtitle="Sony A6400" link="4000-003.min.webp" image="4000-003.min.webp" >}}
|
|
{{< card title="Reference" subtitle="Sony A6400" link="4000-004.min.webp" image="4000-004.min.webp" >}}
|
|
{{< /cards >}}
|
|
|
|
## How-To
|
|
|
|
{{< callout type="info" >}}
|
|
**Hey!** This page is a work in progress. If you'd like to assist in the process of writing, take a look at the [git repository](https://git.myco.systems/mycosystems/midtowndrafting.com)
|
|
{{< /callout >}} |