[nobbis]

DreamFusion doesn't directly build a 3D model from those generated images. It starts with a completely random 3D voxel model, renders it from 6 different angles, then asks Stable Diffusion how plausible an image of "X, side view" it is.

It then sprinkles some noise on the rendering, makes Stable Diffusion improve it a little, then adjusts the voxels to produce that image (using differentiable rendering.)

Rinse and repeat for hours.

[shadowgovt]

Thank you for the clarification; I hadn't grokked the algorithm yet.

That's interesting for a couple of reasons. I can see why that works. It also implies that for closed objects, the voxel data on the interior (where no images can see it) will be complete noise, as there's no signal to pick any color or lack of a voxel.

[nobbis]

Yes, although not complete noise – probably empty. Haven't checked but assume there's regularization of the NeRF parameters.

[FeepingCreature]

    text = f"{ref_text}, front cutaway drawing"

Maybe?