English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent advancements in the text-to-3D task leverage finetuned text-to-imagediffusion models to generate multi-view images, followed by NeRFreconstruction. Yet, existing supervised finetuned (SFT) diffusion models stillsuffer from multi-view inconsistency and the resulting NeRF artifacts. Althoughtraining longer with SFT improves consistency, it also causes distributionshift, which reduces diversity and realistic details. We argue that the SFT ofmulti-view diffusion models resembles the instruction finetuning stage of theLLM alignment pipeline and can benefit from RL finetuning (RLFT) methods.Essentially, RLFT methods optimize models beyond their SFT data distribution byusing their own outputs, effectively mitigating distribution shift. To thisend, we introduce Carve3D, a RLFT method coupled with the Multi-viewReconstruction Consistency (MRC) metric, to improve the consistency ofmulti-view diffusion models. To compute MRC on a set of multi-view images, wecompare them with their corresponding renderings of the reconstructed NeRF atthe same viewpoints. We validate the robustness of MRC with extensiveexperiments conducted under controlled inconsistency levels. We enhance thebase RLFT algorithm to stabilize the training process, reduce distributionshift, and identify scaling laws. Through qualitative and quantitativeexperiments, along with a user study, we demonstrate Carve3D's improvedmulti-view consistency, the resulting superior NeRF reconstruction quality, andminimal distribution shift compared to longer SFT. Project webpage:https://desaixie.github.io/carve-3d.

Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning