English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Generating and editing a 3D scene guided by natural language poses achallenge, primarily due to the complexity of specifying the positionalrelations and volumetric changes within the 3D space. Recent advancements inLarge Language Models (LLMs) have demonstrated impressive reasoning,conversational, and zero-shot generation abilities across various domains.Surprisingly, these models also show great potential in realizing andinterpreting the 3D space. In light of this, we propose a novel language-guidedinteractive 3D generation system, dubbed LI3D, that integrates LLMs as a 3Dlayout interpreter into the off-the-shelf layout-to-3D generative models,allowing users to flexibly and interactively generate visual content.Specifically, we design a versatile layout structure base on the bounding boxesand semantics to prompt the LLMs to model the spatial generation and reasoningfrom language. Our system also incorporates LLaVA, a large language and visionassistant, to provide generative feedback from the visual aspect for improvingthe visual quality of generated content. We validate the effectiveness of LI3D,primarily in 3D generation and editing through multi-round interactions, whichcan be flexibly extended to 2D generation and editing. Various experimentsdemonstrate the potential benefits of incorporating LLMs in generative AI forapplications, e.g., metaverse. Moreover, we benchmark the layout reasoningperformance of LLMs with neural visual artist tasks, revealing their emergentability in the spatial layout domain.

Towards Language-guided Interactive 3D Generation: LLMs as Layout Interpreter with Generative Feedback