Congratulations to WEI Yinyi from the CI3 lab for winning the Best Presentation award at the 2024 IEEE International Conference on Automation in Manufacturing, Transportation and Logistics (iCaMaL2024).
Paper title: Graph-augmented Text-based Floorplan Generation
Brief introduction: In the context of floorplan generation, a recent trend has emerged towards incorporating user-provided descriptive texts to deliver a more flexible and user-centric interface, enabling direct user engagement in the design and configuration processes. However, extracting accurate information from text input to characterize corresponding floorplans is challenging due to the latent representation of semantic, geometric, and topological design details. Conversely, graph-based generation methods, while lacking geometric information, exhibit superior performance owing to the intuitive representation of spatial data in graph structures. Building upon this understanding, this paper proposes a dual-stage framework that augments text input with serialized graphs to leverage the complementary strengths of both modalities. The pre-training stage uses script-generated texts to empower a graph generator to produce corresponding graphs from descriptive texts and familiarize a floorplan generator with concatenated text and graph input. During the fine-tuning stage, the pre-trained graph generator and an alternative graph generator based on prompt-driven large language models provide augmented graphical knowledge for the floorplan generator based on manually annotated descriptive texts. Extensive quantitative and qualitative experiments demonstrate the effectiveness of the proposed approach over the vanilla text-based floorplan generation method.