Varghese Kuruvilla, MS student working with Dr. Ravi Kiran Sarvadevabhatla was awarded the Qualcomm Innovation Fellowship India (2025). Varghese Kuruvillais proposal on PLATO: Generating Objects from Part Lists via Synthesized Layouts was one of the 18 finalized for the award from 150 applications. Here is the proposal written by Varghese Kuruvilla:
AN INTRODUCTION:
Generative AI has transformed digital content creation, enabling text-to-image (T2I) models to synthesize diverse and visually compelling imagery. Apart from scene generation, the problem of controllable object instance generation is also of interest for domains such as game design, animation, product visualization and industrial prototyping.
A controllable way to generate objects is by specifying parts that make up the object (e.g., left arm, head, torso, right leg etc. for a ‘person’). A part-aware generative model would enable users to generate complex objects with precise configurations, providing a measure of control over which parts are visible. For instance, it is possible to generate a cat with only front legs and paws visible by specifying certain parts (Fig. 1). This level of fine-grained control over generated objects can be invaluable for design, asset generation, and automated visual storytelling, where aligning generated content to specific semantic requirements is essential. Beyond improving structural fidelity, part-controllable generation can enable a wide range of practical applications, including modular asset design for animation, interactive character customization, and educational tools involving partial object reasoning.
Modern generative models often struggle to synthesize structured objects from detailed part specifications. They frequently produce anatomically implausible outputs or hallucinated components. To bridge this gap, we propose PLATO: A framework that enables part-controlled object generation.
OUR FRAMEWORK:
We propose PLATO, a novel two-stage framework that enables precise, part-controlled object generation. We envision PLATO consisting of two stages as shown in the figure below.
Our framework consists of two main stages:
- PLayGen (Our novel part layout generator): This takes a list of parts and an object category as input and synthesizes high fidelity layouts of part bounding boxes. This is challenging as the generated layout must be anatomically accurate and adhere to inter-part relationships
- Our Custom Image generation module: In the second stage, PLayGen’s synthesized layout is used to condition a custom-tuned ControlNet-style adapter, enforcing spatial and connectivity constraints.
SOME PROPOSED APPLICATIONS:
- We can take advantage of our two-stage pipeline and enable users to perform part-level edits of the generated layout.
- Fantasy generations:
CONCLUSION:
We propose Plato: A framework for precise, part-controlled object generation. Our design choices enable generation of anatomically accurate layouts and generation of images that adhere to the layouts. To the best of our knowledge, this is the first model that generates instances from part level descriptions and enables applications such as part level editing.
August 2025