Project Overview

Inspiration

Team

Sylvan Shen Rimbik Das

Xinchan JIN Grirish M

Meghna Sonie

Skills

Unity MR Development

AI & XR Pipeline Development

Product Thinking

Timeline

2 weeks tech stack research

3 Day Hackthon

August - October 2024

Role

Lead Unity Developer

AI Feature Prototyper

Project Manager

** This project was developed during XR Hack London 2024 where our team won Meta's Design & Utility Category. The project is currently paused with plans to revisit development in the future. This portfolio entry documents my contributions during the hackathon. For a comprehensive view of the product, please see our pitch deck and demo video. If you're interested in collaborating on future development, feel free to reach out.

Why IntelliSpace?

Market Validation

The journey of IntelliSpace began during a collaborative brainstorming session when our 3D artist, Rimbik Das, proposed exploring the home design space. We were inspired by emerging AI-powered web solutions like RoomGPT that transform users' spaces simply by uploading photos. This sparked our curiosity about the possibilities at the intersection of AI and spatial design powered by mixed reality interactions.

Our brief market exploration revealed a significant gap in the Meta Quest Store's existing home design applications. While current solutions allowed users to select and manipulate predefined furniture from catalogs—functionality we also recognized as essential—we identified an opportunity to differentiate ourselves through advanced AI integration.

The pivotal moment came after discovering the hackathon organizers' AI toolkit demonstration. Though the showcased technology only utilized basic computer vision for object recognition in static screenshots, it triggered a more ambitious vision: What if users could capture any corner of their physical space in real-time through mixed reality and receive AI-generated style transformation recommendations? This would effectively bring the power of solutions like RoomGPT into an immersive, interactive MR environment.

This initial concept rapidly evolved as we explored AI's potential role in home design. We planned to expand our AI capabilities to include personalized feng shui consultations through an AI assistant and I proposed custom texture generation for furniture surfaces. After initial technical feasibility evaluation and resources planning, we finalized our AI feature set to include:

• AI-powered design inspirations using ControlNet API

• Virtual design assistant providing real-time suggestions

• Custom texture generation for personalized aesthetics

While our hackathon approach began from a technology exploration perspective—which successfully helped us identify unique value propositions that existing solutions weren't offering—we recognized the importance of validating our concept through proper market research. If we are going to build a successful product, I understand that human-centered value must drive technology implementation rather than the reverse.

Post-hackathon, we conducted more systematic market research that revealed compelling evidence supporting our concept, and we found that:

• 63% of DIY home designers ultimately regret their projects and often hire professionals to fix mistakes

• Homeowners typically lose 10-15% of their design budgets due to errors in visualization and planning

• Users consistently struggle with three core challenges:

  • Visualization difficulties for furniture placement, dimensions, and styles

  • Professional design services face miscommunication challenges, causing delays and higher costs.

  • Disconnection between creative design concepts and available retail products

Looking for market fit reinforced our value proposition while prompting us to rethink certain aspects of our technology implementation. For instance, we discovered that our custom texture generation feature, while technically impressive, presented significant feasibility challenges due to disconnects in the supply chain between virtual design and physical product availability.

This validation process transformed our understanding of IntelliSpace from a technology showcase into a potential solution for genuine market needs. It reinforced the importance of balancing technological innovation with practical user requirements—a key insight that would guide our approach to further development.

Technical Contributions

During the hackathon, I led the development of AI integration features for IntelliSpace. I have always been eager to expand my Unity capabilities through AI integration, and this project provided the perfect opportunity to apply this combined skillset. Working within the intense time constraints of the hackathon environment, I successfully:

• I Evaluated multiple AI model options to determine the most suitable solution for our design inspiration feature, ultimately selecting ControlNet for its powerful image-to-image capabilities.

  • Implemented ControlNet's line detection model, which excels at preserving structural elements while transforming visual styles—ideal for maintaining furniture placement and room layout while generating new design aesthetics.

• I identified Replicate as the optimal hosting platform, allowing us to access pre-trained ControlNet models through their API infrastructure.

• I developed a custom integration between Unity and the Replicate API within our mixed reality environment, allowing the application to capture room layouts from the MR environment and send them to the AI model for transformation.

Initial Prototype

Hackthon Delivery at London PwC office

Post-hackthon Polishing

AI Feature Prototyping

→

→

Beyond the core ControlNet implementation, I expanded our AI capabilities by:

• Integrated OpenAI's DALL-E API to power our texture customization feature, enabling users to generate unique material finishes through natural language prompts.

• Developed a seamless connection between Unity's UI system and the DALL-E API, creating an intuitive interface for texture generation and application to 3D objects.

• Implemented a texture selection and preview system that allowed users to evaluate AI-generated textures before applying them to their virtual furniture.

→

Initial Prototype

Post-hackthon Polishing

Challenges & Solutions

The integration revealed several areas for improvement that would enhance future iterations:

• A Meta mentor suggested replacing text input with voice commands, which would significantly improve the immersive quality of the experience and simplify user interactions in the MR environment.

• UV mapping presented consistent challenges for procedurally applied textures, particularly with complex furniture geometries.

  • To address this, I implemented a creative workaround for tiling textures by incorporating the keyword "seamless" in generation prompts, which helped DALL-E produce more appropriate textures for 3D application.

Achievement & Reflections

Our team's efforts culminated in winning Meta's Design & Utility Category at XR Hack London 2024, which validated our innovative approach to merging AI with mixed reality for home design. This recognition also earned us the opportunity to showcase IntelliSpace at AWE Vienna 2024, placing our work in front of a global audience of XR professionals.

During these events, we actively engaged with industry practitioners, gathering diverse feedback that proved invaluable for understanding our concept's market potential. These conversations ranged from enthusiastic support to constructive criticism that challenged our assumptions. One particularly memorable interaction came from an interior designer who bluntly stated that professional designers would never adopt our solution. This feedback, though initially discouraging, provided critical perspective on the importance of clearly defining our target users and value proposition—perhaps focusing more on homeowners and retailers rather than design professionals.

Although IntelliSpace remains at the prototype stage without immediate plans for commercialization, the project represents a pivotal moment in my professional development. It served as my introduction to product thinking and demonstrated the significant gap between hackathon projects and market-ready products. Through this experience, I've gained:

• A deeper appreciation for human-centered design, recognizing that technological innovation must address genuine user needs to create lasting value.

• Skills in rapid prototyping and feature prioritization, especially in high-pressure environments with constrained resources.

• An awareness of business viability considerations that must complement technical feasibility.

• A growing interest in entrepreneurial thinking and product development methodology.

The project also highlighted the importance of staying current with the evolving technology landscape, particularly as AI capabilities advance rapidly. Future opportunities will continue to emerge to enhance spatial design applications through multimodal AI systems, and more sophisticated spatial understanding capabilities.

This hackathon experience, while brief, has inspired me to further develop my product design skills alongside my technical expertise—creating solutions that meaningfully bridge technology capabilities with human needs.

*(Left-right: Me, Rimbik Das, Meghna Sonie, Xinchan Jin)

Group photo of all the winners (Global)

Future Plan

As IntelliSpace evolves beyond its hackathon origins and initial showcase to the public, I believe several strategic initiatives will transform this prototype into a viable product:

• Continuously follow developments in generative AI to enhance and update our AI systems and features.

• Conduct comprehensive user testing with our prototype to validate assumptions and refine experiences.

• Gather quantitative data on feature usage and user willingness to pay for various feature sets.

• Explore collaborations with interior design professionals and property developers to validate market fit.