AR Marketing App - Christmas Gamified Experience
A gamified Christmas-themed augmented reality marketing experience built during ARKit's first weeks of availability in September 2017. The app combined horizontal plane detection with face tracking for an interactive campaign celebrating Latvia's 2018 Dog Year. Users scanned surfaces to trigger animated Santa characters that opened surprise boxes, unlocking keys to 50 different dog breed face filter masks. Deployed via TestFlight at a marketing event with 250+ users.
Problem Solved
CatchSmart needed an innovative marketing campaign that would demonstrate technical leadership and create memorable brand experiences during the Christmas season
- •Architected dual AR mode system switching between plane detection and face tracking
- •Managed 50 different 3D dog breed masks with real-time face tracking at 60fps
- •Coordinated with 3D freelancer on asset creation and optimization
- •Built multimedia synchronization of 3D animations, audio, and haptic feedback
- •Deployed and supported 250+ users at marketing event via TestFlight
- ✓Delivered complete gamified AR experience in 3 weeks during ARKit's first month of availability
- ✓Pioneered dual AR mode architecture switching between plane detection and face tracking when simultaneous operation was impossible
- ✓Managed 50 different 3D dog breed masks with real-time face tracking at smooth 60fps
- ✓Created interactive Christmas narrative flow from surface detection through unlock mechanics to face filters
- ✓Deployed to 250+ users via TestFlight open beta at marketing event
- ✓Collaborated with 3D freelancer to create and optimize 50 dog masks plus animated Santa characters
- ✓Solved technical problems with zero community resources when ARKit documentation was sparse and no Stack Overflow answers existed
- ✓Built multimedia experience synchronizing 3D animations, Christmas audio, haptic feedback, and social sharing
Performance
- • 60fps real-time face tracking
- • 3-week development timeline
Scale
- • 50 dog breed face masks
- • 250+ beta testers
Technology Stack
Challenge
ARKit had just been released at WWDC 2017 with sparse documentation, no Stack Overflow answers, and no established community patterns
Solution
Trial and error experimentation, reading Apple framework headers directly, and discovering solutions through necessity that would later become best practices
Impact
Developed ability to pioneer with bleeding-edge technology without safety net of community knowledge
Challenge
ARKit 1.0 couldn't run face tracking and plane detection simultaneously, requiring complete context switch between modes
Solution
Carefully orchestrated state management shutting down plane detection AR session, reinitializing face tracking session, while maintaining game narrative flow through the transition
Impact
Seamless user experience across two fundamentally different AR modes
Challenge
50 different dog breed masks in memory simultaneously risked crashes on older devices
Solution
Painstaking memory optimization, lazy loading strategies, and aggressive polygon reduction on 3D models from freelancer
Impact
Stable operation across all ARKit-capable devices from iPhone 6s to iPhone X
Challenge
OBJ files from freelancer stripped materials and textures during SceneKit conversion, and models were too high-poly for mobile performance
Solution
Manual reconstruction of materials in SceneKit, polygon reduction on each of 50 masks without losing distinctive breed features, maintaining 60fps target
Impact
Production-quality 3D assets running at smooth frame rates on mobile hardware
Challenge
Face tracking required maintaining 60fps or masks would lag behind user movements, breaking immersion
Solution
Continuous performance profiling and optimization, balancing visual quality with frame rate consistency across device range
Impact
Smooth, responsive face tracking experience that felt magical rather than technical
Situation
In September 2017, Apple had just released ARKit weeks earlier, and CatchSmart wanted to create a cutting-edge Christmas marketing campaign that would showcase their innovation capabilities. AR was completely novel to consumers in Latvia, and no established patterns existed for building ARKit applications.
Task
Build a complete gamified AR experience combining surface detection for animated Santa characters with face tracking for dog breed masks, celebrating Latvia's 2018 Dog Year, in just 3 weeks for a marketing event.
Action
Khaled pioneered with technology that was literally weeks old, solving problems that had no Stack Overflow answers by reading Apple framework headers and experimenting. He architected a dual AR mode system that elegantly handled the constraint that ARKit 1.0 couldn't run face tracking and plane detection simultaneously. Working with a 3D freelancer, he managed 50 dog breed masks through an optimization pipeline converting OBJ files to SceneKit, reducing polygon counts, and rebuilding stripped materials. He synchronized 3D animations, Christmas music via AVAudioEngine, haptic feedback via UIFeedbackGenerator, and social sharing into one cohesive experience. The entire multimedia system had to coordinate perfectly while maintaining 60fps face tracking performance.
Result
The app launched at a marketing event where 250+ users experienced AR for the first time. Users were delighted as Santa characters ran across their tables and they transformed into dog breeds. Being first with ARKit in Latvia created significant marketing buzz, demonstrating CatchSmart's technical innovation capabilities. The project proved the ability to rapidly ship production-quality experiences with bleeding-edge frameworks.
Technical
- • ARKit session lifecycle management and dual-mode architecture
- • 3D asset pipeline from OBJ conversion to SceneKit optimization
- • Mobile GPU performance engineering for 60fps requirements
- • Multimedia system orchestration across multiple frameworks
- • Memory optimization for real-time 3D rendering
Soft Skills
- • Pioneering without documentation or community support
- • Rapid prototyping under tight deadlines
- • Collaborating with 3D freelancers and communicating technical constraints
- • User-centric design for novel interaction paradigms
Key Insights
- 💡 Being first with bleeding-edge technology creates disproportionate impact even if implementation isn't perfect
- 💡 AR's power comes from blending digital experiences with physical space in ways that feel magical
- 💡 Technical capability means nothing if users can't access it - onboarding novel interactions is as hard as building them