Exploring a virtual Christmas world with many surprises at home in the warmth of a living room. This was possible with our xmasAR app.
The AR tools from Apple and Google allow what was previously only possible with a great deal more effort: augmented reality through the smartphone lens.
We wanted to experience first-hand how easy or complex it really is to conjure up a virtual Christmas world in our clients living rooms with the Apples ARKit.
And so, we embarked on our journey to the promising world of AR.
1. The Vision
As with all projects, in the beginning there is a vision.
Ours was to invite our customers on a voyage of discovery through a virtual Christmas world.
2. Scene Design
But what does a Christmas world consist of? What is there to discover?
After several brainstorming sessions, we finally decided on a cottage in the woods with a smoking fireplace, in front of it a lantern, a sled and a snowman. It should snow heavily and cause the viewer to seek refuge in the cottage.
Upon entering the cottage, the explorer should find a Christmas tree with presents, a blazing fire in the fireplace, and a TV.
And of course, the scene should also hold some surprises ready for particularly inquisitive explorers...
3. 3D Modeling
Next, it was about 3D modeling. We have modeled all assets such as cottage, Christmas tree, fireplace, etc., using Cinema 4D.
We programmed the iOS version with ARKit 1.5 and SceneKit.
Our conclusion: the Apple Tools deliver what they promise.
In Android, we initially set to ARCore, which unfortunately has not been proven to the same extent. Finally, we rebuilt everything in Unity, which worked very well, but unfortunately was not finished in time for Christmas.
A special challenge was the surface recognition for placing the Christmas world in the environment of the user.
To start the AR app, the star on our Christmas card had to be recognized first. We had integrated this image into the ARKit.
After the image recognition, the surface recognition was carried out so that the 3D model could be placed correctly in the room, then the scene could be initialized and loaded.
The effects took some time: snow, chimney smoke, blazing fire, fog.
But the effects effort was worthwhile, because they finally give the virtual experience the illusion of authenticity.
Also crucial for a deceptively real virtual world is realistic illumination of the scenes.
Accordingly, we spent a lot of time on realistic placement of light and shadow, depending on the explorer's position.
7. Video Editing
The videos, which are played in the scene on the TV, we edited with «After Effects».
8. Sound Editing
Last but not least, it was about sound editing. We opted for ambient background music (in the loop) for outside the cottage, then a slightly louder Christmas classic inside and a rock version of Elvis in the YouTube video as soon as the TV turns on.
And finally, the xmasAR app arrived in the store just in time before the annual App Store Connect winter break.
As another goodie we created iMessage stickers as an app extension.
It's best to try the AR app yourself!
Download, scan the star on the Christmas card and off you go!
You can find more of our references here.