COSMIC PERPECTIVE:

an Ar HLS STREAMING APP

Photographs that trigger text, sprites, and video streamed from the cloud, made to connect and excite the public about spaceflight.
Built with Unity and Vuforia for iOS and Android devices
December 2020
Created for the Cosmic Perspective shop.
Want to build your own app like this? A blank template with instructions is available on Alex’s Github.

*Note: the Augmented Reality items in the Cosmic Perspective Shop are no longer available and this app is not live.

ABOUT

Cosmic Perspective is an incredible organization dedicated to connecting the world to space through story, film, and immersive experiences. They are inspired by many of the things I am, namely the overview effect, collaboration, and sharing empathy. For this project, I was asked if it was viable (technically and financially) to create an augmented reality app that would add extra features and delve further into the stories behind some of the beautiful photographic prints they sold in their shop. This app is another small way to share the joy, delight, and excitement about the wonders of space flight with the public.

GUIDELINES

+ Stream large transparent videos quickly and without sacrificing quality when having over a trigger image
+ With multiple triggers simultaneously, be able to pause video stream when trigger is no longer in frame, and resume when it is.
+ Also display a launch mission patch and text around the accompanying video
+ Accommodate users of all ages with zero AR experience
+ Allow the team at Cosmic Perspective to drag and drop new triggers into the project on their own once handed off.

TARGET AUDIENCE

The Cosmic Perspective app was developed to be suitable for any person with a mobile device built in the past few years, regardless of experience with technology, that has purchased a print from the Cosmic Perspective store or has printed one of the images themselves.

Intended impact

It’s an app meant to require minimal interaction to deliver high quality HD video stories. It is meant to quickly engage the audience and inspiring more curiosity and wonder.

CHALLENGES

1. Though the concept was minimal and Vuforia has vastly simplified that augmented reality process of: [camera captures trigger] -> [video activates], the videos Cosmic Perspective have made are gorgeous, high definition stories of rocket launches, and thus the files are much to large to be hosted locally. For this stage I had to learn the ins and outs of HLS streaming, cloud hosting/access/security, and most importantly, converting video to be streamed.

2. Vuforia nor Unity did not (as of the time I am writing this) have the functionality to play an HLS playlist. I had to find a 3rd party solution that would work for both iOS and Android builds, could be compatible with Vuforia, and was not cost prohibitive.

3. Additionally, the once open source Vuforia SDK had updated and was now closed source. I was no longer able to access the namespace of the 3rd party HLS player within Vuforia, and had to learn about assembly files and compilation order to be able to build a script accessible by each SDK before other scripts were compiled.

CLOSING REMARKS

Despite it’s simple nature, the separate components of this app that needed to come together were uniquely challenging in their own way and I learned a tremendous, tremendous amount about fields I had no experience in. (Video codecs and manually creating HLS playlists is absolutely an art form and anyone who does this full time has my respect.) I also loved getting to dip my toe into cloud access and security concepts, which had inspired me down a separate path once this project was over.

But most importantly, when I was originally commissioned for this back in August 2019 (it went through several stages of being on hold) there wasn’t much information about streaming apps available for small scale/personal-sized Unity projects. It was fun getting to consult with people from different backgrounds who held separate pieces of knowledge that I could synthesize together in my own way to make this work. It also went through several SDK iterations (one with the Chinese-based EasyAR instead of Vuforia), and lots of trial and error with different HLS players.

Because I didn’t want anyone to go through what I had to do, I ended up making several different templates (iOS & Android, Vuforia & EasyAR) with a detailed walkthrough of how to setup a project from scratch all the way to completion, along with FAQs, pricing for different options, notes, and more. Honestly, that part felt the most rewarding! You can find the Unity templates and in depth tutorials on my Github.


*Note: the Augmented Reality items in the Cosmic Perspective Shop are no longer available and this app is not live.