Ben and Zeezee’s initial tech request seemed a little unlikely to
work in the format desired: basically trying to use a phone worn by
the actor to stream 4K video that was originally recorded as 8K,
360 degree video. Phones just don’t have the processing power for
those kinds of resolution, and the wireless connection would need
to be incredibly fast and incredibly reliable for it to look good.
Our project would require investigating other hardware, software
and wireless technologies for a better solution.
As we met a few more times to discuss the project, I gained a
greater understanding of the project’s goals and proposed a
secondary solution using either a phone or a microcontroller to get
the orientation of the actor’s head, and then sending that
orientation to a computer to do the processing. That way, only a
relatively small amount of information would actually need to be
sent wirelessly – which would result in a much more reliable
connection for performances, better processing speed and
potentially a smaller device that the actor would be required to
wear on stage.
Internet discussion threads were a good resource for exploring different technologies. We eventually came up with a path that we believed was feasible in the limited time allotted and went for it. Ben raced the clock to make it to the Micro Center before closing, and returned with the TinyDuino microcontroller we had settled on. Our project was about to move forward. More about that in New Wearable Acting Technology.
Along with the choice of microcontroller, we also needed software to receive the microcontroller data and tie it in with the video. I had had some experience programming with the game engine Unity during the previous summer and knew if its versatile capabilities. I suspected that it might be up to the task. In fact it worked marvelously well with one hitch.
I tried to use Unity’s built-in movie texture program in order to play the 360 video, but that initially produced incredible performance issues. After some online investigation, we discovered the AVPRO video plug-in for Windows, which was designed as an alternative to the built-in functions of Unity; it worked really, really well. It was actually a significant unanticipated expensive for our project, but it was key to making it all work!
The last major tech challenge was the need for the technology to work wirelessly for performances. I had no prior experience programming for any wireless technology so I was concerned that it might be a time sinkhole to figure out and program. However, it turned out to be easier than I had previously imagined because we went with a simple Bluetooth solution.
- Harry Klein