MEGA (Mundo Estrella Galicia) Sport Training (Kinect)

Disclaimer: This is a video preview of the published game. This video not shown the final state art.

3D Wire 2019

Details

Project: Estrella Galicia: MEGA Sport Training
Studio: Flying Beast Labs (Cateffects S.L.)
Genre: Sports, Casual
Platform: Windows, Kinect v2
Job description: Lead Programmer, Tool Programmer, Technical Artist
Technologies: Unity3D, .NET, C#, Git, Source Tree, Hercules Setup Utility
Website:
- https://mundoestrellagalicia.es/
- https://flyingbeastlabs.com/proyectos/mega-sport-training/
- https://flyingbeastlabs.com/3d-wire/

Credits

Lead Programmer, Tool Programmer & Technical Artist
José Miguel Sánchez Fernández

Gameplay Programmer
Diego Santiago Sanz

Producer & Game Designer
Carlos L. Hernando Lopéz

Project Manager, Music Composser & Sound Technician
Daniel Nuñez Martín

Art Director, 2D Art & UX Designer
Alberto de Torres Martínez

3D Art, Modeling, Rigging & Animation
Irene Arnaiz Lopéz

Producer
Cesar Piña Mudarra

Description

Lead Programmer and Tool Programmer for the development of MEGA Sport Training, an interactive experience for MEGA: Mundo Estrella Galicia museum, in A Coruña, Galicia, Spain.

My main task in this project was develop the entire game architecture, focused on modularize all critical parts of the project for ease maintenance, upgrade or deep-changes quickly, the creation of requested tools during the development, and supervising the work of a junior programmer, the person that was develop the 4 mini-games that compose the game (basket, football, hockey and rugby) and help me to integrate the Kinect v2 SDK in Unity and study how implement features like the advanced gesture recognition.

Other tasks that I colaborate was supporting the 2D and 3D artists as Technical Artist, optimizing all 3D assets and materials in-game (combining meshes, fixing materials setup in Unity, and advise the 3D artist, a junior profile, in some tasks involved with optimizing modeling and asset exporting), setup the lighting in all scenes and setup all post-processing effects to fit the limited requeriments of the target machine (an Intel NUC without dedicated graphics card) to achieve a good performance without lost quality visual look & feel.

3D Wire 2019

This same year, we make a variation for the 3DWire (https://3dwire.es/), an important spanish annual event of animation, games and new media that is celebrating in Segovia. Weird Games: Food Training is the new name of the game and have a little visual changes as replacing all sport things like the basket ball by a toaster or the hockey disks by spanish omelets, most of the work as rebranding and applied some gamification rewards for the event games.

The true challenge: Kinect 2.0, Windows 10 and beyond

Also of the difficulties recording custom gestures using the official Kinect SDK tools, implmenting a custom gesture system to avoid some issues with some gesture recognitions, and the many other related and technical issues, the true challenge to develop and maintain this project are the Kinect 2.0 + Windows 10 combo:

  • Kinect 2.0 has been discontinued by Microsoft in fall 2017 and removed support on Windows (drivers, tools, etc...) a few months after publising the tools for Kinect 2 for Windows (supported Windows 8.1 as last official version).
  • Never existed an official SDK for Unity, only a few outdated particular user implementations on the web for Unity 5.6 (we developed the game with Unity 2018.3.5). We using the only full implementation solution available and well documented by examples, that can be found in the Asset Store.
  • No all PC configurations supports Kinect 2 hardware. During the pre-production, we found that only 2 PCs in our studio are capable to run Kinect 2 without problems. Other PCs directly can't work, can't recognized the Kinect hardware. Seems the USB 3.0 in most of them has issues related with power supply or the drivers (in some cases, Kinect burned the AC Power unit on a couple of our PCs).
  • Kinect 2 hardware can't works on current versions of Windows 10. The last supported version is the build 1803 (April 2018 Update). The USB 3.0 drivers has many known issues and one of them affect to Kinect 2 USB connector. The solution initially seems very simple: using the build 1803, but...
  • In Windows 10 you can't full disable Windows Update services. And, later or soon, your Windows 10 will be forced to upgrade to latest version. In our case, disconnect the PC from the net is not a solution. The PC can not we unpluged from the net because need access to internet to send mails and send and read signals from an UDP server. Basically, the solution passed to reroute the all Windows Update known server address to localhost in the Windows hosts file, setup all Windows Update related services to manual and disabled it, and blocking the internal Windows folders related with Windows Update downloads and deployments.
  • The official Kinect 2 AC Power Unit not works on current PC configurations. One of the Kinects, supplied by the museum worked with the Surface 3 AC Power Unit instead of the Kinect official unit. For any reason the Kinect 2 official AC Power Unit not works properly on current PCs.

And the final boss...

But the true challenge was the final issue of the Kinect 2 + Windows 10 combo. At the final stages of the development and testing, we found the game sometimes was freeze without any reason! (very inappropiate for a museum experience).

After 2 weeks of deep debugging and testings in the source code of the Unity implementation of the Kinect SDK, I found that the reason of the game was freezing was the avatar recognition event in a driver level. It's seems that the outdated driver in Windows 10 has any internal memory leak or something related bug.

Because I'm not was capable to fix this, because is an internal driver bug and not a source code issue, I designed a little "trick" to able the game to catch the freeze state and reset himself. I developed a little program that works as a monitor instance of the game. A simple terminal program that run in fist place and launch a new instance of the game as child process. Basically the game instances comunicate with the monitor to tell that it's alive. When the game stop to send signals to the monitor, the monitor understand that the game is freezed, kill the freezed process and launch a new one.

For prevent the user noticed the game reseting process, as idea of Carlos L. Hernando, we empty the entire Windows desktop of icons and hide the task bar, and using the same game splash screen background as desktop background. With this, the user not see any stranger thing except that the game has return to the main menu screen.

P.D.: In a way to avoid several freezes and their resepective resets, I needed to reimplemented all avatar recognition logic system in the game to avoid conitnuous calls to native Kinect avatar recognition functions.