Jeremy Hartmann and Daniel Vogel of the Cheriton School of Computer Science at the University of Waterloo in Canada have developed technology that enables online conversion of a 2D broadcast into an interactive 3D space where viewers can see what is happening from every angle can track. The researchers’ work was “Improved Video Game Streaming by Recreating an Interactive 3D Space for Viewers”. released Last month.
Many multiplayer shooters support a spectator mode that allows you to watch the action from different angles. This can be useful for learning other players’ tactics or watching experienced players play. However, you can use this mode only within the game, that is, to view the game from different angles, install the game on your computer. The work of Canadian researchers describes a technology that allows you to get an effect similar to that of spectator mode, but without having to install the game.
This technology has the potential to make gaming streaming much more enjoyable on popular platforms like Twitch and YouTube. With its help, the video received in real time is converted into 3D space, allowing the viewer to choose the angle of view to see what is happening in the game. This allows viewers to change their viewing angle at will while the streamer they are watching keeps playing.
In the demo video you can see that the 3D room creation is not that fast and for some time now the view has seemed to come from an unfortunate angle with a gray background. However, as the player progresses, there is a gradual formation of untextured models. This is not the same as spectator mode as the user can only see what is around the streamer character. Also, anything outside of the player character’s field of view will not be rendered either.
The lack of textures and animations of character movements spoils the impression of the technology presented, but it is important to understand that it is still in its infancy at this point. Improvements in technology and the use of machine learning algorithms may result in more immersive game streams over time, allowing viewers to truly immerse themselves in gameplay.