Rec Room, a social and fun VR experience

Posted on January 17, 2023 by LimEnQiAngie

What is Rec Room? What parents need to know | Internet Matters

What is Rec Room and what makes it so great?

Rec Room is a free, social virtual reality game that allows people to get together, join one of the many worlds offered, and enjoy a wide variety of activities; most of the said activities and worlds, are also, in fact, user-created.

Rec Room’s content is mostly user-generated, similar to games such as Roblox or VRChat. The users create the world, model the objects, and script the necessary behaviours before sharing their creations for others to enjoy. Players have even gone on to recreate well-beloved games or franchises in Rec Room, such as Five Nights at Freddy’s, Uno, Wordle, Harry Potter and many more. The only limitation of what they can create is their imagination.

With many users developing content daily, it allows for a seemingly limitless number of activities for other players to engage in. Players can bowl with one another, play laser tag, do stand-up comedy and much more. With so many different activities, there is bound to be something for everyone.

_{Rec Room. (2022, December 28). Best Featured Rooms of 2022! [Video]. YouTube. https://youtu.be/AwAFWW87PyY}

But what makes this game different from Roblox or VRChat besides the graphics?

All of the games share a similar concept, players can join their friends in various activities, and they can create content and share it.

However, one of the key differences is that, unlike Roblox or VRChat where the player has to download external tools to create and script the world (VRChat uses Unity, while Roblox uses a tool called Roblox Studio), Rec Room tools are built into the game. This not only allows for a much gentle learning curve for beginners compared to the other two games but also allows users to immediately view and interact with their creations, making the process more efficient.

_{Rec Room. (2020, September 2). How to create in Rec Room using the Maker Pen! [Video]. YouTube. https://youtu.be/x8WeiOfXE20}

An immersive experience

While the game is available on various other devices such as mobile phones, PC, Playstation, and many more, the experience is simply not the same as compared to playing it on a VR headset.

Rec Room makes full use of the motion capture system of a VR headset. Players can freely look around the world just by moving their heads around instead of having to manually pan their screen.

The arm motions through the controllers are also replicated in-game, such as waving your arm in real life will result in your in-game avatar’s arm waving. Users can also use this functionality to interact with in-game objects; for example, to play in-game basketball, players would just need to perform actions they would normally do in real life. This also allows for more expressive and unique interactions with the world and other players.

In the game, there is also an in-game microphone that enables players to communicate with one another. The volume of their voice changes dynamically depending on the distance between their avatars.

All this makes the game feel immersive as the motions and actions in the real world are also closely reflected in the virtual world.

_{Rec Room. (2022, September 11). I 1v1’d THE ORIGINAL BASKETBALL MAP CREATOR | Rec Room Basketball VR Gameplay [Video]. YouTube. https://youtu.be/0sQ3qPlxkwI}

Motion Sickness? What if you don’t have your VR legs?

Motion sickness is something many suffer when they first enter virtual reality, especially so when it is their first time. This happens when users aren’t accustomed to the mixed signals received by the brain from the eyes and the inner ear (contains ‘sensors’ to help keep the body balanced).

Visually, the players see themselves moving in the virtual world, however in reality the body is stationary. This disparity between what the inner ear and eyes senses causes a disconnect among users, which in turn, causes the feeling of motion sickness and nausea. (This can also happen for first-person games, not just VR games)

Of course, one way to fix this would be to use an omnidirectional treadmill, but that’s not exactly something that can be readily obtained by most consumers. (It’s pricey and big)

Rec Room like many other VR games and applications uses the joystick of the controller to move around the world. However, Rec Room also added an additional feature, the ability to “teleport” around as a substitute for walking in the world.

This feature helps to reduce motion sickness as users are no longer forcing their inner ear to constantly adapt to the artificial motion since the visuals comes to them. Thus, making the experience much more enjoyable.

_{BananaVR. (2021, May 19). How To Rec Room – Teleport VS Walking [Video]. YouTube. https://youtu.be/opcQU2AMX8Q}

Why I personally enjoy it so much

During the height of covid lockdowns, Rec Room was one of the few ways I could engage in activities with my friends.

The motions and controls are intuitive as they closely replicated the physical world. With this, my friends and I were able to play a wide variety of games without much confusion.

The user interface was also easy to understand and did not feel intrusive. All I had to do was look at my Avatar’s wrist, and a simple menu will pop up. I was even able to use hand gestures to invoke certain actions, such as shaking hands with another player to become friends or putting up a stop gesture to ignore a player. Not having to open a separate menu to perform those actions made interacting with the world much more enjoyable.

And while the visuals were simplistic, it was visually appealing and the low poly graphics suited the wide and wild variety of activities offered.

I had a fun time with my friends trying out the different activities and getting immersed in the world.

Possible Improvements

While I did have a lot of fun with Rec Room, my biggest gripe was the lack of options to adjust the volume of other players’ microphones.

Some of my friends had a softer mic, and it was hard to hear them from time to time even when their avatars were standing close by. It can also be quite disorienting in VR to hear the different volumes of voices even when the avatars are standing at the same distance from you.

There is a universal volume setting for the players’ voices, however, raising it meant every player’s microphone sound volume will also be raised. The players with a louder mic will still end up talking over those with a softer mic, thus, defeating the purpose of turning up the volume.

A possible solution will be to have a setting similar to what Discord has, where you can adjust the volume of a specific player. This will make it so that we can better hear one another and the experience won’t be ruined when you’re barely able to hear someone, or when someone with a particularly loud mic starts shouting beside you.

*_{Rec Room (n.d.). Comfort and Moderation. https://recroom.com/comfortandsafety}*

Conclusion

Rec Room is able to take advantage of what a virtual headset can offer to provide a fun and immersive VR experience for players. The controls are intuitive, the user interface and experience are pleasant, and it has a wide range of activities for players to engage in.

It is a fun game and a great experience in the virtual world that I, along with many others greatly enjoy!

Resources

Courtney, A. (2022, October 11). Rec Room Vs VRChat: Which Is The Best Social VR Game. VR Lowdown. Retrieved January 17, 2023, from https://vrlowdown.com/rec-room-vs-vrchat/

Create & Learn Team (2022, April 29). Roblox Tutorial: How to Make a Game. Create&Learn. Retrieved January 17, 2023, from https://www.create-learn.us/blog/roblox-tutorial-how-to-make-a-game/

Moss, G. (2018, September 18). Motion Sickness In VR Sucks (Here’s What To Do About It). VR Fitness Insider. https://www.vrfitnessinsider.com/motion-sickness-sucks/

Rec Room. (2017, May 3). How to Rec Room #4 – Stop Gesture [Video]. YouTube. https://youtu.be/2bWTgmAxBes

Rec Room. (2017, April 6). How To Rec Room #3 – Shake Hands to Make Friends [Video]. YouTube. https://youtu.be/epo3Gw9NMDs

Beat Saber: The best of both Virtual Reality and rhythm games.

Posted on January 17, 2023 by luziyi

Introduction:

For anyone who has done some research on gaming with VR systems, Beat Saber is a title that is almost impossible to miss. Officially released in 2019, the rhythm game has since accumulated great popularity, with its Steam version currently sitting at an overwhelmingly positive review with a whooping 60,924 total reviews count^[1] (as of 15^th Jan 2023). Being a huge fan of all kinds of rhythm games myself, I too have tried it out myself and quickly became addicted to it too. In the following passage, I will try to explain what made Beat Saber a successful VR experience and maybe convince you to try it too.

So, what is Beat Saber?

Just as most rhythm games go, Beat Saber requires the player to follow the rhythm of a chosen piece of song, and react to the beats by swinging an imaginary lightsaber. Once the game starts and a song is selected to be played, the player is immediately immersed in a futuristic neon-lit setting. As the music starts blasting, notes represented by glowing blocks will approach the player from the front, requiring the player to swing the controller in a designated direction just as the notes reach the player. Depending on the difficulty chosen, players may also need to squat down to avoid obstacles or maintain various poses.

Adapting the pros of traditional rhythm games into virtual reality

In the world of rhythm games, a deciding factor that determines the satisfaction a player gets is the feedback provided after hitting a correct note. This feedback can appear in various forms depending on the platform of the game, and it is particularly important as it serves to ‘reward’ the player for hitting a right note. Such interactions usually happen through simulating the sense of sight and hearing, but sometimes the sense of touch is involved too. For example in the popular pc rhythm game osu!, notes glow brightly when hit at the right timing and produce a loud ‘cluck’ sound. Notes hit outside of the perfect timing window (aka a good instead of a perfect) will produce a lighter glow as well as a duller sound compared to that of a perfect hit. In the arcade rhythm game Taiko no Tatsujin, the motion of the drum stick bouncing off the drum is in itself a form of feedback through the sense of touch to the player as well.

Compared to the above titles, Beat Saber has done an equally extraordinary job in interacting with the players to provide such positive feedback. When a player hits a note at the correct timing, the glowing block will be sliced swiftly into two halves and subsequently fly over the player’s shoulders. At the same time, a clean cutting sound will be played to indicate the successful hit, further stimulating the release of dopamine. Beat Saber also enables controller vibration when encountering obstacles, which is used to provide negative feedback to the player. Given that Beat Saber is a VR title, all the gameplay happens within the front field of view of the player. Compared to a mobile rhythm game where visual feedback could be blocked by the player’s finger, a note in Beat Saber grows in size as it approaches the player’s player model and eventually takes up a significant portion of the view space. This means that any form of visual stimuli in the game is almost impossible to miss, thus greatly enhancing the intensity of interaction between the game and the player. These interactions are vital in providing positive feedback on the player’s actions and making sure that the players will continue to play the game.

Immersing in the world of music

As a game hosted on VR platforms, Beat Saber also provides a great level of immersion that allows the player to indulge in the world of rhythm and dancing beats. In order to achieve that, Beat Saber implemented a series of outstanding feature that serves to enhance immersion. Firstly, the game places significant emphasis on simplicity and reduces the instances of unnecessary information in most parts of the game. Throughout the duration of a song, the gameplay information displayed on the screen is limited to the models of the sabers, the track the notes travel on, as well as the notes/obstacles. Other than that, the only UI elements shown are the combo multiplier, the current combo, as well as the current score, and the current hp. In terms of representational fidelity, Beat Saber is highly iconic and all representations used in game are very intuitive to the player. By minimizing distractions, the player is encouraged to focus on the notes themselves while dancing to the rhythm.

Beat Saber also makes use of carefully crafted background scenes during gameplay that further enhance the level of immersion experienced by the players. Compared to the bright glowing colors of the notes and the saber, the background usually takes on a darker hue and seemingly lacks in detail. However, throughout the duration of the song, the background will also pulsate and change in color, all in sync with the tempo of the music. Even if it is just a subconscious part of the player’s view space, the background ‘breathes’ with the music and is continuously engaging with the player as well.

Areas for future improvement

As much as I would like to spend multiple hours daily on this amazing VR title, there are quite a few areas that need further improvement. Firstly, Beat Saber on higher difficulties can get very physically intensive, and it can be quite a torture to play in VR headsets that doesn’t come with in-built fans. On top of that, given the headset’s limited battery life, attaching an additional power bank is often needed in order to support longer play duration. All these add up to a decent weight that needs to be carried on the player’s head, and this can be very immersion-breaking when the player begins to experience discomfort from wearing the headset for extended durations.

Beat Saber also sets great limitations on the kind of headphones/earpiece that can be used. Bluetooth earpieces, while convenient, are horrible options due to the unbearable delay they can cause. Beat Saber’s timing window for hitting a note is speculated to be around 44ms^[2], and Bluetooth earpiece has a much longer delay than that. While wired headsets do not have that issue, it is much more cumbersome to wear, and the wire management can get messy if not done properly. A possible improvement to this would be to add a function to detect Bluetooth delay and adjust gameplay accordingly.

Another issue with the game in its current stage is its lack of content, specifically the choices of songs. While the game regularly releases DLC packs with new songs every few months, the songs are largely limited to western pops and rocks, which is a shame for fans of J-pop and K-pop. While modding support is available, it is more troublesome to set up and maintain, and modded songs also come in varying qualities.

Conclusion

Overall, Beat Saber is an absolute must-get for fans of other rhythm games, as it provides a totally new and immersive experience that is different from all existing rhythm games on the market. Even for those who never played rhythm games before, Beat Saber is still worth a try as one of the best-rated VR games available. Just slide on a VR headset, and enjoy the sensory feast while dancing to the beats!

References:

[1]: “Beat saber on steam,” Beat Saber on Steam. [Online]. Available: https://store.steampowered.com/app/620980/Beat_Saber/. [Accessed: 17-Jan-2023].

[2]: “Timing adjustment :: Beat saber feedback & suggestions,” Steam Community. [Online]. Available: https://steamcommunity.com/app/620980/discussions/1/1696046342863490923/?l=japanese&tscn=1553186699. [Accessed: 17-Jan-2023].

IKEA Immerse: XR for Online Shopping in a post-pandemic world

Posted on January 17, 2023 by Peter

The COVID-19 pandemic has greatly changed people’s lives around the world. Despite the real-world inconvenience caused by the coronavirus pandemic, applications utilizing XR technology are booming to create convenience and enhance user experience.

In the post-pandemic world, people are reluctant to walk into crowded stores and rather purchase anything they want online. According to Statista, more than 20% of the sales are bound to take place on the internet by 2022. Brands like IKEA were among the first to realize and jump in on the opportunity. They refined their e-commerce side of things to prove an all-around better experience for their customers.

IKEA has created an entire app from the ground up dedicated to XR. The app, called IKEA Immerse, helps users and customers to create, experience, and share their own configurations in a virtual living and kitchen room set.

Why do I like it?

I like this application because IKEA Immerse invites consumers to experience, design, and create in mixed reality. Instead of choosing from what has been provided, we become part of the designers ourselves. We can feel free to design what we want before purchasing.

Why is it engaging?

Enhancing in-store experience

Thanks to XR technology, IKEA Immerse has made it simpler for customers to view and try different furniture before purchasing. Compared with traditional offline shopping, people can more efficiently try different combinations of colors and styles, and customize their preferred lighting and shading.

Social interaction and sharing

With transferable VR equipment, customers could share their instant feelings with friends more conveniently and at a lower cost. In a virtual environment, people could also interact with furniture items through vision, and obtain instant feedback.

Extending the relationship beyond purchase

XR is still playing its role even after purchasing. After-sale service and support can be far more helpful with XR user manuals, instructions, and feature guidance. Instead of having a user interface directly into these physical products, users could engage with their smart refrigerators or ovens via an XR app on their own digital device.

What features are well done and what can be improved?

I think the following features are well-constructed:

Photorealistic representation of the physical world. All the real items are projected into the virtual world in a 3D and highly realistic manner. Textures are rendered so well that you can feel no visual difference between viewing and experiencing virtual reality and the offline store.
More adaptable and diversified features than the physical world. You can adjust the lighting and shading freely by virtue of real-time rendering, beyond physical limits. Eg, night mode can be chosen during the daytime and you can add more shading without having to draw the curtain.

Meanwhile, I think there is still room for improvement regarding the following features.

Adding a sense of touch. When purchasing furniture, both visual and tactile communications are essential for customers. IKEA Immersive has done quite well on vision, but if customers cannot feel if the bed is soft enough, it would definitely affect their purchase decisions merely based on virtual viewing.
Making it more trustworthy. For online shopping and remote experiencing, it is easier for sellers to exaggerate their product effects and attract customers in a dishonest way. If IKEA Immersive could prove the authenticity of products and experience shown in XR devices, I believe it will gain them more advantages compared with similar products.

References

Bring Pokémon to reality, or walk into their world

Posted on January 16, 2023 by Bo

What is AR?

AR stands for Augmented Reality. It is a technology that superimposes computer-generated images, sounds, or other information over a user’s view of the real world, in order to enhance or augment their perception of it. If we want to catch Pokemons in our real world, we need AR to create those virtual characters. Luckily, Pokémon Go has made this possible for us.

What is Pokemon Go?

Pokemon Go was one of the first mainstream examples of AR technology being used in a mobile game, and helped to popularize the concept of using AR in gaming. The game uses the player’s smartphone camera and GPS location to display Pokemon characters on the screen as if they were in the player’s real-world surroundings. Players can then catch the Pokemon by “throwing” virtual Poke Balls at them.

For me, the story of Pokemon accompanied me throughout my childhood. And The Pokemon Go is the earliest AR technology application I have learned. So let’s go back to 2016 and find out why is this game so engaging?

Why is Pokemon Go engaging?

Pokrmon Go became extremely popular upon its release in 2016, with millions of players worldwide catching Pokemon in their neighborhoods, parks, and other public places. There are many reasons behind its success：

Gamification of reality: The game uses augmented reality technology to superimpose virtual Pokemon characters on the player’s real-world surroundings, which creates a sense of excitement and immersion for players.
Social interaction: The game encourages players to go outside and explore their neighborhoods and communities, providing opportunities for social interaction with other players.
Nostalgia: The game is based on the popular Pokemon franchise which has a large fan base, many of whom grew up playing the original games, watching the anime and collecting the cards. So it brings back childhood memories for many people like me.
Convenience: To catch Pokemon in real life, all you need to do is pick up your phone and use the camera to explore the surrounding scenes
Collecting and Progress: The game has a collecting aspect where players can catch different Pokemon and add them to their collection, this creates a sense of progression and accomplishment for players.
Rewards: The game also rewards players for catching Pokemon, visiting PokeStops, and battling at Gyms, which helps to keep players engaged and motivated to play.

What can be improved?

Gameplay: Some players have complained that the gameplay can become repetitive and lacks depth. Adding more diverse gameplay mechanics and activities could help keep the game interesting for players. We want to see more vivid Pokémon, not Pokémon that only poses with a few fixed actions.
Social features: While the game encourages social interaction, the in-game features that support this are limited. Improving these features, such as the ability to trade Pokemon with other players, would encourage more social play.
Accessibility: Some players have criticized the game for being too physically demanding, as it requires players to walk around to find and catch Pokemon. Improving the accessibility of the game for players with mobility issues, such as providing more PokeStops and Gyms in one area or including more options to play the game from home, could help make the game more inclusive.
AR improvements: Pokemon Go’s AR implementation is not perfect, it can be improved for example by making the Pokemon appear more natural and realistic in the real-world environment, and allowing players to interact with them in more ways.

VR: a way to walk into the game world

Virtual Reality (VR) is a technology that allows users to experience a simulated environment through the use of a headset and other equipment. The user’s movements and actions in the virtual world are tracked and translated into corresponding movements and actions in the simulation. If you have ever imagined entering an exciting game world, like the Pokemon game, then you must not miss the following movie.

What is Ready Player One?

Ready Player One is a science fiction action adventure film directed by Steven Spielberg, and released in 2018. The film is based on the 2011 novel of the same name by Ernest Cline. The movie is set in a dystopian future where the majority of the population spend their time in the virtual reality world of the OASIS, a place to escape the real world. The plot centers around a virtual treasure hunt set up by the creator of the OASIS, in which the winner will inherit his fortune and control of the virtual world. The movie deals with themes such as escapism, the impact of technology on society, and the power of virtual reality.

Why is Ready Player One engaging?

One of the main reasons is the visually stunning and immersive special effects used to bring the virtual world of the OASIS to life. The movie does a great job of capturing the feeling of immersion and presence in the virtual world, making audiences feel like they are actually inside the OASIS. And that’s one of the goals of future VR technology.

Many familiar game characters appeared in this movie. The protagonist in the movie uses VR equipment to enter one wonderful game world after another. I think no one can refuse the exciting experience of driving Gundam in the game world. For those who grew up with games, this movie fulfills their desire to enter the real game world.

This movie has also changed people’s views on VR to a certain extent, presenting people with a wild possibility of the future VR world. We know that some of the wildest ideas in sci-fi movies and TV shows are now reality. People also look forward to one day being able to enter the game world in this movie.

What can be improved?

Although the movie shows us a beautiful world built with VR technology, there are still some shortcomings in the plot. Some of the characters in the movie use simple glasses and hand devices to enter the VR world, and some use expensive equipment to get a more realistic experience. They use 360-degree treadmills to play games indoors, and people who can’t afford such equipment can only run on the street, which has also caused some traffic accidents. But if you want to complete the magical movements of the VR characters in the movie, it is not enough to have a treadmill. It’s like when you do a backflip or something like that, you don’t feel comfortable wearing these devices. You want the device to be able to recognize these difficult movements, but the reality is that such a sensitive device will become a burden on you. Especially wired hand and leg motion capture, their presence will hinder the movement. VR devices still take a long time to develop and update if we want to get that kind of future VR experience.

Unravelling the Mystery: An In-Depth Analysis of The Room VR: A Dark Matter

Posted on January 16, 2023 by junx

XR, or Extended Reality, is a rapidly growing field that encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR) applications. In this blog post, I would like to take a deeper dive into one such application that I have had the pleasure of playing: The Room VR: A Dark Matter.

Engaging Immersion: The Room VR: A Dark Matter’s Attention to Detail and Puzzle Design

(A Room VR: A Dark Matter’s Trailer, https://cdn.cloudflare.steamstatic.com/steam/apps/256776470/movie480.webm?t=1585225680)

The Room VR: A Dark Matter is a VR puzzle game developed by Fireproof Games. The game is set in an abandoned British Institute of Archaeology, where players take on the role of an investigator tasked with solving the mysterious disappearance of the institute’s staff. The game is incredibly engaging because it immerses players in a fully-realized 3D environment, and the puzzles are challenging and satisfying to solve.

The Best Features: Immersion and Interaction

(Cover photo https://cdn.cloudflare.steamstatic.com/steam/apps/1104380/header.jpg?t=1646758373)

The XR experience in The Room VR: A Dark Matter helps to immerse the player in a number of ways. One of the key ways is through the use of a fully-realized 3D environment. The game’s environments are meticulously crafted, with attention to detail in every aspect, from the textures of the walls to the placement of objects. This allows the player to feel as though they are truly inside the game world, and the sense of immersion is heightened by the ability to physically interact with the environment. For example, players can open drawers and manipulate objects, and the realistic physics of the game means that objects will respond in a way that feels natural.

Another aspect of the XR experience that helps to immerse the player is the use of VR-optimized puzzles and interactions. The game’s puzzles are designed specifically for VR, which means that they would not be possible on a traditional screen. This adds an extra layer of immersion, as players must use their physical movements to solve the puzzles, and this sense of physical interaction with the game world makes the experience feel more real.

The game’s atmosphere and sound also play a role in immersing the player. The story and atmosphere of the game are well-executed, making players feel like they are truly investigating a mystery. The game’s original soundtrack also adds to the atmosphere, creating a sense of tension and intrigue that helps to immerse the player in the game world.

All of these factors together create an XR experience that immerses the player in a way that is not possible with traditional 2D games. The ability to physically interact with the game world, the challenging and well-designed puzzles, the well-executed story and atmosphere, and the use of VR-specific features all work together to create an immersive experience that is truly unique to The Room VR: A Dark Matter.

Room for Improvement: Short Length and Intuitive Controls

While The Room VR: A Dark Matter is an engaging and immersive game, there are a few areas where improvements could be made. One such area is the game’s length. While the game is satisfying to play, it can be completed relatively quickly, which may leave some players wanting more. To address this, the developers could consider adding more levels or puzzles to the game or even creating additional downloadable content. This would give players more content to enjoy and would make the game more replayable.

Another area where the game could be improved is in the area of controls. While the game’s controls are functional, they may be difficult for some players to master. Some players may find it challenging to manipulate objects within the game’s environment, which could make the game less enjoyable. To address this, the developers could consider making the controls more intuitive, such as by adding more tutorials or helping options, or by making the controls more customizable.

Additionally, the game could benefit from more variety of puzzles or objects. The game currently features a limited set of objects and puzzles which players can interact with, adding more variety would make the experience more diverse and engaging. Moreover, the game could benefit from a more advanced physics engine, which would make the objects’ interactions more realistic and satisfying.

Lastly, the game could also benefit from more advanced AI for non-playable characters, making the story more engaging and providing a more realistic experience.

Conclusion: Highly Enjoyable VR Experience

In conclusion, The Room VR: A Dark Matter is a well-crafted and engaging VR puzzle game that immerses players in a fully-realized 3D environment. The game’s attention to detail, challenging puzzles, and well-executed story and atmosphere make it a highly enjoyable experience. While the game’s length and controls could be improved, they do not detract from the overall quality of the game. If you’re looking for a VR puzzle game that immerses you in a compelling mystery, The Room VR: A Dark Matter is definitely worth a play.

Resources

Entertainment Software Association. (2019). Essential Facts About the Computer and Video Game Industry. Retrieved from https://www.theesa.com/wp-content/uploads/2019/03/ESA-Essential-Facts-2019.pdf

GN. (2020, March 26). The Room VR: A Dark Matter Review. Retrieved from https://www.ign.com/articles/2020/03/26/the-room-vr-a-dark-matter-review

Immersive Museum Tours with MediaScape XR

Posted on January 15, 2023 by liu yongliang

(The demonstration of the MediaScape XR places the remote visitor in the virtual Sound and Vision museum. [1])

What is it?

The MediaScape XR experience offers users a way to visit a virtual museum and interact with 3D models of cultural artifacts. The users also get to interact with each other and enjoy the experience together. The demo was created by the partnership between Centrum Wiskunde & Informatica (CWI) and the Netherlands Institute for Sound & Vision.

What features make it engaging?

Highly realistic representation of yourself

The users will be captured in 3D and a photorealistic hologram of themselves will be projected into the virtual world [3]. This allows the users to see themselves in the museum, as well as other users nearby. The representation opens up ways to make the experience realistic as the visitors see how the artifacts can interact with their presence and also what others are doing.

(illustration of visitor interaction. Source)

Multi-user setup to allow social interactions

The highlight of the experience is the opportunity to learn more from the 3D recreation of a cultural artifact: a costume once worn by Jereney Kaagman, a member of the Dutch rock band Earth and Fire, for the performance of their song “Weekend” in the 1979 television program [4]. The user gets to observe the costume from different perspectives, and even relive the experience of wearing the costume and performing on stage with their friends. This is amazing because it goes beyond just visual appreciation of the cultural heritage. The users get to experience the historical setup and have their rendition/creative output in the same scene.

(actual performance by Earth and Fire [2])

What features can be improved and how?

Virtualizing museums is one of the most natural applications of XR. We can take on museum tours without leaving our homes and the experience is often claimed to be more immersive. While I think that MediaScape XR has done a good job of showcasing what’s possible in the future, there is still a lot of room for improvement.

Some potential improvements that I can think of are:

Transcending the physical limitations for the museum tour: while I know that XR applications tend to want to reproduce how people visit museums, I wonder if they can do so solving problems that can’t be eliminated in the real world. For example, as a rather lazy person, I would love to be able to navigate the museum without having to walk around. Seated, if possible. Other potential improvements include having the ability to interact with the exhibits with touch. The current MediaScape XR setup requires a significant room space and it is not possible to have the experience at home. Support for more than two users is also difficult due to the equipment needed. I would hope that in the future, the XR experience can truly be a way to explore without ever leaving one’s home.

Ways to tune the crowd/environment at museums: just as how MediaScape XR refers to their application as social XR, museum tours can be a very social activity. This means it would be nice to see what others are looking at, have guided tours by museum staff, and exchange comments with other visitors.

More immersive experience: museums as a way to store artifacts is a somewhat old concept that originated from an era without photos or films. I think that XR applications can do a better job if they don’t just think from the standpoint of how to make artifacts more alive, but also how to make the experience immersive in time and space. For example, allowing visitors to hold conversations with historical figures.

I look forward to further developments of museum tours by MediaScape XR and hope that such tours will become available for museums in Singapore.

References

[1] “DIS wins best demo award at ACM Multimedia 2022,” CWI, [Online]. Available: https://www.cwi.nl/en/news/dis-wins-best-demo-award-at-acm-multimedia-2022/

[2] “Earth & Fire – Weekend (Video),” YouTube, [Online]. Available: https://www.youtube.com/watch?v=4iK5e76auJA

[3] “Sneak peek into future of cultural heritage,” CWI and Netherlands Institute for Sound and Vision, [Online]. Available: https://www.cwi.nl/en/news/cwi-and-the-netherlands-institute-for-sound-vision-gave-a-sneek-peek-into-the-future-of-cultural-heritage/

[4] “With MediaScape XR, Cultural Heritage Gets The Virtual Reality Treatment,” Jingculturecrypto, [Online]. Available: https://jingculturecrypto.com/cwi-netherlands-sound-and-vision-mediascape-xr/

Keep Talking and Nobody Explodes!

Posted on January 15, 2023 by zhanhong (e0544554)

If you’re a gamer, you’ve probably heard about this VR game. I think it’s one of the more innovative works for VR, so please keep reading and (hopefully) nobody explodes.

Introduction

Keep Talking and Nobody Explodes (subsequently shortened as Keep Talking) is a 2015 multiplayer puzzle game initially developed for Virtual Reality (VR) systems. The game is played with at least one “Defuser”, who is given a virtual bomb to defuse, and one or more players as “Experts”, who has to communicate instructions to the Defuser for them to successfully defuse the bomb.

Gameplay footage. Note: The game supports a wide range of controllers.

How this works with VR: the Defuser wears the head-mounted display (HMD), and using the controls, they manipulate the bomb within the game. Examples of interactions include cutting wires, pressing buttons, and flipping switches. Meanwhile, the Experts read off a manual to the Defuser, giving instructions on how to defuse the bomb. Of course, the bomb is different (procedurally generated) each time, which means the players need to communicate to each other, as neither can see each other’s bomb/manual.

Screenshot of what is seen by the Defuser. Promotional image for the game.
Check out the game here.

Why is it engaging?

Firstly, the Defuser is given a lot of pressure in trying to Defuse the bomb within a given amount of time. This goal-oriented behaviour coupled with time pressure and a wealth of interaction types, gives the player strong feelings of satisfaction (if things go well), control, and pressure (if things don’t go well) that increase engagement and hopefully, presence, because the virtual elements command the player’s full attention. Otherwise, they explode (and if done right, the fear of explosion feels very real).

Fidelity

The game is relatively low-fidelity; the visuals are kept simple and distinct for easier visual recognition of patterns, and the controls are not 100% high-fidelity either. You won’t have to actually exert “pressure” (push) to press a button, nor make any cutting motion to cut a wire. However, if you’re using VR controllers, the bomb’s representation in 3D space is quite accurate, such that it feels like you’re handling a real object, and moving/rotating the bomb following the movement of your hands.

This strikes a fine balance between control and frustration. The user is kept in control with regard to rotating the bomb and visually inspecting it, as that is a required interaction to gather information, but there are no unnecessary interactions that create only frustration, that detract from the main point of the game.

Innovative take on VR

Yet, the point of the game is strangely to not be fully present within the virtual environment.

Why? Because while the game is projecting stimuli onto the player, they still have to remain aware of external stimuli — in this case, audio stimuli outside of the VR environment, which are the defusal instructions given by the other players.

This game’s best features are in what it lacks. The creators of the game were inspired by the fact that users had to queue to use VR HMD for demos. Wanting to involve those not using the HMD, they created this game, which creatively turned the lack of HMD into a feature.

Moreover, the use of HMD not only transports the user into the virtual environment, it also removes the user from the actual environment. The creators realised that the HMD not only create stimuli, but also remove stimuli from the actual world, in this case, visual stimuli.

Removing Stimuli

If you’re a gamer from an earlier generation, you may be familiar with split-screen gameplay. One challenge throughout this era of gameplay was in keeping information from others. If you were competing with other players, it was a common tactic to look at their portion on the screen for a tactical advantage, like knowing their location.

Screenshot of a game using 4-way split screen. Source

Using VR technology, Keep Talking used the lack of real-world visual stimuli as a feature. The lack of information from the external world naturally provides immersion for the Defuser, but also perfectly serves as a game mechanism. Perhaps the use of VR HMD may provide a new angle on having multiple screens and differing streams of information in gameplay.

Possible Improvements

As it stands, the game is very modifiable. You may find several custom-made modules (puzzles on the bomb) or modifications to make the game more challenging, such as having the player drive a bus while defusing the bomb. I imagine if there’s a change that could be made, someone could have already made the modification.

The player base seems to adore challenge, so although I mentioned the game does not have high-fidelity interactions above, perhaps it would be a good optional challenge for the game.

In any case, it has been nearly 10 years since the publishing of this game, and VR technology has definitely become more accessible for the general public due to increase in support, production, and popular usage (such as with the advent of the Metaverse).

The original problem that the game sought to address (Lack of VR HMD sets) can now be set aside to pave the road for further innovations, involving more HMDs. Perhaps, it could now offer two-way gameplay, such as two players both performing the role of Defuser AND Expert, each given incomplete information and needing to help each other progress to defuse their own bomb. A similar concept has been done in this puzzle game, We Were Here, in which clues to solve to one player’s puzzle is found in another player’s world, and vice versa.

Altogether, Keep Talking creates a novel experience for co-operative work by encompassing VR’s features (and lack thereof). Apart from recreational activities, it’s been used in work interviews by this company for a quick gauge of how potential hires work with each other, showcasing the use of VR demos beyond pure entertainment. Do give the game a try if you haven’t already, and try not to explode!

HoloLamp

Posted on January 13, 2023 by justin.g

What is HoloLamp?

HoloLamp is a portable device that leverages on PICO projectors to offer glasses-free and hands-free augmented reality experiences.

HoloLamp’s technology warps the image being displayed so that the image users see projected is what their eyes think it would see if it was in 3D.

The lamp unit simply requires a HDMI output and a USB port and comes with an SDK in Unity for developers to build applications with HoloLamp.

Why is Holo Lamp Engaging?

HoloLamp is a interesting concept to me because it takes away all the hardware around XR at the moment like glasses or a mobile device with a camera required to experience XR, and is still capable of delivering an immersive 3D experience right in front of the guest.

Some features of HoloLamp include:

Pico Projector
Video Capture
Face Tracking Technology
Spatial Mapping
3D holograms

Use Cases of HoloLamp (Dining & Special Effects)

One use case that HoloLamp has been demonstrated for that is interesting, is its use at food establishments.

We can expect to see HoloLamp making waves in the gastronomic space as despite it having reached many heights and elevated magnitudes from orthodox dining, it may plateau in the near future. By using a tool like HoloLamp, we are able to further enhance the gastronomic experience farther from just how the dish tastes, looks and is presented. This extension can be used pre, mid and post dining to tell an even more compelling storyline, as well as curate interactive dining experiences without being limited by manpower.

Another use case as demonstrated by HoloLamp is for bringing miniatures to life with its 3D algorithms.

In the video above, it is fascinating to see that HoloLamp is able to map the space around the LEGO miniatures and project 3D special effects with reference to the miniatures.

Feature Discussion

Overall, the base functionalities of HoloLamp remain unchanged. However we can discuss the feature they have been adapted into, for use in the above applications.

Dining Use Case

In the dining use case, it is able to project a to-scale representation of the food, as well as a the background and explanation of the dishes. A 3D chef avatar can also be seen assisting the user and maintaining eye contact with the guest. Interestingly, it also allows user to interact with the projection by adding or removing dishes to their order.

Unlike QR code menus, having the HoloLamp placed on the table allows to-scale food portions that can help users decide how much food they really need. A large pro compared to paper menus is that this alternative is potentially more sustainable, but also makes menu updates easier and minimises the cost of reprinting.

Special Effects Use Case

In the miniature special effects use case, HoloLamp is used to map the objects on the table to superimpose effects. In one example, it is also able to model a 3D hologram amongst the miniatures. HoloLamp creates an optical 3D illusion using Spatial Augmented Reality to detect objects on the table.

Spatial Mapping that is then used to program interactions with HoloLamp. Each different color is a data point representing an object that HoloLamp considers as a rigid piece. These captured objects are then tracked in real time.

This opens up limitless possibilities for interactions with a wide variety of objects on the surface with HoloLamp

Feature Improvements

Add in more hand detection gestures and actions
- Despite already supporting some interactions, we can consider further adding-on to the facial recognition technology, and expand it to detect more types of motion within its surroundings.
- Since the guest is so close to the table and has the ability to touch the projected image, adding more interactions would open up a new layer of interaction
- As seen in the dining demo, it seems that touch-button interactions are supported, perhaps in the future, guests can interact with a slider to adjust the food portion size and see the food get adjusted in real-time, to properly cater to guests of varying appetites
Pair this technology with AR glasses
- Although it defeats the purpose of HoloLamp, AR glasses of today have become less of an obstruction to the experience, hence it may be wise to adopt a piece of hardware to expand on immersion and experience.
- By pairing it with some type of AR glasses that could be specific to that food establishment, restaurants can add a new layer of immersion apart from the table
- For example, flying whales and seafood walking around the guest can be seen when dining at e.g a seafood restaurant with the AR glasses
Add room-wide eye-tracking for 3D perception across all viewers
- Notably, the 3D technology/perception can only work for one viewer in the room. Any other viewer will see the projection in 2D. As such, by bridging this gap, we may be able to

More Possible Use Cases to leverage on HoloLamp

Before I round off this post, I’d like to propose a use case for HoloLamp despite technological restrictions that exist in today’s world.

A potential use case of the HoloLamp can be found augmenting table-top game experiences like Dungeons and Dragons.

Dungeons and Dragons is a role-playing table-top game that involves a dungeon master and its players. The story line is presented by the dungeon master and its players can react to the story lin and make decisions that can alter the path of the story. During the gameplay, the dungeon master typically relies on visual aids such as table-top maps; props modelling player characters; terrain like trees, hills and water bodies; all at varying fidelities. Some examples are shown below.

In its most primitive, maps can take the form of grids and manually demarcated spaces with pen or markers. And a step above that would be high fidelity models of trees and other objects

An (almost) overkill table setup for a dungeons and dragons game. Uses a 46″ TV, covered in fake leather, placed atop a 2×4″ raised platform (Source)

As seen above, existing solutions for improving the fidelity and immersion of the game can get quite over the top and hardware intensive. I believe that by using HoloLamp, we are able to capitalise on its 3D, spatial mapping and facial recognition capabilities to build more experiences with minimal hardware.

Conclusion

HoloLamp is a very interesting piece of technology that could have been deemed advanced for its time. Although this was released around 2017/2018, no further development has been found ever since. Pico projector technology has also recently seen an influx in innovation, bringing to light projectors like LUMOS, Kodak, Xiaomi and Anker. Similarly, consumers are being exposed to more XR applications and is starting to be applied in many ways. For example, how XR has been propagated by Web3.0 and the Metaverse.

Perhaps with the changing technological landscape and consumer exposure, we can see the return of technologies like HoloLamp and the advent of new ideas that arise from similar technology in the near future.

I would say that at this point, the lifeline of XR experiences like HoloLamp are dependant on who is willing to take up the challenge to embrace this technology and if they have a compelling enough market to capture.

Resources

https://hololamp.io/

https://www.youtube.com/@hololamp4467

https://www.youtube.com/watch?v=Q84sNac3R80

https://www.reddit.com/r/DnD/comments/l7cpf7/oc_finally_finished_my_digital_battle_map_setup/

https://electricalfundablog.com/technologies-behind-pico-projector/

AR, The New Driving Force

Posted on January 13, 2023 by JietNg

Eyelights in action

Introduction to EyeLights

Smartphones have brought us convenience and beneficial impacts, but it has also brought about fatal distractions. Research has shown that 1 out of 4 accidents is due to distractions on the phone while driving. Accidents also negatively affect the environment through the up to 75 million tonnes of C02 emitted by car repairs and car manufacturing. Until the introduction of EyeLights, which utilizes AR to display visualizations of information from a smartphone to prevent any unnecessary distractions from the road. Users are able to view information, such as their speed, arrows to signify the direction to take, indicators that notify that they are too close to another car in front of them as well as other features. As a motor enthusiast myself, I look forward to any technological advancements to better the experience of driving, which explains why EyeLights caught my attention. Though the display that EyeLights provide is nothing compared to Iron Man’s helmet, it still signals the start of the futuristic display design that may be used by future vehicles, which I am all for. Additionally, it makes use of commonly known design language to perform different actions to interact with the connected smartphone such as:

Turning on EyeLights: Hold button for 2 seconds
Pause/Resume music, receive/hang up calls: Press button once

This design would thus allow for a fuss-free onboarding of new users.

EyeLights’ experience

It presents a high level of engagement as users are able to view the extra layer of virtual information to make decisions in real life, such as making the turn when EyeLights display an arrow to turn the corner. On top of that, users are able to interact with supported functions of the connected smartphone device, such as activating the voice assistant or displaying the map.

However, this would mean that the UI design of the visualization layer has to be iconic enough such that users would understand what the different symbols represent. The reason is because different countries might adopt different road signs and symbols that EyeLights might not have considered (EyeLights is based in France). This could lead to the unintended effect of confusing the new user and might cause a fatal accident.

Therefore, an improvement would be to present tooltips for first-time users to understand all the visual icons of EyeLights so that the visual icons would rather help them than be a distraction. An even better improvement would be to support functions based on the location the user is driving in so that visual cues of EyeLights compliments the road system of the location.

Overall, though some improvements can be made to EyeLights for a more enhanced experience for motorists, it still serves as an effective AR application that could disrupt how road information could be visualized to users, which would soon include car drivers too.

Citations

Impact. EyeLights | Affichage tête haute Moto. (n.d.). Retrieved January 20, 2023, from https://eye-lights.com/en/pages/notre-impact

Mamun. (2022, May 30). Eyelights Eyeride motorcycle helmet user guide. Manuals+. Retrieved January 20, 2023, from https://manuals.plus/eyelights/eyeride-motorcycle-helmet-manual#axzz7qtvC27l0

XR Gaming’s Finest: The Immersive Experience of Beat Saber and Rumble

Posted on January 12, 2023 by lio

XR, or Extended Reality, is a rapidly growing field that encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies. These technologies have the potential to revolutionize how we interact with the digital world and offer new and exciting ways for people to engage with content. Let us will explore two VR games that I particularly enjoy: Beat Saber (2019) and Rumble (2022).

Beat Saber Trailer

Beat Saber

First, let’s take a look at Beat Saber. Beat Saber was revolutionary for its time, being one of the first games to fully embrace the VR medium, making good use of its interaction types and immersive qualities. Developed by Beat Games, Beat Saber is a VR rhythm game that has players slash through blocks representing musical beats with a pair of glowing swords. The game is set in a futuristic, neon-lit world, and players must slash the blocks in time with the beat of the music, while also avoiding obstacles that fly towards them.

What I like about Beat Saber is the way it immerses players in the game’s world. The neon-lit visuals and electronic music are incredibly engaging, and the sense of motion and physicality created by the VR headset and controllers makes it feel like you’re really in the game. The scoring system rewards long and accurate swings, encouraging players to commit to the satisfying motion of slashing that is unique to this medium. They also borrow the concept of sabers from the lightsabers in Star Wars, which gives players something familiar and intuitive to use, while also letting players live out the fantasy of wielding the sci-fi weapon.

One area where Beat Saber could be improved is in the variety of its content. While the game has a large selection of levels and songs, the gameplay itself does not change much from level to level. Adding more variety in gameplay, such as different types of blocks or obstacles, could make the game feel fresh and keep players engaged for longer. Examples include targets that players need to stab or look at, which are still immersive interactions that the VR medium offers.

Rumble Trailer

Rumble

Now, let us take a look at Rumble. Rumble is one of the up-and-coming VR games that takes gesture controls to a whole new level. Developed by Buckethead Entertainment, Rumble is a VR one-on-one fighting game where players use martial arts-like movements to move the earth around them. Players erect walls and hurl rocks at their opponents, damaging them and knocking them out.

Everything I liked about Beat Saber can be seen implemented in Rumble as well. The act of controlling and dodging objects in the space around you is incredibly immersive. The gesture controls encourage exaggerated and fun movements that make the player feel like they are actually fighting. And of course, the main mechanic borrows itself from earth bending in Avatar: The Last Airbender, again being familiar, intuitive, and fascinating.

On top of that, Rumble goes above and beyond in its polish. The UI is unobtrusive – you look down to see your health in an arc on the ground, and the enemy’s health is displayed above them when you damage them, but other than that there is no UI at all which makes the experience extremely immersive. The particle effects help sell the movement and collision of the earth in the game in a very aesthetic manner. It is also surprisingly realistic, where rocks lift up dust from the ground as they emerge, and explode into a cloud of dust when they collide. This is not only immersive but also a surprising hidden mechanic of the game, obscuring vision in the middle of battle. Lastly, there are also small interactions like the ability to fistbump your opponent after a fight that really puts the cherry on top.

Conclusion

I’m addicted please send help.