Experience cultural heritage at home thanks to AR and VR

Two teams from the School of Advanced Technology at Xi’an Jiaotong-Liverpool University designed applications to give users an at-home museum experience.

CubeMuseum and CoVisit VR Museum use augmented reality (AR) and virtual reality (VR), respectively. With these two applications, people can enjoy museums and antiques without leaving home – especially useful during the pandemic restrictions.

The applications can also be placed in museums to make the visits more fun and interactive.

Their works joined competitors from more than 600 colleges and universities across the country and won first prize at the 4th China Competition on Virtual Reality (CCVR 2021).

CubeMuseum

CubeMuseum is a creative AR souvenir. Each side of the cube contains a removable card with a picture of a relic. By scanning a card, users can see a 3D visualisation of an antique with additional information.

“When developing and evaluating the app, we conducted user experience testing, which confirmed that compared to some conventional methods, CubeMuseum is an enjoyable experience that can improve people’s understanding of the antiques,” says Ningning Xu, leader of the AR team.

CoVisit VR Museum

CoVisit VR Museum is an application that multiple players can enjoy simultaneously. Users enter a virtual museum through their VR or computer terminal and freely explore their favourite collections.

They can view every antique’s information in both Chinese and English, including its size, age, location and how it was discovered. They can also take the antiques to their own virtual museums and put up an exhibition.

In the VR space, users can communicate with and visit each other and interact with the antiques.

Mingze Wu, leader of the VR team, says that since the two projects both belonged to the field of human-computer interaction, they are user-centric and user-experience-oriented. For example, in CoVisit VR Museum, since the virtual museum is extensive, users will be “teleported” when they want to move quickly, so they don’t need a big room to experience it.

“For both apps, we digitised antiques that not everyone could experience and presented them virtually and attractively. We hope our applications could help preserve and promote cultural heritage,” adds Xu.

From coding to art design, every step was done by the team members, and each person was responsible for a different part.

Although the projects’ goals were clear, the methods of achieving them varied, Wu says: “Clarifying and piecing together our ideas was a challenge for both teams.”

Xu explains how they overcame this difficulty: “We first designed a prototype. Based on that, we discussed every function with our supervisor—what it would bring to users, whether users would like it, how learning and entertainment would coexist, so on and so forth. Edutainment has always been a hot topic in the field of education.”

Take the past into the future

Xu says: “Many AR applications today don’t stay popular for long. So later in these apps’ lifecycles, we’ll involve users in the design process and let them choose their favourite elements to create their own way of learning about and preserving cultural heritage.

“We’d love to offer opportunities to recreate. Users will be able to experience the antiques’ charm when they create, and in this way, our application will remain relevant,” says Xu.

Wu believed that the key to CoVisit VR Museum’s success was the idea behind the whole design, which was to promote cultural exploration. The app’s goal informed how the game’s mechanism would motivate users to explore.

Though the technical aspects were complex, these were just the building blocks. The foundation, he says, is the cultural relics themselves. “To raise awareness and make more people interested in history and heritage, we designers should improve our cultural knowledge,” says Wu.

Dr Yue Li and Professor Haining Liang from the Department of Computing at the School of Advanced Technology were the teams’ supervisors. They both spoke highly of the students’ performance.

Dr Li says: “Development involving VR and AR, especially for cultural heritage preservation, not only requires excellent coding skills but also skills in 3D modelling, mapping, animation, art, game logic and other areas, as well as close cooperation. It was the first time XJTLU students participated in the competition, and their teamwork was wonderful.”

She adds: “Many of them were new to the lab and unfamiliar with the development process or even the tools. But all of them were quick learners, picking up new skills to solve the problems they came across. They also did a lot of research to introduce new functions. I appreciate how they took the initiative.”

From left to right are Year 4 students Xingbo Wei, Letian Xie, Ruizhe Kan majored in Information and Computing Science, graduate student Ningning Xu majored in Applied Informatics, Dr Yue Li, Year 4 student Jiangyue Li, Year 3 students Ruixiang Zhao and Mingze Wu majored in Information and Computing Science.

By Huatian Jin

Translated by Xiangyin Han

Photos courtesy of Yue Li

Edited by Patricia Pieterse