Adolfo Almeida graduated with a master’s degree in Computer Engineering in 2020 and has been working at MultiChoice for the past two years. He hails from Angola. His project focused on developing a video recommendation system using a range of deep learning features extracted from video content. What makes this system novel is that it actually watches thousands of movies, making use of the same sound and visual perceptions that a human would. Based on what one has watched in the past, it makes recommendations within a particular genre. It differed from other systems in use at the time as those systems made use of both text and metadata.

THE VALUE OF A RECOMMENDATION SYSTEM

Following the popularisation of media streaming, a number of video streaming services are continuously buying new video content to generate a potential profit. As such, the newly added content has to be handled well to be recommended to suitable users. Video-sharing websites rely heavily on video recommendation systems to assist users to discover videos they may enjoy.

With the enormous increase in the number of new videos constantly being uploaded, some video streaming services have to deal with unrated, unaudited and completely new content, which they know nothing about. Recent studies on video content analysis and video-retrieval tasks use various types of deep learning features extracted using pre-trained models due to their outstanding performance in different domains, compared to hand-crafted features.

A video recommendation system is a user-level video-filtering service that helps users explore the videos that are available to watch. It offers a more personalised experience for users by recommending the most relevant and appropriate videos for them to watch. In order to do this, algorithms are used to analyse the information about the videos and the users, as well as the past interactions between them.

Existing recommendation systems use one of three approaches: the collaborative filtering recommendation method, the content-based recommendation method and the hybrid recommendation method, which is a combination of the first two.

Most video streaming services that use a video recommendation system to compute the video relevance based on user feedback use the collaborative filtering method because of its state-of-the-art accuracy. This feedback is used to model the user-video preference, and compute video-to-video relevance scores to provide personalised recommendations. However, this approach suffers from the new item cold-start problem.

Adolfo’s research project addressed the new item cold-start problem by exploring the potential of various deep learning features to provide video recommendations. Various deep learning features extracted from multi-modal, extremely high-dimensional information from videos are used to enhance the quality of recommendations. These features included those that capture the visual appearance, audio and motion information from video content. 

In the process, Adolfo compared deep learning features against genre features and hand-crafted features. He also explored different fusion methods to evaluate how well these feature modalities can be combined to fully exploit the complementary information captured by them. This was done to improve the  recommendation quality in terms of accuracy and beyondaccuracy metrics. Experiments on a real-world video dataset for movie recommendations show that deep learning features outperform handcrafted features.

Finally, he performed an ablation study to empirically assess the importance of using a diverse range of video content features on the overall recommendation quality, while taking full advantage of the available data. The results suggested that the fusion of visual, audio and action features provide more accurate video recommendations to users when compared to the fusion of only visual and audio features. In addition, the combination of various deep learning features with hand-crafted features and textual metadata yields significant improvement in recommendations compared to combining only deep learning features.