Nominated by: RAI – Roberto Rossetto
Nominee: RAI – Federico Maria Pandolfi
Title: Visual Search: Requirements, Architectures and Use Cases to Visually Search through Broadcast Programmes
The proposed project aims to solve a significant problem that historically affects the production and the archives departments of broadcasting companies: the lack of metadata and the consequent difficulty to retrieve the needed multimedia contents. To be more specific, this is particularly true for example when it comes to the raw footage that is used by the newsrooms to edit the news to be aired. Usually, this type of footage is either not or only partially annotated and it is generally difficult to access. The result is that the footage often gets discarded or lost, causing a loss of money and resources.
A feasible solution to this issue is presented in this project and involves state-of-the-art technologies in the visual search field. The project is not only conceived to be a mere visual search tool but it is instead designed to be a framework ready to be included in the production workflow. It consist in two parts: a search engine and a workflow manager. It was designed to be as modular as possible, allowing the replacement of the search engine with different or newer solutions if needed.
The idea behind this project is that it is possible to add metadata to the non-annotated raw footage simply by linking it to the corresponding aired material which, on the contrary, is thoroughly annotated. This approach theoretically goes well beyond our specific use-case but we decided to start with a practical and well-confined scenario and extend afterwards the framework’s capabilities (e.g. logo detection, image search over pictures catalog, …).
The engine of the whole framework relies in a software module developed by the British company Visual Atoms, whose employees are world leading experts in image and video analysis and computer visions. Experimental results regarding this search engine module are already available and have been obtained using the opinion of a significant number of skilled professionals who deal daily with the catalogue, annotation and metadata, so as to evaluate the framework in the most objective way possible.
What’s around the search engine is a workflow manager, which manages the source files, folder structure, performs required conversions and tracks the state of each step. This manager is made mostly using the open-source software called “Apache AirFlow”, a new and efficient platform to manage workflows, part of the Apache Software Foundation. The workflow manager on the other hand is still in early development. Therefore there is still not enough evidence of its efficiency and performance under any workload.
The project is being carried out since September 2016 (approx. 20 months to date) and its prototype is still under development by Rai Teche, together with a small scientific newsroom (TG Leonardo) that represents fairly well a typical newsroom but has a much smaller footprint.
Early empirical results demonstrate that the correlation between raw and aired material can be achieved with acceptable precision. This allows metadata sharing between annotated (aired) and non-annotated (raw) material.
