Archive Achievement Awards 2019

Artificial Intelligence at the service of the RTS Audiovisual Archives

by Pietro Rezzonico, RTS

Artificial Intelligence at the service of the RTS Audiovisual Archives – Pietro Rezzonico, RTS

One of the objectives of RTS is to be able to explore and exploit its audiovisual archives and to open them to the public. Faced with the lack of metadata on content, we have set up a development team over the past two and a half years that uses artificial intelligence (AI) technologies to perform automatic metadata extraction to achieve this objective. The first development was the synchronization of videos between different versions of a program, such as live feeds and clean feeds. Based on the exact visual recognition, it was a quick win that helped documentalists by saving manual work time and generates a basic segmentation for all programs.

Secondly, we have developed quality control algorithms to identify specific defects on the files resulting from the digitalization. The manual quality control of 400,000 digital files would mean 20 years of work. Instead of that, an interface allows humans to examine the results of the algorithms in a quickest mode. Based on these successes, we have developed a unique solution that we label Visual Feature Extraction Pipeline. This tool implements recognition capabilities using Machine Learning techniques to identify patterns within facial, object and landmark features. All algorithms are based on features that are extracted once at the beginning of the pipeline. Three applications were developed and are in production:

1) The facial recognition. To perform the facial recognition, a public figures database was created by documentalists. It contains now 5,500 people. The results of the facial recognition are integrated in the digital assets management system. This automatic metadata extraction helps the documentalists during the indexation process and makes data searchable by the end users in the search interface.

2) The visual search, which makes it possible to search for similar faces, landmarks or objects from a given shot or image. It allows a powerful exploration and exploitation of the audiovisual content by finding similar undescribed images or finding images with rights issues. At the end 2019, 100% of the RTS video archives will be searchable through this revolutionary way.

3) Classification allows content to be automatically categorized using custom classifiers, which can be trained with objects or landmarks features. The first implementation is carried out by our documentalists who are creating a sport classifier, in order to automatically categorize and index old sport collections for which no metadata are available.

Several new uses are planned, such as the traceability of the use of archives and the speaker identification, which are currently under development. What we have learned: the AI technologies can really help us in our goal to open up the archives by extracting metadata automatically, enrich our legacy archives, and giving discoverability to our archival content. Internal developments allow us to be independent of commercial solutions, to protect our sensitive data and to meet our specific needs (local public figures, classifiers). The use of open source tools opens collaboration possibilities with other public broadcaster. Putting into production the AI based applications early in the development process allows early feedback and results in a user focused development. The costs of ownerships are more than half as high as with a commercial solution.

SHORTLISTED CANDIDATES

Gender Equality Monitor

by David Doukhan, INA

Gender Equality Monitor – David Doukhan, INA

Gender Equality Monitor (GEM) is a project initiated in July 2017 at INA (the French National Audiovisual Institute). It aims to describe automatically representation and treatment differences existing between women and men in French-language media such as TV, radio, newspapers and song lyrics collections. The ambition of this project is to analyze several million documents sampled over a period of more than 80 years, in order to produce the most comprehensive description of the representation of men and women in French media.

Automatic audiovisual indexing methods based on recent advances in machine learning (speaker gender detection, face recognition and characterization, speech-to-text and spoken language understanding) are proposed to deal with this challenging amount of data.

A large-scale study focusing on women and men’s speech time has been conducted using one million hours of audiovisual material, sampled from 1995 to 2018 over 21 French radio stations and 34 TV channels. Speech-time, described using Women Speech Time Percentage (WSTP), was obtained using inaSpeechSegmenter: an open-source machine learning software built at INA, which automatically detects women and men voices in sound signals. WSTP variations have been analyzed across channels, years, hours, and regions. Key findings include the fact that men spoke twice as much as women on TV and on radio in 2018, and spoke three times as much than women before 2004. Only one musical radio station out of the 55 channels considered is associated with a larger women speech-time. WSTP is lower during high-ratings time-slots on private channels. WSTP is higher for channels aimed at a female audience, and lower for sport and cultural thematic channels. Detailed estimates resulting from these analyses have been released in open-data, thus allowing academics, journalists and audiovisual professionals to combine it with other structured information sources (channel governance, ratings, political events…).

A smaller-scale study was conducted using face detection and gender classification software realized at INA. Facial exposition time was compared to speech time estimates. Early findings suggest women visual presence on French TV is larger than women speaking-time on all the considered channels. Additional preliminary work based on speech-to-text and lexicometry allowed to provide insights on the words chosen to refer to male and female speakers, showing clear differences between channels. More exhaustive analyses are being carried on to confirm these trends and monitor their evolution over time.

Based on these extremely encouraging results, the GEM project has recently obtained funding from ANR (the French National Research Agency). Starting in January 2020 and for a duration of 42 months, it will involve a transdisciplinary consortium of seven members: two major audiovisual media actors (INA and online streaming platform Deezer), two STEM laboratories specialized in automatic information extraction from text and speech (LIUM, LIMSI) and three humanities laboratories specialized in the study of gender and media (CARISM, LERASS, ENS LYON). The project will be based on three complementary lines of work: formalizing descriptors relevant for the quantification of representation differences between genders, implementing these descriptors using information extraction methods, and carrying out quantitative studies based on the exploitation of the descriptors obtained automatically.

A Searchable SVT

by Anna Abenius & Henrik Johansson, SVT

A Searchable SVT – Anna Abenius & Henrik Johansson, SVT

The purpose of the project, A Searchable SVT, was to improve the possibilities to navigate through online program material from the perspective of the audience. Without a lot of money, the project tried to find ways to improve searchability without re-building all systems and processes. The project focused on the lack of cast and crew names online, especially for Program Acquisitions. During the last decades these names have not been stored in the archive based on the lack of re-use rights.

One important issue that complicates this project is that metadata production at SVT is a cooperation between many different departments involved in producing, curating and archiving metadata such as Archive, Communication, Acquisition, Reruns, Legal departments, Online publishing including Open archive, IT Development and production teams, in total around 17 different teams and functions.
After some research the project found out that it was quite easy to export the information from the archive system to SVT Play (our main online platform). The problem was that the whole process for applying metadata was based on archive purposes and therefore was made after a production was published. And in the case of Program Acquisition this would mean a completely new task. The challenge was mainly to change the work process and inform and convince everybody that this was the way to go forward.

The focus was to move the input of metadata from afterwards to before publication. An important member of the project was the head of SVT Play who managed to convince selected parts of the organisation (Fiction and Entertainment) about the benefits of providing this metadata before publication. This had always been a challenge for the Archive, but now when the Program Acquisition and production teams could see the need from the audience and the possibilities to reach out to a wider audience, the interest was increasingly rising.

As a result of metadata input earlier in the workflow, a lot of other developments also needs to take place. A system that already have been developed and implemented as a result of the project is ”The Cred”, an application that makes it easier to apply cast and crew to the archive system. The application is developed for the program acquisition department, but there are already plans to develop it further.

During the project we learned that many teams and functions at our company are involved and directly affected by the same metadata flow. We also learned that even if it will be a lot of new processes and needs for metadata with online publishing it’s also a very good opportunity for the archive to get better data, and to get it faster as well.

Knowyourcarrier.com

by Marlies Criel, VIAA

Knowyourcarrier.com – Marlies Criel, VIAA

In October 2018, VIAA launched a website to help everyone with any kind of audiovisual material in their attic or basement. The website has three direct purposes: 1) help people identify which audiovisual carrier type they hold, 2) help to establish its heritage value and 3) advise on how it can be digitised and how the carrier as well as the resulting files should be preserved. Indirectly, the site raises the awareness about audiovisual heritage and brings specialised professional knowledge much closer to a general audience.

The tool has many extras and the use of a decision tree, enriched with photographs and fun facts, makes the tool easy to use. With binary yes-no questions and simple questions about the shape or size of a carrier, you can identify your own carrier and determine its heritage value. Furthermore, the tool helps with customised advice on preservation and digitisation. You will not only find out whether you have a wax cylinder, a video CD or maybe even a Betacam in your hand, you will also find guidelines for a correct preservation. To top this, we’ve added a template and specification table that support you in formulating specific questions to digitisation companies. By launching knowyourcarrier.com, we share expert knowledge about the process of preservation with everyone in an innovative way.

Since our foundation in 2012, VIAA has digitised a broad scope and large volume of audiovisual materials from broadcasters, museums, archives and government bodies. In each digitisation project we tackled new types of carriers, leading to an extensive knowledge about digitisation and sustainable preservation for several types of carriers, according to the current international standards (including IASA TC04 and IASA TC06). We noticed that, in many organisations, there is a lack of knowledge about carrier types and how to best preserve or digitise them. On top of that, we often receive questions from individuals about the nature of certain carriers and if we could help to preserve or to digitise them. We all know valuable sound and image is not only kept by professional organisations like broadcasters, museums and archives, but also stored by individuals everywhere in society. Sometimes it gets lost, because people have carriers, but don’t have the players anymore, don’t know that what they own is actually of heritage value, and just ‘get rid of old stuff’. For example: an individual may have the only copy of programmes that a television archive did not preserve. Or a private collection of magnetic tapes with heritage value could benefit from professional care, which is also the scope of UNESCO’s recently launched Magnetic Tape Alert Project. ‘Knowyourcarrier.com’ brings a responsible and correct answer to the many questions that one may have about audiovisual preservation.

We launched knowyourcarrier.com on Saturday, 27 October 2018, World Day for Audiovisual Heritage, with as a central theme: ‘Your story is moving’. This was very appropriate, since we help a great number of potential storytellers to safeguard their sources for future generations with the tool. The website has its own link, but is also accessible through the VIAA website. To increase the website’s findability, we also offer iFrames. With this feature, the website can be integrated in another website. The tool is available in both Dutch and English, making it relevant to an international community.

SHORTLISTED CANDIDATES

Give us the GIFs: Crowd-Curating Iterative Born-Digital Heritage at Scale

by Rasa Bocyte, Brigitte Jansen & Johan Oomen, Netherlands Institute for Sound and Vision

Give us the GIFs: Crowd-Curating Iterative Born-Digital Heritage at Scale – Rasa Bocyte, Brigitte Jansen & Johan Oomen, Netherlands Institute for Sound and Vision

Archiving born-digital collections

Sound and Vision established its digital archive of radio and television broadcasts as of 1999. It started collecting web video in 2008, websites in 2013 and video games in 2016. Since 2012, the research team has been investigating possibilities for archiving more complex interactive websites and virtual reality productions. Initially, from a collection strategy point of view, born-digital collections have been treated as “contextual” materials that support and add value to the radio and television collections which were regarded as the core collection of Sound and Vision. Given the many changes in media production and consumption, this differentiation between core and contextual collections has become obsolete. In the latest Collection Policy, published earlier this year, a more substantive order was adopted. Topical pillars now form the foundation of the collection policy. These are populated with collections that encompass various media types that range from linear television to web-video and physical artefacts to video games.

Reconceptualising the Collection Policy into four broad pillars that encompass media in all its possible shapes and sizes, opened the way to include new artefacts that would have never been considered as part of a core collection before. In particular, it lowered the archival threshold for born-digital artefacts like GIFs that previously would not have been recognised as a prominent element in the digital media landscape.

Preserving GIFs

GIFs have become an inseparable part of digital culture, spreading like a virus across the web. Despite that, GIFs have not been widely recognised as cultural heritage and have not found their way into archival collections. We are eager to challenge this. We consider GIFs as a broader cultural phenomenon that appropriates easily-recognisable cues from audiovisual culture to convey messages that immediately move us.

They are part of a remix culture online that takes fragments from popular television shows and renders them into memes with silly captions, they show footage of celebrities with funny facial expressions that we can all relate to, or they capture memorable moments that we want to watch over and over again. The list goes on and on, but what all of these GIFs have in common is that they serve as a reflection of the current zeitgeist in the digital communication sphere. As an organisation tasked with preserving Dutch media culture, we cannot overlook GIFs as an important part of it that needs to be preserved and represented in our collections.

So what kind of GIFs could be collected for a Dutch GIF collection? That turns out to be quite a tricky question. GIFs have no national or thematic boundaries. Good luck finding GIFs tagged “Dutch” or “Netherlands”! Is it even possible or necessary to determine the “dutchness” of a GIF? We have scavenged through the web to find what we think best matches our selection criteria. But we needed help figuring out what fits into the collection and what doesn’t. And this is where the help of our users comes in! On the 26-28th of October 2018, we used our Instagram channel to let the Dutch public decide what should be included in the GIF collection at Sound and Vision. This way, the audience received the opportunity to cast their vote and choose which GIFs should be preserved for the future.

Artifacts detection by AI: Quality Check of video files from mass digitization

by Denise Barcella, RTS

Artifacts detection by AI: Quality Check of video files from mass digitization – Denise Barcella, RTS

The RTS Quality Check project uses the potential of artificial intelligence (AI), that is to say computer vision, to ease quality control of video files produced by mass digitization. Five algorithms are currently checking almost 400’000 files, resulting from the digitization of 120’000 analogue carriers (mainly Betacam SP), so as to ease human work. Humans have only to examine the results of the algorithms, and, decide if a new digitization is needed.

These 5 algorithms have been calibrated by our project, with the main common anomalies or artefacts that we found in our digitized tv archive collection and categorized according to their type. An interface was also designed to display the results of the algorithms so that humans can make the right decisions easily and efficiently. The main goal was to avoid any loss of our unique audiovisual heritage, because unfortunately, the quality control was not done thoroughly enough during the mass digitization and this for lack of financial and technical means at the time, and because there are threats on the original carriers, like time passing by.

What was a challenge is that RTS Archive Department had to find a solution to carry out this quality control in a context of cost-saving measures and after digitization (which made the process much more complicated). As the potential of AI was raising in almost any field, we decided to try using it in the context of the Quality Control.

TV Guide 1968

by Jakub Adamus & Jakub Hosek, CTV

TV Guide 1968 – Jakub Adamus & Jakub Hosek, CTV

TV Guide 1968 represents so far the biggest web project the Czech TV archives has participated on. The idea of transformation of old paper TV guide into its digital interactive version was born as the 50th anniversary of the Prague Spring was coming closer. The more often we were leafing through old issues to search for relevant information about the then broadcast, the more intensely we were dreaming about its online version available for all users searching for the same information that would like same as the old paper version. But we didn’t want only a textual database, we decided to create a fully-fledged AV portal and publish there as many AV materials, sound files, photos and articles as we could. The guide was being released week after week during the whole year 2018. Due to IPR reasons we couldn’t publish everything that survived, but still it stands for very complex source of broadcast of Czechoslovak TV in the important year 1968.

SHORTLISTED CANDIDATES

TILT

by Immy Verdonschot & Maartje Hülsenbeck, Netherlands Institute for Sound and Vision

TILT – Immy Verdonschot & Maartje Hülsenbeck, Netherlands Institute for Sound and Vision

TILT is an online innovative storytelling platform with which we tell stories about man, media and society. Due to its technology we can share our archive by telling these stories about past, present and future and putting them into context. With copyright and Intellectual Property issues, we of course have to be careful. That’s why we keep a close eye on citation law and only use short fragments of our archive to get our point across. There’s also the question of ethics when telling a story: is it newsworthy enough or could usage of the footage harm someone? We also have a privacy check to make sure none of the footage we use for example show children (under age). These are all questions we’re dealing with when writing a story and take into close consideration on whether or not to use (archival) footage. But all of it comes together in great stories about media in one form or another, showing the relevance of our archive as well as show the relevance of certain subjects.

Sound and Vision is an institute for media and archiving. To expand on our position and mission, we wanted to publish articles about man, media and society. This way we don’t only show that we’re up to date with media culture, but that we want to participate in it. Our target group is everyone with an above average interest in media, which may vary from newspapers to vlogs, from television to games and the web. Our primary focus will be on millennials (18-36). TILT is the unique result of that: an innovative story platform built out of cards. We make stories with snackable cards that you’ll still want to read and watch – despite the short attention span. The vertical orientation and swipe interaction make it ideal to read with one hand. A visual language and strong emphasis on attractive footage makes the ‘instagram worthy’ story platform unresistable for this generation. The stories on the cards come to life with smart combinations of (archival) video, audio, lively text, photography, interactive polls and/or questions. And if you don’t feel like reading it right away, you can always choose the shorter version or ‘snooze’ it, but sending a reminder to yourself.

ARchive

by Sarah-Haye Aziz and Massimiliano Babbucci, RSI

ARchive – Sarah-Haye Aziz & Massimiliano Babbucci, RSI

RSI experience on getting the new generation close to the archival contents.

ARchive is a new way of consuming A/V content that anyone can use simply switching on the mobile device and point in all direction discovering contents with high added value, both historical and emotional. A walkthrough in augmented reality world. Linking the new technology to the world of archival content and bring the history to user direct in the field.

Thanks to the use of augmented reality technology and GPS, we offer to the user the ability to access A/V contents of our archives linked to the region and in particular to those contents strongly related to the position in which the user is. The ARchive application is able in real time to determine the position of the user and place all the A/V contents selected from our archives in the space around. Each content has his own set of metadata like title, description, coordinates that allow the user to make the experience more complete.

Awards 2019
Awards 2019

FIAT/IFTA is humbled to award the 2019 Lifetime Achievement Award to Jean Varra.

Since 2014 FIAT/IFTA has also given a special Lifetime Achievement Award to a person with special merits for FIAT/IFTA and the field of audiovisual archiving as a whole.

FIAT/IFTA is humbled to award Jean Varra the 2019 Lifetime Achievement Award.

As you know, for a few years, FIAT/IFTA also gives an Award to a special person. It is a Lifetime Honorary Award with which we honour someone who has got special merits for this federation and for the field of audiovisual archiving as a whole.

This year in Dubrovnik, we would like to honour someone whose career started already in 1975, that is 44 years ago. In fact, he started his career in audiovisual archiving thanks to a small coincidence. He had applied to two organisations, but the institution he has worked for was the quickest to reply to his application.

For the first three years of his career, he focused on training young professionals. He discovered that he liked it and he would keep on doing it for the rest of his professional life, and he does it until today. Literally, hundreds of students have been in his classroom and will acknowledge the thoroughness of his teaching. You can imagine the impact he has had on a whole generation of audiovisual archivists!

In the eighties and nineties, the man we honour tonight was a privileged witness of how digital

image and sound conquered media production. But the last 15 years of his career were entirely dedicated to the archives. From the late nineties he was the main driver behind one of the earliest and probably also the largest audiovisual preservation and digitisation plans in the world. From his working place in Bry-sur-Marne, but also all over the world, he provided decisive input for uncountable trainings, collection assessments, digitisation plans, European research projects and so on. One can recognize his influence in audiovisual archives from Cuba, via South Africa and Madagascar, up until Cambodia.

But this man is not only appreciated because of his unmeasurable knowledge and professionalism. He is also a warm personality, a true gentleman, and a very welcome guest on a long series of FIAT/IFTA conferences, where his presentations were always of exceptionally high quality. We honour this man because he has strived for quality with unwavering force, because he constantly conveyed this mentality to the ones who surrounded him and because he has always done so with great kindness and modesty.

The 2019 FIAT/IFTA Lifetime Honorary Award goes to … Mr. Jean Varra.

RECENT EDITIONS