It had never been done before, and the idea would make many a broadcaster鈥檚 eyes go wide with wonder. Use artificial intelligence (AI) to help create two nights of experimental programming on network 91热爆 TV on 91热爆 Four, by delving into the treasures of the vast 91热爆 Archive. And then dig deeper, and use AI and machine learning to create mini-91热爆 Four style segments in a programme presented by Hannah Fry, to illustrate how the technology works.
So why were we doing it? Here at R&D we鈥檙e already exploring how AI and machine learning technologies could transform the future of media production, and are always looking to work with programme makers to support and future proof the 91热爆.
The 91热爆 has one of the largest broadcast archives in the world, and manually searching millions of programme hours is near impossible. It鈥檇 take decades for one person to watch them all.
The increasing digitisation of the archive gives us the opportunity to develop AI tech to help filmmakers and schedulers find hidden gems - programmes and content that may otherwise not be seen again, or overlooked. And we are able to utilize the speed of machines to help us, as a computer鈥檚 gaze is much faster than ours and can scan a years鈥� worth of TV in a couple of days.
R&D have previously worked with 91热爆 Four, so it was a great to collaborate again and we agreed with 91热爆 Four Editor Cassian Harrison that artificial intelligence, an emerging trend, would be a great choice. Two ideas were finally selected that could make their working lives easier and complement their schedule best. And so 91热爆 4.1 was born.
First, we would use machine learning to pick out likely shows for an evening鈥檚 entertainment on 91热爆 Four from the vast 91热爆 Archive. The channel's scheduler would then use this list to select what would be broadcast.
Now 91热爆 Four is quite a distinctive channel, so we鈥檝e had to create the new technology ourselves, drawing on the work of developers around the world. It鈥檚 been done in-house by R&D and can鈥檛 be bought off the shelf. It鈥檚 in its early stages and we鈥檝e been working with academics to refine it, including one from Mexico who is working on a PhD about teaching computers to enjoy television.
The AI examined the programmes that 91热爆 Four had shown in the past and their attributes, analyzing their descriptions and subjects 鈥� be it music, history or science. Computers trawled through more than 270,000 programmes across the archive that were available in digital form, ranking the top 150 most relevant factual ones by what you could call their '91热爆 Four-ness'. Schedulers then used the list to select those to show across two nights, the 4th and 5th September 2018.
Now to the second idea, and really challenging part of the project, to use AI to help make TV in Made by Machine: When AI met the Archive. Our aim was to highlight how the technology worked, and using it to create mini-91热爆 Four style compilations from these top 150 programmes seemed like the perfect way to do it. But how would the machine 'watch' them? What would it see? And could it be trained to select and edit video in the beautiful way people do?
The aim was not only for the AI to create segments, but to let people 'see' inside the mind of the machine as it did so, albeit in a simplified way. Viewers could watch its processes and also spot the limits of its learning.
How did it do that? It broke each of those top 150 91热爆 Four-like shows into bite-size chunks (more than 15,000 in total), and then chained them back together.
And to show a range of different ways computers could analyse video and then try to link it together to form some sort of story or narrative, R&D technologists devised four techniques:
- Object & Scene Recognition: where the AI learns to identify what a scene consists of, including the type of landscape, identifying objects, whether people are featured and what they might be wearing. You then see how it attempts to create a compilation where each scene follows on from the last in some way.
- Subtitle Analysis: the AI uses natural language processing as it scans the subtitles of archive programmes and then looks for connections between words, topics and themes as it pieces footage together.
- Visual Energy (or Dynamism): the AI analysing video frame by frame to try to detect whether there鈥檚 a lot of activity on screen (high energy) or not (low energy). It then tries to create a compilation with shifting pace starting slow, building up energy then dropping back for a breather before shifting to a climax.
- Finally the AI draws on what it鈥檚 learnt using all three techniques to create a new piece of content.
For each we tried to show on screen the decisions that were being made, in the form of simplified text that was generated by the AI. The video compilations in the programme are virtually as made by the AI. The selection of all the clips and pretty much the editing were all done by the AI, with minimal human intervention.
We tested each of these techniques by creating compilations of up to 15 minutes. It didn鈥檛 always go to plan. The machine didn鈥檛 always get it right, and sometimes got stuck in its own a data loops. At one point it had a particular fascination with buses, another test run focused almost solely on clips from one documentary about the garden of an English country house, with barely a mention of the other 150 programmes. But there were fantastic moments of unexpected juxtaposition, and wonderful snippets of archive programmes which our team had never seen before and which we鈥檇 now like to watch in their entirety!
The AI reflects its training data, so any bias in its training was replicated. So it failed to spot mobile phones, but picked up cell phones (incorrectly in some case). And the machine's choices were guided by our engineers. For example, we opted for the bite-size chunks of programme to be created with durations between 25 seconds to just under two minutes. It also found some programmes that wouldn鈥檛 usually be picked up for scheduling due to complicated contractual and rights issues, so we filtered those out. Multiple versions were made using each technique, and then one of each was chosen by 91热爆 Four to go into Made by Machine: When AI met the Archive.
These sections were incorporated into the programme together with presenter Dr Hannah Fry and a virtual host, '91热爆 4.1' as well as colleagues across the 91热爆, including the team at 91热爆 Archives in West London. As experimental as the programme was, we didn鈥檛 have the timeframe or resources for everything to be created by AI and machine learning. The virtual host '91热爆 4.1鈥� was made by , in the spirit of 91热爆 Four and the programme. We also worked closely our colleagues in 91热爆 Television & Media Operations to produce the programme production, with graphics designed with .
Made by Machine: When AI met the Archive at 9pm on 91热爆 Four, 5th September 2018 as part of 91热爆 4.1 AI TV.
- -
- 91热爆 R&D - Using Artificial Intelligence to Search the Archive
- 91热爆 R&D - AI Opportunities: Transforming Coverage of Live Events
- 91热爆 R&D - Artificial Intelligence in Broadcasting
- 91热爆 R&D - 91热爆 RemArc - how our archives can help people with dementia
- Machine Learning and Artificial Intelligence Training and Skills from the 91热爆 Academy including:
-
Internet Research and Future Services section
The Internet Research and Future Services section is an interdisciplinary team of researchers, technologists, designers, and data scientists who carry out original research to solve problems for the 91热爆. Our work focuses on the intersection of audience needs and public service values, with digital media and machine learning. We develop research insights, prototypes and systems using experimental approaches and emerging technologies.