Indexing news with AI

Asharq News is a 24/7, multi-platform Arabic news service reaching across the Arab world and beyond. When it required an automated way to describe and accurately summarise the 1,500 hours of archive and interviews it records each month for use by journalists, editors and producers, it partnered with Newsbridge to use its MXT-1 AI indexing technology.

This generative AI tech has been a game changer for Asharq News, enabling it to produce accurate, unbiased content descriptions in multiple languages, 40 times faster than a human could. MXT-1 enables it to index at scale in record time and search its large video collections as easily as it searches the web. The project is currently in the final stages of review before rollout.

The context

As a leading MENA news source, Asharq News conducts about 70 interviews with economic experts daily, each 5-10 minutes long. To successfully build a news story, it frequently taps into its archived footage. Ensuring that each piece of content is indexed with an accurate summary is critical.

The challenge

Kathey Battrick, Senior Manager, Library and Media Management at Asharq News, who helmed this project, speaks about the challenges.

“Asharq News deals with a massive volume of content, indexing around 1,500 hours every month. If content descriptions are missing or indexing is inaccurate, it takes additional time to find content and start crafting stories. Being able to quickly publish quality content to leverage the momentum of global news is a key requirement for the channel. Adopting such technology means we have descriptive metadata available to search within a matter of minutes,” she says.

In addition, Asharq News has a hybrid approach to indexing media assets in both Arabic and English. It uses AI to generate multilingual, timestamped metadata. Librarians review video footage and create descriptions so that content is thoroughly searchable and can be repurposed in the future.

Manually indexing media content in two languages, however, is challenging. Transcribing and summarising just one hour in a single language can take as long as eight hours for a human to complete. In addition, content descriptions must have a neutral tone of voice. When indexing is performed by a person, the descriptions can be biased based on their perspective. Creating this level of detail manually would require a large team of archivists. This in turn increases the risk of introducing spelling mistakes and inconsistencies in metadata entry and naming conventions.

Asharq News produces media content in Arabic, while many of the journalists and news producers search for content in English. The channel therefore needed a robust, efficient and accurate solution to produce scene descriptions of archived content in both languages.

The solution

Asharq News tapped Newsbridge, its longtime technology partner, for a solution that would enhance its multilingual media indexing process. “Newsbridge was already exploring the use of natural language models to generate human-like descriptions of video content. Our news requirements just acted as a catalyst for them to create MXT-1, Newsbridge’s generative and multimodal AI indexing technology,” explains Battrick.

MXT-1 combines multiple AI modalities, including computer vision and speech processing, with natural language models. The technology transcribes, detects and summarises people, objects, logos, context, actions, landmarks and landscapes. It describes the audiovisual content then gives it a one-line editorial summary, all with no human intervention. It can index more than 500 hours of video a minute.

The collaboration

Asharq News and Newsbridge worked together to train the MXT-1 technology on hundreds of hours of audiovisual content, leveraging AI transformers to make it particularly efficient at describing content with the same tone of voice and expressions used by the media organisation.

Every interview that Asharq News records is automatically transcribed and summarised by MXT-1, with the Asharq team supervising. The AI model is continuously fine-tuned to ensure that it outputs descriptions with the same grammar, syntax and tone of voice as Asharq News archivists. MXT-1 creates the content descriptions 40 times faster than real time.

“The use of AI can also facilitate the creation of your own datasets relevant to your content. In our use case, this is a dataset of Arab public figures and business leaders. Beyond summarising the transcript, MXT-1 leverages multimodal AI technology to recognise and add the names of Arab public figures and business leaders to the summary based on facial recognition, ensuring a consistent approach to names and designations,” explains Battrick.

The innovation

Asharq News is taking the industry forward in terms of technology deployment, with state-of-the-art infrastructure such as highly advanced studio equipment, the latest broadcast technology, efficient asset management and high-speed internet connectivity. It has the latest graphics, augmented reality and virtual reality systems, as well as a motion-tracking system that allows high fluidity while displaying visual content. It accommodates traditional as well as new media platforms, including mobile TV, broadband and IPTV, allowing content to be distributed across different platforms.

MXT-1 brings a special approach to AI indexing for Asharq News. The technology detects people, objects, logos, context, actions, landmarks, landscapes and text, as well as transcribing, translating and summarising audio-to-text in multiple languages. While traditional AI providers in the market can generate keywords and speech-to-text content, MXT-1 uses natural language to link everything together and describe a scene, resulting in more accurate descriptions and faster, more affordable AI indexing.

The technology offers Asharq News a significant leap forward in AI indexing technology based on its ability to describe scenes in natural language, which ChatGPT and other generative AIs currently on the market do not offer. Moreover, it improves upon the current state of AI indexing, which produces a jumble of tags that fail to give content owners the level of information they need.

MXT-1 bundles all the latest evolutions of Newsbridge’s multifaceted AI and can be trained and fine-tuned by an end user with multimodal rules and a custom thesaurus. It generates metadata with confidence scores of over 90%, ensuring it is accurate. Through a user-friendly interface, Asharq News can assess and monitor metadata output, meaning it is always in control and gets reliable results.

At a time when sustainability is important, MXT-1 technology dramatically reduces energy consumption, making it seven times as cost-efficient as mono or unimodal AI systems. This is achieved by maximising the use of energy-efficient CPUs over GPUs, coupled with a smart frame sampling technology.

Transforming multilanguage media indexing

When news producers at Asharq News are building a story, they need to be able to quickly find relevant sequences of footage. With MXT-1 media summarisation, the media company has achieved an important efficiency breakthrough in content indexing and discovery workflow, with news teams no longer needing to wait two or three days for an expert interview to be fully catalogued.

While most AI systems generate tags in English, MXT-1 generates actual descriptions in multiple languages, leveraging natural language models. Asharq News is currently beta testing the AI media summarisation capability for interviews in English, and plans to fine-tune the summaries and add support for Arabic when the language is released alongside French, Spanish, German, Italian and Portuguese in the near future.

Battrick says the collaboration with Newsbridge on MXT-1 media summarisation offers “a fast and efficient way of enhancing the cataloguing of our content … the user interface is intuitive, and we are able to smoothly train the AI to detect and recognise who is being interviewed to optimise our archiving workflow. We are able to create a rich multilingual archive, leveraging translations based on the Wikidata knowledge graph. This means that reporters will be able to search for content in both Arabic and English. Through a customised thesaurus, Asharq News can easily manage people’s names, titles and aliases in two languages.”

As an early adopter of MXT-1 generative AI indexing technology, working closely with Newsbridge to specifically develop media summarisation, Asharq News is working to transform how producers, writers and reporters working across its broadcast, social media and digital platforms find content.