A surprisingly low number of citizens is aware that in the Netherlands both the national and local government broadcast all of their meetings and debates. Even fewer people actually sit down to look at these live or recorded video streams, which is a pity as a majority of the discussions directly affect our daily lives. Therefore, in an attempt to increase participation and bridge the gap between government and citizens, we decided to commit our expertise towards improving the accessibility of these video recordings. Supported with funding from SBIR (Small Business Innovation Research), a government initiative to promote human-centered AI for the public sector, we conducted a 4 month study that finished with a prototype dashboard (see also VITE-A2).
Approach to creating AI based highlights
The original goal was to deploy our state-of-the-art emotion recognition combined with an analysis of the available transcripts, a strength of our sister company Parabots. We reasoned that based on emotions from speech and text, we could create a summary with the most relevant scenes. This proved to be very complex, due to the content of the debates and the quality of the data.
However, the idea of finding important fragments out of large datasets is still very relevant. We therefore focussed on the detection and segmentation of bite sized statements. We gained relevant insights in what would be required to further develop this. In addition, we developed an efficient and personalized way to present it.
Resulting in a prototype dashboard
Ultimately we combined our efforts in the design of a prototype dashboard. The video below provides a clear picture of what we believe to be its purpose. For different (personalized) topics we present a timeline, background information and related topics. Short video statements are meant to spike interest and invite a user to watch (parts of) the original debate. The video fragments can be sorted on relevant metrics such as emotional intensity. Finally, the voting is made accessible and all information from the dashboard can easily be shared across social media platforms to increase engagement of civilians with governmental decisions. An online tool like this could help users come into contact with complex data in an accessible way.