Methods for Analyzing Multimedia Data (CSS Summer School, Berlin)

August 5, 2019

The 3rd edition of the Summer School Series on Methods for Computational Social Science focusing on Methods for Analyzing Multimedia Data took place in Berlin, Germany, July 22-27, 2019, with the participation of DNDS students Hao Cui, Rebeka O. Szabó, and Milán Janosov.

The school hosted about 40 students from about ten disciplines, such as mathematics, sociology, physics, and computer science, doing an excellent job in bridging interdisciplinarity gaps and bringing the various branches of computational social science together. This year’s edition was focusing on multimedia data for social sciences, which topic was introduced by roughly ten leading researchers through lectures and practical hands-on session, covering image, social media, and even video-based data sources. We learned about neural networks and deep learning based computer vision techniques, such as image classification, and their potential use for studying social media data or analyzing urban spaces. As moving towards social media, its links to health and human well-being were also highlighted in several talks. We also discussed data exploration and visualization by relying on artificial intelligence, and how to use AI to analyze sound, detect emotion in voice, and combined that with image recognition.

Extending the scope of the school beyond the lectures, from the first day on we split up into groups to carry out mini-projects mentored by the lecturers. The group of Milán Janosov, together with Rui Sun from the University of Amsterdam, and Rishabh Kaushal from the IIIT, Delhi focused on understanding the role of uploaded images and texts in the success of online petitions. The project was supervised by Miriam Redi from the Wikimedia Foundation, and won the best project award of the summer school.

The group of Rebeka O. Szabó (CEU), Severin Engelmann (Technical University Munich), Kim Smeenk (Kim is from the University of Gröningen) and Thien-Huong Ninh (Cosumnes River College) was working on which aesthetic features of images underly convergence and divergence in aesthetic empiricism. In other words, the team intended to identify those characteristics of images that can explain why people tend to agree or disagree on the beauty and quality of the images. In addition, the group of Hao Cui (CEU) and three sociologists from Pennsylvania State University, Trinity College Dublin and Ferdowsi University of Mashhad, supervised by Luca Maria Aiello (Senior Research Scientist at Nokia Bell Labs), was working on Textual Markers of Social Interactions. They analyzed hate/offensive speech from user comments posted on Reddit and quantified the effects of Jihadi terrorism on hate/offensive speech towards immigrants.  

Finally, in the name of the participants, I would like to thank the organizers for putting together this great event, especially Claudia Wagner (GESIS - Leibniz Institute for Social Sciences) and Nicola Perra (Greenwich University).

Blog post by Milán Janosov