Abstract
Social networks, involving people and their interactions are at core of human society. But many current computational social methods focus more on the individual than their interactions. Deep neural networks have been successfully applied to tasks such as natural language processing, dialog modeling, or analyzing sentiments in a conversation. In these areas, we will often encounter data that originate from multiple sources. These signals can interact with each other synchronously, but detecting such synchrony may prove challenging.
In this work we focus on investigating how deep neural network architectures can help us better understand synchrony in social contexts. We investigate different coupled sequential models such as an end-to-end connected gated recurrent unit (GRU), an inherently coupled GRU, message-passing, the role of attention and the use of transformer networks for coupling.
We evaluate the effectiveness of our coupling models on multiple datasets. We first test on synthesized sequential coupled data as a sanity-check and then move on to more realistic data. We test our models on three different real-world datasets collected in the context of various social interactions. In two of the datasets, we predict the rapport between two persons based on data extracted from the video of them interacting. In the third dataset, we predict friendship/familiarity between two people based on their interaction. We present the findings from the work and conclude that the coupled transformer network performs the best.
Library of Congress Subject Headings
Social interaction--Computer simulation; Neural networks (Computer science); Intelligent agents (Computer software)
Publication Date
5-2021
Document Type
Thesis
Student Type
Graduate
Degree Name
Computer Science (MS)
Department, Program, or Center
Computer Science (GCCIS)
Advisor
Ifeoma Nwogu
Advisor/Committee Member
Joe Geigel
Advisor/Committee Member
Christopher Homan
Recommended Citation
Sharma, Srijan Kumar, "Modeling Dyadic Human Interaction using Sequential Neural Network" (2021). Thesis. Rochester Institute of Technology. Accessed from
https://repository.rit.edu/theses/10813
Campus
RIT – Main Campus
Plan Codes
COMPSCI-MS
