TY - GEN
T1 - Measuring Speaking Time from Privacy-Preserving Videos
AU - Maeda, Shun
AU - Gu, Chunzhi
AU - Zhang, Chao
N1 - Publisher Copyright:
© 2022 SPIE.
PY - 2022
Y1 - 2022
N2 - The ongoing pandemic caused by the COVID-19 virus is challenging many aspects of daily life such as restricting the conversation time. A vision-based face analyzing system is considerable for measuring and managing the person-wise speaking time, however, pointing a camera to people directly would be offensive and intrusive. In addition, privacy contents such as the identifiable face of the speakers should not be recorded during measuring. In this paper, we adopt a deep multimodal clustering method, called DMC, to perform unsupervised audiovisual learning for matching preprocessed audio with corresponding locations at videos. We set the camera above the speakers, and by feeding a pair of captured audio and visual data to a pre-trained DMC, a series of heatmaps that identify the location of the speaking people can be generated. Eventually, the speaking time measurement of each speaker can be achieved by accumulating the lasting speaking time of the corresponding heatmap.
AB - The ongoing pandemic caused by the COVID-19 virus is challenging many aspects of daily life such as restricting the conversation time. A vision-based face analyzing system is considerable for measuring and managing the person-wise speaking time, however, pointing a camera to people directly would be offensive and intrusive. In addition, privacy contents such as the identifiable face of the speakers should not be recorded during measuring. In this paper, we adopt a deep multimodal clustering method, called DMC, to perform unsupervised audiovisual learning for matching preprocessed audio with corresponding locations at videos. We set the camera above the speakers, and by feeding a pair of captured audio and visual data to a pre-trained DMC, a series of heatmaps that identify the location of the speaking people can be generated. Eventually, the speaking time measurement of each speaker can be achieved by accumulating the lasting speaking time of the corresponding heatmap.
KW - Deep Multi-modal Clustering
KW - Speaking time measurement
KW - Video-based sound source localization
UR - http://www.scopus.com/inward/record.url?scp=85131822698&partnerID=8YFLogxK
U2 - 10.1117/12.2625956
DO - 10.1117/12.2625956
M3 - 会議への寄与
AN - SCOPUS:85131822698
T3 - Proceedings of SPIE - The International Society for Optical Engineering
BT - International Workshop on Advanced Imaging Technology, IWAIT 2022
A2 - Nakajima, Masayuki
A2 - Muramatsu, Shogo
A2 - Kim, Jae-Gon
A2 - Guo, Jing-Ming
A2 - Kemao, Qian
PB - SPIE
T2 - 2022 International Workshop on Advanced Imaging Technology, IWAIT 2022
Y2 - 4 January 2022 through 6 January 2022
ER -