Home >

news Help

Publication Information


Title
Japanese: 
English:Deep Modular Multimodal Fusion on Multiple Sensors for Volcano Activity Recognition 
Author
Japanese: LE HIEP VINH, 村田剛志, Masato Iguchi.  
English: Hiep Le, Tsuyoshi MURATA, Masato Iguchi.  
Language English 
Journal/Book name
Japanese: 
English:Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2018. Lecture Notes in Computer Science 
Volume, Number, Page Vol. 11053        pp. 602–617
Published date Jan. 18, 2019 
Publisher
Japanese: 
English: 
Conference name
Japanese: 
English:European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2018) 
Conference site
Japanese: 
English:Dublin 
Official URL https://link.springer.com/chapter/10.1007/978-3-030-10997-4_37
 
DOI https://doi.org/10.1007/978-3-030-10997-4_37
Abstract Nowadays, with the development of sensor techniques and the growth in a number of volcanic monitoring systems, more and more data about volcanic sensor signals are gathered. This results in a need for mining these data to study the mechanism of the volcanic eruption. This paper focuses on Volcano Activity Recognition (VAR) where the inputs are multiple sensor data obtained from the volcanic monitoring system in the form of time series. And the output of this research is the volcano status which is either explosive or not explosive. It is hard even for experts to extract handcrafted features from these time series. To solve this problem, we propose a deep neural network architecture called VolNet which adapts Convolutional Neural Network for each time series to extract non-handcrafted feature representation which is considered powerful to discriminate between classes. By taking advantages of VolNet as a building block, we propose a simple but effective fusion model called Deep Modular Multimodal Fusion (DMMF) which adapts data grouping as the guidance to design the architecture of fusion model. Different from conventional multimodal fusion where the features are concatenated all at once at the fusion step, DMMF fuses relevant modalities in different modules separately in a hierarchical fashion. We conducted extensive experiments to demonstrate the effectiveness of VolNet and DMMF on the volcanic sensor datasets obtained from Sakurajima volcano, which are the biggest volcanic sensor datasets in Japan. The experiments showed that DMMF outperformed the current state-of-the-art fusion model with the increase of F-score up to 1.9% on average.

©2007 Tokyo Institute of Technology All rights reserved.