Home >

news Help

Publication Information


Title
Japanese:Fixed-Weight Difference Target Propagation 
English:Fixed-Weight Difference Target Propagation 
Author
Japanese: 澁谷 辰吉, 井上 中順, 川上 玲, 佐藤 育郎.  
English: Tatsukichi Shibuya, Nakamasa Inoue, Rei Kawakami, Ikuro Sato.  
Language English 
Journal/Book name
Japanese: 
English:Proceedings of the AAAI Conference on Artificial Intelligence 
Volume, Number, Page vol. 37    no. 8    pp. 9811-9819
Published date Feb. 2023 
Publisher
Japanese: 
English: 
Conference name
Japanese: 
English:the 37th AAAI Conference on Artificial Intelligence 
Conference site
Japanese: 
English:Washington, DC 
Abstract Target Propagation (TP) is a biologically more plausible algorithm than the error backpropagation (BP) to train deep networks, and improving the practicality of TP is an open issue. TP methods require the feedforward and feedback networks to form layer-wise autoencoders for propagating the target values generated at the output layer. However, this causes certain drawbacks; e.g., careful hyperparameter tuning is required to synchronize the feedforward and feedback training, and frequent updates of the feedback path are usually needed more than that of the feedforward path. Learning of the feedforward and feedback networks is sufficient to make TP methods capable of training, but is having these layer-wise autoencoders a necessary condition for TP to work? We answer this question by presenting Fixed-Weight Difference Target Propagation (FW-DTP) that keeps the feedback weights constant during training. We confirmed that this simple method, which naturally resolves the abovementioned problems of TP, can still deliver informative target values to hidden layers for a given task; indeed, FW-DTP consistently achieves higher test performance than a baseline, the Difference Target Propagation (DTP), on four classification datasets. We also present a novel propagation architecture that explains the exact form of the feedback function of DTP to analyze FW-DTP.

©2007 Tokyo Institute of Technology All rights reserved.