12/5/2024 Ikhwan Jeon: Training convolutional neural networks with blurry images enables the learning of more human-aligned visual representations
CCN Brown Bag
Ikhwan Jeon
Graduate Student, Tong Lab
Date: Thursday, December 5, 2024
Time: 12:10PM-1:00PM
Location: 316 Wilson Hall
Training convolutional neural networks with blurry images enables the learning of more human-aligned visual representations
Although convolutional neural networks (CNNs) can achieve human-level object recognition performance on natural images, research has revealed systematic deviations between CNNs and human visual processing, including susceptibility to visual noise (Jang et al., 2021) and insensitivity to shape information (Geirhos et al., 2019). However, recent work has shown that these deviations can be reduced by providing CNNs with auxiliary training on blurry images (Jang and Tong, 2024). In this work, we further demonstrate how blur training can help better align the internal representations of CNNs with human perception by evaluating the quality of metameric stimuli generated from CNNs (Feather et al., 2023). The metamers of CNNs are defined as image pairs that produce nearly identical responses; such metamers can be generated by modifying an initially random noise image until it produces CNN responses that closely mimic the responses to a reference object image. Here, we evaluated whether the metamers of CNNs are also metameric to human perception. If two different systems tend to see things in the same way, then metamers can provide a measure of their degree of alignment. To investigate the potential benefits of training CNNs on blur regarding representational alignment, we generated metamers from both standard and blur-trained CNNs, evaluating both RGB and grayscale-trained models. We then had human observers and cross-validated CNN models classify the metamers generated from a given CNN. Across all conditions, the metamers of blur-trained models were more recognizable than those generated from standard clear-trained CNNs. Our results reveal a general benefit of blur training for creating recognizable CNN metamers, indicating better alignment between the internal representations of CNNs and the human visual system.
Questions? Contact Isabel Gauthier.