Object-based attention
Objects are the units through which we interact with the world and perform tasks. The role of the attention mechanism in this process is to make the objects in the sequence the figure, in order to facilitate interacting with the object and performing the task. Here we model a general-purpose object-based attention mechanism, leveraging recent developments in deep learning. We specifically focus on three subtasks underlying object-based attention: (a) solving the binding problem, grouping features of each object, and using object-centric representation learning; (b) capturing the interaction between a (largely) bottom-up mechanism for recognition and a top-down mechanism for attention planning using encoder–decoder models; and (c) learning to sequentially sample objects using end-to-end training.
- Adeli, H., Ahn, S., & Zelinsky, G. J. (2023). A brain-inspired object-based attention network for multiobject recognition and visual reasoning. Journal of Vision, 23(5), 16-16. (paper)
- Adeli, H., Ahn, S., Kriegeskorte, N., & Zelinsky, G. J. (August 2023). Self-supervised transformers predict dynamics of object-based attention in humans. In Proceedings of the Cognitive Computational Neuroscience (CCN 2023) conference (pp. 1-3). (paper)
- Ahn, S., Adeli, H., & Zelinsky, G. J. (August 2023). Using generated object reconstructions to study object-based attention. In Proceedings of the Cognitive Computational Neuroscience (CCN 2023) conference (pp. 1-3). (paper)
Modeling human eye-movements
Visual attention controls most of the human behavior, being able to predict movements of attention would allow applications to anticipate the behaviors made in response to image content. We take a synergistic computational and behavioral approach for modeling the movements of human attention and predict the movements of human attention in various visual tasks from categorical visual search to free-viewing to reading.
- Mondal, S., Yang, Z., Ahn, S., Samaras, D., Zelinsky, G., & Hoai, M. (2023). Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1441-1450). (paper)
- Yang, Z., Huang, L., Chen, Y., Wei, Z., Ahn, S., Zelinsky, G., … & Hoai, M. (2020). Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 193-202). (paper)
- Adeli, H., Vitu, F., & Zelinsky, G. J. (2017). A model of the superior colliculus predicts fixation locations during scene viewing and visual search. Journal of Neuroscience, 37(6), 1453-1467. (Paper)