COCO-Search 18
Download: https://sites.google.com/view/cocosearch/
Description: COCO-Search18 is a laboratory-quality dataset of goal-directed behavior large enough to train deep-network models. It consists of the eye gaze behavior from 10 people searching for each of 18 target-object categories in 6202 natural-scene images, yielding ~300,000 search fixations. COCO-Search18 is now part of the MIT/Tuebingen Saliency Benchmark, previously the MIT Saliency Benchmark but renamed to reflect the new group that will be managing the competition. As part of this re-organization, the benchmark will broaden its scope to go beyond purely spatial fixation prediction and even beyond the free-viewing task. COCO-Search18 is partly responsible for this broadening, and represents a significant expansion of the benchmark into goal-directed search behavior. The training, validation, and test images in COCO-Search18 are already freely available as part of COCO. Researchers are also free to see and use COCO-Search18’s training and validation search fixations, but the fixations on the test images are withheld. As part of a separate benchmark track, it will be possible to upload predictions and have them evaluated on the test dataset. In this initial stage of release, only fixations made on target-present search trials are available at this time (stay tuned for release of the target-absent fixations). We hope you enjoy using COCO-Search18!
Paper:
Chen, Y., Yang, Z., Ahn, S., Samaras, D., Hoai, M., & Zelinsky, G. (2021). COCO-Search18 Fixation Dataset for Predicting Goal-directed Attention Control. Scientific Reports, 11 (1), 1-11, 2021. https://www.nature.com/articles/s41598-021-87715-9
Yang, Z., Huang, L., Chen, Y., Wei, Z., Ahn, S., Zelinsky, G., Samaras, D., & Hoai, M. (2020). Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 193-202).
SBU-68
Download: SBU-68
Description: This dataset consisted of crossly-cropped images of 68 object categories that were distributed across three levels of a category hierarchy. There were 48 subordinate-level categories, which were grouped into 16 basic-level categories, which were grouped into 4 superordinate-level categories. A categorical search task was used, and the participants were 26 Stony Brook University undergraduates.
Paper : (2019) Modelling attention control using a convolutional neural network designed after the ventral visual pathway, Visual Cognition, DOI:10.1080/
Microwave-Clock Search Dataset (MCS)
Download: MCS.zip
Description page: Microwave Clock Search
Paper : Zelinsky, G., Yang, Z., Huang, L., Chen, Y., Ahn, S., Wei, Z., … & Hoai, M. (2019). Benchmarking Gaze Prediction for Categorical Visual Search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops
SBU-CES (Stony Brook University – Categorical and Exemplar Search)
Download: SBU-CES.zip
Paper: Adeli, H., Vitu, F., & Zelinsky, G. J. (2017). A model of the superior colliculus predicts fixation locations during scene viewing and visual search. Journal of Neuroscience, 37(6), 1453-1467.
Stony Brook University Real-world Clutter Dataset (SBU-RwC90)
Download: SBU-RwC90.zip
Description page: Image Clutter and Proto-object Segmentation
Paper: Yu, C. P., Hua, W. Y., Samaras, D., & Zelinsky, G. (2013). Modeling clutter perception using parametric proto-object partitioning. In Advances in neural information processing systems (pp. 118-126).