website templates

COGNITIVE VISION
AND PERCEPTION

Deep Semantics Integrating AI and Vision for
Reasoning about Space, Action and Motion

www.cognitive-vision.org

Cognitive Vision presents an emerging line of research bringing together a novel and unique combination of methodologies from Artificial Intelligence, Vision and Machine Learning, Cognitive Science and Psychology, Visual Perception, and Spatial Cognition and Computation.

WE DEVELOP

General methods for the processing and semantic interpretation of dynamic visuo-spatial imagery with a particular emphasis on the ability to abstract, learn, and reason with cognitively rooted structured characterisations of commonsense knowledge pertaining to space and motion.


Semantic interpretation of dynamic visual imagery calls for general and systematic methods integrating techniques in knowledge representation and computer vision. Our research emphasises deep semantics, denoting the existence of declarative models -e.g., pertaining space and motion- and corresponding formalisation and methods supporting (domain-independent) reasoning capabilities such as semantic question-answering, relational visuospatial learning, and (non-monotonic) visuospatial abduction.

Presently, we emphasise applications of developed methods and tools in two settings:
(1). explainable visual abduction for active visual sensemaking and control (emphasising human-centred, ethical considerations in autonomous driving); and (2). semantic interpretation of multimodal human behavioural stimuli (emphasising AI foundations for empirically-driven research behavioural research in psychology, social sciences, visual art)

SELECT REFERENCES


Suchan, J., Bhatt, M., Vardarajan, S. (2019). Out of Sight But Not Out of Mind: An Answer Set Programming Based Online Abduction Framework for Visual Sensemaking in Autonomous Driving. IJCAI 2019:  the 28th International Joint Conference on Artificial Intelligence (IJCAI) 2019, August 10 - 16, Macao.  (Distinguished Paper Nomination - Received honourable Mention)

Bhatt, M., Suchan, J.,  Vardarajan, S. (2019). Deep Semantics for Explainable Visuospatial Intelligence: Perspectives on Integrating Commonsense Spatial Abstractions and Low-Level Neural Features. NeSy 2019: In Proceedings of 14th International Workshop on Neural-Symbolic Learning and Reasoning (NeSy), at IJCAI 2019., August 12, 2019 (accepted for publication - to appear).

J. Suchan., M. Bhatt, Walega, P., Schultz, C. (2018). Visual Explanation by High-Level Abduction: On Answer-Set Programming Driven Reasoning about Moving Objects. In AAAI 2018: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, February 2-7, 2018, New Orleans, USA.

Mehul Bhatt, Kristian Kersting (2018): Semantic Interpretation of Multi-Modal Human-Behaviour Data - Making Sense of Events, Activities, Processes. KI / Artificial Intelligence, 31(4): 317-320 (2017)

Suchan, J., Bhatt, M., Vardarajan, S., Amirshahi, S. A., and Yu, S. (2018). Semantic Analysis of (Reflectional) Visual Symmetry: A Human-Centred Computational Model for Declarative Explainability. Advances in Cognitive Systems, Vol 6: 65:84, 2018.

Jakob Suchan, Mehul Bhatt (2017): Deep Semantic Abstractions of Everyday Human Activities - On Commonsense Representations of Human Interactions. ROBOT (1) 2017: 477-488

J. Suchan., M. Bhatt. (2017). Commonsense Scene Semantics for Cognitive Robotics: Towards Grounding Embodied Visuo-Locomotive Interactions. In ICCV 2017 Workshop: Vision in Practice on Autonomous Robots (ViPAR), International Conference on Computer Vision (ICCV), Venice, Italy.

Jakob Suchan. Declarative Reasoning about Space and Motion with Video. KI, 31(4):321–330, 2017

Suchan, J., Bhatt, M. (2016). Semantic Question-Answering with Video and Eye-Tracking Data: AI Foundations for Human Visual Perception Driven Cognitive Film Studies. IJCAI 2016: 25th International Joint Conference on Artificial Intelligence, New York City, USA.

Michael Spranger, Jakob Suchan, Mehul Bhatt (2016): Robust Natural Language Processing - Combining Reasoning, Cognitive Semantics, and Construction Grammar for Spatial Language. IJCAI 2016: 2908-2914

Suchan, J., Bhatt, M., Santos, P. (2014). Perceptual Narratives of Space and Motion for Semantic Interpretation of Visual Data, in: Proceedings of International Workshop on Computer Vision + Ontology Applied Cross-Disciplinary Technologies (CONTACT). ECCV 2014 -- European Conference on Computer Vision, Zurich, Switzerland.

Suchan, J., Spranger, M., Bhatt, M., Eppe, M. (2014). Grounding Dynamic Spatial Relations for Embodied (Robot) Interaction, in: 13th Pacific Rim International Conference on Artificial Intelligence (PRICAI-2014), Queensland, Australia, 2014.

Bhatt, M., Suchan, J., Schultz, C. (2013). Cognitive Interpretation of Everyday Activities - Toward Perceptual Narrative Based Visuo-Spatial Scene Interpretation. Computational Models of Narrative (CMN) 2013., a satellite workshop of CogSci 2013: The 35th meeting of the Cognitive Science Society., Editors: M. Finlayson., B. Fisseni., Benedikt Löwe., J. C. Meister. OASIcs proceedings volume. OpenAccess Series in Informatics (OASIcs). Dagstuhl, Germany

Bhatt, M., Schultz, C., Freksa, C. (2013). The `Space' in Spatial Assistance Systems: Conception, Formalisation, and Computation. in Thora Tenbrink, Jan Wiener, Christophe Claramunt (editors). Representing space in cognition: Interrelations of behavior, language, and formal models. Series: Explorations in Language and Space. Oxford University Press, 2012. 978-0-19-967991-1.

Bhatt, M. (2012). Reasoning about Space, Actions and Change: A Paradigm for Applications of Spatial Reasoning. in: Hazarika, S. (editor). Qualitative Spatio-Temporal Representation and Reasoning: Trends and Future Directions. IGI Global (PA, USA). DOI: 10.4018/978-1-61692-868-1. ISBN13: 978161692868.

Bhatt, M., Guesgen, H., Woelfl, S., Hazarika, S. (2011). Qualitative Spatial and Temporal Reasoning: Emerging Applications, Trends and Future Directions. Journal of Spatial Cognition and Computation. Issue: Emerging Applications of Spatial and Temporal Reasoning. 11(1). ISSN: 1387-5868 print/1542-7633 online, Taylor & Francis Group 2011.

Bhatt, M., Lee, J. H., Schultz, C. (2009). CLP(QS): A Declarative Spatial Reasoning Framework. Proceedings of the 10th International Conference on Spatial Information Theory (COSIT 11). Belfast, Maine.

Bhatt, M., and Loke, S.  (2008). Modelling Dynamic Spatial Systems in the Situation Calculus. Spatial Cognition and Computation, 8(1-2):86–130, 2008.

© Copyright 2019 •  CoDesign Lab EU  
Cognitive Vision - All Rights Reserved