A Knowledge Modeling Framework for Household Action Recognition and Task Representation: The Concept Hierarchy

The Concept Hierarchy is a human-understandable and explainable framework for knowledge representation. Apart from geometric data, the framework attaches semantic and state information to environment entities by classifying and describing them using concepts and attributes. The framework allows the definition of complex data types, such as templated sequences and associative containers; and also describes how attribute changes propagate to other related ones. Autonomous systems use this semantic information to generalize to unseen entities and scenes and to plan and reason in indoor environments while solving unforeseen circumstances due to unexpected entity states. The framework also defines tasks, actions, skills, affordances, and contexts. This theoretical modeling is validated with typical applications of autonomous systems: environment discretization, action and skill recognition in human manipulation, reorganization of identified repetitive knowledge, and task verification.