Map as a By-product: Collective Landmark Mapping from IMU Data and User-provided Texts in Situated Tasks

This paper presents Collective Landmark Mapper, a novel map-as-a-by-product system for generating semantic landmark maps of indoor environments. Consider users engaged in situated tasks that require them to navigate these environments and regularly take notes on their smartphones. Collective Landmark Mapper exploits the smartphone's IMU data and the user's free text input during these tasks to identify a set of landmarks encountered by the user. The identified landmarks are then aggregated across multiple users to generate a unified map representing the positions and semantic information of all landmarks. In developing the proposed system, we focused specifically on retail applications and conducted a formative interview with stakeholders to confirm their practical needs that motivate the map-as-a-byproduct approach. Our user study demonstrates the feasibility of the proposed system and its superior mapping performance in two different setups: creating a product availability map from restocking checklist tasks at a retail store and constructing a room usage map from office inspection tasks, further demonstrating the potential applicability to non-retail applications.