ROMJIST Volume 25, No. 1, 2022, pp. 80-91
Adrian AIORDĂCHIOAE, Radu-Daniel VATAVU LifeTags++: A Multi-User, Multi-Device, and Multi-Perspective System for Recording and Abstracting Visual Life with Tag Clouds
ABSTRACT: We present LifeTags++, a multi-user multi-perspective lifelogging and life abstraction system designed to automatically record, extract, process, and store visual concepts from a wide range of video camera devices and systems, including personal mobile and wearable devices, video cameras worn on the body, and public IP video cameras from the environment. Other features of LifeTags++ include integration with mobile crowdsensing to accept queries and deliver notifications about specific concepts of interest entered by users. We present technical details regarding the design and engineering of LifeTags++ as a distributed, cloud-based software architecture accommodating mobile and wearable devices and public ambient video cameras. Also, we report the results of an evaluation study with a dataset of 18,780 visual concepts extracted from a number of nearly 1,000 snapshots captured from three video cameras simultaneously recording the life of a user from three different perspectives: egocentric eye-level provided by a pair of video camera glasses, egocentric back view with a wearable clip-on camera, and exocentric ambient from a public IP camera. Our results show the benefits of employing multi-perspective life abstraction for creating rich lifelogs of visual concepts. Based on our results, we present opportunities for future lifelogging systems that employ mobile devices, wearable cameras, and personal robots in the form of drones in conjunction with ambient sensing and computing infrastructure.KEYWORDS: Mobile, distributed systems, cloud computing; Lifelogging; Smartglasses; Camera glasses; Image processing; Smart wearables; Assistive visionRead full text (pdf)