Aparajita Saraf

IMU2CLIP: Language-grounded Motion Sensor Translation with Multimodal Contrastive Learning

We present IMU2CLIP, a novel pre-training approach to align Inertial Measurement Unit (IMU) motion sensor recordings with text and video, by projecting them into the joint representation space of Contrastive Language-Image Pre-training (CLIP). The proposed approach allows IMU2CLIP to translate human…