36 accepted papers
Multimodal learning, while contributing to numerous success stories across various fields, faces the challenge of prohibitively expensive manual annotation. To address the scarcity of annotated data, a popular solution is unsupervised domain adaptation, which has been extensively studied in unimodal
Visual prostheses hold great promise for restoring vision in blind individuals. While researchers have successfully utilized M/EEG signals to evoke visual perceptions during the brain decoding stage of visual prostheses, the complementary process of converting images into M/EEG signals in the brain …
Advanced end-to-end autonomous driving systems predict other vehicles
Graph domain adaptation (GDA) transfers knowledge from a labeled source graph to an unlabeled target graph to alleviate label scarcity. In multi-view graphs, the challenge of mitigating domain shift is constrained by structural information across various views. Moreover, within each view, structures…
Multimodal learning has celebrated remarkable success across diverse areas, yet faces the challenge of prohibitively expensive data collection and annotation when adapting models to new environments. In this context, domain adaptation has gained growing popularity as a technique for knowledge transf…
We present lightweight flow matching multilingual text-to-speech (TTS) systems for Ojibwe, Mi’kmaq, and Maliseet, three Indigenous languages in North America. Our results show that training a multilingual TTS model on three typologically similar languages can improve the performance over monolingual…
Trustworthy multi-modal learning integrates multiple sources of data reliably. However, the current methods still focus on performance improvement by developing deep multi-modal networks. These approaches frequently encounter challenges due to the inherent non-convex nature of deep neural networks a…
Electroadhesion (EA), as an electrostatically driven, controllable adhesion technology, has unique attributes such as low noise, robust adaptability, and energy efficiency. However, its adhesion pressure is still low (0.1~10kPa) which may significantly limit its applications. This paper presents an
This paper introduces LingGym, a new benchmark that evaluates LLMs’ capacity for meta-linguistic reasoning using Interlinear Glossed Text (IGT) and grammatical descriptions extracted from 18 typologically diverse reference grammars. Unlike previous work that focuses on specific downstream tasks, we
Soft robots and smart materials have seen rapid advancements in recent years, with significant potential applications in medical devices. Liquid crystal elastomers (LCEs) exhibit unique attributes of large deformations and diverse actuation modes, facilitating controllable bending in soft medical ca
Soft pneumatic robots are gaining significant attention due to their compliance and adaptability in unstructured environments. While emerging dual-chamber soft pneumatic robots can achieve complex 3D deformations beyond conventional single-axis bending, real-time proprioception remains challenging d
Multi-view clustering can partition data samples into their categories by learning a consensus representation in an unsupervised way and has received more and more attention in recent years. However, there is an untrusted fusion problem. The reasons for this problem are as follows: 1) The current me…
We present ZIPA, a family of efficient speech models that advances the state-of-the-art performance of crosslinguistic phone recognition. We first curated IPA PACK++, a large-scale multilingual speech corpus with 17,000+ hours of normalized phone transcriptions and a novel evaluation set capturing u…
The multi-view hash method converts heterogeneous data from multiple views into binary hash codes, which is one of the critical technologies in multimedia retrieval. However, the current methods mainly explore the complementarity among multiple views while lacking confidence in learning and fusion.…
Soft pneumatic actuators can accomplish various customizable deformation/motion through the distribution of cavities and gradients in stiffness. However, traditional manufacturing methods, say molding, struggle to produce soft actuators with both complex cavities and desirable stiffness distribution…
Deep neural networks (DNNs) have achieved significant advancements in click-through rate (CTR) prediction by demonstrating strong generalization on training data. However, in real-world scenarios, the assumption of independent and identically distributed (i.i.d.) conditions, which is fundamental to…
Human Mesh Recovery (HMR) is the task of estimating a parameterized 3D human mesh from an image. There is a kind of methods first training a regression model for this problem, then further optimizing the pretrained regression model for any specific sample individually at test time. However, the pret…
In this project, we demonstrate that phoneme-based models for speech processing can achieve strong crosslinguistic generalizability to unseen languages. We curated the IPAPACK, a massively multilingual speech corpora with phonemic transcriptions, encompassing more than 115 languages from diverse lan…
Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence length. In contrast, recurrent neural networks (RNNs) exhibit linear scaling in memory and computational requirements but st…
Natural eyeball motions in humanoid robots can contribute to friendly communication, thus improving the human-robot interaction. In this paper, we develop antagonist-agonist artificial muscles for humanoid eyeball motions, by using dielectric elastomer actuators (DEAs). Inspired by human eyeballs, t…
Inducing semantic representations directly from speech signals is a highly challenging task but has many useful applications in speech mining and spoken language understanding. This study tackles the unsupervised learning of semantic representations for spoken utterances. Through converting speech s…
Dielectric elastomer actuators (DEAs) have been widely employed to drive various soft robots, due to their quiet fast muscle-like behavior. It is significant but challenging to model and control these soft actuators, due to their viscoelastic property, irregular geometry, complex structure, etc. In…
The task of phone-to-audio alignment has many applications in speech research. Here we introduce two Wav2Vec2-based models for both text-dependent and text-independent phone-to-audio alignment. The proposed Wav2Vec2-FS, a semi-supervised model, directly learns phone-to-audio alignment through contra…
As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multilingual settings. The BigScience workshop, a 1-year international and multidisciplinary initiative, was formed with the goal of researching and training large lan…
An individual’s variation in writing style is often a function of both social and personal attributes. While structured social variation has been extensively studied, e.g., gender based variation, far less is known about how to characterize individual styles due to their idiosyncratic nature. We int…
Cross-modal hashing has received increasing research attentions due to its less storage and efficient retrieval. However, most existing cross-modal hashing methods focus only on exploring multi-modal information, while underestimate the significance of local and Euclidean structure information on th…
New words are regularly introduced to communities, yet not all of these words persist in a community’s lexicon. Among the many factors contributing to lexical change, we focus on the understudied effect of social networks. We conduct a large-scale analysis of over 80k neologisms in 4420 online commu…
Earthworm-like soft robots have been widely studied for various applications, such as medical endoscopy and pipeline inspection. Many actuation modes have been chosen to drive the soft robots, including pneumatic actuators, dielectric elastomeric actuators, and shape memory actuators. Pneumatic actu…
Dielectric elastomer actuators (DEAs) have been widely employed as artificial muscles in soft robots. Due to material viscoelasticity and nonlinear electromechanical coupling, it is challenging to accurately model a viscoelastic DEA, especially when the actuator is of a complex or irregular configur
B-mode ultrasound tongue imaging is widely used in the speech production field. However, efficient interpretation is in a great need for the tongue image sequences. Inspired by the recent success of unsupervised deep learning approach, we explore unsupervised convolutional network architecture for t…
A challenge in speech production research is to predict future tongue movements based on a short period of past tongue movements. This study tackles speaker-dependent tongue motion prediction problem in unlabeled ultrasound videos with convolutional long short-term memory (ConvLSTM) networks. The mo…
Soft robots have recently evoked extensive attention due to their abilities to work effectively in unstructured environments. As an actuation technology of soft robots, dielectric elastomers exhibit many intriguing attributes such as large strain and high energy density. This work presents a novel d…
Soft-bodied robots, due to their intrinsic compliance, have shown great potential for operating within unstructured environment and interacting with unknown objects. This letter deals with automatic design and fabrication of soft robots. From a structure point of view, we synthesize a soft cable-dri
Frogs are capable of multiple locomotion modes including jumping and swimming, which enables them to adapt to various environmental conditions. This paper demonstrates a frog-inspired robot, which can mimic the swimming motion of a natural frog. The robot is developed based on dielectric elastomer a…
Soft actuators play an important role in producing motions in soft robots, and dielectric elastomers have shown great promise because of their considerable voltage-induced deformation. In particular, air-filled dielectric elastomer actuators have been well studied, where the air inside provides pres…
Although dielectric elastomers have been extensively studied recently, to date there has been little research into application of dielectric elastomer actuators to undersea robots. This letter focuses on development of a jellyfish robot using a dielectric elastomer actuator, which exhibits muscle-li