AI RESEARCH

OmniEncoder: See, Hear, and Feel Continuous Motion Like Humans With One Encoder

arXiv CS.CV

ArXi:2605.01506v1 Announce Type: new Recent advances in omni-modal large language models have enabled remarkable progress in joint vision-audio understanding. However, prevailing architectures rely on modality-specific encoders with a \emph{video-coarse, audio-dense} design -- sampling visual frames at 1--2 fps while processing audio waveforms at 25 fps -- resulting in systems that perceive video \emph{frame by frame, modality by modality} rather than holistically as humans do.