In this paper, we present JEPA-ISAC, a novel ISAC (Integrated Sensing and Communication) framework designed to enhance Distributed Multimodal Sensing Networks (DMSNs) by integrating JEPA (Joint Embedding Predictive Architecture). First, JEPA facilitates cross-modal semantic sharing by enabling the system to reconstruct the sensing data of certain distributed nodes from related nodes or modalities in a self-supervised learning manner. This approach reduces redundant transmissions, thereby optimizing communication efficiency and minimizing traffic. Additionally, our framework leverages freely available RF signals (CSI, RSSI) of a communication system as a complementary modality to enhance sensing performance of each distributed node without increasing hardware complexity. Besides, we also demonstrate a trade-off between the number of sensing modalities and JEPA model complexity, simplifying model requirements and reducing training time while maintaining high performance in tasks such as cross-modal data reconstruction. This work significantly advances the capabilities of DMSNs by improving communication efficiency and sensing accuracy.

We propose (??name), a JEPA-aided multimodal ISAC system to facilitate hyper-intelligent & efficient multimodal Distributed Sensing.

Integrated Sensing and Communication (ISAC) systems have become crucial for modern applications such as autonomous driving, smart cities, and environmental monitoring. However, traditional ISAC systems are limited by the transmission of raw data, single-modal sensing, and surface-level data integration, resulting in high communication costs, computational inefficiencies, and limited ability to handle complex prediction tasks. To overcome these limitations, we propose JEPA-ISAC, a novel framework that incorporates Joint Embedding Predictive Architecture (JEPA) to enable efficient semantic communication and advanced multimodal sensing.

(we may do it finally)

Additionally, our framework leverages freely available RF signals (CSI, RSSI) of a communication system as a complementary modality to enhance sensing performance of each distributed node without increasing hardware complexity.