may include topics like:
Traditional plain ISAC
multimodal ISAC
(semantic) ISAC
multimodal semantic comunications
Integrated Sensing and Communication (ISAC) has been an evolving field with various stages of development, marked by different approaches in combining communication and sensing functions to enhance efficiency in applications such as autonomous driving, smart cities, and environmental monitoring. Below is a breakdown of the key stages of ISAC development leading to your current work on the JEPA-aided Multimodal Semantic ISAC framework.
The early stages of ISAC focused on single-modal systems, primarily relying on radio frequency (RF) signals for both sensing and communication. The goal was to reuse the same hardware and signal for dual purposes, which could enhance the spectral efficiency of the system.
Focus: Using RF signals to sense the environment while transmitting communication data.
Key challenges: Single-modal sensing led to limited accuracy in detecting and understanding complex environments due to the lack of multimodal data. The reliance on RF signals also restricted the system’s ability to interpret non-RF information, such as images or point clouds.
Reference:
To improve sensing accuracy, multimodal fusion techniques were introduced. These approaches combined data from multiple sensors—such as cameras, LiDAR, and RF sensors—to provide a more comprehensive view of the environment.
Focus: Directly transmitting and fusing raw data from different sensor modalities.
Key challenges: While this approach improved accuracy, it came with significant communication overhead due to the large amount of raw data transmitted. The computational load increased as all data had to be processed centrally, limiting scalability and efficiency in real-time applications.
Reference: