PyTorchKR Papers

최신 논문

총 420편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2501.14520

Scene Understanding Enabled Semantic Communication with Open Channel Coding

As communication systems transition from symbol transmission to conveying meaningful information, sixth-generation (6G) networks emphasize semantic communication. This approach prioritizes high-level semantic information, improving robustness and reducing redundancy across modalities like text, speech, and images. However, traditional semantic communication faces limitations, including static coding strategies, poor generalization, and reliance on task-specific knowledge bases that hinder adaptability. To overcome these challenges, we propose a novel system combining scene understanding, Large Language Models (LLMs), and open channel coding, named \textbf{OpenSC}. Traditional systems rely on fixed domain-specific knowledge bases, limiting their ability to generalize. Our open channel coding approach leverages shared, publicly available knowledge, enabling flexible, adaptive encoding. This dynamic system reduces reliance on static task-specific data, enhancing adaptability across diverse tasks and environments. Additionally, we use scene graphs for structured semantic encoding, capturing object relationships and context to improve tasks like Visual Question Answering (VQA). Our approach selectively encodes key semantic elements, minimizing redundancy and improving transmission efficiency. Experimental results show significant improvements in both semantic understanding and efficiency, advancing the potential of adaptive, generalizable semantic communication in 6G networks.

arXiv:2501.10654

Efficient Transmission of Radiomaps via Physics-Enhanced Semantic Communications

Enriching information of spectrum coverage, radiomap plays an important role in many wireless communication applications, such as resource allocation and network optimization. To enable real-time, distributed spectrum management, particularly in the scenarios with unstable and dynamic environments, the efficient transmission of spectrum coverage information for radiomaps from edge devices to the central server emerges as a critical problem. In this work, we propose an innovative physics-enhanced semantic communication framework tailored for efficient radiomap transmission based on generative learning models. Specifically, instead of bit-wise message passing, we only transmit the key "semantics" in radiomaps characterized by the radio propagation behavior and surrounding environments, where semantic compression schemes are utilized to reduce the communication overhead. Incorporating the novel concepts of Radio Depth Maps, the radiomaps are reconstructed from the delivered semantic information backboned on the conditional generative adversarial networks. Our framework is further extended to facilitate its implementation in the scenarios of multi-user edge computing, by integrating with federated learning for collaborative model training while preserving the data privacy. Experimental results show that our approach achieves high accuracy in radio coverage information recovery at ultra-high bandwidth efficiency, which has great potentials in many wireless-generated data transmission applications.

arXiv:2501.12988

Large Language Model-Based Semantic Communication System for Image Transmission

The remarkable success of Large Language Models (LLMs) in understanding and generating various data types, such as images and text, has demonstrated their ability to process and extract semantic information across diverse domains. This transformative capability lays the foundation for semantic communications, enabling highly efficient and intelligent communication systems. In this work, we present a novel OFDM-based semantic communication framework for image transmission. We propose an innovative semantic encoder design that leverages the ability of LLMs to extract the meaning of transmitted data rather than focusing on its raw representation. On the receiver side, we design an LLM-based semantic decoder capable of comprehending context and generating the most appropriate representation to fit the given context. We evaluate our proposed system under different scenarios, including Urban Macro-cell environments with varying speed ranges. The evaluation metrics demonstrate that our proposed system reduces the data size 4250 times, while achieving a higher data rate compared to conventional communication methods. This approach offers a robust and scalable solution to unlock the full potential of 6G connectivity.

arXiv:2501.12502

Sequence Spreading-Based Semantic Communication Under High RF Interference

In the evolving landscape of wireless communications, semantic communication (SemCom) has recently emerged as a 6G enabler that prioritizes the transmission of meaning and contextual relevance over conventional bit-centric metrics. However, the deployment of SemCom systems in industrial settings presents considerable challenges, such as high radio frequency interference (RFI), that can adversely affect system performance. To address this problem, in this work, we propose a novel approach based on integrating sequence spreading techniques with SemCom to enhance system robustness against such adverse conditions and enable scalable multi-user (MU) SemCom. In addition, we propose a novel signal refining network (SRN) to refine the received signal after despreading and equalization. The proposed network eliminates the need for computationally intensive end-to-end (E2E) training while improving performance metrics, achieving a 25% gain in BLEU score and a 12% increase in semantic similarity compared to E2E training using the same bandwidth.

arXiv:2501.09918공식 코드 · Papers with Code

GenSC-6G: A Prototype Testbed for Integrated Generative AI, Quantum, and Semantic Communication

We introduce a prototyping testbed, GenSC-6G, developed to generate a comprehensive dataset that supports the integration of generative artificial intelligence (AI), quantum computing, and semantic communication for emerging sixth-generation (6G) applications. The GenSC-6G dataset is designed with noise-augmented synthetic data optimized for semantic decoding, classification, and localization tasks, significantly enhancing flexibility for diverse AI-driven communication applications. This adaptable prototype supports seamless modifications across baseline models, communication modules, and goal-oriented decoders. Case studies demonstrate its application in lightweight classification, semantic upsampling, and edge-based language inference under noise conditions. The GenSC-6G dataset serves as a scalable and robust resource for developing goal-oriented communication systems tailored to the growing demands of 6G networks.

arXiv:2501.05859공식 코드 · Papers with Code

Large Model Empowered Streaming Speech Semantic Communications

In this paper, we introduce a large model-empowered streaming semantic communication system for speech transmission across various languages, named LSSC-ST. Specifically, we devise an edge-device collaborative semantic communication architecture by offloading the intricate semantic extraction and channel coding modules to edge servers, thereby reducing the computational burden on local devices. To support multilingual speech transmission, pre-trained large speech models are utilized to learn unified semantic features from speech in different languages, breaking the constraint of a single input language and enhancing the practicality of the LSSC-ST. Moreover, the input speech is sequentially streamed into the developed system as short speech segments, which enables low transmission latency without degrading the quality of the produced speech. A novel dynamic speech segmentation algorithm is proposed to further reduce the transmission latency by adaptively adjusting the duration of speech segments. According to simulation results, the LSSC-ST provides more accurate speech transmission and achieves a streaming manner with lower latency compared to the existing non-streaming semantic communication systems.

arXiv:2501.09396

Joint Transmission and Deblurring: A Semantic Communication Approach Using Events

Deep learning-based joint source-channel coding (JSCC) is emerging as a promising technology for effective image transmission. However, most existing approaches focus on transmitting clear images, overlooking real-world challenges such as motion blur caused by camera shaking or fast-moving objects. Motion blur often degrades image quality, making transmission and reconstruction more challenging. Event cameras, which asynchronously record pixel intensity changes with extremely low latency, have shown great potential for motion deblurring tasks. However, the efficient transmission of the abundant data generated by event cameras remains a significant challenge. In this work, we propose a novel JSCC framework for the joint transmission of blurry images and events, aimed at achieving high-quality reconstructions under limited channel bandwidth. This approach is designed as a deblurring task-oriented JSCC system. Since RGB cameras and event cameras capture the same scene through different modalities, their outputs contain both shared and domain-specific information. To avoid repeatedly transmitting the shared information, we extract and transmit their shared information and domain-specific information, respectively. At the receiver, the received signals are processed by a deblurring decoder to generate clear images. Additionally, we introduce a multi-stage training strategy to train the proposed model. Simulation results demonstrate that our method significantly outperforms existing JSCC-based image transmission schemes, addressing motion blur effectively.

arXiv:2501.04732

SNR-EQ-JSCC: Joint Source-Channel Coding with SNR-Based Embedding and Query

Coping with the impact of dynamic channels is a critical issue in joint source-channel coding (JSCC)-based semantic communication systems. In this paper, we propose a lightweight channel-adaptive semantic coding architecture called SNR-EQ-JSCC. It is built upon the generic Transformer model and achieves channel adaptation (CA) by Embedding the signal-to-noise ratio (SNR) into the attention blocks and dynamically adjusting attention scores through channel-adaptive Queries. Meanwhile, penalty terms are introduced in the loss function to stabilize the training process. Considering that instantaneous SNR feedback may be imperfect, we propose an alternative method that uses only the average SNR, which requires no retraining of SNR-EQ-JSCC. Simulation results conducted on image transmission demonstrate that the proposed SNR-EQJSCC outperforms the state-of-the-art SwinJSCC in peak signal-to-noise ratio (PSNR) and perception metrics while only requiring 0.05% of the storage overhead and 6.38% of the computational complexity for CA. Moreover, the channel-adaptive query method demonstrates significant improvements in perception metrics. When instantaneous SNR feedback is imperfect, SNR-EQ-JSCC using only the average SNR still surpasses baseline schemes.

arXiv:2501.04480

Research on environment perception and behavior prediction of intelligent UAV based on semantic communication

The convergence of drone delivery systems, virtual worlds, and blockchain has transformed logistics and supply chain management, providing a fast, and environmentally friendly alternative to traditional ground transportation methods;Provide users with a real-world experience, virtual service providers need to collect up-to-the-minute delivery information from edge devices. To address this challenge, 1) a reinforcement learning approach is introduced to enable drones with fast training capabilities and the ability to autonomously adapt to new virtual scenarios for effective resource allocation.2) A semantic communication framework for meta-universes is proposed, which utilizes the extraction of semantic information to reduce the communication cost and incentivize the transmission of information for meta-universe services.3) In order to ensure that user information security, a lightweight authentication and key agreement scheme is designed between the drone and the user by introducing blockchain technology. In our experiments, the drone adaptation performance is improved by about 35\%, and the local offloading rate can reach 90\% with the increase of the number of base stations. The semantic communication system proposed in this paper is compared with the Cross Entropy baseline model. Introducing blockchain technology the throughput of the transaction is maintained at a stable value with different number of drones.

arXiv:2501.01616

Digital-Analog Transmission based Emergency Semantic Communications

Emergency Wireless Communication (EWC) networks adopt the User Datagram Protocol (UDP) to transmit scene images in real time for quickly assessing the extent of the damage. However, existing UDP-based EWC exhibits suboptimal performance under poor channel conditions since UDP lacks an Automatic Repeat reQuest (ARQ) mechanism. In addition, future EWC systems must not only enhance human decisionmaking during emergency response operations but also support Artificial Intelligence (AI)-driven approaches to improve rescue efficiency. The Deep Learning-based Semantic Communication (DL-based SemCom) emerges as a robust, efficient, and taskoriented transmission scheme, suitable for deployment in UDP based EWC. Due to the constraints in hardware capabilities and transmission resources, the EWC transmitter is unable to integrate sufficiently powerful NN model, thereby failing to achieve ideal performance under EWC scene. For EWC scene, we propose a performance-constrained semantic coding model, which considers the effects of the semantic noise and the channel noise. Then, we derive Cramer-Rao lower bound of the proposed semantic coding model, as guidance for the design of semantic codec to enhance its adaptability to semantic noise as well as channel noise. To further improve the system performance, we propose Digital-Analog transmission based Emergency Semantic Communication (DAESemCom) framework, which integrates the analog DL-based semantic coding and the digital Distributed Source Coding (DSC) schemes to leverage their respective advantages. The simulation results show that the proposed DA-ESemCom framework outperforms the classical Separated Source-Channel Coding (SSCC) and other DL-based Joint Source-Channel Coding (DL-based JSCC) schemes in terms of fidelity and detection performances.

arXiv:2501.02273

Blind Training for Channel-Adaptive Digital Semantic Communications

Semantic encoders and decoders for digital semantic communication (SC) often struggle to adapt to variations in unpredictable channel environments and diverse system designs. To address these challenges, this paper proposes a novel framework for training semantic encoders and decoders to enable channel-adaptive digital SC. The core idea is to use binary symmetric channel (BSC) as a universal representation of generic digital communications, eliminating the need to specify channel environments or system designs. Based on this idea, our framework employs parallel BSCs to equivalently model the relationship between the encoder's output and the decoder's input. The bit-flip probabilities of these BSCs are treated as trainable parameters during end-to-end training, with varying levels of regularization applied to address diverse requirements in practical systems. The advantage of our framework is justified by developing a training-aware communication strategy for the inference stage. This strategy makes communication bit errors align with the pre-trained bit-flip probabilities by adaptively selecting power and modulation levels based on practical requirements and channel conditions. Simulation results demonstrate that the proposed framework outperforms existing training approaches in terms of both task performance and power consumption.

arXiv:2501.02006

Multi-Task Semantic Communication With Graph Attention-Based Feature Correlation Extraction

Multi-task semantic communication can serve multiple learning tasks using a shared encoder model. Existing models have overlooked the intricate relationships between features extracted during an encoding process of tasks. This paper presents a new graph attention inter-block (GAI) module to the encoder/transmitter of a multi-task semantic communication system, which enriches the features for multiple tasks by embedding the intermediate outputs of encoding in the features, compared to the existing techniques. The key idea is that we interpret the outputs of the intermediate feature extraction blocks of the encoder as the nodes of a graph to capture the correlations of the intermediate features. Another important aspect is that we refine the node representation using a graph attention mechanism to extract the correlations and a multi-layer perceptron network to associate the node representations with different tasks. Consequently, the intermediate features are weighted and embedded into the features transmitted for executing multiple tasks at the receiver. Experiments demonstrate that the proposed model surpasses the most competitive and publicly available models by 11.4% on the CityScapes 2Task dataset and outperforms the established state-of-the-art by 3.97% on the NYU V2 3Task dataset, respectively, when the bandwidth ratio of the communication channel (i.e., compression level for transmission over the channel) is as constrained as 1 12 .

arXiv:2501.01172

ROME: Robust Model Ensembling for Semantic Communication Against Semantic Jamming Attacks

Recently, semantic communication (SC) has garnered increasing attention for its efficiency, yet it remains vulnerable to semantic jamming attacks. These attacks entail introducing crafted perturbation signals to legitimate signals over the wireless channel, thereby misleading the receivers' semantic interpretation. This paper investigates the above issue from a practical perspective. Contrasting with previous studies focusing on power-fixed attacks, we extensively consider a more challenging scenario of power-variable attacks by devising an innovative attack model named Adjustable Perturbation Generator (APG), which is capable of generating semantic jamming signals of various power levels. To combat semantic jamming attacks, we propose a novel framework called Robust Model Ensembling (ROME) for secure semantic communication. Specifically, ROME can detect the presence of semantic jamming attacks and their power levels. When high-power jamming attacks are detected, ROME adapts to raise its robustness at the cost of generalization ability, and thus effectively accommodating the attacks. Furthermore, we theoretically analyze the robustness of the system, demonstrating its superiority in combating semantic jamming attacks via adaptive robustness. Simulation results show that the proposed ROME approach exhibits significant adaptability and delivers graceful robustness and generalization ability under power-variable semantic jamming attacks.

arXiv:2501.01048

Power-Efficient Optimization for Coexisting Semantic and Bit-Based Users in NOMA Networks

Semantic communications, which focus on transmitting the semantic meaning of data, have been proposed as a novel paradigm for achieving efficient and relevant communication. Meanwhile, non-orthogonal multiple access (NOMA) enhances spectral efficiency by allowing multiple users to share the same spectrum. However, semantic communications are unlikely to fully replace conventional bit-level communications in the near future, as the latter remain dominant. Therefore, integrating semantic users into a NOMA network alongside conventional bit-based users becomes a meaningful approach to improve both transmission and spectrum efficiency. Nonetheless, due to the lack of a mathematical model that accurately characterizes the relationship between the performance of semantic transceivers and wireless resource allocation, enhancing performance through resource optimization remains a challenge. Moreover, successive interference cancellation (SIC), a key technique in NOMA, introduces additional complexity in system design and implementation. To address these challenges, this paper first improves the deep semantic communication (DeepSC) transceiver to make it adaptive to varying wireless transmission conditions. Subsequently, a data-driven regression approach is employed to develop a mathematical model that captures the impact of wireless resources on semantic transceiver performance. In parallel, a multi-cluster hybrid NOMA (H-NOMA) framework is proposed, where each cluster consists of one semantic user and one bit-based user, to mitigate the complexity introduced by SIC. A total transmit power minimization problem is then formulated by jointly optimizing the beamforming design, bandwidth allocation, and semantic symbol factor.

arXiv:2412.20937

Generative AI Empowered Semantic Feature Multiple Access (SFMA) Over Wireless Networks

This paper investigates a novel generative artificial intelligence (GAI) empowered multi-user semantic communication system called semantic feature multiple access (SFMA) for video transmission, which comprises a base station (BS) and paired users. The BS generates and combines semantic information of several frames simultaneously requested by paired users into a single signal. Users recover their frames from this combined signal and input the recovered frames into a GAI-based video frame interpolation model to generate the intermediate frame. To optimize transmission rates and temporal gaps between simultaneously transmitted frames, we formulate an optimization problem to maximize the system sum rate while minimizing temporal gaps. Since the standard signal-to-interference-plus-noise ratio (SINR) equation does not accurately capture the performance of our semantic communication system, we introduce a weight parameter into the SINR equation to better represent the system's performance. Due to its dependence on transmit power, we propose a three-step solution. First, we develop a user pairing algorithm that pairs two users with the highest preference value, a weighted combination of semantic transmission rate and temporal gap. Second, we optimize inter-group power allocation by formulating an optimization problem that allocates proper transmit power across all user groups to maximize system sum rates while satisfying each user's minimum rate requirement. Third, we address intra-group power allocation to enhance each user's performance. Simulation results demonstrate that our method improves transmission rates by up to 24.8%, 45.8%, and 66.1% compared to fixed-power non-orthogonal multiple access (F-NOMA), orthogonal joint source-channel coding (O-JSCC), and orthogonal frequency division multiple access (OFDMA), respectively.

arXiv:2412.19979

Explainable Semantic Federated Learning Enabled Industrial Edge Network for Fire Surveillance

In fire surveillance, Industrial Internet of Things (IIoT) devices require transmitting large monitoring data frequently, which leads to huge consumption of spectrum resources. Hence, we propose an Industrial Edge Semantic Network (IESN) to allow IIoT devices to send warnings through Semantic communication (SC). Thus, we should consider (1) Data privacy and security. (2) SC model adaptation for heterogeneous devices. (3) Explainability of semantics. Therefore, first, we present an eXplainable Semantic Federated Learning (XSFL) to train the SC model, thus ensuring data privacy and security. Then, we present an Adaptive Client Training (ACT) strategy to provide a specific SC model for each device according to its Fisher information matrix, thus overcoming the heterogeneity. Next, an Explainable SC (ESC) mechanism is designed, which introduces a leakyReLU-based activation mapping to explain the relationship between the extracted semantics and monitoring data. Finally, simulation results demonstrate the effectiveness of XSFL.

arXiv:2412.18876

Towards Compatible Semantic Communication: A Perspective on Digital Coding and Modulation

Semantic communication (SC) is emerging as a pivotal innovation within the 6G framework, aimed at enabling more intelligent transmission. This development has led to numerous studies focused on designing advanced systems through powerful deep learning techniques. Nevertheless, many of these approaches envision an analog transmission manner by formulating the transmitted signals as continuous-valued semantic representation vectors, limiting their compatibility with existing digital systems. To enhance compatibility, it is essential to explore digitized SC systems. This article systematically identifies two promising paradigms for designing digital SC: probabilistic and deterministic approaches, according to the modulation strategies. For both, we first provide a comprehensive analysis of the methodologies. Then, we put forward the principles of designing digital SC systems with a specific focus on informativeness and robustness of semantic representations to enhance performance, along with constellation design. Additionally, we present a case study to demonstrate the effectiveness of these methods. Moreover, this article also explores the intrinsic advantages and opportunities provided by digital SC systems, and then outlines several potential research directions for future investigation.

arXiv:2412.14538

Overview of AI and Communication for 6G Network: Fundamentals, Challenges, and Future Research Opportunities

With the growing demand for seamless connectivity and intelligent communication, the integration of artificial intelligence (AI) and sixth-generation (6G) communication networks has emerged as a transformative paradigm. By embedding AI capabilities across various network layers, this integration enables optimized resource allocation, improved efficiency, and enhanced system robust performance, particularly in intricate and dynamic environments. This paper presents a comprehensive overview of AI and communication for 6G networks, with a focus on emphasizing their foundational principles, inherent challenges, and future research opportunities. We first review the integration of AI and communications in the context of 6G, exploring the driving factors behind incorporating AI into wireless communications, as well as the vision for the convergence of AI and 6G. The discourse then transitions to a detailed exposition of the envisioned integration of AI within 6G networks, delineated across three progressive developmental stages. The first stage, AI for Network, focuses on employing AI to augment network performance, optimize efficiency, and enhance user service experiences. The second stage, Network for AI, highlights the role of the network in facilitating and buttressing AI operations and presents key enabling technologies, such as digital twins for AI and semantic communication. In the final stage, AI as a Service, it is anticipated that future 6G networks will innately provide AI functions as services, supporting application scenarios like immersive communication and intelligent industrial robots. In addition, we conduct an in-depth analysis of the critical challenges faced by the integration of AI and communications in 6G. Finally, we outline promising future research opportunities that are expected to drive the development and refinement of AI and 6G communications.

arXiv:2412.11857

Scalable Data Transmission Framework for Earth Observation Satellites with Channel Adaptation

The immense volume of data generated by Earth observation (EO) satellites presents significant challenges in transmitting it to Earth over rate-limited satellite-to-ground communication links. This paper presents an efficient downlink framework for multi-spectral satellite images, leveraging adaptive transmission techniques based on pixel importance and link capacity. By integrating semantic communication principles, the framework prioritizes critical information, such as changed multi-spectral pixels, to optimize data transmission. The process involves preprocessing, assessing pixel importance to encode only significant changes, and dynamically adjusting transmissions to match channel conditions. Experimental results on the real dataset and simulated link demonstrate that the proposed approach ensures high-quality data delivery while significantly reducing number of transmitted data, making it highly suitable for satellite-based EO applications.

arXiv:2412.07173

Semantic Communications for Digital Signals via Carrier Images

Most of current semantic communication (SemCom) frameworks focus on the image transmission, which, however, do not address the problem on how to deliver digital signals without any semantic features. This paper proposes a novel SemCom approach to transmit digital signals by using the image as the carrier signal. Specifically, the proposed approach encodes the digital signal as a binary stream and maps it to mask locations on an image. This allows binary data to be visually represented, enabling the use of existing model, pre-trained Masked Autoencoders (MAE), which are optimized for masked image reconstruction, as the SemCom encoder and decoder. Since MAE can both process and recover masked images, this approach allows for the joint transmission of digital signals and images without incurring significant communication overheads. In addition, considering the mask tokens transmission encoded by the MAE still faces extra costs, we design a sparse encoding module at the transmitter to encode the mask tokens into a sparse matrix, and it can be recovered at the receiver. Thus, this approach simply needs to transmit the latent representations of the unmasked patches and a sparse matrix, which further reduce the transmission overhead compared with the original MAE encoder. Simulation results show that the approach maintains reliable transmission even in a high mask ratio of images.