PyTorchKR Papers

최신 논문

총 500편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2502.01031공식 코드 · Papers with Code

DiffIM: Differentiable Influence Minimization with Surrogate Modeling and Continuous Relaxation

In social networks, people influence each other through social links, which can be represented as propagation among nodes in graphs. Influence minimization (IMIN) is the problem of manipulating the structures of an input graph (e.g., removing edges) to reduce the propagation among nodes. IMIN can represent time-critical real-world applications, such as rumor blocking, but IMIN is theoretically difficult and computationally expensive. Moreover, the discrete nature of IMIN hinders the usage of powerful machine learning techniques, which requires differentiable computation. In this work, we propose DiffIM, a novel method for IMIN with two differentiable schemes for acceleration: (1) surrogate modeling for efficient influence estimation, which avoids time-consuming simulations (e.g., Monte Carlo), and (2) the continuous relaxation of decisions, which avoids the evaluation of individual discrete decisions (e.g., removing an edge). We further propose a third accelerating scheme, gradient-driven selection, that chooses edges instantly based on gradients without optimization (spec., gradient descent iterations) on each test instance. Through extensive experiments on real-world graphs, we show that each proposed scheme significantly improves speed with little (or even no) IMIN performance degradation. Our method is Pareto-optimal (i.e., no baseline is faster and more effective than it) and typically several orders of magnitude (spec., up to 15,160X) faster than the most effective baseline while being more effective.

arXiv:2501.19180

Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning

Large language models (LLMs) are vital for a wide range of applications yet remain susceptible to jailbreak threats, which could lead to the generation of inappropriate responses. Conventional defenses, such as refusal and adversarial training, often fail to cover corner cases or rare domains, leaving LLMs still vulnerable to more sophisticated attacks. We propose a novel defense strategy, Safety Chain-of-Thought (SCoT), which harnesses the enhanced \textit{reasoning capabilities} of LLMs for proactive assessment of harmful inputs, rather than simply blocking them. SCoT augments any refusal training datasets to critically analyze the intent behind each request before generating answers. By employing proactive reasoning, SCoT enhances the generalization of LLMs across varied harmful queries and scenarios not covered in the safety alignment corpus. Additionally, it generates detailed refusals specifying the rules violated. Comparative evaluations show that SCoT significantly surpasses existing defenses, reducing vulnerability to out-of-distribution issues and adversarial manipulations while maintaining strong general capabilities.

arXiv:2501.17808

Replacing the Gallium Oxide Shell with Conductive Ag: Toward a Printable and Recyclable Composite for Highly Stretchable Electronics, Electromagnetic Shielding, and Thermal Interfaces

Liquid metal (LM)-based composites hold promise for soft electronics due to their high conductivity and fluidic nature. However, the presence of {\alpha}_Ga2O3 and GaOOH layers around LM droplets impairs conductivity and performance. We tackle this issue by replacing the oxide layer with conductive silver (Ag) using an ultrasonic_assisted galvanic replacement reaction. The Ag_coated nanoparticles form aggregated, porous microparticles that are mixed with styrene_isoprene_styrene (SIS) polymers, resulting in a digitally printable composite with superior electrical conductivity and electromechanical properties compared to conventional fillers. Adding more LM enhances these properties further. The composite achieves EMI shielding effectiveness (SE) exceeding 75 dB in the X_band frequency range, even at 200 per cent strain, meeting stringent military and medical standards. It is applicable in wireless communications and Bluetooth signal blocking and as a thermal interface material (TIM). Additionally, we highlight its recyclability using a biodegradable solvent, underscoring its eco_friendly potential. This composite represents a significant advancement in stretchable electronics and EMI shielding, with implications for wearable and bioelectronic applications.

arXiv:2501.14753

ABACUS: A FinOps Service for Cloud Cost Optimization

In recent years, as more enterprises have moved their infrastructure to the cloud, significant challenges have emerged in achieving holistic cloud spend visibility and cost optimization. FinOps practices provide a way for enterprises to achieve these business goals by optimizing cloud costs and bringing accountability to cloud spend. This paper presents ABACUS - Automated Budget Analysis and Cloud Usage Surveillance, a FinOps solution for optimizing cloud costs by setting budgets, enforcing those budgets through blocking new deployments, and alerting appropriate teams if spending breaches a budget threshold. ABACUS also leverages best practices like Infrastructure-as-Code to alert engineering teams of the expected cost of deployment before resources are deployed in the cloud. Finally, future research directions are proposed to advance the state of the art in this important field.

arXiv:2501.14432

CAMEO: Autocorrelation-Preserving Line Simplification for Lossy Time Series Compression

Time series data from a variety of sensors and IoT devices need effective compression to reduce storage and I/O bandwidth requirements. While most time series databases and systems rely on lossless compression, lossy techniques offer even greater space-saving with a small loss in precision. However, the unknown impact on downstream analytics applications requires a semi-manual trial-and-error exploration. We initiate work on lossy compression that provides guarantees on complex statistical features (which are strongly correlated with the accuracy of the downstream analytics). Specifically, we propose a new lossy compression method that provides guarantees on the autocorrelation and partial-autocorrelation functions (ACF/PACF) of a time series. Our method leverages line simplification techniques as well as incremental maintenance of aggregates, blocking, and parallelization strategies for effective and efficient compression. The results show that our method improves compression ratios by 2x on average and up to 54x on selected datasets, compared to previous lossy and lossless compression methods. Moreover, we maintain -- and sometimes even improve -- the forecasting accuracy by preserving the autocorrelation properties of the time series. Our framework is extensible to multivariate time series and other statistical features of the time series.

arXiv:2501.10063

Hybrid Parallel Collaborative Simulation Framework Integrating Device Physics with Circuit Dynamics for PDAE-Modeled Power Electronic Equipment

Optimizing high-performance power electronic equipment, such as power converters, requires multiscale simulations that incorporate the physics of power semiconductor devices and the dynamics of other circuit components, especially in conducting Design of Experiments (DoEs), defining the safe operating area of devices, and analyzing failures related to semiconductor devices. However, current methodologies either overlook the intricacies of device physics or do not achieve satisfactory computational speeds. To bridge this gap, this paper proposes a Hybrid-Parallel Collaborative (HPC) framework specifically designed to analyze the Partial Differential Algebraic Equation (PDAE) modeled power electronic equipment, integrating the device physics and circuit dynamics. The HPC framework employs a dynamic iteration to tackle the challenges inherent in solving the coupled nonlinear PDAE system, and utilizes a hybrid-parallel computing strategy to reduce computing time. Physics-based system partitioning along with hybrid-process-thread parallelization on shared and distributed memory are employed, facilitating the simulation of hundreds of partial differential equations (PDEs)-modeled devices simultaneously without compromising speed. Experiments based on the hybrid line commutated converter and reverse-blocking integrated gate-commutated thyristors are conducted under 3 typical real-world scenarios: semiconductor device optimization for the converter; converter design optimization; and device failure analysis. The HPC framework delivers simulation speed up to 60 times faster than the leading commercial software, while maintaining carrier-level accuracy in the experiments. This shows great potential for comprehensive analysis and collaborative optimization of devices and electronic power equipment, particularly in extreme conditions and failure scenarios.

arXiv:2501.08647

Broadband measurements and analysis of human blocking in a 60 GHz indoor radio channel

New millimeter-wave wireless communication systems can be strongly impacted by the blockage introduced by the human body. At 60 GHz, the coverage of these systems is relatively limited due to high propagation losses. Thus, beamforming allows to find a reflective path to replace the blocked one. In this work, the study focuses on the impact of a human blocker in a meeting room, to evaluate the blocking losses introduced by the human body at 60 GHz. The results obtained in terms of path loss and channel impulse response show that the attenuation by the human body is between 24 and 26 dB. Moreover, the results show that the use of beamforming allows to exploit the reflected paths to replace the direct link that can be blocked by the human body.

arXiv:2501.07927공식 코드 · Papers with Code

Gandalf the Red: Adaptive Security for LLMs

Current evaluations of defenses against prompt attacks in large language model (LLM) applications often overlook two critical factors: the dynamic nature of adversarial behavior and the usability penalties imposed on legitimate users by restrictive defenses. We propose D-SEC (Dynamic Security Utility Threat Model), which explicitly separates attackers from legitimate users, models multi-step interactions, and expresses the security-utility in an optimizable form. We further address the shortcomings in existing evaluations by introducing Gandalf, a crowd-sourced, gamified red-teaming platform designed to generate realistic, adaptive attack. Using Gandalf, we collect and release a dataset of 279k prompt attacks. Complemented by benign user data, our analysis reveals the interplay between security and utility, showing that defenses integrated in the LLM (e.g., system prompts) can degrade usability even without blocking requests. We demonstrate that restricted application domains, defense-in-depth, and adaptive defenses are effective strategies for building secure and useful LLM applications.

arXiv:2501.05673

Network Diffuser for Placing-Scheduling Service Function Chains with Inverse Demonstration

Network services are increasingly managed by considering chained-up virtual network functions and relevant traffic flows, known as the Service Function Chains (SFCs). To deal with sequential arrivals of SFCs in an online fashion, we must consider two closely-coupled problems - an SFC placement problem that maps SFCs to servers/links in the network and an SFC scheduling problem that determines when each SFC is executed. Solving the whole SFC problem targeting these two optimizations jointly is extremely challenging. In this paper, we propose a novel network diffuser using conditional generative modeling for this SFC placing-scheduling optimization. Recent advances in generative AI and diffusion models have made it possible to generate high-quality images/videos and decision trajectories from language description. We formulate the SFC optimization as a problem of generating a state sequence for planning and perform graph diffusion on the state trajectories to enable extraction of SFC decisions, with SFC optimization constraints and objectives as conditions. To address the lack of demonstration data due to NP-hardness and exponential problem space of the SFC optimization, we also propose a novel and somewhat maverick approach -- Rather than solving instances of this difficult optimization, we start with randomly-generated solutions as input, and then determine appropriate SFC optimization problems that render these solutions feasible. This inverse demonstration enables us to obtain sufficient expert demonstrations, i.e., problem-solution pairs, through further optimization. In our numerical evaluations, the proposed network diffuser outperforms learning and heuristic baselines, by $\sim$20\% improvement in SFC reward and $\sim$50\% reduction in SFC waiting time and blocking rate.

arXiv:2501.03905

mFabric: An Efficient and Scalable Fabric for Mixture-of-Experts Training

Mixture-of-Expert (MoE) models outperform conventional models by selectively activating different subnets, named \emph{experts}, on a per-token basis. This gated computation generates dynamic communications that cannot be determined beforehand, challenging the existing GPU interconnects that remain \emph{static} during the distributed training process. In this paper, we advocate for a first-of-its-kind system, called mFabric, that unlocks topology reconfiguration \emph{during} distributed MoE training. Towards this vision, we first perform a production measurement study and show that the MoE dynamic communication pattern has \emph{strong locality}, alleviating the requirement of global reconfiguration. Based on this, we design and implement a \emph{regionally reconfigurable high-bandwidth domain} on top of existing electrical interconnects using optical circuit switching (OCS), achieving scalability while maintaining rapid adaptability. We have built a fully functional mFabric prototype with commodity hardware and a customized collective communication runtime that trains state-of-the-art MoE models with \emph{in-training} topology reconfiguration across 32 A100 GPUs. Large-scale packet-level simulations show that mFabric delivers comparable performance as the non-blocking fat-tree fabric while boosting the training cost efficiency (e.g., performance per dollar) of four representative MoE models by 1.2$\times$--1.5$\times$ and 1.9$\times$--2.3$\times$ at 100 Gbps and 400 Gbps link bandwidths, respectively.

arXiv:2412.12194

BlockDoor: Blocking Backdoor Based Watermarks in Deep Neural Networks

Adoption of machine learning models across industries have turned Neural Networks (DNNs) into a prized Intellectual Property (IP), which needs to be protected from being stolen or being used without authorization. This topic gave rise to multiple watermarking schemes, through which, one can establish the ownership of a model. Watermarking using backdooring is the most well established method available in the literature, with specific works demonstrating the difficulty in removing the watermarks, embedded as backdoors within the weights of the network. However, in our work, we have identified a critical flaw in the design of the watermark verification with backdoors, pertaining to the behaviour of the samples of the Trigger Set, which acts as the secret key. In this paper, we present BlockDoor, which is a comprehensive package of techniques that is used as a wrapper to block all three different kinds of Trigger samples, which are used in the literature as means to embed watermarks within the trained neural networks as backdoors. The framework implemented through BlockDoor is able to detect potential Trigger samples, through separate functions for adversarial noise based triggers, out-of-distribution triggers and random label based triggers. Apart from a simple Denial-of-Service for a potential Trigger sample, our approach is also able to modify the Trigger samples for correct machine learning functionality. Extensive evaluation of BlockDoor establishes that it is able to significantly reduce the watermark validation accuracy of the Trigger set by up to $98\%$ without compromising on functionality, delivering up to a less than $1\%$ drop on the clean samples. BlockDoor has been tested on multiple datasets and neural architectures.

arXiv:2412.11468

Block-Based Multi-Scale Image Rescaling

Image rescaling (IR) seeks to determine the optimal low-resolution (LR) representation of a high-resolution (HR) image to reconstruct a high-quality super-resolution (SR) image. Typically, HR images with resolutions exceeding 2K possess rich information that is unevenly distributed across the image. Traditional image rescaling methods often fall short because they focus solely on the overall scaling rate, ignoring the varying amounts of information in different parts of the image. To address this limitation, we propose a Block-Based Multi-Scale Image Rescaling Framework (BBMR), tailored for IR tasks involving HR images of 2K resolution and higher. BBMR consists of two main components: the Downscaling Module and the Upscaling Module. In the Downscaling Module, the HR image is segmented into sub-blocks of equal size, with each sub-block receiving a dynamically allocated scaling rate while maintaining a constant overall scaling rate. For the Upscaling Module, we introduce the Joint Super-Resolution method (JointSR), which performs SR on these sub-blocks with varying scaling rates and effectively eliminates blocking artifacts. Experimental results demonstrate that BBMR significantly enhances the SR image quality on the of 2K and 4K test dataset compared to initial network image rescaling methods.

arXiv:2412.09920

Precision-Enhanced Human-Object Contact Detection via Depth-Aware Perspective Interaction and Object Texture Restoration

Human-object contact (HOT) is designed to accurately identify the areas where humans and objects come into contact. Current methods frequently fail to account for scenarios where objects are frequently blocking the view, resulting in inaccurate identification of contact areas. To tackle this problem, we suggest using a perspective interaction HOT detector called PIHOT, which utilizes a depth map generation model to offer depth information of humans and objects related to the camera, thereby preventing false interaction detection. Furthermore, we use mask dilatation and object restoration techniques to restore the texture details in covered areas, improve the boundaries between objects, and enhance the perception of humans interacting with objects. Moreover, a spatial awareness perception is intended to concentrate on the characteristic features close to the points of contact. The experimental results show that the PIHOT algorithm achieves state-of-the-art performance on three benchmark datasets for HOT detection tasks. Compared to the most recent DHOT, our method enjoys an average improvement of 13%, 27.5%, 16%, and 18.5% on SC-Acc., C-Acc., mIoU, and wIoU metrics, respectively.

arXiv:2412.07430

Knowledge Graph Guided Evaluation of Abstention Techniques

To deploy language models safely, it is crucial that they abstain from responding to inappropriate requests. Several prior studies test the safety promises of models based on their effectiveness in blocking malicious requests. In this work, we focus on evaluating the underlying techniques that cause models to abstain. We create SELECT, a benchmark derived from a set of benign concepts (e.g., "rivers") from a knowledge graph. The nature of SELECT enables us to isolate the effects of abstention techniques from other safety training procedures, as well as evaluate their generalization and specificity. Using SELECT, we benchmark different abstention techniques over six open-weight and closed-source models. We find that the examined techniques indeed cause models to abstain with over $80\%$ abstention rates. However, these techniques are not as effective for descendants of the target concepts, with refusal rates declining by $19\%$. We also characterize the generalization-vs-specificity trade-offs for different techniques. Overall, no single technique is invariably better than the others. Our findings call for a careful evaluation of different aspects of abstention, and hopefully inform practitioners of various trade-offs involved.

arXiv:2412.04318

The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation

This paper introduces the counter-intuitive generalization results of overfitting pre-trained large language models (LLMs) on very small datasets. In the setting of open-ended text generation, it is well-documented that LLMs tend to generate repetitive and dull sequences, a phenomenon that is especially apparent when generating using greedy decoding. This issue persists even with state-of-the-art LLMs containing billions of parameters, trained via next-token prediction on large datasets. We find that by further fine-tuning these models to achieve a near-zero training loss on a small set of samples -- a process we refer to as hyperfitting -- the long-sequence generative capabilities are greatly enhanced. Greedy decoding with these Hyperfitted models even outperform Top-P sampling over long-sequences, both in terms of diversity and human preferences. This phenomenon extends to LLMs of various sizes, different domains, and even autoregressive image generation. We further find this phenomena to be distinctly different from that of Grokking and double descent. Surprisingly, our experiments indicate that hyperfitted models rarely fall into repeating sequences they were trained on, and even explicitly blocking these sequences results in high-quality output. All hyperfitted models produce extremely low-entropy predictions, often allocating nearly all probability to a single token.

arXiv:2412.00449

Effect of Correlated Building Blockages on the Ergodic Capacity of mmWave Systems in Urban Scenarios

The mmWave bands, considered to support the forthcoming generation of mobile communications technologies, have a well-known vulnerability to blockages. Recent works in the literature analyze the blockage probability considering independence or correlation among the blocking elements of the different links. In this letter, we characterize the effect of blockages and their correlation on the ergodic capacity. We carry out the analysis for urban scenarios, where the considered blocking elements are buildings that are primarily parallel to the streets. We also present numerical simulations based on actual building features of the city of Chicago to validate the obtained expressions.

arXiv:2412.00442

Analysis of Blocking in mmWave Cellular Systems: Characterization of the LOS and NLOS Intervals in Urban Scenarios

In the millimeter waves (mmWave) bands considered for 5G and beyond, the use of very high frequencies results in the interruption of communication whenever there is no line of sight between the transmitter and the receiver. Blockages have been modeled in the literature so far using tools such as stochastic geometry and random shape theory. Using these tools, in this paper, we characterize the lengths of the segments in line-of-sight (LOS) and in non-line-of-sight (NLOS) statistically in an urban scenario where buildings (with random positions, lengths, and heights) are deployed in parallel directions configuring streets.

arXiv:2411.16911

Avoiding Deadlocks Is Not Enough: Analysis and Resolution of Blocked Airplanes

This paper is devoted to the analysis and resolution of a pathological phenomenon in airplane encounters called blocking mode. As autonomy in airplane systems increases, a pathological phenomenon can be observed in two-aircraft encounter scenarios, where airplanes stick together and fly in parallel for an extended period. This parallel flight results in a temporary blocking that significantly delays progress. In contrast to widely studied deadlocks in multi-robot systems, such transient blocking is often overlooked in existing literature. Since such prolonged parallel flying places high-speed airplanes at elevated risks of near-miss collisions, encounter conflicts must be resolved as quickly as possible in the context of aviation. We develop a mathematical model for a two-airplane encounter system that replicates this blocking phenomenon. Using this model, we analyze the conditions under which blocking occurs, quantify the duration of the blocking period, and demonstrate that the blocking condition is significantly less restrictive than that of deadlock. Based on these analytical insights, we propose an intention-aware strategy with an adaptive priority mechanism that enables efficient resolution of ongoing blocking phenomena while also incidentally eliminating deadlocks. Notably, the developed strategy does not rely on central coordination and communications that can be unreliable in harsh situations. The analytical findings and the proposed resolution strategy are validated through extensive simulations.

arXiv:2411.10629

Leveraging large language models for efficient representation learning for entity resolution

In this paper, the authors propose TriBERTa, a supervised entity resolution system that utilizes a pre-trained large language model and a triplet loss function to learn representations for entity matching. The system consists of two steps: first, name entity records are fed into a Sentence Bidirectional Encoder Representations from Transformers (SBERT) model to generate vector representations, which are then fine-tuned using contrastive learning based on a triplet loss function. Fine-tuned representations are used as input for entity matching tasks, and the results show that the proposed approach outperforms state-of-the-art representations, including SBERT without fine-tuning and conventional Term Frequency-Inverse Document Frequency (TF-IDF), by a margin of 3 - 19%. Additionally, the representations generated by TriBERTa demonstrated increased robustness, maintaining consistently higher performance across a range of datasets. The authors also discussed the importance of entity resolution in today's data-driven landscape and the challenges that arise when identifying and reconciling duplicate data across different sources. They also described the ER process, which involves several crucial steps, including blocking, entity matching, and clustering.

arXiv:2411.10200

Block based Adaptive Compressive Sensing with Sampling Rate Control

Compressive sensing (CS), acquiring and reconstructing signals below the Nyquist rate, has great potential in image and video acquisition to exploit data redundancy and greatly reduce the amount of sampled data. To further reduce the sampled data while keeping the video quality, this paper explores the temporal redundancy in video CS and proposes a block based adaptive compressive sensing framework with a sampling rate (SR) control strategy. To avoid redundant compression of non-moving regions, we first incorporate moving block detection between consecutive frames, and only transmit the measurements of moving blocks. The non-moving regions are reconstructed from the previous frame. In addition, we propose a block storage system and a dynamic threshold to achieve adaptive SR allocation to each frame based on the area of moving regions and target SR for controlling the average SR within the target SR. Finally, to reduce blocking artifacts and improve reconstruction quality, we adopt a cooperative reconstruction of the moving and non-moving blocks by referring to the measurements of the non-moving blocks from the previous frame. Extensive experiments have demonstrated that this work is able to control SR and obtain better performance than existing works.