A hybrid beamformer (HBF) is designed for integrated sensing and communication (ISAC)-aided millimeter wave (mmWave) systems. The ISAC base station (BS), relying on a limited number of radio frequency (RF) chains, supports multiple communication users (CUs) and simultaneously detects the radar target (RT). To maximize the probability of detection (PD) of the RT, and achieve rate fairness among the CUs, we formulate two problems for the optimization of the RF and baseband (BB) transmit precoders (TPCs): PD-maximization (PD-max) and geometric mean rate-maximization (GMR-max), while ensuring the quality of services (QoS) of the RT and CUs. Both problems are highly non-convex due to the intractable expressions of the PD and GMR and also due to the non-convex unity magnitude constraints imposed on each element of the RF TPC. To solve these problems, we first transform the intractable expressions into their tractable counterparts and propose a power-efficient bisection search and majorization and minimization-based alternating algorithms for the PD-max and GMR-max problems, respectively. Furthermore, both algorithms optimize the BB TPC and RF TPCs in an alternating fashion via the successive convex approximation (SCA) and penalty-based Riemannian conjugate gradient (PRCG) techniques, respectively. Specifically, in the PRCG method, we initially add all the constraints except for the unity magnitude constraint to the objective function as a penalty term and subsequently employ the RCG method for optimizing the RF TPC. Finally, we present our simulation results and compare them to the benchmarks for demonstrating the efficacy of the proposed algorithms.
In studies on complex network systems using graph theory, eigen-analysis is typically performed on an undirected graph model of the network. However, when analyzing cascading failures in a power system, the interactions among failures suggest the need for a directed graph beyond the topology of the power system to model directions of failure propagation. To accurately quantify failure interactions for effective mitigation strategies, this paper proposes a stochastic interaction graph model and associated eigen-analysis. Different types of modes on failure propagations are defined and characterized by the eigenvalues of a stochastic interaction matrix, whose absolute values are unity, zero, or in between. Finding and interpreting these modes helps identify the probable patterns of failure propagation, either local or widespread, and the participating components based on eigenvectors. Then, by lowering the failure probabilities of critical components highly participating in a mode of widespread failures, cascading can be mitigated. The validity of the proposed stochastic interaction graph model, eigen-analysis and the resulting mitigation strategies is demonstrated using simulated cascading failure data on an NPCC 140-bus system.
This paper addresses two minimum reaching time control problems within the context of finite stable systems. The well-known Variable Structure Control (VSC) and Unity Vector Control (UVC) strategies are analyzed, with the primary objective of designing optimal and robust state feedback gains that ensure minimum finite time convergence to the origin. This is achieved in the presence of convex bounded parameter uncertainty and norm-bounded exogenous disturbances. In both cases, the optimality conditions are expressed through Linear Matrix Inequalities (LMIs), which are solved efficiently within the framework of multivariable systems using existing numerical tools. The theoretical results are demonstrated with two practically motivated examples.
This paper introduces Unity RL Playground, an open-source reinforcement learning framework built on top of Unity ML-Agents. Unity RL Playground automates the process of training mobile robots to perform various locomotion tasks such as walking, running, and jumping in simulation, with the potential for seamless transfer to real hardware. Key features include one-click training for imported robot models, universal compatibility with diverse robot configurations, multi-mode motion learning capabilities, and extreme performance testing to aid in robot design optimization and morphological evolution. The attached video can be found at https://linqi-ye.github.io/video/iros25.mp4 and the code is coming soon.
We develop a formal economic framework to analyze whether neural scaling laws in artificial intelligence will activate Jevons' Paradox in labor markets, potentially leading to increased AI adoption and human labor substitution. By using a time-varying elasticity of substitution (VES) approach, we establish analytical conditions under which AI systems transition from complementing to substituting for human labor. Our model formalizes four interconnected mechanisms: (1) exponential growth in computational capacity ($C(t) = C(0) \cdot e^{g \cdot t}$); (2) logarithmic scaling of AI capabilities with computation ($\sigma(t) = \delta \cdot \ln(C(t)/C(0))$); (3) declining AI prices ($p_A(t) = p_A(0) \cdot e^{-d \cdot t}$); and (4) a resulting compound effect parameter ($\phi = \delta \cdot g$) that governs market transformation dynamics. We identify five distinct phases of AI market penetration, demonstrating that complete market transformation requires the elasticity of substitution to exceed unity ($\sigma > 1$), with the timing determined primarily by the compound parameter $\phi$ rather than price competition alone. These findings provide an analytical framing for evaluating industry claims about AI substitution effects, especially on the role of quality versus price in the technological transition.
We present a novel form of scalable knowledge representation about agents in a simulated democracy, e-polis, where real users respond to social challenges associated with democratic institutions, structured as Smart Spatial Types, a new type of Smart Building that changes architectural form according to the philosophical doctrine of a visitor. At the end of the game players vote on the Smart City that results from their collective choices. Our approach uses deductive systems in an unusual way: by integrating a model of democracy with a model of a Smart City we are able to prove quality aspects of the simulated democracy in different urban and social settings, while adding ease and flexibility to the development. Second, we can infer and reason with abstract knowledge, which is a limitation of the Unity platform; third, our system enables real-time decision-making and adaptation of the game flow based on the player's abstract state, paving the road to explainability. Scalability is achieved by maintaining a dual-layer knowledge representation mechanism for reasoning about the simulated democracy that functions in a similar way to a two-level cache. The lower layer knows about the current state of the game by continually processing a high rate of events produced by the in-built physics engine of the Unity platform, e.g., it knows of the position of a player in space, in terms of his coordinates x,y,z as well as their choices for each challenge. The higher layer knows of easily-retrievable, user-defined abstract knowledge about current and historical states, e.g., it knows of the political doctrine of a Smart Spatial Type, a player's philosophical doctrine, and the collective philosophical doctrine of a community players with respect to current social issues.
In this document, a nonlinear control law for a grid-tied converter is introduced. The converter topology consists of a voltage source inverter (VSI) linked to the grid through an inductive-capacitive second-order filter, its input being connected to a capacitive DC-link supplied by a renewable energy-based input power source. In order to achieve good performance in presence of large state excursions caused mainly by substantial set-point modifications and/or considerable disturbances, a nonlinear control law based on a complex-valued feedback linearization strategy is designed. Specifically, a flat output is adopted, which is given by the summation of the energy stored in the DC-link capacitor and in the output filter's inductor and capacitor, as well as by the reactive energy at the output. After linearizing the system through a pertinent coordinate transformation and a nonlinear feedback, a linear trajectory tracking control law is implemented. The performance of the system controlled by applying the proposed strategy is tested via simulation for a very weak grid of unity X/R ratio, yielding satisfactory results.
In precision agriculture, the scarcity of labeled data and significant covariate shifts pose unique challenges for training machine learning models. This scarcity is particularly problematic due to the dynamic nature of the environment and the evolving appearance of agricultural subjects as living things. We propose a novel system for generating realistic synthetic data to address these challenges. Utilizing a vineyard simulator based on the Unity engine, our system employs a cut-and-paste technique with geometrical consistency considerations to produce accurate photo-realistic images and labels from synthetic environments to train detection algorithms. This approach generates diverse data samples across various viewpoints and lighting conditions. We demonstrate considerable performance improvements in training a state-of-the-art detector by applying our method to table grapes cultivation. The combination of techniques can be easily automated, an increasingly important consideration for adoption in agricultural practice.
Temporal sentence grounding aims to detect event timestamps described by the natural language query from given untrimmed videos. The existing fully-supervised setting achieves great results but requires expensive annotation costs; while the weakly-supervised setting adopts cheap labels but performs poorly. To pursue high performance with less annotation costs, this paper introduces an intermediate partially-supervised setting, i.e., only short-clip is available during training. To make full use of partial labels, we specially design one contrast-unity framework, with the two-stage goal of implicit-explicit progressive grounding. In the implicit stage, we align event-query representations at fine granularity using comprehensive quadruple contrastive learning: event-query gather, event-background separation, intra-cluster compactness and inter-cluster separability. Then, high-quality representations bring acceptable grounding pseudo-labels. In the explicit stage, to explicitly optimize grounding objectives, we train one fully-supervised model using obtained pseudo-labels for grounding refinement and denoising. Extensive experiments and thoroughly ablations on Charades-STA and ActivityNet Captions demonstrate the significance of partial supervision, as well as our superior performance.
This paper introduces IRIS, an immersive Robot Interaction System leveraging Extended Reality (XR), designed for robot data collection and interaction across multiple simulators, benchmarks, and real-world scenarios. While existing XR-based data collection systems provide efficient and intuitive solutions for large-scale data collection, they are often challenging to reproduce and reuse. This limitation arises because current systems are highly tailored to simulator-specific use cases and environments. IRIS is a novel, easily extendable framework that already supports multiple simulators, benchmarks, and even headsets. Furthermore, IRIS is able to include additional information from real-world sensors, such as point clouds captured through depth cameras. A unified scene specification is generated directly from simulators or real-world sensors and transmitted to XR headsets, creating identical scenes in XR. This specification allows IRIS to support any of the objects, assets, and robots provided by the simulators. In addition, IRIS introduces shared spatial anchors and a robust communication protocol that links simulations between multiple XR headsets. This feature enables multiple XR headsets to share a synchronized scene, facilitating collaborative and multi-user data collection. IRIS can be deployed on any device that supports the Unity Framework, encompassing the vast majority of commercially available headsets. In this work, IRIS was deployed and tested on the Meta Quest 3 and the HoloLens 2. IRIS showcased its versatility across a wide range of real-world and simulated scenarios, using current popular robot simulators such as MuJoCo, IsaacSim, CoppeliaSim, and Genesis. In addition, a user study evaluates IRIS on a data collection task for the LIBERO benchmark. The study shows that IRIS significantly outperforms the baseline in both objective and subjective metrics.
The interaction with Large Language Models (LLMs) through instructions has been extensively investigated in the research community. While instructions have been widely used as the guidelines for task solving, this paper further notices that both instructions and parameters are the compression of task data. Therefore, they could be strongly correlated and can be learned to predict one from the other. This paper proposes a novel neural network framework, SHIP (\textbf{Sh}uttle between the \textbf{I}nstructions and the \textbf{P}arameters), to model and learn the mutual mappings between the instructions and the parameters of LLMs. We verify that SHIP can effectively map one of the instructions/parameters to the other by evaluating it on the tasks of instruction deduction and induction. The results show that SHIP performs better than existing baseline methods in terms of deductive capabilities while significantly surpassing them in inductive capabilities. Moreover, SHIP can effectively combine the two mapping processes to perform excellent inductive reasoning. The code and data for this paper are released at https://anonymous.4open.science/r/Shuttle-Between-Instructions-Parameters/.
Computer vision for real-world applications faces data acquisition challenges, including accessibility, high costs, difficulty in obtaining diversity in scenarios or environmental conditions. Synthetic data usage has surged as a solution to these obstacles. Leveraging game engines for synthetic dataset creation effectively enriches training datasets with increased diversity and richness. The choice of the game engine, pivotal for generating photorealistic simulations, may influence synthetic data quality. This study compares Unity Engine’s and Unreal Engine’s capabilities in generating synthetic maritime datasets to support ship recognition applications. To this end, the realworld maritime dataset ShipSG has been replicated in the corresponding game engines to create the same scenarios. The performance of the generated synthetic datasets is benchmarked against the real-world ShipSG dataset using the object recognition model YOLOv8. Furthermore, the comparison evaluates various photorealistic parameters found in the dataset images to determine the optimal configuration for improving performance with YOLOv8. The datasets generated using the Unity Engine, with all photorealistic effects present and the one with no lens distortion, achieved the highest accuracy in ship recognition with a mAP of 72.3%. Both configurations of the synthetic datasets were utilised to augment the ShipSG dataset to train YOLOv8. The configuration with all photorealistic parameters in place provides the highest mAP increase, of 0.4% compared with YOLOv8 performance on ShipSG when no synthetic data is used. This evidence underscores that utilising game engines can effectively support and enhance ship recognition tasks.
Objective: This study explores a novel deep learning approach for EEG analysis and perceptual state guidance, inspired by Level of Detail (LOD) theory. The goal is to improve perceptual state identification accuracy and advance personalized psychological therapy. Methods: Portable EEG devices and music rhythm signals were used for data collection. LOD theory was applied to dynamically adjust EEG signal processing, extracting core perceptual features. A Unity-based software system integrated EEG data with audio materials. The deep learning model combined a CNN for feature extraction and classification, and a DQN for reinforcement learning to optimize rhythm adjustments. Results: The CNN achieved 94.05% accuracy in perceptual state classification. The DQN guided subjects to target states with a 92.45% success rate, averaging 13.2 rhythm cycles. However, only 50% of users reported psychological alignment with the target state, indicating room for improvement. Discussion: The results validate the potential of LOD-based EEG biofeedback. Limitations include dataset source, label subjectivity, and reward function optimization. Future work will expand to diverse subjects, incorporate varied musical elements, and refine reward functions for better generalization and personalization.
Person Re-identification (ReID) aims to retrieve the specific person across non-overlapping cameras, which greatly helps intelligent transportation systems. As we all know, Convolutional Neural Networks (CNNs) and Transformers have the unique strengths to extract local and global features, respectively. Considering this fact, we focus on the mutual fusion between them to learn more comprehensive representations for persons. In particular, we utilize the complementary integration of deep features from different model structures. We propose a novel fusion framework called FusionReID to unify the strengths of CNNs and Transformers for image-based person ReID. More specifically, we first deploy a Dual-branch Feature Extraction (DFE) to extract features through CNNs and Transformers from a single image. Moreover, we design a novel Dual-attention Mutual Fusion (DMF) to achieve sufficient feature fusions. The DMF comprises Local Refinement Units (LRU) and Heterogenous Transmission Modules (HTM). LRU utilizes depth-separable convolutions to align deep features in channel dimensions and spatial sizes. HTM consists of a Shared Encoding Unit (SEU) and two Mutual Fusion Units (MFU). Through the continuous stacking of HTM, deep features after LRU are repeatedly utilized to generate more discriminative features. Extensive experiments on three public ReID benchmarks demonstrate that our method can attain superior performances than most state-of-the-arts. The source code is available at https://github.com/924973292/FusionReID.
In this research, we have derived a mathematical model for within human dynamics of COVID-19 infection using delay differential equations. The new model considers a 'latent period' and 'the time for immune response' as delay parameters, allowing us to study the effects of time delays in human COVID-19 infection. We have determined the equilibrium points and analyzed their stability. The disease-free equilibrium is stable when the basic reproduction number, $R_0$, is below unity. Stability switch of the endemic equilibrium occurs through Hopf-bifurcation. This study shows that the effect of latent delay is stabilizing whereas immune response delay has a destabilizing nature.
Reinforcement Learning (RL) policies are designed to predict actions based on current observations to maximize cumulative future rewards. In real-world applications (i.e., non-simulated environments), sensors are essential for measuring the current state and providing the observations on which RL policies rely to make decisions. A significant challenge in deploying RL policies in real-world scenarios is handling sensor dropouts, which can result from hardware malfunctions, physical damage, or environmental factors like dust on a camera lens. A common strategy to mitigate this issue is the use of backup sensors, though this comes with added costs. This paper explores the optimization of backup sensor configurations to maximize expected returns while keeping costs below a specified threshold, C. Our approach uses a second-order approximation of expected returns and includes penalties for exceeding cost constraints. We then optimize this quadratic program using Tabu Search, a meta-heuristic algorithm. The approach is evaluated across eight OpenAI Gym environments and a custom Unity-based robotic environment (RobotArmGrasping). Empirical results demonstrate that our quadratic program effectively approximates real expected returns, facilitating the identification of optimal sensor configurations.
Objective:To develop a no-reference image quality assessment method using automated distortion recognition to boost MRI-guided radiotherapy precision.Methods:We analyzed 106,000 MR images from 10 patients with liver metastasis,captured with the Elekta Unity MR-LINAC.Our No-Reference Quality Assessment Model includes:1)image preprocessing to enhance visibility of key diagnostic features;2)feature extraction and directional analysis using MSCN coefficients across four directions to capture textural attributes and gradients,vital for identifying image features and potential distortions;3)integrative Quality Index(QI)calculation,which integrates features via AGGD parameter estimation and K-means clustering.The QI,based on a weighted MAD computation of directional scores,provides a comprehensive image quality measure,robust against outliers.LOO-CV assessed model generalizability and performance.Tumor tracking algorithm performance was compared with and without preprocessing to verify tracking accuracy enhancements.Results:Preprocessing significantly improved image quality,with the QI showing substantial positive changes and surpassing other metrics.After normalization,the QI's average value was 79.6 times higher than CNR,indicating improved image definition and contrast.It also showed higher sensitivity in detail recognition with average values 6.5 times and 1.7 times higher than Tenengrad gradient and entropy.The tumor tracking algorithm confirmed significant tracking accuracy improvements with preprocessed images,validating preprocessing effectiveness.Conclusions:This study introduces a novel no-reference image quality evaluation method based on automated distortion recognition,offering a new quality control tool for MRIgRT tumor tracking.It enhances clinical application accuracy and facilitates medical image quality assessment standardization, with significant clinical and research value.
In the context of medical Augmented Reality (AR) applications, object tracking is a key challenge and requires a significant amount of annotation masks. As segmentation foundation models like the Segment Anything Model (SAM) begin to emerge, zero-shot segmentation requires only minimal human participation obtaining high-quality object masks. We introduce a HoloLens-Object-Labeling (HOLa) Unity and Python application based on the SAM-Track algorithm that offers fully automatic single object annotation for HoloLens 2 while requiring minimal human participation. HOLa does not have to be adjusted to a specific image appearance and could thus alleviate AR research in any application field. We evaluate HOLa for different degrees of image complexity in open liver surgery and in medical phantom experiments. Using HOLa for image annotation can increase the labeling speed by more than 500 times while providing Dice scores between 0.875 and 0.982, which are comparable to human annotators. Our code is publicly available at: https://github.com/mschwimmbeck/HOLa
Long duration spaceflight missions can affect the cognitive and behavioral activities of astronauts due to changes in gravity. The microgravity significantly impacts the central nervous system physiology which causes the degradation in the performance and lead to potential risk in the space exploration. The aim of this study was to evaluate functional connectivity at simulated space conditions using an unloading harness system to mimic the body-weight distribution related to Earth, Mars, and International Space Station. A unity model with six directional arrows to imagine six different motor imagery tasks associated with arms and legs were designed for the Oculus Rift S virtual reality headset for testing. An Electroencephalogram (EEG) and functional near infrared spectroscopy (fNIRS) signals were recorded from 10 participants in the distributed weight conditions related to Earth, Mars, and International Space station using the g.Nautilus fNIRS system at sampling rate of 500 Hz. The magnitude squared coherence were estimated from left vs right hemisphere of the brain that represents functional connectivity. The EEG coherence was the higher which shows the strong functional connectivity and fNIRS coherence was lower shows weak functional connectivity between left vs right hemisphere of the brain, during all the tasks and trials irrespective of the simulated space conditions. Further analysis of functional connectivity needed between the intra-regions of the brain.
Autonomous and semi-autonomous systems are using deep learning models to improve decision-making. However, deep classifiers can be overly confident in their incorrect predictions, a major issue especially in safety-critical domains. The present study introduces three foundational desiderata for developing real-world risk-aware classification systems. Expanding upon the previously proposed Evidential Deep Learning (EDL), we demonstrate the unity between these principles and EDL's operational attributes. We then augment EDL empowering autonomous agents to exercise discretion during structured decision-making when uncertainty and risks are inherent. We rigorously examine empirical scenarios to substantiate these theoretical innovations. In contrast to existing risk-aware classifiers, our proposed methodologies consistently exhibit superior performance, underscoring their transformative potential in risk-conscious classification strategies.