In this paper, we present the use of Reinforcement Learning (RL) based on Robust Model Predictive Control (RMPC) for the control of an Autonomous Surface Vehicle (ASV). The RL-MPC strategy is utilized for obstacle avoidance and target (set-point) tracking. A scenario-tree robust MPC is used to handle potential failures of the ship thrusters. Besides, the wind and ocean current are considered as unknown stochastic disturbances in the real system, which are handled via constraints tightening. The tightening and other cost parameters are adjusted by RL, using a Q-learning technique. An economic cost is considered, minimizing the time and energy required to achieve the ship missions. The method is illustrated in simulation on a nonlinear 3-DOF model of a scaled version of the Cybership 2.
In the application of computer-vision based displacement measurement, an optical target is usually required to prove the reference. In the case that the optical target cannot be attached to the measuring objective, edge detection, feature matching and template matching are the most common approaches in target-less photogrammetry. However, their performance significantly relies on parameter settings. This becomes problematic in dynamic scenes where complicated background texture exists and varies over time. To tackle this issue, we propose virtual point tracking for real-time target-less dynamic displacement measurement, incorporating deep learning techniques and domain knowledge. Our approach consists of three steps: 1) automatic calibration for detection of region of interest; 2) virtual point detection for each video frame using deep convolutional neural network; 3) domain-knowledge based rule engine for point tracking in adjacent frames. The proposed approach can be executed on an edge computer in a real-time manner (i.e. over 30 frames per second). We demonstrate our approach for a railway application, where the lateral displacement of the wheel on the rail is measured during operation. We also implement an algorithm using template matching and line detection as the baseline for comparison. The numerical experiments have been performed to evaluate the performance and the latency of our approach in the harsh railway environment with noisy and varying backgrounds.
Extracting maximum power available from photovoltaic arrays requires the system operating at the maximum power point (MPP). Therefore, finding the MPP is necessary for efficient operation of PV arrays. The MPP changes with multiple environmental factors, mainly temperature and irradiance. Traditionally, the incremental conductance technique with fixed step size was used to find the MPP, which suffers from a trade-off between speed of convergence and accuracy. In this work, we propose an incremental conductance maximum power point tracking (MPPT) algorithm with a variable step size, which adaptively changes the step size after each iteration based on how far away the current operating point is from a new MPP. This mitigates the aforementioned trade-off drastically by achieving faster convergence speed without the loss of accuracy. A series of simulations involving variations in temperature and irradiance were performed using MATLAB, and the speed of convergence and accuracy were compared with the traditional IC technique.
This paper presents a combined power system with a common dc bus that contains solar power, wind power, battery storage, and a constant dc load (CDL). In wind system, an AC-DC uncontrolled rectifier is used at the first stage and the DC-DC converter is controlled by a maximum power point tracker (MPPT) at second stage. In the solar system, two cascaded boost converters are controlled through a sliding mode controller (SMC) to regulate the power flow to the load. A supervisory control strategy is also introduced to maximize the simultaneous energy harvesting from both renewable sources and balance the energy between the sources, battery, and the load. According to the level of power generation available at each renewable energy source, the state of charge in the battery, and the load requirement, the controller results in four contingencies. Simulation results show the accurate operation of the supervisory controller and functionality of the maximum power point tracking algorithm for solar and for wind power.
This paper proposes a novel topology for grid connected photovoltaic (PV) system based on modular multilevel converter (MMC). In this topology, a PV array is connected to capacitors of each submodule (SM) of the MMC through a DC-DC boost converter with maximum power point tracking (MPPT) control. This topology will maximize the efficiency of the system in the case of partial shading conditions, as it can regulate the SM capacitor voltages independently from each other to realize distributed MPPT. A model predictive control is used to track the AC output current, balance the SMs capacitor voltages, and to mitigate the circulating current. The proposed PV generation topology with 7 level MMC system validity has been verified by simulations via MATLAB/Simulink toolbox under normal operation, partial shading and PV array failure.
Closed loop anesthesia delivery (CLAD) systems can help anesthesiologists efficiently achieve and maintain desired anesthetic depth over an extended period of time. A typical CLAD system would use an anesthetic marker, calculated from physiological signals, as real-time feedback to adjust anesthetic dosage towards achieving a desired set-point of the marker. Since control strategies for CLAD vary across the systems reported in recent literature, a comparative analysis of common control strategies can be useful. For a nonlinear plant model based on well-established models of compartmental pharmacokinetics and sigmoid-Emax pharmacodynamics, we numerically analyze the set-point tracking performance of three output-feedback linear control strategies: proportional-integral-derivative (PID) control, linear quadratic Gaussian (LQG) control, and an LQG with integral action (ILQG). Specifically, we numerically simulate multiple CLAD sessions for the scenario where the plant model parameters are unavailable for a patient and the controller is designed based on a nominal model and controller gains are held constant throughout a session. Based on the numerical analyses performed here, conditioned on our choice of model and controllers, we infer that in terms of accuracy and bias PID control performs better than ILQG which in turn performs better than LQG. In the case of noisy observations, ILQG can be tuned to provide a smoother infusion rate while achieving comparable steady-state response with respect to PID. The numerical analyses framework and findings, reported here, can help CLAD developers in their choice of control strategies. This paper may also serve as a tutorial paper for teaching control theory for CLAD.
In this work, the autonomous control of a quadrotor-manipulator unmanned aerial vehicle is treated using an extended dynamic model. Due to persistent aerodynamic disturbances and dynamic couplings, the control of a quadrotor-manipulator unit is cumbersome. Active disturbance rejection control (ADRC) is proposed for the cancellation of the undesirable dynamics while a PD controller is developed for set-point tracking. Parameters were tuned using Gradient Descent-Sequential Quadratic Programming in Simulink design optimization
One of the most important issues in the operation of a photovoltaic (PV) system is extracting maximum power from the PV array, especially in partial shading condition (PSC). Under PSC, P-V characteristic of PV arrays will have multiple peak points, only one of which is global maximum. Conventional maximum power point tracking (MPPT) methods are not able to extract maximum power in this condition. In this paper, a novel two-stage MPPT method is presented to overcome this drawback. In the first stage, a method is proposed to determine the occurrence of PSC, and in the second stage, using a new algorithm that is based on ramp change of the duty cycle and continuous sampling from the P-V characteristic of the array, global maximum power point of array is reached. P&O algorithm is then re-activated to trace small changes of the new MPP. Open loop operation of the proposed method makes its implementation cheap and simple. The method is robust in the face of changing environmental conditions and array characteristics, and has minimum negative impact on the connected power system. Simulations in Matlab/Simulink and experimental results validate the performance of the proposed methods.
P-V characteristic of photovoltaic (PV) arrays under partially shaded conditions (PSC) has multiple peaks, and conventional maximum power point tracking (MPPT) algorithms may fail to track the global maximum power point (GMPP) because of their insufficient intelligence in discriminating the local and global peaks. This paper proposes a novel fast MPPT method to achieve GMPP of PV array under all PSCs. The proposed method reduces the number of required samples and increases the speed of GMPP tracking based on comprehensive study of I-V and P-V characteristics of PV array. Performance of the proposed method has been evaluated in simulations of different PSCs. Also, its performance has been compared with two selected methods in the literature through simulations. Comparisons highlight the superiority of the proposed method.
The performance of a photovoltaic system is subject to varying environmental conditions, and it becomes more challenging to track the maximum power point (MPP) and maintain the optimal performance when partial shading occurs. In this paper, we propose an enhanced maximum power point tracking (MPPT) method utilizing the state estimation by the sequential Monte Carlo (SMC) filtering which is assisted by the prediction of MPP via an artificial neural network (ANN). A state-space model for the sequential estimation of MPP is proposed in the framework of incremental conductance (I-C) MPPT approach, and the ANN model based on the observed voltage and current or irradiance data predicts the global MPP (GMPP) to refine the estimation by SMC. Moreover, a quick irrandiance change detection method is applied, such that the SMC-based MPPT method resorts to the assistance from ANN only when partial shading is detected. Simulation results show that the proposed enhanced MPPT method achieves high efficiency and is robust to rapid irradiance change under different noise levels.
In this paper, we propose a concept to design, track, and compare application-specific feature definitions expressed as sets of critical points. Our work has been inspired by the observation that in many applications a large variety of different feature definitions for the same concept are used. Often, these definitions compete with each other and it is unclear which definition should be used in which context. A prominent example is the definition of cyclones in climate research. Despite the differences, frequently these feature definitions can be related to topological concepts. In our approach, we provide a cyclone tracking framework that supports interactive feature definition and comparison based on a precomputed tracking graph that stores all extremal points as well as their temporal correspondents. The framework combines a set of independent building blocks: critical point extraction, critical point tracking, feature definition, and track exploration. One of the major advantages of such an approach is the flexibility it provides, that is, each block is exchangeable. Moreover, it also enables us to perform the most expensive analysis, the construction of a full tracking graph, as a prepossessing step, while keeping the feature definition interactive. Different feature definitions can be explored and compared interactively based on this tracking graph. Features are specified by rules for grouping critical points, while feature tracking corresponds to filtering and querying the full tracking graph by specific requests. We demonstrate this method for cyclone identification and tracking in the context of climate research.
Conventional SLAM techniques strongly rely on scene rigidity to solve data association, ignoring dynamic parts of the scene. In this work we present Semi-Direct DefSLAM (SD-DefSLAM), a novel monocular deformable SLAM method able to map highly deforming environments, built on top of DefSLAM. To robustly solve data association in challenging deforming scenes, SD-DefSLAM combines direct and indirect methods: an enhanced illumination-invariant Lucas-Kanade tracker for data association, geometric Bundle Adjustment for pose and deformable map estimation, and bag-of-words based on feature descriptors for camera relocation. Dynamic objects are detected and segmented-out using a CNN trained for the specific application domain. We thoroughly evaluate our system in two public datasets. The mandala dataset is a SLAM benchmark with increasingly aggressive deformations. The Hamlyn dataset contains intracorporeal sequences that pose serious real-life challenges beyond deformation like weak texture, specular reflections, surgical tools and occlusions. Our results show that SD-DefSLAM outperforms DefSLAM in point tracking, reconstruction accuracy and scale drift thanks to the improvement in all the data association steps, being the first system able to robustly perform SLAM inside the human body.
Unlike conventional frame-based sensors, event-based visual sensors output information through spikes at a high temporal resolution. By only encoding changes in pixel intensity, they showcase a low-power consuming, low-latency approach to visual information sensing. To use this information for higher sensory tasks like object recognition and tracking, an essential simplification step is the extraction and learning of features. An ideal feature descriptor must be robust to changes involving (i) local transformations and (ii) re-appearances of a local event pattern. To that end, we propose a novel spatiotemporal feature representation learning algorithm based on slow feature analysis (SFA). Using SFA, smoothly changing linear projections are learnt which are robust to local visual transformations. In order to determine if the features can learn to be invariant to various visual transformations, feature point tracking tasks are used for evaluation. Extensive experiments across two datasets demonstrate the adaptability of the spatiotemporal feature learner to translation, scaling and rotational transformations of the feature points. More importantly, we find that the obtained feature representations are able to exploit the high temporal resolution of such event-based cameras in generating better feature tracks.
Visual SLAM shows significant progress in recent years due to high attention from vision community but still, challenges remain for low-textured environments. Feature based visual SLAMs do not produce reliable camera and structure estimates due to insufficient features in a low-textured environment. Moreover, existing visual SLAMs produce partial reconstruction when the number of 3D-2D correspondences is insufficient for incremental camera estimation using bundle adjustment. This paper presents Edge SLAM, a feature based monocular visual SLAM which mitigates the above mentioned problems. Our proposed Edge SLAM pipeline detects edge points from images and tracks those using optical flow for point correspondence. We further refine these point correspondences using geometrical relationship among three views. Owing to our edge-point tracking, we use a robust method for two-view initialization for bundle adjustment. Our proposed SLAM also identifies the potential situations where estimating a new camera into the existing reconstruction is becoming unreliable and we adopt a novel method to estimate the new camera reliably using a local optimization technique. We present an extensive evaluation of our proposed SLAM pipeline with most popular open datasets and compare with the state-of-the art. Experimental result indicates that our Edge SLAM is robust and works reliably well for both textured and less-textured environment in comparison to existing state-of-the-art SLAMs.
This paper proposes a resilient-backpropagation-neural-network-(Rprop-NN) based algorithm for Photovoltaic (PV) maximum power point tracking (MPPT). A supervision mechanism is proposed to calibrate the Rprop-NN-MPPT reference and limit short-circuit current caused by incorrect prediction. Conventional MPPT algorithms (e.g., perturb and observe (P&O), hill climbing, and incremental conductance (Inc-Cond) etc.) are trial-and-error-based, which may result in steady-state oscillations and loss of tracking direction under fast-changing ambient environment. In addition, partial shading is also a challenge due to the difficulty of finding the global maximum power point on a multi-peak characteristic curve. As an attempt to address the aforementioned issues, a novel Rprop-NN MPPT algorithm is developed and elaborated in this work. Multiple case studies are carried out to verify the effectiveness of the proposed algorithm.
In this paper, a neural-network (NN)-based online optimal control method (NN-OPT) is proposed for ultra-capacitors (UCs) energy storage system (ESS) in hybrid AC/DC microgrids involving multiple distributed generations (e.g., Photovoltaic (PV) system, battery storage, diesel generator). Conventional control strategies usually produce large disturbances to buses during charging and discharging (C&D) processes of UCs, which significantly degrades the power quality and system performance, especially under fast C&D modes. Therefore, the optimal control theory is adopted to optimize the C&D profile as well as to suppress the disturbances caused by UCs implementation. Specifically, an NN-based intelligent algorithm is developed to learn the optimal control policy for bidirectional-converter-interfaced UCs. The inaccuracies of system modeling are also considered in the control design. Since the designed NN-OPT method is decentralized that only requires the local measurements, plug & play of UCs can be easily realized with minimal communication efforts. In addition, the PV system is under the maximum power point tracking (MPPT) control to extract the maximum benefit. Both islanded and grid-tied modes are considered during the controller design. Extensive case studies have been conducted to evaluate the effectiveness of the proposed method.
Motor control is a set of time-varying muscle excitations which generate desired motions for a biomechanical system. Muscle excitations cannot be directly measured from live subjects. An alternative approach is to estimate muscle activations using inverse motion-driven simulation. In this article, we propose a deep reinforcement learning method to estimate the muscle excitations in simulated biomechanical systems. Here, we introduce a custom-made reward function which incentivizes faster point-to-point tracking of target motion. Moreover, we deploy two new techniques, namely, episode-based hard update and dual buffer experience replay, to avoid feedback training loops. The proposed method is tested in four simulated 2D and 3D environments with 6 to 24 axial muscles. The results show that the models were able to learn muscle excitations for given motions after nearly 100,000 simulated steps. Moreover, the root mean square error in point-to-point reaching of the target across experiments was less than 1% of the length of the domain of motion. Our reinforcement learning method is far from the conventional dynamic approaches as the muscle control is derived functionally by a set of distributed neurons. This can open paths for neural activity interpretation of this phenomenon.
Feature or interest points typically use information aggregation in 2D patches which does not remain stable at object boundaries when there is object motion against a significantly varying background. Level or iso-intensity curves are much more stable under such conditions, especially the longer ones. In this paper, we identify stable portions on long iso-curves and detect corners on them. Further, the iso-curve associated with a corner is used to discard portions from the background and improve matching. Such CoMIC (Corners on Maximally-stable Iso-intensity Curves) points yield superior results at the object boundary regions compared to state-of-the-art detectors while performing comparably at the interior regions as well. This is illustrated in exhaustive matching experiments for both boundary and non-boundary regions in applications such as stereo and point tracking for structure from motion in video sequences.
Egocentric videos are characterised by their ability to have the first person view. With the popularity of Google Glass and GoPro, use of egocentric videos is on the rise. Recognizing action of the wearer from egocentric videos is an important problem. Unstructured movement of the camera due to natural head motion of the wearer causes sharp changes in the visual field of the egocentric camera causing many standard third person action recognition techniques to perform poorly on such videos. Objects present in the scene and hand gestures of the wearer are the most important cues for first person action recognition but are difficult to segment and recognize in an egocentric video. We propose a novel representation of the first person actions derived from feature trajectories. The features are simple to compute using standard point tracking and does not assume segmentation of hand/objects or recognizing object or hand pose unlike in many previous approaches. We train a bag of words classifier with the proposed features and report a performance improvement of more than 11% on publicly available datasets. Although not designed for the particular case, we show that our technique can also recognize wearer's actions when hands or objects are not visible.
We present techniques for improving performance driven facial animation, emotion recognition, and facial key-point or landmark prediction using learned identity invariant representations. Established approaches to these problems can work well if sufficient examples and labels for a particular identity are available and factors of variation are highly controlled. However, labeled examples of facial expressions, emotions and key-points for new individuals are difficult and costly to obtain. In this paper we improve the ability of techniques to generalize to new and unseen individuals by explicitly modeling previously seen variations related to identity and expression. We use a weakly-supervised approach in which identity labels are used to learn the different factors of variation linked to identity separately from factors related to expression. We show how probabilistic modeling of these sources of variation allows one to learn identity-invariant representations for expressions which can then be used to identity-normalize various procedures for facial expression analysis and animation control. We also show how to extend the widely used techniques of active appearance models and constrained local models through replacing the underlying point distribution models which are typically constructed using principal component analysis with identity-expression factorized representations. We present a wide variety of experiments in which we consistently improve performance on emotion recognition, markerless performance-driven facial animation and facial key-point tracking.