**39** Publications

- Page
**1**of**2** - Next Page

Comput Biol Med 2021 Apr 30;134:104432. Epub 2021 Apr 30.

Missouri University of Science and Technology, Rolla, MO, 65409, USA. Electronic address:

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1016/j.compbiomed.2021.104432 | DOI Listing |

April 2021

IEEE Trans Neural Netw Learn Syst 2021 Mar 12;PP. Epub 2021 Mar 12.

This article intends to address an online optimal adaptive regulation of nonlinear discrete-time systems in affine form and with partially uncertain dynamics using a multilayer neural network (MNN). The actor-critic framework estimates both the optimal control input and value function. Instantaneous control input error and temporal difference are used to tune the weights of the critic and actor networks, respectively. The selection of the basis functions and their derivatives are not required in the proposed approach. The state vector, critic, and actor NN weights are proven to be bounded using the Lyapunov method. Our approach can be extended to neural networks with an arbitrary number of hidden layers. We have demonstrated our approach via a simulation example.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2021.3061414 | DOI Listing |

March 2021

IEEE Trans Neural Netw Learn Syst 2020 Oct 28;PP. Epub 2020 Oct 28.

This article investigates adaptive robust controller design for discrete-time (DT) affine nonlinear systems using an adaptive dynamic programming. A novel adaptive interleaved reinforcement learning algorithm is developed for finding a robust controller of DT affine nonlinear systems subject to matched or unmatched uncertainties. To this end, the robust control problem is converted into the optimal control problem for nominal systems by selecting an appropriate utility function. The performance evaluation and control policy update combined with neural networks approximation are alternately implemented at each time step for solving a simplified Hamilton-Jacobi-Bellman (HJB) equation such that the uniformly ultimately bounded (UUB) stability of DT affine nonlinear systems can be guaranteed, allowing for all realization of unknown bounded uncertainties. The rigorously theoretical proofs of convergence of the proposed interleaved RL algorithm and UUB stability of uncertain systems are provided. Simulation results are given to verify the effectiveness of the proposed method.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2020.3027653 | DOI Listing |

October 2020

IEEE Trans Neural Netw Learn Syst 2020 Sep 17;PP. Epub 2020 Sep 17.

In this article, a novel learning methodology is introduced for the problem of classification in the context of high-dimensional data. In particular, the challenges introduced by high-dimensional data sets are addressed by formulating a L₁ regularized zero-sum game where optimal sparsity is estimated through a two-player game between the penalty coefficients/sparsity parameters and the deep neural network weights. In order to solve this game, a distributed learning methodology is proposed where additional variables are utilized to derive layerwise cost functions. Finally, an alternating minimization approach developed to solve the problem where the Nash solution provides optimal sparsity and compensation through the classifier. The proposed learning approach is implemented in a parallel and distributed environment through a novel computational algorithm. The efficiency of the approach is demonstrated both theoretically and empirically with nine data sets.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2020.3017434 | DOI Listing |

September 2020

IEEE Trans Cybern 2020 Jul 8;PP. Epub 2020 Jul 8.

In this article, we introduce a novel approximate optimal decentralized control scheme for uncertain input-affine nonlinear-interconnected systems. In the proposed scheme, we design a controller and an event-triggering mechanism (ETM) at each subsystem to optimize a local performance index and reduce redundant control updates, respectively. To this end, we formulate a noncooperative dynamic game at every subsystem in which we collectively model the interconnection inputs and the event-triggering error as adversarial players that deteriorate the subsystem performance and model the control policy as the performance optimizer, competing against these adversarial players. To obtain a solution to this game, one has to solve the associated Hamilton-Jacobi-Isaac (HJI) equation, which does not have a closed-form solution even when the subsystem dynamics are accurately known. In this context, we introduce an event-driven off-policy integral reinforcement learning (OIRL) approach to learn an approximate solution to this HJI equation using artificial neural networks (NNs). We then use this NN approximated solution to design the control policy and event-triggering threshold at each subsystem. In the learning framework, we guarantee the Zeno-free behavior of the ETMs at each subsystem using the exploration policies. Finally, we derive sufficient conditions to guarantee uniform ultimate bounded regulation of the controlled system states and demonstrate the efficacy of the proposed framework with numerical examples.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TCYB.2020.2991166 | DOI Listing |

July 2020

IEEE Trans Neural Netw Learn Syst 2020 Jan 18;31(1):235-245. Epub 2019 Mar 18.

In networked control systems (NCS), a certain class of attacks on the communication network is known to raise traffic flows causing delays and packet losses to increase. This paper presents a novel neural network (NN)-based attack detection and estimation scheme that captures the abnormal traffic flow due to a class of attacks on the communication links within the feedback loop of an NCS. By modeling the unknown network flow as a nonlinear function at the bottleneck node and using a NN observer, the network attack detection residual is defined and utilized to determine the onset of an attack in the communication network when the residual exceeds a predefined threshold. Upon detection, another NN is used to estimate the flow injected by the attack. For the physical system, we develop an attack detection scheme by using an adaptive dynamic programming-based optimal event-triggered NN controller in the presence of network delays and packet losses. Attacks on the network as well as on the sensors of the physical system can be detected and estimated with the proposed scheme. The simulation results confirm theoretical conclusions.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2019.2900430 | DOI Listing |

January 2020

IEEE Trans Neural Netw Learn Syst 2019 06 12;30(6):1651-1658. Epub 2018 Oct 12.

In this paper, adaptive neural networks (NNs) are employed in the event-triggered feedback control framework to enable a robot manipulator to track a predefined trajectory. In the proposed output feedback control scheme, the joint velocities of the robot manipulator are reconstructed using a nonlinear NN observer by using the joint position measurements. Two different configurations are proposed for the implementation of the controller depending on whether the observer is co-located with the sensor or the controller in the feedback control loop. Besides the observer NN, a second NN is utilized to compensate the effects of nonlinearities in the robot dynamics via the feedback control. For both the configurations, by utilizing observer NN and the second NN, torque input is computed by the controller. The Lyapunov stability method is employed to determine the event-triggering condition, weight update rules for the controller, and the observer for both the configurations. The tracking performance of the robot manipulator with the two configurations is analyzed, wherein it is demonstrated that all the signals in the closed-loop system composed of the robotic system, the observer, the event-sampling mechanism, and the controller are locally uniformly ultimately bounded in the presence of bounded disturbance torque. To demonstrate the efficacy of the proposed design, simulation results are presented.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2018.2870661 | DOI Listing |

June 2019

IEEE Trans Neural Netw Learn Syst 2019 May 8;30(5):1512-1522. Epub 2018 Oct 8.

In this paper, approximate optimal distributed control schemes for a class of nonlinear interconnected systems with strong interconnections are presented using continuous and event-sampled feedback information. The optimal control design is formulated as an N -player nonzero-sum game where the control policies of the subsystems act as players. An approximate Nash equilibrium solution to the game, which is the solution to the coupled Hamilton-Jacobi equation, is obtained using the approximate dynamic programming-based approach. A critic neural network (NN) at each subsystem is utilized to approximate the Nash solution and novel event-sampling conditions, that are decentralized, are designed to asynchronously orchestrate the sampling and transmission of state vector at each subsystem. To ensure the local ultimate boundedness of the closed-loop system state and NN parameter estimation errors, a hybrid-learning scheme is introduced and the stability is guaranteed using Lyapunov-based stability analysis. Finally, implementation of the proposed event-based distributed control scheme for linear interconnected systems is discussed. For completeness, Zeno-free behavior of the event-sampled system is shown analytically and a numerical example is included to support the analytical results.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2018.2869896 | DOI Listing |

May 2019

IEEE Trans Cybern 2019 Nov 24;49(11):3923-3933. Epub 2018 Jul 24.

This paper presents novel cooperative tracking control for a class of input-constrained multiagent systems with a dynamic leader. Each follower agent is described by a high-order nonlinear dynamics in strict feedback form with input constraints. Our main contribution lies in presenting a system transformation method that can convert the input-constrained state feedback cooperative tracking control of agents into an unconstrained output feedback control of agents with dynamics in Brunovsky normal form. As a result, the original problem is simplified to be a simple stabilization of the transformed system for the agents. Thus, the use of the backstepping scheme is obviated, and the synthesis and computation are extremely simplified. It is strictly proved that all follower agents can synchronize to the leader with bounded synchronization errors, and all other signals in the closed-loop system are semi-global uniformly ultimately bounded. Finally, numerical analysis is carried out to validate the theoretical results and demonstrate the effectiveness of the proposed approach.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TCYB.2018.2853623 | DOI Listing |

November 2019

IEEE Trans Cybern 2018 Sep 7;48(9):2510-2519. Epub 2017 Sep 7.

In this paper, a distributed control scheme for an interconnected system composed of uncertain input affine nonlinear subsystems with event triggered state feedback is presented by using a novel hybrid learning scheme-based approximate dynamic programming with online exploration. First, an approximate solution to the Hamilton-Jacobi-Bellman equation is generated with event sampled neural network (NN) approximation and subsequently, a near optimal control policy for each subsystem is derived. Artificial NNs are utilized as function approximators to develop a suite of identifiers and learn the dynamics of each subsystem. The NN weight tuning rules for the identifier and event-triggering condition are derived using Lyapunov stability theory. Taking into account, the effects of NN approximation of system dynamics and boot-strapping, a novel NN weight update is presented to approximate the optimal value function. Finally, a novel strategy to incorporate exploration in online control framework, using identifiers, is introduced to reduce the overall cost at the expense of additional computations during the initial online learning phase. System states and the NN weight estimation errors are regulated and local uniformly ultimately bounded results are achieved. The analytical results are substantiated using simulation studies.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TCYB.2017.2741342 | DOI Listing |

September 2018

IEEE Trans Neural Netw Learn Syst 2018 08 29;29(8):3669-3681. Epub 2017 Aug 29.

In this paper, an adaptive dynamic programming-based near optimal boundary controller is developed for partial differential equations (PDEs) modeled by the uncertain Burgers' equation under Neumann boundary condition in 2-D. Initially, Hamilton-Jacobi-Bellman equation is derived in infinite-dimensional space. Subsequently, a novel neural network (NN) identifier is introduced to approximate the nonlinear dynamics in the 2-D PDE. The optimal control input is derived by online estimation of the value function through an additional NN-based forward-in-time estimation and approximated dynamic model. Novel update laws are developed for estimation of the identifier and value function online. The designed control policy can be applied using a finite number of actuators at the boundaries. Local ultimate boundedness of the closed-loop system is studied in detail using Lyapunov theory. Simulation results confirm the optimizing performance of the proposed controller on an unstable 2-D Burgers' equation.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2017.2736786 | DOI Listing |

August 2018

IEEE Trans Cybern 2018 Jul 21;48(7):2001-2011. Epub 2017 Jul 21.

In this paper, a novel adaptive control strategy is presented for the tracking control of a class of multi-input-multioutput uncertain nonlinear systems with external disturbances to place user-defined time-varying constraints on the system state. Our contribution includes a step forward beyond the usual stabilization result to show that the states of the plant converge asymptotically, as well as remain within user-defined time-varying bounds. To achieve the new results, an error transformation technique is first established to generate an equivalent nonlinear system from the original one, whose asymptotic stability guarantees both the satisfaction of the time-varying restrictions and the asymptotic tracking performance of the original system. The uncertainties of the transformed system are overcome by an online neural network (NN) approximator, while the external disturbances and NN reconstruction error are compensated by the robust integral of the sign of the error signal. Via standard Lyapunov method, asymptotic tracking performance is theoretically guaranteed, and all the closed-loop signals are bounded. The requirement for a prior knowledge of bounds of uncertain terms is relaxed. Finally, simulation results demonstrate the merits of the proposed controller.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TCYB.2017.2726039 | DOI Listing |

July 2018

IEEE Trans Neural Netw Learn Syst 2018 07 8;29(7):2846-2856. Epub 2017 Jun 8.

This paper presents an approximate optimal distributed control scheme for a known interconnected system composed of input affine nonlinear subsystems using event-triggered state and output feedback via a novel hybrid learning scheme. First, the cost function for the overall system is redefined as the sum of cost functions of individual subsystems. A distributed optimal control policy for the interconnected system is developed using the optimal value function of each subsystem. To generate the optimal control policy, forward-in-time, neural networks are employed to reconstruct the unknown optimal value function at each subsystem online. In order to retain the advantages of event-triggered feedback for an adaptive optimal controller, a novel hybrid learning scheme is proposed to reduce the convergence time for the learning algorithm. The development is based on the observation that, in the event-triggered feedback, the sampling instants are dynamic and results in variable interevent time. To relax the requirement of entire state measurements, an extended nonlinear observer is designed at each subsystem to recover the system internal states from the measurable feedback. Using a Lyapunov-based analysis, it is demonstrated that the system states and the observer errors remain locally uniformly ultimately bounded and the control policy converges to a neighborhood of the optimal policy. Simulation results are presented to demonstrate the performance of the developed controller.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2017.2693205 | DOI Listing |

July 2018

IEEE Trans Neural Netw Learn Syst 2018 05 11;29(5):1850-1863. Epub 2017 Apr 11.

In this paper, a novel event-triggered implementation of a tracking controller for an uncertain strict-feedback system is presented. Neural networks (NNs) are utilized in the backstepping approach to design a control input by approximating unknown dynamics of the strict-feedback nonlinear system with event-sampled inputs. The system state vector is assumed to be unknown and an NN observer is used to estimate the state vector. By using the estimated state vector and backstepping design approach, an event-sampled controller is introduced. As part of the controller design, first, input-to-state-like stability for a continuously sampled controller that has been injected with bounded measurement errors is demonstrated, and subsequently, an event-execution control law is derived, such that the measurement errors are guaranteed to remain bounded. Lyapunov theory is used to demonstrate that the tracking errors, the observer estimation errors, and the NN weight estimation errors for each NN are locally uniformly ultimately bounded in the presence bounded disturbances, NN reconstruction errors, as well as errors introduced by event sampling. Simulation results are provided to illustrate the effectiveness of the proposed controllers.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2017.2678922 | DOI Listing |

May 2018

IEEE Trans Neural Netw Learn Syst 2018 04 6;29(4):1263-1274. Epub 2017 Mar 6.

In this paper, neurodynamic programming-based output feedback boundary control of distributed parameter systems governed by uncertain coupled semilinear parabolic partial differential equations (PDEs) under Neumann or Dirichlet boundary control conditions is introduced. First, Hamilton-Jacobi-Bellman (HJB) equation is formulated in the original PDE domain and the optimal control policy is derived using the value functional as the solution of the HJB equation. Subsequently, a novel observer is developed to estimate the system states given the uncertain nonlinearity in PDE dynamics and measured outputs. Consequently, the suboptimal boundary control policy is obtained by forward-in-time estimation of the value functional using a neural network (NN)-based online approximator and estimated state vector obtained from the NN observer. Novel adaptive tuning laws in continuous time are proposed for learning the value functional online to satisfy the HJB equation along system trajectories while ensuring the closed-loop stability. Local uniformly ultimate boundedness of the closed-loop system is verified by using Lyapunov theory. The performance of the proposed controller is verified via simulation on an unstable coupled diffusion reaction process.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2017.2669941 | DOI Listing |

April 2018

IEEE Trans Neural Netw Learn Syst 2018 04 2;29(4):1213-1225. Epub 2017 Mar 2.

This paper develops a near optimal boundary control method for distributed parameter systems governed by uncertain linear 1-D parabolic partial differential equations (PDE) by using approximate dynamic programming. A quadratic surface integral is proposed to express the optimal cost functional for the infinite-dimensional state space. Accordingly, the Hamilton-Jacobi-Bellman (HJB) equation is formulated in the infinite-dimensional domain without using any model reduction. Subsequently, a neural network identifier is developed to estimate the unknown spatially varying coefficient in PDE dynamics. Novel tuning law is proposed to guarantee the boundedness of identifier approximation error in the PDE domain. A radial basis network (RBN) is subsequently proposed to generate an approximate solution for the optimal surface kernel function online. The tuning law for near optimal RBN weights is created, such that the HJB equation error is minimized while the dynamics are identified and closed-loop system remains stable. Ultimate boundedness (UB) of the closed-loop system is verified by using the Lyapunov theory. The performance of the proposed controller is successfully confirmed by simulation on an unstable diffusion-reaction process.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2017.2669944 | DOI Listing |

April 2018

IEEE Trans Neural Netw Learn Syst 2017 03 7;28(3):639-652. Epub 2016 Apr 7.

This paper presents an approximate optimal control of nonlinear continuous-time systems in affine form by using the adaptive dynamic programming (ADP) with event-sampled state and input vectors. The knowledge of the system dynamics is relaxed by using a neural network (NN) identifier with event-sampled inputs. The value function, which becomes an approximate solution to the Hamilton-Jacobi-Bellman equation, is generated by using event-sampled NN approximator. Subsequently, the NN identifier and the approximated value function are utilized to obtain the optimal control policy. Both the identifier and value function approximator weights are tuned only at the event-sampled instants leading to an aperiodic update scheme. A novel adaptive event sampling condition is designed to determine the sampling instants, such that the approximation accuracy and the stability are maintained. A positive lower bound on the minimum inter-sample time is guaranteed to avoid accumulation point, and the dependence of inter-sample time upon the NN weight estimates is analyzed. A local ultimate boundedness of the resulting nonlinear impulsive dynamical closed-loop system is shown. Finally, a numerical example is utilized to evaluate the performance of the near-optimal design. The net result is the design of an event-sampled ADP-based controller for nonlinear continuous-time systems.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2016.2539366 | DOI Listing |

March 2017

IEEE Trans Cybern 2017 Feb 11;47(2):425-438. Epub 2016 Feb 11.

In this paper, an event-driven stochastic adaptive dynamic programming (ADP)-based technique is introduced for nonlinear systems with a communication network within its feedback loop. A near optimal control policy is designed using an actor-critic framework and ADP with event sampled state vector. First, the system dynamics are approximated by using a novel neural network (NN) identifier with event sampled state vector. The optimal control policy is generated via an actor NN by using the NN identifier and value function approximated by a critic NN through ADP. The stochastic NN identifier, actor, and critic NN weights are tuned at the event sampled instants leading to aperiodic weight tuning laws. Above all, an adaptive event sampling condition based on estimated NN weights is designed by using the Lyapunov technique to ensure ultimate boundedness of all the closed-loop signals along with the approximation accuracy. The net result is event-driven stochastic ADP technique that can significantly reduce the computation and network transmissions. Finally, the analytical design is substantiated with simulation results.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TCYB.2016.2519445 | DOI Listing |

February 2017

IEEE Trans Neural Netw Learn Syst 2016 Jan 26;27(1):151-64. Epub 2015 Oct 26.

This paper presents a novel adaptive neural network (NN) control of single-input and single-output uncertain nonlinear discrete-time systems under event sampled NN inputs. In this control scheme, the feedback signals are transmitted, and the NN weights are tuned in an aperiodic manner at the event sampled instants. After reviewing the NN approximation property with event sampled inputs, an adaptive state estimator (SE), consisting of linearly parameterized NNs, is utilized to approximate the unknown system dynamics in an event sampled context. The SE is viewed as a model and its approximated dynamics and the state vector, during any two events, are utilized for the event-triggered controller design. An adaptive event-trigger condition is derived by using both the estimated NN weights and a dead-zone operator to determine the event sampling instants. This condition both facilitates the NN approximation and reduces the transmission of feedback signals. The ultimate boundedness of both the NN weight estimation error and the system state vector is demonstrated through the Lyapunov approach. As expected, during an initial online learning phase, events are observed more frequently. Over time with the convergence of the NN weights, the inter-event times increase, thereby lowering the number of triggered events. These claims are illustrated through the simulation results.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2015.2472290 | DOI Listing |

January 2016

IEEE Trans Neural Netw Learn Syst 2015 Dec 1;26(12):3278-86. Epub 2015 Sep 1.

This paper presents a novel state-feedback control scheme for the tracking control of a class of multi-input multioutput continuous-time nonlinear systems with unknown dynamics and bounded disturbances. First, the control law consisting of the robust integral of a neural network (NN) output plus sign of the tracking error feedback multiplied with an adaptive gain is introduced. The NN in the control law learns the system dynamics in an online manner, while the NN residual reconstruction errors and the bounded disturbances are overcome by the error sign signal. Since both of the NN output and the error sign signal are included in the integral, the continuity of the control input is ensured. The controller structure and the NN weight update law are novel in contrast with the previous effort, and the semiglobal asymptotic tracking performance is still guaranteed by using the Lyapunov analysis. In addition, the NN weights and all other signals are proved to be bounded simultaneously. The proposed approach also relaxes the need for the upper bounds of certain terms, which are usually required in the previous designs. Finally, the theoretical results are substantiated with simulations.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2015.2470175 | DOI Listing |

December 2015

IEEE Trans Neural Netw Learn Syst 2016 09 13;27(9):1801-15. Epub 2015 Aug 13.

This paper presents an event-triggered near optimal control of uncertain nonlinear discrete-time systems. Event-driven neurodynamic programming (NDP) is utilized to design the control policy. A neural network (NN)-based identifier, with event-based state and input vectors, is utilized to learn the system dynamics. An actor-critic framework is used to learn the cost function and the optimal control input. The NN weights of the identifier, the critic, and the actor NNs are tuned aperiodically once every triggered instant. An adaptive event-trigger condition to decide the trigger instants is derived. Thus, a suitable number of events are generated to ensure a desired accuracy of approximation. A near optimal performance is achieved without using value and/or policy iterations. A detailed analysis of nontrivial inter-event times with an explicit formula to show the reduction in computation is also derived. The Lyapunov technique is used in conjunction with the event-trigger condition to guarantee the ultimate boundedness of the closed-loop system. The simulation results are included to verify the performance of the controller. The net result is the development of event-driven NDP.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2015.2453320 | DOI Listing |

September 2016

IEEE Trans Neural Netw Learn Syst 2015 Oct 23;26(10):2535-49. Epub 2015 Jun 23.

This paper proposes a novel optimal tracking control scheme for nonlinear continuous-time systems in strict-feedback form with uncertain dynamics. The optimal tracking problem is transformed into an equivalent optimal regulation problem through a feedforward adaptive control input that is generated by modifying the standard backstepping technique. Subsequently, a neural network-based optimal control scheme is introduced to estimate the cost, or value function, over an infinite horizon for the resulting nonlinear continuous-time systems in affine form when the internal dynamics are unknown. The estimated cost function is then used to obtain the optimal feedback control input; therefore, the overall optimal control input for the nonlinear continuous-time system in strict-feedback form includes the feedforward plus the optimal feedback terms. It is shown that the estimated cost function minimizes the Hamilton-Jacobi-Bellman estimation error in a forward-in-time manner without using any value or policy iterations. Finally, optimal output feedback control is introduced through the design of a suitable observer. Lyapunov theory is utilized to show the overall stability of the proposed schemes without requiring an initial admissible controller. Simulation examples are provided to validate the theoretical results.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2015.2441712 | DOI Listing |

October 2015

IEEE Trans Neural Netw Learn Syst 2016 Mar 14;27(3):497-509. Epub 2015 Apr 14.

This paper presents a novel approximation-based event-triggered control of multi-input multi-output uncertain nonlinear continuous-time systems in affine form. The controller is approximated using a linearly parameterized neural network (NN) in the context of event-based sampling. After revisiting the NN approximation property in the context of event-based sampling, an event-triggered condition is proposed using the Lyapunov technique to reduce the network resource utilization and to generate the required number of events for the NN approximation. In addition, a novel weight update law for aperiodic tuning of the NN weights at triggered instants is proposed to relax the knowledge of complete system dynamics and to reduce the computation when compared with the traditional NN-based control. Nonetheless, a nonzero positive lower bound for the inter-event times is guaranteed to avoid the accumulation of events or Zeno behavior. For analyzing the stability, the event-triggered system is modeled as a nonlinear impulsive dynamical system and the Lyapunov technique is used to show local ultimate boundedness of all signals. Furthermore, in order to overcome the unnecessary triggered events when the system states are inside the ultimate bound, a dead-zone operator is used to reset the event-trigger errors to zero. Finally, the analytical design is substantiated with numerical results.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2015.2416259 | DOI Listing |

March 2016

IEEE Trans Neural Netw Learn Syst 2015 Aug 18;26(8):1776-88. Epub 2015 Mar 18.

The output feedback-based near-optimal regulation of uncertain and quantized nonlinear discrete-time systems in affine form with control constraint over finite horizon is addressed in this paper. First, the effect of input constraint is handled using a nonquadratic cost functional. Next, a neural network (NN)-based Luenberger observer is proposed to reconstruct both the system states and the control coefficient matrix so that a separate identifier is not needed. Then, approximate dynamic programming-based actor-critic framework is utilized to approximate the time-varying solution of the Hamilton-Jacobi-Bellman using NNs with constant weights and time-dependent activation functions. A new error term is defined and incorporated in the NN update law so that the terminal constraint error is also minimized over time. Finally, a novel dynamic quantizer for the control inputs with adaptive step size is designed to eliminate the quantization error overtime, thus overcoming the drawback of the traditional uniform quantizer. The proposed scheme functions in a forward-in-time manner without offline training phase. Lyapunov analysis is used to investigate the stability. Simulation results are given to show the effectiveness and feasibility of the proposed method.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2015.2409301 | DOI Listing |

August 2015

IEEE Trans Neural Netw Learn Syst 2015 Mar;26(3):486-99

In this paper, the finite-horizon optimal control design for nonlinear discrete-time systems in affine form is presented. In contrast with the traditional approximate dynamic programming methodology, which requires at least partial knowledge of the system dynamics, in this paper, the complete system dynamics are relaxed utilizing a neural network (NN)-based identifier to learn the control coefficient matrix. The identifier is then used together with the actor-critic-based scheme to learn the time-varying solution, referred to as the value function, of the Hamilton-Jacobi-Bellman (HJB) equation in an online and forward-in-time manner. Since the solution of HJB is time-varying, NNs with constant weights and time-varying activation functions are considered. To properly satisfy the terminal constraint, an additional error term is incorporated in the novel update law such that the terminal constraint error is also minimized over time. Policy and/or value iterations are not needed and the NN weights are updated once a sampling instant. The uniform ultimate boundedness of the closed-loop system is verified by standard Lyapunov stability theory under nonautonomous analysis. Numerical examples are provided to illustrate the effectiveness of the proposed method.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2014.2315646 | DOI Listing |

March 2015

IEEE Trans Neural Netw Learn Syst 2015 Mar;26(3):472-85

The stochastic optimal control of nonlinear networked control systems (NNCSs) using neuro-dynamic programming (NDP) over a finite time horizon is a challenging problem due to terminal constraints, system uncertainties, and unknown network imperfections, such as network-induced delays and packet losses. Since the traditional iteration or time-based infinite horizon NDP schemes are unsuitable for NNCS with terminal constraints, a novel time-based NDP scheme is developed to solve finite horizon optimal control of NNCS by mitigating the above-mentioned challenges. First, an online neural network (NN) identifier is introduced to approximate the control coefficient matrix that is subsequently utilized in conjunction with the critic and actor NNs to determine a time-based stochastic optimal control input over finite horizon in a forward-in-time and online manner. Eventually, Lyapunov theory is used to show that all closed-loop signals and NN weights are uniformly ultimately bounded with ultimate bounds being a function of initial conditions and final time. Moreover, the approximated control input converges close to optimal value within finite time. The simulation results are included to show the effectiveness of the proposed scheme.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2014.2315622 | DOI Listing |

March 2015

IEEE Trans Neural Netw Learn Syst 2013 Jul;24(7):1061-73

Helicopter unmanned aerial vehicles (UAVs) are widely used for both military and civilian operations. Because the helicopter UAVs are underactuated nonlinear mechanical systems, high-performance controller design for them presents a challenge. This paper introduces an optimal controller design via an output feedback for trajectory tracking of a helicopter UAV, using a neural network (NN). The output-feedback control system utilizes the backstepping methodology, employing kinematic and dynamic controllers and an NN observer. The online approximator-based dynamic controller learns the infinite-horizon Hamilton-Jacobi-Bellman equation in continuous time and calculates the corresponding optimal control input by minimizing a cost function, forward-in-time, without using the value and policy iterations. Optimal tracking is accomplished by using a single NN utilized for the cost function approximation. The overall closed-loop system stability is demonstrated using Lyapunov analysis. Finally, simulation results are provided to demonstrate the effectiveness of the proposed control design for trajectory tracking.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2013.2251747 | DOI Listing |

July 2013

IEEE Trans Neural Netw Learn Syst 2014 May;25(5):908-19

Measured data or states for a nonlinear dynamic system is usually contaminated by outliers. Identifying and removing outliers will make the data (or system states) more trustworthy and reliable since outliers in the measured data (or states) can cause missed or false alarms during fault diagnosis. In addition, faults can make the system states nonstationary needing a novel analytical model-based fault detection (FD) framework. In this paper, an online outlier identification and removal (OIR) scheme is proposed for a nonlinear dynamic system. Since the dynamics of the system can experience unknown changes due to faults, traditional observer-based techniques cannot be used to remove the outliers. The OIR scheme uses a neural network (NN) to estimate the actual system states from measured system states involving outliers. With this method, the outlier detection is performed online at each time instant by finding the difference between the estimated and the measured states and comparing its median with its standard deviation over a moving time window. The NN weight update law in OIR is designed such that the detected outliers will have no effect on the state estimation, which is subsequently used for model-based fault diagnosis. In addition, since the OIR estimator cannot distinguish between the faulty or healthy operating conditions, a separate model-based observer is designed for fault diagnosis, which uses the OIR scheme as a preprocessing unit to improve the FD performance. The stability analysis of both OIR and fault diagnosis schemes are introduced. Finally, a three-tank benchmarking system and a simple linear system are used to verify the proposed scheme in simulations, and then the scheme is applied on an axial piston pump testbed. The scheme can be applied to nonlinear systems whose dynamics and underlying distribution of states are subjected to change due to both unknown faults and operating conditions.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2013.2283456 | DOI Listing |

May 2014

IEEE Trans Neural Netw Learn Syst 2013 Mar;24(3):471-84

The stochastic optimal controller design for the nonlinear networked control system (NNCS) with uncertain system dynamics is a challenging problem due to the presence of both system nonlinearities and communication network imperfections, such as random delays and packet losses, which are not unknown a priori. In the recent literature, neuro dynamic programming (NDP) techniques, based on value and policy iterations, have been widely reported to solve the optimal control of general affine nonlinear systems. However, for realtime control, value and policy iterations-based methodology are not suitable and time-based NDP techniques are preferred. In addition, output feedback-based controller designs are preferred for implementation. Therefore, in this paper, a novel NNCS representation incorporating the system uncertainties and network imperfections is introduced first by using input and output measurements for facilitating output feedback. Then, an online neural network (NN) identifier is introduced to estimate the control coefficient matrix, which is subsequently utilized for the controller design. Subsequently, the critic and action NNs are employed along with the NN identifier to determine the forward-in-time, time-based stochastic optimal control of NNCS without using value and policy iterations. Here, the value function and control inputs are updated once a sampling instant. By using novel NN weight update laws, Lyapunov theory is used to show that all the closed-loop signals and NN weights are uniformly ultimately bounded in the mean while the approximated control input converges close to its target value with time. Simulation results are included to show the effectiveness of the proposed scheme.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2012.2234133 | DOI Listing |

March 2013

IEEE Trans Neural Netw Learn Syst 2012 Jul;23(7):1118-29

In this paper, the Hamilton-Jacobi-Bellman equation is solved forward-in-time for the optimal control of a class of general affine nonlinear discrete-time systems without using value and policy iterations. The proposed approach, referred to as adaptive dynamic programming, uses two neural networks (NNs), to solve the infinite horizon optimal regulation control of affine nonlinear discrete-time systems in the presence of unknown internal dynamics and a known control coefficient matrix. One NN approximates the cost function and is referred to as the critic NN, while the second NN generates the control input and is referred to as the action NN. The cost function and policy are updated once at the sampling instant and thus the proposed approach can be referred to as time-based ADP. Novel update laws for tuning the unknown weights of the NNs online are derived. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded and that the approximated control signal approaches the optimal control input with small bounded error over time. In the absence of disturbances, an optimal control is demonstrated. Simulation results are included to show the effectiveness of the approach. The end result is the systematic design of an optimal controller with guaranteed convergence that is suitable for hardware implementation.

## Download full-text PDF |
Source |
---|---|

http://dx.doi.org/10.1109/TNNLS.2012.2196708 | DOI Listing |

July 2012