Model-based shared control approach for a power wheelchair driving assistance system using a force feedback joystick

This paper presents a haptic-based assistance system (AS) for power wheelchair users designed using the model-based shared control approach. The idea is to combine robust control with a high-level driving supervisor in order to successfully share control authority between the wheelchair user and the assistance system. This shared control strategy is composed of two parts, namely an operational part and a tactical part. Through the haptic joystick interface, this assistance system aims to reduce user’s effort when manipulating the joystick, guide the user to avoid any potential collisions, and maintain the active participation of the user in driving the wheelchair. In the operational part, an optimal Takagi-Sugeno fuzzy control approach is proposed to deal with the time-varying user’s intention represented by his desired longitudinal and angular position errors and velocities and hand torques. The control design is formulated as an LMI optimization problem which can be easily solved with numerical solvers. Two unknown input observers for Takagi-Sugeno fuzzy model have been designed to estimate the user’s intention in order to generate an assistance torque via a haptic force feedback joystick. The control supervisor in the tactical part, aims to provide a decision-making signal which allows for the conflict management based on the user hand torque estimation. A specific algorithm has been developed to solve the conflict between the user’s desired actions and the suggestions from the assistance system to ensure the user remains the final decision-maker. Experimental results show the effectiveness and the validity of the proposed assistance system.


Introduction
Commercialized since the 1950s (Woods and Watson, 2003), power wheelchairs now play an important role in the daily life of people with limited mobility. Generally, users drive the wheelchair by manipulating a joystick (or other input device) and the controller converts the joystick commands into the voltages applied to the electric motors to move the wheelchair.

Related studies on power wheelchair assistance system
There is much research on improving user safety and comfort. Wheelchairs can be equipped with many sensors/actuators and different interfaces to increase the situation awareness of users and assist them to avoid any potential accidents or even replace them to perform many different tasks (Simpson, 2009).
Users with severe impairments are often provided with a system that allows them to choose the desired destination/task. The system then performs the latter as an intelligent actuator (Iturrate et al., 2009;Leishman et al., 2010;Carlson and DelMillan, 2013;Carrington et al., 2014;Jain and Argall, 2014). To enhance user comfort during these autonomous navigation tasks, wheelchair acceleration and jerking have been considered in the synthesis of path-following controllers of autonomous wheelchairs (Park and Kuipers, 2011;Morales et al., 2013;Nguyen et al., 2018).
For another group of users with better wheelchair control skills, the authors in Demeester et al. (2010) found that the autonomous functions may be irritating because the assistance is too safe for them. A comprehensive survey in Viswanathan et al. (2017) presents an overall picture of the needs and desires of wheelchair users. The findings suggest that the AS should be customizable to the specific problems of each individual. They also note that users "preferred to drive themselves rather than to be driven" or "preferred the semi-autonomous mode of control over fully autonomous mode, regardless of their cognitive status". Conflict issues were observed when the user wanted to go "his/her own way" and the AS let them get closer to the obstacles, if necessary. Besides that, users preferred to maintain the wheelchair speed and obtain steering assistance to avoid the obstacles without being stopped. Another suggestion is that the human-machine interaction performed with various feedback interfaces can be employed to increase user situation awareness. However, "communication interfaces must be carefully designed" to reduce user's workload, as user sensory channels are heavily loaded during the driving task. The study cited above suggests that assistance with both speed and direction may provide users with "a higher level of perceived control of the wheelchair". These findings confirm the central role of users in designing assistive systems for power wheelchairs where "the most important design aim should be to maximize and augment the skills of the user, not replace them" (Nisbet, 2002).
To satisfy the above-mentioned user preferences, the so-called "shared control" scheme has been proposed. The authors in Vander Poorten et al. (2012) provide the haptic forces for avoiding obstacles using the kinematic properties and the dimensions of the wheelchair. In Devigne et al. (2016), Devigne et al. (2018), the haptic forces are provided thanks to the allowed and forbidden areas in the joystick's velocity map relating to the obstacles detected. This approach was also extended in Devigne et al. (2019) to assist the wheelchair user in avoiding negative obstacles that occur when the ground drops off and thus evade a falling or tipping hazard for the wheelchair. The number of collisions, the travel time, and the joystick motions were reduced by adapting the Vector Field Histogram method to design the haptic force feedback in Morere et al. (2015).
Combining a standard joystick with a wearable vibrotactile armband device may allow users to overrule the guidance of the AS Devigne et al. (2019). However, guidance for the obstacle avoidance function was not successful because the participant found uncomfortable with the sustained vibration. The researchers in Kucukyilmaz and Demiris (2018) proposed a Gaussian process regression model to mimic the haptic force provided by human remote assistance. One important finding is that the efforts of the user in manipulating the joystick were reduced with haptic feedback. This suggests that an AS should continuously provide haptic forces not only for obstacle avoidance but also to reduce user effort.

Proposed methodology and contribution
The above review has shown that most of the existing power wheelchair AS using a haptic joystick address several aspects of the AS such as obstacle avoidance and haptic interaction. However, none of them provide a general method to design a power wheelchair AS. In this paper, we propose a model-based shared control approach, which is a systematic method, to design an AS with a haptic force feedback joystick for a power wheelchair. This new approach allows to consider such an interaction through a user's hand-joystick model introduced to represent the dynamics of the user-joystick-wheelchair system. Hence, the assistance actions can be computed according to the user's real-time behaviors. This proposed user's intention-based shared control approach for a power wheelchair, offers an effective solution to integrate the user's difficulties at the neuromuscular system in the driving assistance system design and allows to personalize the assistance performance according to the user's preferences. Figure 1 presents the overall system structure of the proposed model-based shared control approach for a power wheelchair driving assistance system using a force feedback joystick. Firstly, the human factors are considered in an augmented dynamic model of a userjoystick-wheelchair system. The wheelchair user model in the navigation task consists of a path planner, a path-following controller, and a neuromuscular model.
Secondly, the proposed AS consists of a conflict management function and two main operating modes: Normal Driving Assistance (NDA) and Obstacle Avoidance Assistance (OAA). The NDA operating mode is used when no risk of collision is detected. From the measurable states of the joystick and the wheelchair, the AS predicts the intentions of the user using Unknown Input Observers (UIO) and then assists the user in achieving their goal by applying haptic torques to the joystick thanks to the Longitudinal and Angular controllers, L-Assist and A-Assist, respectively. This haptic assistance reduces user effort when manipulating the joystick and, therefore, improves user comfort.
When the risk detection unit detects a risk of collision, the OAA operating mode is activated. The AS finds an optimal, collision-free path (thanks to the path planning unit) that is not too far from the actual path of the wheelchair. The path-following controller in OAA operating mode makes the wheelchair follow this optimal path like an autonomous robot by applying haptic torques to the joystick to modify the speed and direction of the wheelchair. This action also aims to intuitively inform the user about the risk of collision, suggest an alternative path, and guide the user to avoid the obstacle. If the user agrees with the suggestions of the AS, they work together until the risk is eliminated. On the other hand, if the user does not agree with the system suggestions (expressed through user's hand torques applied to the joystick which are estimated by the UIO), the conflict management unit will switch AS to NDA operating mode to give the user the possibility to override the assistance system, resolve conflicts and thus improve the situation awareness for navigation safety and the acceptability of the system. Extending our preliminary results from (Nguyen et al., 2020) in which the augmented dynamic model of the user-joystick-wheelchair system was developed, this paper presents the design steps of a Frontiers in Control Engineering frontiersin.org 02 completed AS (as shown in Figure 1) and includes extensive experimental results with the Human-Hardware-in-the-loop simulation to validate our proposed approach. The main contributions of this paper are summarized as follows.
1. A new model-based shared control approach for the design of an assistance system with a haptic force feedback control authority allocation between the power wheelchair user and the assistance system is proposed. 2. The proposed approach considers a user's driving model interacting with his joystick and includes information about the user's real-time intentions in the control design process, which offers an effective solution for the shared control issue, especially the personalization of the assistance system and conflicting situations avoidance. 3. The control design is based on a user-in-the-loop wheelchair system by combining robust control (operational part) with a high-level driving supervisor (tactical part) to improve the mutual user-automation understanding and to enhance the user situation awareness, especially during obstacle avoidance. 4. The consideration of the whole user-joystick-wheelchair system is particularly interesting from the point of view of the automatic control design. The closed-loop stability of the whole user-joystickwheelchair system can be rigorously guaranteed using the Lyapunov stability argument. 5. The proposed new shared control and the user's intention estimation approaches have been successfully validated with a human-hardware-in-the-loop simulator.
The remainder of this paper is organized as follows. The next section summarizes the development of the dynamic user-joystickwheelchair model. Based on this whole model, the design of the shared control-based power wheelchair driving assistance system is presented in Section 3 and Section 4. Then, the human-hardware-in-the-loop validation results are presented in Section 5. Finally, some conclusions and future works are presented in the last section.

User-joystick-wheelchair dynamic model
This section summarizes the development of the dynamic userjoystick-wheelchair model presented in Nguyen et al. (2020) 2.1 User-joystick-wheelchair modeling The power wheelchair is modelled as a unicycle robot, as shown in Figure 2, and its movements can be described by the following system under the no-slip assumption where (x, y, φ) is the actual position of the wheelchair in the global coordinate system. v, ω are the actual longitudinal and angular velocities of the wheelchair, respectively. At each moment, the users determine the desired trajectory C which they want to track in the navigation task. To represent user intentions, we introduced a point P (x P , y P , φ P ) along the path C representing the desired wheelchair positions, together with the desired longitudinal and angular velocities (v r , ω r ) as shown in Figure 2. The position errors of point P in the wheelchair body coordinate system are given by As shown in Figure 2, e 1 represents the look-ahead distance which is the distance between the wheelchair center of gravity CG and the desired position P in the longitudinal direction. e 2 and e 3 are the lateral and angular errors, respectively, which the user tries to minimize. Therefore (e 1 , e 2 , e 3 ) and (v r , ω r ) can be interpreted as the intention of the user in the next few moments, in terms of position and orientation.
To follow the desired path at the desired velocity v r , the user applies longitudinal and angular muscular torques, T h,l , T h,a respectively, to the joystick. The neuromuscular dynamics of these forces can be considered as a first-order transfer function  as follows where i = l, a denotes the force of the user's hand acting on the joystick for the longitudinal and lateral components. τ m is the time constant representing the user's processing time delay for transmitting information and the time reaction of the muscles. T h,i,ref are the desired torques of the user which can be modelled as the proportional-integral controller to minimize the velocity, lateral and angular errors as follows T h,a,ref k re2 e 2 + k re3 e 3 where e v = v r − v is the velocity error. k p and k i are the parameters representing how the user reacts to the desired velocity. k re2 and k re3 are the parameters representing the effort of the user to change the wheelchair direction. These parameters allow to simulate the user driving behaviour during navigation task. Frontiers in Control Engineering frontiersin.org The hand-joystick dynamic model is represented by a secondorder system (Marouf et al., 2017), as in (6). It should be noted that the joystick can rotate about its two axes (x − axis and y − axis) which are in the same direction as the body coordinate system of the wheelchair as shown in Figure 2.
where θ i (i = l, a) are the rotation angles of the joystick. J i , K i , B i are the equivalent moment of inertia, stiffness, and viscosity of the hand-joystick system, respectively. T a,a , T a,l are the assistance haptic torques applied to the joystick in both directions. The dynamics of the wheelchair can be linked to the joystick angles as first-order systems as follows where k yv , k xω are the coefficients related to the power amplifier characteristics of the joystick and the low-level controller. τ yv , τ xω are time constants of the dynamic response of the wheelchair.

State-space representation of the userjoystick-wheelchair model
The dynamics of the lateral position and linear speed errors can be described by For the sake of simplicity, we assume that _ v r 0 and e 3 is small (sin e 3 ≈ e 3 ). The dynamic model of the user-joystick-wheelchair system can be separated into two parts: longitudinal and angular. From (9), (7) and (3), the longitudinal dynamics can be represented in the following form is the system state vector and the system matrices are given by Kinematic model of the power wheelchair and joystick coordinate system with the applied torques.

FIGURE 3
Path planning with the multiple choices.

FIGURE 4
The detailed algorithm in the conflict management block contains four variables (isconflict, iscollis, solve_conflict and risk.).

Frontiers in Control Engineering frontiersin.org
In this model, the desired velocity v r of the user is considered as the unknown input and needs to be estimated, whereas the assistance torque T a,l is the control input. Only the joystick position θ l measurement is available in the real wheelchair system and therefore the output matrix is given as C l [ 0 0 1 0 0 ]. The angular part of this human-wheelchair model has a similar form where x a [ e 2 e 3 ω θ a _ θ a T h,a ] T is the system state vector and the system matrices are given by: In this model, the desired rotation speed of the user ω r is considered as the unknown input whereas the assistance torque T a,a is the control input. For this system, only the joystick position θ a and the real yaw rate ω can be directly measured and therefore, the output matrix is C a 0 0 1 0 0 0 0 0 0 1 0 0 .The longitudinal dynamics (10) is a linear time invariant (LTI) system whereas the angular dynamics (11) is non-linear with the variant parameters v r , e 1 in the system matrix A a .
In the next section, we present the Tagaki-Sugeno fuzzy model representing this system under the sum of multiple linear subsystems to deal with this non-linearity and then we will present the appropriate observers and controllers designed for our proposed assistance system.

Shared control for a power wheelchair driving assistance system
This section details the control design of the proposed shared control-based assistance system for power wheelchairs where the overall architecture is represented in Figure 1. Two unknown input observers for the longitudinal (L-UIO) and angular (A-UIO) system dynamics (respectively represented by (10); (11)), were designed to estimate simultaneously the state variables as well as the unknown inputs. The L-UIO estimates the state vector x l and the user desired velocity v r , considered as an unknown input, and the A-UIO estimates the state vector x a and the user desired yaw rate ω r , considered as an unknown input. Two controllers, L-Assist and A-Assist were designed using models (10) and (11) to provide the assistance torques T a,l and T a,a applied to the joystick.
L-Assist auto and A-Assist auto are two parts of a path-following controller that drive the wheelchair along the optimal path like an autonomous robot by providing the assistance torques T a,l and T a,a in OAA operating mode. They are designed based on the reduced versions of models (10) and (11) in which the user behavior model Virtual reality simulator for an electric wheelchair including a virtual scenario, a force feedback joystick, observers/controllers, and electronic boards.

FIGURE 6
Simulation-experiment scenario. The green dot-line is the predefined reference path and the white square represents the wheelchair.

Frontiers in Control Engineering
frontiersin.org 05 is not integrated. We use the same symbols T a,i , i = l, a for the assistance torques in both operating modes. The conflict management function makes decisions about the assistance torques to be applied to the joystick, based on the risk of collision and conflict between the user and the AS. The conflict is detected by evaluating the estimated hand torquesT h,i , i l, a and the assistance torques T a,auto as in Figure 1.T h,i are estimated using the user hand torques observer block, which is also designed using the unknown input observers based on the hand-joystick dynamic models described in (6).
In the following, the unknown input proportional-integral (PI) observer for the Tagaki-Sugeno (T-S) fuzzy models to estimate the user intention and the system states is Firstly presented. Then the LMI-based fuzzy optimal shared control approach to provide assistance torques on the joystick is given. Finally, the application of these approaches to the user-joystick-wheelchair dynamic system is presented.

T-S fuzzy observers design for User's intention estimation
3.1.1 LMI-based unknown input observer design Consider the following continuous-time T-S fuzzy system described by where x(t) ∈ R nx is the system state, u(t) ∈ R nu is the control input, y(t) ∈ R nq is the vector of outputs, z(t) ∈ R nz is the vector of performance outputs, θ ∈ R n θ is the vector of the premise variable and f(t) is the unknown input vector. The system matrices of . . , r} represent the set of r local linear subsystems. The scalar membership functions η i (θ) satisfy the following convex sum property To estimate simultaneously the states and unknown inputs of system 12, the Proportional-Integral observer for the T-S fuzzy model is proposed as follows (Ichalal et al., 2009) wherex,f andŷ are the estimated vectors of the system state, unknown input, and outputs, respectively. The vector of the premise variables θ is supposed to be measurable. The observer's gain matrices K Pi , K Ii need to be designed to make the estimation errors converge to zero. Suppose that the unknown input f(t) in 12 varies slowly ( _ f(t) ≈ 0), then an augmented model can be deduced from 12 as follow Similarly, observer 14 can be rewritten in augmented form as follows The design of this observer must not only ensure asymptotic stability but also satisfy the performance measures. In engineering practice, the most well-known criteria is the convergence rate of the observer (or decay rate of the estimation error). The observer design conditions are formulated in the following theorem. Theorem 1. Tanaka et al. (1998) Given positive scalar α > 0. The decay rate of system 17 is at least α if there exists a matrix P o P T o > 0 and M i , i = {1, . . . , r} satisfy the following LMI conditions: where H(X) X + X T , i, j {1, . . . , r}, j > i. Then the observer's gain matrices can be found by

User intention estimation
In this paper, user intention is defined by the position errors e 2 , e 3 and desired speeds ω r , v r . e 2 , e 3 and ω r are considered in angular dynamic model 11 which is non-linear with two time-varying parameters v r , e 1 . T-S model 12 has been adapted to represent model (11) with v r as the unique premise variable whereas e 1 is considered as a constant. As v r cannot be measured directly, the membership functions were calculated based on the estimated valuev r from the L-UIO block in Figure 1 which is bounded v min ≤v r ≤v max withv min 0.05m/s andv max 0.3m/s. The membership functions are Theorem 1 can now be applied to the T-S model of the user-joystickwheelchair system 12 to design the Fuzzy unknown input observer (14) for the A-UIO block. The L-UIO block estimates the desired velocity v r and the states of the longitudinal dynamic system 10. This is a linear time-invariant system, a special case of fuzzy model 12 with r = 1. Therefore, Theorem 1 can be applied to design the L-UIO block.

FIGURE 9
Estimation of joystick angles where the observers play the role of filters.

FIGURE 8
Simulation and experimental results of the four tests to identify the parameters of the rotational part of the user model.

Frontiers in Control Engineering
frontiersin.org 3.2 LMI-based fuzzy optimal shared control design

Control problem formulation
For the controller design, we consider the fuzzy state feedback controller which is designed based on the following parallel distributed compensation (PDC) law where L i is the feedback gain matrix corresponding to ith subsystem and the Lyapunov function We consider the following control problem.

Problem
Find the feedback gains matrices L i and matrix P c such that the controller (21) stabilizes system (12) while minimizing the performance index: along the solution of (12), where Q = Q T > 0, R = R T > 0 are the weighting matrices with appropriate dimensions which should be appropriately chosen for shared control purposes.

Optimal controller design
The following theorem provides the LMI-based conditions for designing a state feedback controller to solve the above control problem. Theorem 2. Given positive scalar γ > 0 and two matrices Q, R with appropriate dimensions. If there exists a matrix P c P T c > 0 and N i , i 1, . . . , r { }satisfy the following LMI-based optimization: . . , r}, j > i, and the symbol + stands for matrix blocks that can be deduced by symmetry. Then the performance index J will be less than γ and the PDC control law 21 with feedback gain matrices given by solves the above control problem (Li et al., 2000). Theorem 2 is then applied to the above T-S representation of angular dynamic model 11 to design state feedback controller (21) for the A-Assist block. The position errors and user effort should be minimized and therefore the performance matrix can be defined as: The fuzzy optimal control presented in 3.2 is reduced to the wellknown classical LQR control in the case of a linear system. Therefore, it can be applied to design the L-Assist block based on the linear longitudinal dynamic model (10). Remark 1. The LMI conditions in Theorem 2 and Theorem 1 can be solved numerically with available solvers. In this work, the feedback 4 Cooperative control strategy for shared control authority

Driving assistance operating modes
This part presents the design of both Normal Driving Assistance (NDA) and Obstacle Avoidance Assistance (OAA) operating modes.
The NDA operating mode consists of two unknown input observers (L-UIO and A-UIO) and two controllers (L-Assist and A-Assist) as shown in Figure 1. The main objective of this configuration is to estimate the user intention using the observers. The controllers then provide the haptic torques to assist the user in achieving their intentions.
The OAA operating mode is activated when a risk of collision is detected to inform the user about this risk and help them follow collision-free path proposed. This operating mode is deactivated when the risk is eliminated or a conflict between the user and AS is detected.

Conflict detection based on the user's hand torque estimation
One important function of our proposed AS is the conflict management function. It is necessary to estimate the user hand torques to detect conflicts between the user and the AS. We exploit here the joystick dynamics 6) where the assistance torques T a,i , i = l, a are applied to the joystick in both directions and are considered as the known inputs. System 6) is represented by state-space model with the state variable Under the assumption that only the joystick angles θ i are measurable, the PI-unknown input observer in 3.1 can be applied to estimate the user hand torques T h,i , i = l, a which are considered as the unknown inputs.

Path following controller
The main difference between the NDA and OAA operating modes is that latter does not consider the user model but works as an autonomous robot. In other words, the augmented user-joystickwheelchair system is reduced to only a joystick-wheelchair system and user actions are considered as unknown inputs as in 4.1.1. Therefore, to design the A-Assist auto block in OAA operating mode, we considered the reduced versions of the angular dynamic model (11) by setting the parameters k re2 , k re3 to zero and removing the neuromuscular model 3). This leads to reduce the size of the system matrices. However, the nature of the system was conserved, which means that the reduced version of (11) is still a non-linear system with the varying parameters v r , e 1 in the system matrix and the control input is still the angular assistance torque T a,a . Furthermore, the position errors e 2 , e 3 and reference yaw rate ω r are given by the path planning as shown in Figure 1, and do not, therefore, need to be estimated. Thus T-S representation 12 can be applied to this reduced system, where v r is still the premise variable within a range [v rmin v rmax ] and the membership functions are the same as (20). The optimal fuzzy controller can then be designed as in 3.2 to minimize the positions errors e 2 , e 3 with the performance output matrix C zi 1 0 0 0 0 0 1 0 0 0 to provide the assistance torques T a,a in OAA operating mode. A similar approach can be adapted to design the L-Assist auto block. By setting the parameters k p , k i to zeros and removing the neuromuscular dynamics, the longitudinal model (10) is reduced but is always a linear system with the assistance torque T a,l as the control input. The reference velocity v r is given by the path planning block as shown in Figure 1. Therefore the LQR optimal controller in Section 3.2.2 can be applied to design the L-Assist auto providing the assistance torque T a,l .
The human hand torque inputs are not considered in designing the controllers in OAA mode. Generally, it is reasonable to design a robust controller that rejects the effect of noise/disturbance to obtain the desired system performance. However, in this case, the main purpose of the above controllers was not to reject human actions but to communicate the intention of the AS to the user. If the AS rejects the user torque, it can make them feel confused if they suddenly feel a considerable reverse torque. Therefore, the above controller design did not consider the "external disturbance" of human hand torques.

Risk management
As mentioned in Section 1.2, the risk management block must detect the risk of collision and find the optimal alternative solution to avoid the obstacle. This section details each function of this block.

Potential collision detection
In this section, we assume that the obstacle positions and geometry are known beforehand. In a real confined environment, a few centimetres can affect collision or passage in the navigation tasks

FIGURE 11
Estimation of the position errors with and without assistance torques.

Frontiers in Control Engineering
frontiersin.org 09 (Vander Poorten et al., 2012). Therefore, we also consider the wheelchair geometry in detecting potential collisions. A similar approach can be found in Vander Poorten et al. (2012) where the time-to-collision of a predicted circular path was used to formulate the guidance torques applied to the joystick. However, we argue that the users can change their desired trajectories very quickly, within a second. As a result, a collision detected in the next, 5s, for example, has no practical meaning. Therefore, we consider here a "prediction horizon" varying from 1sec to 1.5sec depending on the linear velocity of the wheelchair.
To predict the wheelchair position in the next few seconds, we have two options. The first one is to use the user intention estimation results as in Section 3.1.2. Although this method can estimate the desired position of the user in the next few moments based on the joystick dynamics, this position may not be the future position of the wheelchair. The second option is to use the joystick position, which is roughly proportional to the linear and angular velocities of the wheelchair, and kinematic model 1). Although this method is purely kinematic and neglects the joystick dynamics, it is widely accepted in prior studies (Vander Poorten et al., 2012;Devigne et al., 2016;Devigne et al., 2018;Kucukyilmaz and Demiris, 2018;Devigne et al., 2019). In this paper, this method is used to predict the future position of the wheelchair.
To perform the collision check, we defined all obstacles as connected segments. The wheelchair geometry has been simplified to a rectangular box and the collision check task was achieved by checking the intersections between the future positions of the wheelchair and the known edges of the obstacles.

FIGURE 12
Estimation of user hand torques and assistance torques.

FIGURE 13
Estimation of user hand torques and assistance torques where the parameters could be adapted to user need.

Frontiers in Control Engineering
frontiersin.org

Path planning
Once the potential collision is detected, the AS must find an alternative path. This path must be collision-free and not be too far from the actual future path of the wheelchair so that it is feasible and does not require too much user effort. Figure 3 illustrates our idea about the alternative solutions for obstacle avoidance. With the potential collision of the wheelchair (the red one), there are, for example, two possible substitute solutions. The yellow one is too far from the intention of the user, so the green one is selected as the optimal position because it satisfies both the above requirements. We only consider here obstacles near the wheelchair and hence, the proposed path is only "optimal" in this context. By using the joystick positions which imply (v, ω) pair, potential collisions can be detected as described in Section 4.2.1. The optimal path finding problem can now be reduced to find an optimal (v opt , ω opt ) pair that satisfies the two requirements above. We used here a "scan method" that numerically scans all the possible solutions, within mechanical limitations, of the pair (v, ω) and, minimizes the cost function J q(v opt − v) 2 + r(ω opt − ω) 2 to find the "best" one that is not only a collision-free path but is also the closest to the actual movements of the wheelchair. By varying weighting coefficient pair (q, r) we can satisfy user preference: "maintaining the wheelchair speed and getting steering assistance to avoid the obstacles without being stopped" (Nisbet, 2002). Finally, the output of the path planning function is not only (v opt , ω opt ) but also the new position errors e 2 , e 3 between the optimal and the actual position of the wheelchair (the green and blue ones, respectively, in Figure 3). With the given "prediction horizon" and the pair (v opt , ω opt ), these new position errors e 2 , e 3 can be found using the kinematic model 1).

Conflict management
The objective of this block is to detect any disagreement between the user and the AS expressed through the interaction torques applied to the haptic joystick. This topic is often neglected in the literature even though it is one of the greatest issues in driving power wheelchairs (Viswanathan et al., 2017). In this research, conflict between the user and the AS is defined as the combination of the three following conditions.
1. The hand torques T h,i and the assistance torques T a,i are in opposite directions, T h,i · T a,i < 0, i = l, a; 2. In both longitudinal and lateral directions, the differences between the hand and assistance torques are greater than a predefined threshold torque T cf , |T a,i − T h,i | > T cf , i = l, a; 3. Conditions 1 and 2 occur and are maintained for a period greater than a given period t thr .
The conflict conditions can be easily detected when the assistance and hand torques are known by the controllers and observers. However, the remaining problem is managing the risk and the conflict so that once the conflict is detected, the AS can be changed from OAA to NDA operating mode. Figure 4 shows the detailed operation of the conflict management block. The proposed algorithm uses four control variables: conflict, iscollis, risk, and solve_conflict to solve this problem. The detailed operational steps of this algorithm can be described as follows.
1. When there is no potential collision detected (iscollis = 0), the variables are set to 0 and the AS always stays in NDA operating mode. 2. When a potential collision is detected (iscollis = 1) and solve_ conflict is still zero, the risk variable is set to 1. Then if isconflict is still zero OAA operating mode is activated. The user is now expected to sense the haptic torques that communicate a collision-free path. The user then make their decision by applying the appropriate torques to the joystick. a. If there is no conflict between the user and the assistance torques (isconflict = 0), it means that the user agrees with the proposed

FIGURE 14
Path taken with the position of the wheelchair where the obstacle avoidance assistance is activated when the risk of collision is detected.
Frontiers in Control Engineering frontiersin.org path and the AS will stay in autonomous mode until no risk of collision is detected. The control loop is then repeated from step 1. b. However, if a conflict is detected (isconflict = 1), it means that the user does not agree with the proposed path, and the assistance is switched to NDA operating mode to follow the user preferences after setting the variables isconflict to zero and solve_conflict to 1 to indicate that the conflict is solved for the next calculation loop. 3. After re-activating the NDA operating mode, as the conflict has been solved, even if a potential collision is still detected by the sensors (iscollis is still 1), the risk variable is set to 0 and the AS stays in this operating mode. Once the potential collision is eliminated iscollis = 0, the control loop is repeated from step 1.

Human-hardware-in-the-loop simulation
In this section, we present the experiments conducted to validate the proposed AS. Firstly, a virtual-reality simulator is introduced to integrate the user and the haptic joystick into the numerical simulation loop. The user-joystick wheelchair model was validated by identifying the parameters of the proposed models through measured inputs/ outputs data. Then two functions of the NDA operating mode were validated: i) the estimation of user's intention, and ii) the shared control-based assistance. Finally, the OAA mode was integrated into the validation tests to verify the efficiency of the proposed system.

Virtual reality simulator platform
To validate the proposed approach, we developed a virtual-reality simulator to integrate the wheelchair user and the force feedback joystick into the control loop. The general architecture of this simulator, as shown in Figure 5, includes four (04) main elements: a haptic force feedback joystick, an Arduino Mega 2,560 electronic board, the controllers and observers and the virtual-reality environment.
1. The force feedback joystick is a modified version of a commercial joystick. It is equipped with two electric direct

FIGURE 15
Evolution of the estimated hand torques, assistance torques and the variables of the risk-conflict management functions.
Frontiers in Control Engineering frontiersin.org current motors that work with a nominal voltage of 12 V. Each motor is attached to an axis to provide the user with assistance torques. To control the assistance torques, we used here a dual H-bridge power electronics board and the pulse-width modulation (PWM) method. Two potential dividers were used as the joystick position sensors to measure the rotation angles of the joystick. The maximum rotation angles of this joystick are 20 o for each axis. The electronic board and two position sensors were connected to the Arduino Mega 2,560 board. 2. The Arduino board plays an intermediate role in this system. It reads the joystick positions using its Analog-Digital converters, converts the values to integer format, and then sends them to Simulink through the UDP connection thanks to its Ethernet shield. It receives the assistance torque values from Simulink through the UDP connection, converts them to the corresponding PWM values, and produces these torques by controlling two DC motors using its PWM modules. The source code for these Arduino functions was developed in Simulink thanks to the Arduino Support Package for Simulink. The sample time for the Arduino program was set to 0.01s.

The observer/controllers were implemented in Simulink
Desktop Realtime to work in real-time conditions. The sample time was set to 0.01s. Models 1), 7), 8) were used to simulate the movement of the wheelchair in the virtual environment. 4. A virtual-reality scenario simulated the environment so that the user could experience different situations to verify the functions of the proposed AS. It was designed using the Simulink 3D toolbox, as in Figure 6.

User-joystick wheelchair model validation
By using the virtual-reality simulator platform, the joystick and wheelchair users are involved in the control loop. The augmented model proposed in Section 2 consists of the user-path following model and dynamic model of the joystick which are now replaced by the real objects. Therefore, the parameters of these two models need to be identified.
To obtain data for this parameter identification step, four pathfollowing experiments have been conducted on the virtual simulator platform where the predefined path is shown by the green line from point S to point I in Figure 6. In user-joystick-whelchair model 11, e 1 = 0.1m was chosen as about half of the distance travelled in 1 s by a power wheelchair in a confined environment. This was based on the data published in Gillham et al. (2018) where 14 power wheelchair users ware required to carry out a navigation task in real-world tests. The data shows that the participants finished a course of 10.5m within about 60s. The obstacles in our virtual scenario were also adapted from the experimental setup in Gillham et al. (2018) where the participants commented that this setup was similar to the difficulties in their daily lives. We recorded the position of the wheelchair x, y, φ, the position errors e 2 , e 3 from the virtual-reality environment, and the joystick angles θ a , θ l from the joystick position sensors. Then the model parameters were identified by using the Parameter Estimation toolbox in Simulink. Figure 7 and Figure 8 present the estimation results of the proposed user-joystick model and the real responses of the participant. The responses of the model follow the trend of the real movements. There are some moments, for both velocity and yaw rate, where the simulated and real responses do not match. This may be due to the uncertainties in the way the user perceives their position errors and their reactions. In the proposed model, it is assumed that the reference points are located at a fixed distance in front of the

FIGURE 16
Evolution of the estimated hand torques and assistance torques when a risk of collision is detected.

Frontiers in Control Engineering
frontiersin.org wheelchair, so the user may choose a different point depending on the future curvature of the path. Table 1 summarizes the identified parameters from the above experiments.

Validation of the Normal Driving Assistance operating mode
The experiments in this section were conducted to validate two functions of the proposed AS in NDA operating mode: estimating the user intention and reducing user effort when manipulating the joystick. The wheelchair user was required to go from point S to point D two times in the virtual environment in Figure 6 without colliding with the virtual walls. The first time was without assistance torques and the second time was with assistance torques. With the assumption that e 1 = 0.1m, user intention in this context is short-term intent. This means that we try to estimate what a human wants in the next about one second. In addition, differing from the experiments in Section 5.2 where the path was predefined, there ws no predefined path in these tests and the user was free to choose their desired movements to avoid collisions. Figure 9 shows the filtered and raw results of the joystick angles during the two tests, with and without assistance. We can see that the recorded signal data are quite biased, especially the longitudinal angle of θ y , and the estimated joystick angles converge to the real measured ones. Figure 10 presents the estimated results of the desired movements of the user including longitudinal velocityv r and yaw rateω r . Although these real corresponding user values cannot be measured explicitly, these estimated results were compared with the real measurements on the wheelchair. It was expected that the estimated desired velocity of the user would be a little bit faster than the real one. However, the choice of the observer gains is constrained by the convergence speed and the noise sensitivity of the estimated results. Although increasing these gains can make the estimated ones converge faster, as in the simulation, it also makes the estimated results to be too sensitive to the measurement noise, as in Figure 10. As a result, the estimated values of the desired movements of the user are nearly the same as the real movements of the wheelchair. Therefore, we can practically use the longitudinal velocity v of the wheelchair as the premise variable instead of the estimated desired velocityv r of the user. Figure 11 shows the estimated position errors with and without assistance torques. It can be seen that the lateral position errorê 2 and the angular position errorê 3 with assistance were reduced considerably compared to the case without assistance. Figure 12 shows the hand and assistance torques with and without assistance. With assistance torques, in both the longitudinal and angular directions, the user only provided about half of the necessary torques compared to the case without assistance.

Shared control-based assistance validation
To adapt the assistance torques to user needs (e.g.they may require assistance depending on their health at a specific moment), the assistance should be adjustable. By varying the parameters k re2 , k re3 , k I , k P of the user -path -following controller, the proposed AS adjusts the assistance torques according to the desires of the user. To illustrate this result, we repeated the same experiment one more times and the parameters k re2 , k re3 , k I , k P were reduced to 50% of their values in Table 1. As can be seen in Figure 13 the hand torque was reduced considerably compared to the case in Figure 12. The assistance torques now account for the majority of the total torque necessary to manipulate the joystick.

Validation of obstacle avoidance assistance operating mode
In this section, we present the experiments conducted to validate the proposed AS in the OAA mode. The participant was required to repeat the same path as in the previous experiment without knowing that an additional obstacle had been inserted at the point O as in Figure 6. This obstacle was not visible to the participant but detected by the AS. The goal of this setup was to simulate a circumstance where the situation awareness of the user was different from that of the AS to validate the conflict management function of the proposed AS. The torque threshold T cf was set to 0.2N and t thr was set to 1.5s and both were determined empirically. The pair (q, r) for the optimal path design was set to (25,1) to make sure the steering assistance had a higher priority than the speed assistance.  Figure 14 shows the path taken in the scenario designed and the position of the wheelchair when the risk of collision was detected. The risk management block detected the potential collisions and activated the OAA mode 4 times. In this experiment, all the potential collisions involved the front corners of the wheelchair. When there was no risk of a collision, the system provided about 50% of the required torque to manipulate the joystick, as shown in Figure 15. Otherwise, when a risk of collision appeared, the OAA mode was activated at 7.7s, 14s, 22.5s and 29s. The zoom of these moments is presented in Figure 16.
In the longitudinal direction, the wheelchair user always seemed to disagree with the assistance torque. The 4 times a potential collision was detected, the longitudinal hand torque was always contrary to the assistance one and the user feel an resistance torque acting on his hand on the joystick. The response of the longitudinal velocity is consistent with the proposed optimal solution in Figure 15 where the real velocity decreased in accordance with the suggestion from the AS.
In the lateral direction, the user agreed more with the suggestion made by system. The hand torque had the same direction as the assistance for the first three potential collisions detected and the user feels a drop of assist torque at the moment where the AS swicth from OAA to NDA mode. Although the longitudinal hand and assistance torques were opposite these three times, no conflict was found between the user and the system. The reason is that with the assistance in the lateral direction, the system found that the risk of collision was eliminated and then it switched to NDA operating mode. Figure 15 illustrates these results when only two flags iscollis and risk are turned on, while the flags isconflict and solve conflict are still off.
The conflict between user intentions and the AS can be seen at point P4 in Figure 14 when the wheelchair approaches the additional obstacle described above. In this case, the proposed AS tried to guide the user along a different collision-free path by providing the haptic torque which gradually increases, as in Figure 15 and Figure 16. The user felt significant resistance at the joystick where AS had been trying to inform the optimal path. However, the user, unaware of the presence of the obstacle, tried to follow their initial intention by applying an opposing torque to the joystick. Once the three conditions in Section 4.3 were verified, the conflict between the user and the system was confirmed and the flag isconflict switched on, as in Figure 15. In this case, the system then had to switch to NDA operating mode, the user would feel a sudden drop in haptic torque at the moment of switching when AS waited for the user's actions. . Therefore the flag solve conflict was switched on and the isconflict and risk flags turned off. The flag solve conflict is maintained until the iscollis flag is turned off to avoid continuously switching between NDA and OAA operating modes.
It should be noted that when the assistance mode is switched between NDA and OAA modes, the initial conditions of the observers are reset to zero, and therefore, the estimated torques are also reset to zero, as shown in Figure 15.

Toward a real-life application
The experimental results confirmed the operation of the assistance system in both NDA and OAA operating modes in a simulated environment. In a real scenario, where the environment is generally more complicated and external disturbances affect the user and the wheelchair dynamics, this system is also expected to work well with an appropriate configuration. Firstly, the parameters k re2 , k re3 , k I , k P of the user driving model must be calibrated according to the user's profile. The parameters can even be changed when the wheelchair is moved with a simple hardware (e.g., a potentiometer). In OAA mode, the effectiveness of the conflict management algorithm was proven by the results of the experiments. The performance of this algorithm is based on the T-S fuzzy unknown input PI observer where the convergence has been demonstrated. This algorithm is designed to cover all cases of relationships between the hand and the assistance torques. However, the authors point out two main reasons that can lead to abnormal behaviour of the AS. The first reason is related to the presence of faults in the sensors which lead to a bad torques estimation or perception of the situation. This cause can be corrected by improving the reliability of the sensors. The second reason is that the optimal path proposed by the AS is only valid for nearby obstacles. This means that this path may not be optimal if the user's anticipation is much higher and therefore conflicts may occur, which decreases user comfort. This problem should be solved by combining this system with a more efficient trajectory planner with long-range performance sensor.

Conclusion
In this paper, a new model-based shared control approach for power wheelchair assistance through a force feedback joystick has been proposed. Based on the user's intention estimation, this proposed assistance system is able to reduce the user effort when manipulating the joystick, guide them to avoid a potential collision and preserve their role as the final decision maker. By introducing an augmented dynamical model of the user-wheelchair system, the unknown input observers were designed to estimate user intention. The optimal T-S fuzzy controller has been designed to handle the time-varying user's intention and to provide haptic torque feedback to the user. To manage the control authority between the user and the automation, a cooperative control strategy has been proposed, and two main operating modes were defined: 1) the Normal Driving Assistance (NDA) mode and ii) the Obstacle Avoidance Assistance (OAA) mode. For the OAA mode, a path following controller was designed to provide an appropriate haptic guidance. We have also developed conflict management functions based on the user's haptic torque applied to the joystick, estimated using the designed UIO. Experiments were conducted using a human-hardware-in-the-loop simulator with a haptic joystick developed in our laboratory to validate the proposed approach. The experimental results have demonstrated the effectiveness of the proposed approach and confirmed all the intended functions. The conflict management function allowed users to communicate their intentions and overrule the guidance of the system effectively to maintain their active participation in driving the wheelchair.
Future works will be focused on the evaluation of the proposed assistance system by conducting experiments on the LAMIH PSCHITT-PMR full scaled wheelchair dynamic simulator for real-world experiments with the participation of able-bodied and disabled peoples.

Data availability statement
Publicly available datasets were analyzed in this study. This data can be found here: https://pubmed.ncbi.nlm.nih.gov/29552641/.

Frontiers in Control Engineering
frontiersin.org