Automerlin mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
179
AUTOMERLIN MOBILE ROBOT’S BILATERAL TELECONTROL WITH
RANDOM DELAY
Aamir Shahzad
Department of Mechanical Engineering, The University of Lahore, Lahore,
(Pakistan)
E-mail: aamir.shahzad1@me.uol.edu.pk
Muhammad Salahudin
Department of Mechanical Engineering, The University of Lahore, Lahore,
(Pakistan)
E-mail: muhammadsalahuddin@hotmail.com
Iqbal Hussain
Department of Mechanical Engineering, The University of Lahore, Lahore,
(Pakistan)
E-mail: iqbal.hussain@me.uol.edu.pk
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
180
ABSTRACT
The main focus of this work is to design a bilateral telecontrol of a mobile robot
AutoMerlin through the Internet. The Internet has an inherent delay, packet
drop, out of order data transmission, duplication, and other impediments as a
communication channel. These factors cause the system to become unstable
and difficult to control through the Internet. The velocity tracking becomes
really hard and the force feedback also rises to an unacceptable level due to
delay and other impediments. In order to address these issues, a power based
TDPC (Time Domain Passivity Control) has been utilized in this work for the
development of stable telecontrol. This approach is based on energy. The
energy has been classified as positive and negative energy to make passivity
analysis independent of monitoring of net system energy in real time. Thus,
monitoring the net energy output at each port enables the extension of TDPC
for delayed systems called TDPN (Time Delay Power Network). TDPN helps
in velocity/force tracking. It transmits velocity/force unaltered by rejecting the
active energy. PO (Passivity Observers) indicate the active behavior and the PC
(Passivity Controllers) dissipate extra surplus energy to keep the system stable
and passive all times. The performance has been tested and plotted to show
the effectiveness of the bilateral controller under random delay and other
limitations.
KEYWORDS
Telecontrol; Haptic force; Joystick; Unstructured environment; Slave robot; Time delay
power network; Random delay.
1. INTRODUCTION
Bilateral telecontrol of a mobile robot can be defined as control of the robot
from a remote location while receiving force feedback from it as shown in
Figure 1. The complete configuration of telecontrol comprises of a human
operator, a haptic device connected to a computer having client algorithm,
communication medium, slave robot equipped with server algorithm and
remote environment [1-4]. The human operator applies the required
maneuvers to the master haptic device which translates it into inputs for the
slave robot in the remote environment. The desired inputs of the master
device travel through some communication medium to the slave robot. These
commands/instructions are executed by the slave robot in order to manipulate
the remote environment. The effect of the environment is a reactive force on
the slave robot. This force is sent as force feedback from a remote location to
master haptic device. This force feedback is played over the haptic joystick.
The human operator experiences the force feedback via haptic device and gets
a sensation of actually manipulating the remote location directly. Telecontrol
is a combination of different subsystems that exchange energy. The energy is
exchanged by forwarding velocity and receiving force feedback [5-7]. Passivity
control is based on system’s net energy and it is an efficient tool for the stability
analysis of the bilateral telecontrol. The combination of passive subsystems is
always a passive system [8]. Hence, to ensure the passivity of the system, the
subsystems can be analyzed to be passive all times. The stability based on the
mathematical model of the system imposes conservative and strict rules on the
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
181
performance of the system. It cannot easily tolerate the delay in the system and
also requires accurate information about the remote environment [9-10]. The
passivity is independent of a mathematical model of the system and is based
on energy balance.
The master and slave are passive as they dissipate energy but when they are
connected by means of any communication medium, then that medium can
behave actively by inducing surplus energy in the system. The time delay is the
main source of instability and activity too. A passivity-based approach using
wave variables has been proposed by Niemeyer [11]. Wave variables have been
utilized to develop the teleoperation with force reflection. Similarly, the
scattering approach has been presented by Anderson and Spong [12]. These
approaches have guaranteed the passive telecontrol at the cost of over
dissipation of energy. This over dissipation of energy resulted in conservative
performance [13-15]. To cope with these issues a remarkable approach based
on TDPC has been presented by Ryu and Hannaford [16]. TDPC has two main
elements called PO (Passivity Observer) and PC (Passivity Controller). PO
keeps the track of energy entering and leaving the system to estimate net
energy. The PC takes all the needed measures to dissipate surplus energy
introduced in the control loop by various means. TDPC is based on calculation
of net energy in real time to perform necessary control action. But in case of
time delay due to distance or communication through a shared medium, the
observation of net energy is not possible in real time. Hence the controller
cannot take any action against the active energy. Therefore, Artigas has
presented an extension to TDPC for the delayed system [17]. By using an
Electrical/ Mechanical analogy, it has been proven that instability occurs due
to non-passive communication block during
Figure 1. The bilateral telecontrol of AutoMerlin.
bilateral telecontrol with constant or random delay. A communication block
has been modelled as a lossless entity which acts as a passive element even with
time delay [18]. Teleoperator’s stability has been guaranteed without limiting
the bandwidth. This control approach ensures passivity of all subsystems in the
closed-loop i.e. master, communication block and slave. Once the passivity is
attained, the system is stable even though there are limitations and
disturbances. These limitations are human operator dynamics, variable and
unknown communication delay. Force feedback has been modelled as
summation of two forces i.e. the virtual force and friction. The virtual force is
based obstacles in the vicinity of the robot in a remote environment and friction
is between ground and wheels. The virtual force calculated due to obstacles is
a function of distance and velocity travelled by mobile robot. The complete
method and its elaboration have been given in [4]. This force acts on the robot
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
182
while it is moving in a remote environment is played over the haptic device as
force feedback to the human operator. The velocity, force telecontrol is shown
in Fig. 2. The slave robot is receiving the velocity command from the master
device and delayed environmental force from the slave is reflected back.
Section I of this paper is Introduction about telecontrol, different approaches
used in teleoperation and their pros and cons. Problem description has been
presented in Section II with different plots to clearly illustrate the issues.
Section III briefly describes the TDPN. It has mathematical modelling for the
passivity of the network with random delay. The energy relations have been
explained in it. Section IV has experimental results to show the performance
of the teleoperation with TDPN. Section V has Conclusion and Future Work.
2. PROBLEM DESCRIPTION
The objective of telecontrol is to establish a close coordination among the
interacting subsystems i.e. the haptic device and slave robot along with some
limitations like limited bandwidth and random or constant time delay.
Therefore, to elaborate the problem, there are some plots which have been
included in this section to realize the actual issues. The blue line in the Figure
3, shows the master haptic device’s velocity and the green line represents the
velocity of the slave robot. It is vivid that the slave velocity is more than the
desired velocity set by the master device. This is due to the activity of the
Internet because the master velocity travels through it and it has the delay in
it and also other limitations like duplication, drop in packets and change in the
order of data etc. [4]. The blue line indicates energy input at the master side
while the green line is showing energy output and net energy is represented
by a red line in Figure 4.
Figure 2. Block diagram of telecontrol.
It is clear from the plot that the net energy is negative and it is accumulating
because active energy is not being dissipated. Figure 5, shows the two forces
i.e. virtual force on the slave robot and force feedback on master haptic device.
During certain intervals like after 4 seconds, the force feedback is larger as
compared to the environmental force acting on the slave. Due to surge in force,
the energy output is greater as compared to the energy input. This implies
that the shared medium i.e. Internet is adding energy into the system to make
it active and unstable. The energy comparison between net energy, input
energy and output energy from slave to master has been plotted in the Figure
6.
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
183
Figure 3. Linear velocities of both robots.
Figure 4. Energy flow master to slave.
Figure 5. Force comparison on both robots.
Figure 6. Energy flow slave to master.
Figure 7. A Two-port network as TDPN.
3. TDPN (TIME DELAY POWER NETWORK)
The communication medium is adding active energy in the passive telecontrol
system while reciprocating the velocity/force. Therefore, in order to solve the
issue of active energy, the communication medium has been modelled as a
Two-port network in which velocity/force move contrary to each other as
depicted in Fig. 7. Each port corresponds to a real-time velocity/force signal
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
184
and its conjugate delayed signal. The velocity force multiplication is power and
the accumulation of this power over time results in energy. This modification
of the communication medium is called the TDPN. TDPN acts in the same
manner as the conventional communication medium but it assists in resolving
the issue of random time delay inherent to a network. Instead of exchange of
velocity/force, a non-conventional concept is used i.e. transfer of energy at each
port. Positive and negative energies are segregated at each port. Positive
energy is getting in while negative energy is getting out at each port. The
analysis of energy is done with respect to input energy at the output of each
port. PO calculates the net energy output at each port. Whenever PO finds out
an active energy presence, then the relevant PC takes the corrective measures
to dump extra active energy to make the system passive again. This leads to a
stable telecontrol as shown in Fig. 8. This approach separates the energy flow
in the backward and forward direction and there is no connection between
these two quantities for passivity analysis of the whole system.
A. Passivity of TDPN
Figure 7 shows the velocities and forces entering and leaving a Two-port
network. Power is the product of these two variables. The total power of the
network can be written as given in (1).
)()()( tPtPtP
SMN
(1)
(M=master) is the power on the master side of a Two-port network and
)(tP
S
(S=slave) is the power on the slave side while
)(tP
N
(N=network) is the
power of the network. The energy at both sides of the network is described as
dPtE
t
MM
0
)()(
(2)
dPtE
t
SS
0
)()(
(3)
dVftE
t
MMM
)()()(
0
(4)
dVftE
t
SSS
)()()(
0
(5)
)(tE
M
,
)(tE
S
are the energies on master side and slave side respectively. Hence,
to keep the network passive, the following condition should prevail.
0,0)( ttP
N
(6)
)(tE
M
,
)(tE
S
are not available due to time delay simultaneously. Therefore, in
order to solve this issue the positive and negative power has been taken at each
port so that net energy can be calculated in the presence of delay.
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
185
0)()(
&0)()(
tVtf
ttPtP
MM
MM
(7)
0)()(
&0)()(
tVtf
ttPtP
MM
MM
(8)
0)()(
&0)()(
tVtf
ttPtP
SS
SS
(9)
0)()(
&0)()(
tVtf
ttPtP
SS
SS
(10)
Both robots positive and negative energies are
0)()(
0
tdPtE
t
MM
(11)
0)()(
0
tdPtE
t
MM
(12)
0)()(
0
tdPtE
t
SS
(13)
0)()(
0
tdPtE
t
SS
(14)
)(tE
in
is positive and
)(tE
out
is negative entering and leaving the port
respectively at each side.
0)()(
ttEtE
M
in
M
(15)
0)()(
ttEtE
M
out
M
(16)
0)()(
ttEtE
s
in
S
(17)
0)()(
ttEtE
M
out
S
(18)
)()()( tEtEtE
SMN
(19)
Overall net energy of whole system is given in (20).
)()(
)()()(
tEtE
tEtEtE
out
S
in
S
out
M
in
MN
(20)
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
186
)(tE
SM
is the energy from master haptic device to slave robot and
)(tE
MS
from slave robot to haptic device.
)()()( tEtEtE
MSSMN
(21)
)()()( tEtEtE
out
S
in
MSM
(22)
)()()( tEtEtE
out
M
in
SMS
(23)
The network is passive until the (24) and (25) inequalities are satisfied.
0)(
tE
SM
(24)
0)(
tE
MS
(25)
Df is the delay time from master to slave called forward delay and Db is the
backward delay from slave robot to haptic device. The net energies with a
forward delay Df and a backward delay Db are given in (26) and (27).
)()()( tEDtEtE
out
Sf
in
MSM
(26)
)()()( tEDtEtE
out
Mb
in
SMS
(27)
B. Passivity observer
)(nE
Obs
M
,
)(nE
Obs
S
are observers at each side as given in (28) and (29). n
represents the random time interval between two sample time.
M
in (28) is
master controller and
S
in (29) is slave controller.
))1()1()(
)(()1()(
2
nVnnE
DnEnEnE
MM
out
S
f
in
M
Obs
M
Obs
M
(28)
))1()1()(
)(()1()(
2
nfnnE
DnEnEnE
SS
out
M
b
in
S
Obs
S
Obs
S
(29)
C. Passivity Controller
To realize a stable telecontrol, it is necessary to dump active energy which is
introduced by the communication medium into the telecontrol system.
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
187
Figure 8. Master and slave controllers.
Designed
M
and
S
behave as dissipative elements on master and slave side
of the TDPN respectively. The dissipation action reduces the effect of surplus
energy to minimal.
M
,
S
provide necessary controller action when the value
of the observed energy is negative. The corrective measures are applied to
velocity and force feedback as given in (32) and (33) and shown in Figure 8.
)(
)(
0)(0
)(
2
nV
nE
nEif
n
M
Obs
M
Obs
M
M
(30)
)(
)(
0)(0
)(
2
nf
nE
nEif
n
S
Obs
S
Obs
S
S
(31)
)()()()( nVnnfnf
MMMM
(32)
)()()()( nfnnVnV
SSSS
(33)
4. EXPERIMENTAL RESULTS
The performance of passivity control with TDPN for telecontrol of mobile
robot AutoMerlin has been plotted and presented in this section. The
performance of the controller has been tested without and with the obstacles
around the robot in the remote environment. The first test run was performed
when the environment was free of obstacles. Fig. 9, shows offset between two
velocities due to the time delay. The dots represent the random time delay
between two sample times. It is vivid from Fig. 9, that there is no surge in slave
velocity after passing through a communication channel with random delay.
Blue plot is energy input and green plot energy output on master and slave
side respectively in Fig. 10. The output energy has surge in it due to active
energy. Hence, its value is greater than the input energy as depicted in the
Fig. 10. The slave controller measures the active energy and diffuses its effect
by dissipating it so that the forward communication always remains passive as
drawn in Fig. 11, by the blue line. Force feedback is sent back from the remote
environment to master haptic device. Fig. 12, is showing the force on the
remote robot as a blue line and force feedback on the haptic device as a green
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
188
line with the random delay indicated by dots. Fig. 13, shows the input energy
entering the port from the slave side with the blue line and green line plot is
the output energy on the other side. The surplus energy is being added by the
network and due to this addition, the system output energy is greater than
respective input energy. As the force on both sides is similar, the master
controller is dissipating the effect of active energy as shown in Fig. 14.
Whenever there is an active behavior by the network on either or both side the
master and/or the slave controllers dissipate the same amount of active energy
so that the system remains passive and hence stable.
Figure 9. Linear velocities of both robots.
Figure 10. Energy flow master to slave.
Figure 11.
S
controller.
Figure 12. Force comparison on both robots.
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
189
Figure 13. Energy flow slave to master.
Figure 14.
M
controller.
The second test run was done when there were obstacles in the environment to
evaluate the performance of the controller. The following plots show the
performance of the controller with obstacles around the operational area of the
remote slave robot. The Fig. 15, shows that there is no change in velocities and
Fig .18, shows the force on both the slave robot and the master robot is the
same. The dots indicate the random intervals. The rise in slave force is due to
the unstructured environment with obstacles. This rise in force is vividly seen
in Fig. 18. Similarly, the plots show that whenever there is an active behavior
both
M
and
S
dissipate energy so that a stable and passive telecontrol can be
performed
.
Figure 15. Linear velocities of both robots.
Figure 16. Energy flow master to slave.
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
190
Figure 17.
S
controller.
Figure 18. Force comparison on both robots.
Figure 19. Energy flow slave to master.
Figure 20.
M
controller.
5. CONCLUSION AND FUTURE WORK
The performance of the designed controller is excellent and it is keeping the
system stable and passive all times. There is no compromise on unknown
parameters because the controller is not based on the mathematical model of
the system. It only functions when required. In order to extend this work, there
are different tasks under planning like the addition of more robots for the
telecontrol of a team of robots for multi-tasking.
6. REFERENCES
[1] A. Shahzad and H. Roth. "Teleoperation of mobile robot using event based
controller and real time force feedback," Scientific Cooperations, International
Workshops on Electrical and Computer Engineering Subfields, Koc University,
Istanbul/Turkey, 22-23 August 2014.
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
191
[2] A. Shahzad, R. Al-jarrah and H. Roth, "Telecontrol of AutoMerlin robot by
employing fuzzy logic", International Journal of Mechanical Engineering and
Robotics Research (IJMERR), Volume 5, No. 1, January, 2016.
[3] A. Shahzad, R. Al-jarrah and H. Roth, "Teleoperation of AutoMerlin by
inculcating FIN algorithm", International Journal of Mechanical Engineering and
Robotics Research (IJMERR), Volume 5, No. 1, January, 2016.
[4] A. Shahzad, H. Roth, "Bilateral telecontrol of AutoMerlin mobile robot", 9Th
IEEE international conference on open source systems and technologies, Lahore,
Pakistan, 17-19 December, 2015.
[5] V. Chawda and M. K. O’Malley," Position synchronization in bilateral
teleoperation under time-varying communication delays," Ieee/asme transactions
on Mechatronics, Volume 20, Issue 1, Feburary, 2015.
[6] R. Anderson and M. Spong, “Bilateral control of teleoperators with time
delay,” IEEE Trans. Autom. Control, vol. 34, no. 5, pp. 494–501, May.1989.
[7] R. J. Anderson and M. W. Spong, “Asymptotic stability for force reflecting
teleoperators with time delay,” Int. J. Robot. Res., vol. 11, no.2, pp.135–149,
1992.
[8] H. K. Khalil, Nonlinear Systems, vol. 3. Upper Saddle River, NJ, USA,Prentice-
hall, 2002.
[9] K. Hashtrudi-Zaad and S. E. Salcudean, “Analysis of control architectures for
teleoperation systems with impedance/admittance master and slave
manipulators,” Int. J. Robot. Res., vol. 20, no. 6, pp. 419–445, 2001.
[10] A. Haddadi, “Stability, performance, and implementation issues in bilateral
teleoperation control and haptic simulation systems,” Ph.D. dissertation, Dept.
Electr. Comput. Eng., Queen’s Univ., Kingston, Canada, 2012.
[11] G. Niemeyer and J.-J. Slotine, “Stable adaptive teleoperation,” IEEE
J. Ocean. Eng., vol. 16, no. 1, pp. 152–162, Jan. 1991.
[12] R. Anderson and M. Spong, “Bilateral control of teleoperators with time
delay,” IEEE Trans. Autom. Control, vol. 34, no. 5, pp. 494–501, May.1989.
[13] P. Arcara and C. Melchiorri, “Control schemes for teleoperation with time
delay: A comparative study,” Robot. Auton. Syst., vol. 38, no. 1, pp.49–64, 2002.
[14] P. F. Hokayem and M. Spong, “Bilateral teleoperation: An historical survey,”
Automatica, vol. 42, no. 12, pp. 2035–2057, 2006.
[15] E. Nuno, L. Basa ˜ nez, and R. Ortega, “Passivity-based control for bilateral
teleoperation: A tutorial,” Automatica, vol. 47, no. 3, pp. 485–495, 2011.
[16] J. Ryu, D. Kwon, and B. Hannaford, “Stable teleoperation with time domain
passivity control,” IEEE Trans. Robot. Autom., vol. 20, no.2, pp.365–373, Apr.
2004.J. Ryu, J. Artigas, and C. Preusche, “A passive bilateral control scheme
for a teleoperator with time-varying communication delay,” Mechatronics, vol.
20, no. 7, pp. 812–823, 2010.
[17] Y. Ye, Y.-J. Pan and T. Hilliard, “Bilateral teleoperation with time-varying
delay: A communication channel passification approach,” IEEE/ASME Trans.
Mechatronics, vol. 18, no. 4, pp. 1431–1434, Aug. 2013.
Automerlin Mobile robot’s bilateral telecontrol with random delay
DOI: http://dx.doi.org/10.17993/3ctecno.2019.specialissue.16
192