Environment Independent Directional Gesture Recognition Technique for Robots Using Multiple Data Fusion

A technique is presented here for directional gesture recognition by robots. The usual technique employed now is using camera vision and image processing. One major disadvantage with that is the environmental constrain. The machine vision system has a lot of lighting constrains. It is therefore only possible to use that technique in a conditioned environment, where the lighting is compatible with camera system used. The technique presented here is designed to work in any environment. It does not employ machine vision. It utilizes a set of sensors fixed on the hands of a human to identify the direction in which the hand is pointing. This technique uses cylindrical coordinate system to precisely find the direction. A programmed computing block in the robot identifies the direction accurately within the given range.


Introduction
The demand for human interaction with robots is increasing with time. One of the most challenging and interesting field of research in this direction has been developing technologies for robots to understand human gestures. This paper presents a technique for robots to sense directional gestures. There are similar techniques that employ machine vision for the same purpose, but they have their own constrains due to environment. The paper technique presented here overcomes that constrain to a great extend.
A set of sensors along with transmitters and receivers are fixed on the robots and on the hands of the human whose gesture should be recognized.

The Basic Working Principle
The system uses two points on the human hand as two ends of a line segment. One point is close to the elbow and another point is close to wrist. The direction in which this line segment points with reference to the location of the robot can be calculated by the processing unit in the robot. The precise sensing of the two points is done by a pair of set of three sensors. To locate each point in space with respect to the robot, a set of three sensors are used. Cylindrical coordinate system is used for the calculation of the direction with respect to the point where the robot is located. Once the direction where the hand is pointing is found out, the direction in which the robot needs to move can be decided by the processing unit.
The robot is assumed as the centre of the cylindrical coordinate. The two points are defined in space using cylindrical coordinates. Three sensors are used to find the three cylindrical coordinate points: A Direction of Arrival (DoA) sensor for finding the angle, a simple electromagnetic transceiver for finding the distance between the point and robot and a height detecting sensor (preferably laser sensor) to detect the height of the point above the ground. Thus all the three coordinate values required to define a point in cylindrical coordinate are obtained. In Fig  1., the robot is located at point O. One of the two points is P. OP is the distance between the robot and the sensor point. The distance can be obtained by electromagnetic transceiver. The heightdetecting sensor will give the height PQ. From OP and PQ, the distance OQ can be found. The angle is given by DoA sensor. Thus a point P in the cylindrical coordinate can be defined with respect to the robot (origin). Thus the co-ordinates of the two points on the hand of the person who is pointing are found out using the above explained technique. Let the two points be P 1 and P 2 . Here point P 1 is the point closer to the elbow and P 2 is the point closer to the wrist. The robot is located at point O. Thus the direction in which the robot has to go is a projection of the line segment joining point P1 and P2. Thus the robot is programmed to reach the point on the ground that is the vertical projection of the point P 2 . Then the robot proceeds from P 2 in the direction of P 1 P 2 .

Direction of Arrival
The most important coordinate value required in terms of precisely acquiring the position of the point is DQJOH ³3´. This angle is obtained by a DoA sensor. To eliminate ambiguity of whether the obtained signal is stray or not, a pair of DoA sensor can be used for a single point. In this case the DoA sensor is positioned on the robot. It is a simple electromagnetic receiver with precise narrowband frequency range. The corresponding electromagnetic transmitter is positioned on the hand of the person whose gesture is to be sensed and it generates the matching narrowband signal.

Distance Between Origin and Point
The distance between origin and the point is ³X´ DQG LW is obtained by employing an electromagnet or acoustic transmitter and receiver and measuring the strength of the received signal. The strength of the signal received is inversely proportional to the distance between the transmitter and receiver.

Height of The Point Above Ground
The height ³K´ of the point above the ground is measured either by a high precision ultrasonic sensor or by a laser sensor. There are also other mechanisms for measuring height above the ground, which can be employed based on factors like size, compatibility and cost.

Calculations To Find The Coordinate Values
The algorithm presented here takes only the x and y coordinates the two coordinates on ground, into consideration for calculation and determination of which direction is gestured. The same algorithm can be extended for threedimensional directional gesture recognition by just taking into consideration the height coordinate. The obtained cylindrical coordinates are first converted into Cartesian coordinates for the sake of simplified and understandable mathematical calculation. In Fig 2., the known quantities are D, h and ', which are values obtained from sensors. The three cylindrical coordinates are angular FRRUGLQDWH ³'´ UDGLDO GLVWDQFH ³X´ DQG YHUWLFDO KHLJKW ³K´ The only unknown value of the three F\OLQGULFDO FRRUGLQDWH YDOXHV LV UDGLDO GLVWDQFH ³X´ This value can be found by a simple Pythagoras formula: Thus after all three cylindrical coordinate values are obtained, it is converted into corresponding Cartesian values x, y and z by conventional formula. As shown in Fig 2., the Cartesian coordinates of the points P 1 and P 2 are found.

Algorithm To Find The Directional Vector
The ultimate objective of the proposed mechanism is to make the robot to identify the direction pointed by the human controller or instructor. Therefore the directional vector of the line segment joining P 1 and P 2 is required to be found. In Fig 3., the top view of the robot and the points P 1 and P 2 are shown. The values of all the three Cartesian coordinates of both the point P 1 and P 2 are known. Only the x and y coordinate values of point P1 and P2 are sufficient to find the two dimensional directional gesture for land roving robots. Three-dimensional calculation is required only for flying robots. Since the x and y coordinate values of both P 1 and P 2 are known, the directional vector of the line segment P 1 P 2 is found out by simple geometrical formula. The robot now has to reach any point close to point P 2 and then proceed moving in the direction P 1 P 2 . The directional convention is that the robot should move only from P 1 to P 2 and further and not from P 2 to P 1 . This is done by assigning a prefixed tag on any values acquired from the sensor corresponding to any one point.

Algorithm To Find The Clearance Point
In the case of a practical land robot, the robot is expected to reach a point on the extended part of the line segment P 1 P 2 . The robot should reach the point P 3 , which is the clearance point, typically as shown in Fig 3. The FOHDUDQFH GLVWDQFH ³n´ should be very small when compared to the length of P 1 P 2 . The mathematical algorithm to find out the coordinate of P 3 is simple. First, the clearance distance that is desired, that is the vDOXH ³n´ is required as input. This length or clearance distance n may vary from system to system. The first step is to find the equation if the circle with point P 2 as the center and the length n as radius. The obtained equation of the circle is solved with the equation of the line joining P 1 P 2 to find the two points where the circle intersects with the line P 1 P 2 . These points are P 3 and P 4 . The point P 3 is farther from point P 1 when compared to point P 4 . Thus the point P 3 is selected as the point where the land roving robot should come. The above algorithm is explained with an illustration below. (2) The equation of circle with P 2 as center and 0.5 as radius is obtained. That equation is: When Eqn.(2) and Eqn. (3) is solved we get two coordinate values. The two coordinate values corresponding to P 3 and P 4 respectively. The two point coordinates are (2.538, 5.132) and (3.465, 4.876). The point P 3 is assigned to the point which does not lie on the line segment P 1 P 2 . Therefore, within the two points P 3 and P 4 , the point whose x and y coordinate values lays within the x and y coordinate values of P 1 and P 2 is eliminated. The eliminated point is P 4 . The other point whose x and y coordinate value lies outside the x and y coordinate value of P 1 and P 2 corresponds to point P 3 . Therefore, the robot is expected to come to position P 3 . The robot now has to proceed from point P 3 in the direction along line segment P 1 P 2 away from point P 2 .

Setup of The Direction of Arrival Sensor
The DoA sensor consists of an array of sensing elements placed close to each other. Since in this case, the signal is expected from any direction, that is 360 degrees, a minimum three arrays is mandatory In Fig 5., the DoA shown uses four array sensors. The DoA, as mentioned above senses from which direction the signal arrives. As shown in Fig 4., the source emits radiation which is first received by the sensing element closest to the source. In this case, it is the top most sensing element (6 th sensing element) of the right side array that first receives the signal. All the other sensing elements receive the signal after that sensing element. Therefore by using conventional time delay analysis, it is possible to detect which sensing element first receives the signal and which one receives following the first reception. If the time gap between the consecutive receptions of the signal by the sensing elements is measureable, then by employing a simple algorithm it is possible to deduce the Direction of Arrival with great accuracy. A simple convention of DoA sensor is presented in presented in Fig 4. There are twelve sensing elements, which are numbered from 1 to 12. Here the sensing element 1 corresponds to 0 0 and 360 0 . Since there are 12 sensing elements, the angle between two sensing element is 30 0 . In this illustration, it is assumed that the 6 th sensing element is closest to the source. Therefore, it is the first element that receives the signal. The reading of the DoA sensor is therefore 150 0 . It is obtained as the product of the number of sensing element and the angle between two sensing elements, which in this case is 150 0 (6*30 0 ).

Conclusion
This mechanism presented here is an inchoate form of the idea. The overall system can be made very compatible and precise if the best in class building components are used. A small computing device should be attached to the sensing and transmitting components to control and synchronies them for better results. The system can be dynamically used to control server robots in hotels, offices if the two dimensional algorithm for land roving robots is employed. The three dimensional algorithm is employed for flying robots. For this the height dimension is taken into calculation. In both the cases the robot can be dynamically controlled on how fast the sensing elements receive the transmitted signals, how fast the transmitting signals transmit the signal and how fast the signals can be conditioned and processed. Timing is the most important factor when it comes to practical implementation of this mechanism.