LOCATION ANALYSIS ON SMART HOUSE USING PROJECTIVE TRANSFORMATION

In this paper, a method of location analysis for smart house is proposed. The proposed method uses projective transformation to process the input from visual sensor for determining coordinate of resident and also the entire device inside the smart house. With a good calculated coordinate, each device function in the smart house can be optimized for the good of the resident. From the experiment results, the proposed method successfully maps all coordinates of any device in the smart house up to 81% accuracy


Introduction
Smart house, or also known as automated house, is a building that uses a smart system to integrate its component with implementing pervasive computing in all aspect to make people activities in home easier to do. The rapid development of the Internet, network, and various smart device technologies are enabled various household appliance and electronic component to be integrated into the system. They are emerged many smart house projects to be developed intensively by many research group. The research grows in developing smarter, more comfort, and more secure house to simplify the life of its residence, ultimately with less energy consumption to support the house [1].
One of the most important aspect in the smart house is the location control. The location control (or locator) enables the smart house to monitor the user's location. The user's location will help the system to determine which services and devices need to be enabled to help the user and which device need to put on standby condition to save energy. Kraum in [2] classify basic location prediction by 6 types: proximity, trilateration, hyperbolic Lateration, triangulation, fingerprinting, and dead reckoning. Each type has its own tech-nique and equipment, which has their own advantages.
In this paper, a method for location analysis based on proximity is presented. The method uses images/video from camera to determine the user's location in a room. The proposed method focused on solving the perspective deviance in the images due to the camera location. It then use the user coordinate that analyzed from the images to find the nearest device that needs to be activated. The proposed method is built based on OSGi framework that proposed in [3].
Location is one of the most important contexts in ubiquitous computing, especially in smart house implementation. In smart house implementation, the resident's location informa-tion could bring more information. Some informa-tion like resident's activity, resident's interaction, resident's transportation can be analyzed and extracted from location information [4].
Because of its important role in smart house implementation, most of smart house research and implementation also include the location analysis research. There are also some smart house researches that use location information as their main source to extract its system parameter to make the system responses. Suk Lee, Kyoung Nam Ha, and Kyung Chang Lee [5] researched a non-terminal-based approach using an array of pyroelectric infrared sensors to detect smart house residents and their location. They use PIR sensor to track resident's movement and make some movement prediction that is used for making the smart house responses.
The resident's tracking system is also developed by The Pervasive Computing Laboratory at the University of Florida Smart environment is built to help elderly people to live independently at home. In the development, ultrasonic sensor technology is implemented to track elderly activity in the House of Matilda (a mock up house). OS-Gi-based framework is also described to combine ultrasonic sensor technology with other devices in the system [6].
In reviewing localization method and modeling a synchronize wireless device system based on location-aware service, Dept. of Inf. Eng. & Comput. Sci., Univ. of Trento, Trento, Italy research about as flexible and transparent tools toward the paradigm of a totally automatic/autonomic environment. The research focused on user localization to minimize the invasisveness of the monitoring system of the smart house [7].

Location Modelling
The first thing to do to start the location analysis is to create a model of the location. In this step, the proposed method creates an assumption that the method will be implemented in indoor room with square shape. The proposed method uses im-ages from video camera to view the room condition. The images then divided into grids to give us a more detailed coordinate, with the top left most section as the starting coordinate. The illustration of the location model can be seen in Figure 2.
Images taken from video camera most likely will have some perspective view depend on the camera location (as seen in Figure 3). It will cause many future problem because the different perspective view can change the standard information about the room. The inconsistency will caused malfunction in some device that need location information, especially in multi-room smart house. Unfortunately, there is no way to make sure that the camera in each room will be placed in the ideal position (in term of height, angle of inclination, or horizontal position), to solve the problem the proposed method uses projective transformation to project standard location model into any perspective view.

Projective Transformation
The idea of projective transformation is to change the perspective of a geometric shape using transformation matrix. The uses of matrix guarantee the invers ability of projection and keep the related information. The proposed method first creates a location model using normal perspective (the image taken from vertical view). The location model has four endpoint for each side (named p1, p2, p3, p4). The proposed method then checked on the input image to find four corresponding endpoint (named p1', p2', p3', p4'). Due to the variation of image perspective, the proposed method needs help from user to determine four endpoint in the image.
Having four pairs of endpoints, the proposed method now needs to find the transformation matrix that satisfies equation (1). The equation then further written as the equation (2), which represents the relation of each x and y coordinate. Using four endpoints pair, the proposed method then has eight equations to find the element of transformation matrix as given by equation (1).
where x, y is the coordinate of original matrix, u, and v, w is the perspective coordinate, and a, b, c, d, e, f, g, and h is the element of transformation matrix. Whereas x' and y' can be calculated using equation (2).
To finish the equation, a Gauss Jordan Elimination and pivoting [8] are needed to find the exact transformation matrix. The transformation matrix then is used to map all grids in location model to its respective coordinate in the projective image. The illustration of this process can be seen in Figure 3.
After finding the transformation matrix, the only problem is how to create a good grid that represents the position in the smart house. The grid must have a good representation of user's movement, which the average step of a men is 78 centimeters and the average of women is 70 centimeters. To create a grid that represents user's movement, the proposed method uses 50x50 centimeters tile to divide the grid into smaller coordinate. The illustration in Figure 4 show the grid allocation in 4x3 meters room model. The allocation then will be transformed using transformation matrix in equation (1) to match the input image. With the location layout and model now has been completed, the process move to identify user's location.

User's Location Identification
To find user's location, some images from video camera are used by the proposed method as the input data. The proposed method then does three processing steps to convert input image to identify the user's coordinate. These steps consist of Human Detection, Ground Finding, and Coordinate projection (the illustration can be seen in Figure  5). For each step, a meaningful information will be extracted from the raw data to be used to determine user coordinate.

Human Detection
The idea of human detection is to isolate the user texture from the rest of the room. The proposed method uses Histogram Oriented Gradient (HOG) descriptor method that proposed in [9] to do that. The idea of HOG is to split the images into many small region and analyze the distribution of gradient intensity in each region. To give a better invariance for different lightning intensity inside the room, the HOG descriptor also looks for intensity distribution in the neighborhood area, and does image normalization to the region that out of pattern with its neighbor. The HOG method then finds the histogram pattern of a human and isolates it from the background, leaving only the meaningful information of the user as a detection window.  The HOG method has a good performance and speed, but it may has a problem to identify two or more people that too close to each other. To avoid the misdetection, the proposed method will ignore the bad case and use only the last known well-separated identification. To do so, the detection process will continue to be done in every interval time to give a good update. The interval can be vary depend on the user implementation. The result of human detection will be used in Ground Finding step.

Ground Finding
The next step in the location identification is ground finding. In this step, the detection window extracted by the previous step will be processed further to find the part that represents the ground. The proposed method will chose the middle part of the bottom of detection window as the ground. The process is simple and takes less time.

Ground Coordinate Projection
In this point, the proposed method projects the ground location into the coordinate that generated by the Projective Transformation process. The process will produces the exact location of the user in the room. The location information later can be used by the proposed method to interact with the other device in the room.

Device Discovery
The proposed method use the assumption that the user more likely will only interacts with devices that has static location (e.g. air conditioner, television, computer, etc.). The first thing to do is the user must register all devices to its own static coordinate. The proposed method then compute the distance between user's location an all the devices in the room (using Euclidean distance) to makes distance list. Depend on the user or system preference; the proposed method will lists all devices that close to the user as the closest device. The al-gorithm to determine the closest device is shown in Algorithm 1.
The algorithm has O(n) complexity, which is linear growth. The linear complexity makes the algorithm able to run fast even when the room packed with dozen of device. Furthermore, to improve the performance further, the proposed method only updates the device discovery when the user moves to another coordinate in the grid. Moreover, this step of process remains inactive and only returns the previous list when invoked.
In some special occasion, the static device might be moved to another place. Depend on the distance of the movement, the proposed method might still be recognizes the device (it will auto remove and add the device in the different location), or needs to be recalibrated by the user. Nevertheless, that small change will not effected the system performance by mean effect.

Implementation
The proposed method is implemented using Java 7 with external library of JavaCV 1.4 to works on the image processing. The source of input is one surveillance camera, TPLINK model no. TLSC-3430, with the resolution 1.3 megapixels. To stream the video from the camera, the proposed method uses HTTP protocol with video mp4 format at 320x240 pixels resolution. The architecture of the code implementation can be seen in Figure 6 (as package id.ac.ui.smarthome.-locator).
When run, the user needs to give some information to calibrate the program, such as the edge coordinate, the size of the room, and the devices location. After that the main program will show the surveillance window of a room, with the information of user's location if available.
Screenshot of the main window can be seen in Figure 7, with the explanation as follow: a) Shows the location of the user in the room. When there is no user in the room, this information will returns with last known coordinate; b) Shows the list of the closest device(s) from the user; c) Shows the list of all devices in the room. The window has another feature to show which device is closer and which is farther to the user. The closer the device to the user, the greener the highlight back- ground. And the farther the device to the user, the darker the highlight background; d) Shows some button to manage the device list. It has functionality such as add, modify, and remove device; e) Shows the general information of the perspective grid, like the edges coordinate and room size; f) Shows the perspective grid of the room. The grid will generated and create an overlay on the input images/video; g) Shows the main window with input video from camera. Moreover, the bigger images and perspective variation of the main window can be seen in Figure 8.

Results and Analysis
To ensure that the implementation is worked well, the proposed method would be tested in a designned simulation that could represent the real environment where the system would be implemented. The simulation has four scenario, which are: resident's location detection, devices abstraction with device addition, device movement detection, and devices abstraction with device removal. From those scenarios, there will be evaluation on how good the system. There would be three result values from the test; appropriate, quite appropriate, and not appropriate. However, for the final evaluation, we just use two values; appropriate and not appropriate. The quite appropriate value is divided equally into two other values, and then the result would be counted as final evaluation result.

Resident's Location Detection
This simulation chooses five coordinates and five tiles area in each coordinate during the test. From the combination, it will get 25 results that can be evaluated. The results of this simulation can be seen in Table 1. From the test, there are 5 appropriate results and 20 quite appropriate results. As the designed in the beginning of this section, the quite appro-  priate result would be divided equally to appropriate and not appropriate result. Then, there are 15 appropriate results and 10 not appropriate results.

Devices Abstraction with Device Addition
This simulation chooses three coordinates and four device addition in each coordinate during the test. The device added randomly one by one. From the combination, there are 12 results that can be evaluated. The result can be seen in Table 2.
All of the test results show that the system ran well and produced appropriate results. In the 12 tests, the system succeeded to detect new device that being added.

Device Movement Detection
In this scenario, the simulation would move selec-ted device and looked for the system response. The simulation would do five movements with same resident's position. The results can be seen in Table 3. In this simulation, all of the test results also show that the system ran well and produced appropriate results for five-movement case.

Device Abstraction with Device Removal
This simulation chooses three coordinates and four devices removal in each coordinate during the test. The device randomly removed one by one. From the combination, the test would get 12 results that could be evaluated. The result can be seen in Table 4.
All of the test results show that the system ran well and produced appropriate results. In the 12 tests, the system succeeded to detect new device that being removed.   From the all test, the proposed method gets 44 appropriate results and 10 not appropriate results over 54 tests. It means that practically the system did their job well on 81.48% and the system makes mistakes on 18.52% of their job. It could be said that the proposed method has promising result to be implemented.

Conclusion
In this paper, a method to identify user's location is proposed. The proposed method use projective transformation to detect and identify user's locateon in a perspective view. The user's location information is used for identify the nearest devices from the user. After it is tested with various scenarios, the proposed method has good response to solve the problem, with 81% result appropriate with the expected result. In the future, more ex-