New indoor navigation system for visually impaired people using visible light communication

Abstract

In this study, we propose an indoor navigation system that utilizes visible light communication technology, which employs LED lights and a geomagnetic correction method, aimed at supporting visually impaired people who travel indoors. To verify the effectiveness of this system, we conducted an experiment targeting visually impaired people. Although acquiring accurate positional information and detecting directions indoors is difficult, we confirmed that using this system, accurate positional information and travel direction can be obtained utilizing visible light communication technology, which employs LED lights, and correcting the values of the geomagnetic sensor integrated in a smartphone.

1. Introduction

Pedestrian support for visually impaired people involves the use of textured paving blocks, guide dogs, GPS-based voice navigation systems, among others. On the other hand, studies aimed at visually impaired people report that there is a need for voice information inside buildings [1], and that, in the future, we will need adequate indoor pedestrian support systems in large commercial facilities, such as shopping centers and underground shopping malls. However, compared to public spaces and transport facilities, no progress is being made in providing commercial facilities with textured paving blocks [2], and although guide dogs are effective on obstacle-free safe walkways, they cannot locate a person’s destination. Moreover, the GPS’ inability to give an accurate position indoors is equally a problem. Therefore, positioning methods using radio waves emitted from a wireless LAN access point are increasingly employed. However, this method has encountered issues with fluctuating positional accuracy due to reflected signals from the wireless LAN, obstacles, or the surrounding environment [3]. Studies into guidance systems using tactile maps, which are effective in creating mental maps, are also underway. However, it takes time to understand a tactile map by touch, and therefore, they are difficult to use while on the move. To address these kinds of issues, our study aims to create a usable system that enables visually impaired people, especially the blind, to travel indoors unaided. A variety of data is required to travel indoors, such as the accurate current position, travel direction, distance to the destination, and information about the barriers and surroundings. This study concentrates on solving problems relating to the current position, the travel direction, and the distance. As an indoor positioning method, we focussed on visible light communication technology using the ubiquitous LED lighting. As LED lighting is often installed in pathway ceilings, accurate positional information can be sent naturally from above the user’s head. Further, we used the geomagnetic sensor in the already widespread smartphone for a method to survey the travel direction. However, because of situations where geomagnetic sensors cannot detect the accurate direction due to the effect of, for example [4], rebars, and because it has been reported that they are not adequate to maintain the correct direction when walking long distances [5–7], we worked on simply improving the directional accuracy by obtaining and correcting the geomagnetic information beneath the LED lights in advance.

This article is structured as follows. In section 2, we present the necessary components of an indoor navigation system, namely, visible light communication technology and indoor map data, and then, we will discuss in detail the correction method for the pedestrian’s position and direction detection. In section 3, we present the design and implementation method of the indoor navigation system, and in section 4, we present the test results for this indoor navigation system. Section 5 discusses this system, and section 6 presents the conclusions and future research.

2. Our approach

In this section, we describe the positioning method and the method for creating indoor map data through visible light communication, components of our ‘indoor navigation system’ and the direction detection method through geomagnetism.

2.1 Positioning for visible light communication

As mentioned previously, methods used for indoor positioning make estimates from the position or radio wave strength of the wireless LAN access point or use wireless active tags. However, these methods have issues with fluctuating positional accuracy due to environmental factors such as obstacles. This makes them difficult to use as a support for visually impaired people in their travels. Therefore, we have focussed on visible light communication technology using LED lights. When using visible light communication, it becomes possible to identify the user’s position within a range of 1–2m. Moreover, visible light communications technology enables users to find out which floor they are located on. The indoor positioning system by visible light communication, Wireless LAN and, RFID which use is being investigated, are shown in Table 1[8–10]. Further, this is shown concretely in Figure 1.

Figure 1

Using visible light communication.

Table 1 Indoor positioning system

Visible light communication is a communication method that uses light perceptible to the human eye, and being already standardized as a communication method by Japan Electronics and Information Technology Industries Association in Japan, it can be used as a visible light ID system (CP-1222). The specifications of the visible light ID system are shown in Table 2, but for this study, we somewhat improved the system. Information can be embedded directly into the visible light ID system and sent or the positional information, for example, latitude and longitude can be obtained through an external server after a unique ID has been sent and received.

Table 2 Specification of Visible light ID system

2.2 Creation of indoor map data

It is difficult for visually impaired people to look directly at maps, but accurate indoor map data are necessary to calculate and provide accurate positional information, travel direction, and distance. Indoor map data can also be represented in a local coordinate system that does not use geographic references. However, indoor map data, beginning with map portal sites, are often expressed using a geographic coordinate system. When considering outdoor–indoor seamless navigation, for example, integration with a geographic coordinate system is preferable. For that reason, we decided on a geographic coordinate system for the indoor map data used in this study. Indoor map data are built with the method for mapping each floor plan managed by the Legal Affairs Bureau, as proposed by Nakajima [11]. The LED lights from nodes and network data linking the LED paths are created on the basis of the voice guidance’s need to retrieve the route to the destination. The small round points in Figure 2 right are LED lights.

Figure 2

Indoor map data.

2.3 Compensated geomagnetic sensing

For example, in car navigation systems, the direction in which a user is travelling is detected from GPS and the acceleration sensors. For travelling indoors, Nordin and Ali [12] propose a method to infer the travel direction using node information. However, both methods have a drawback in that the direction is not known unless the user starts travelling. Therefore, we decided to show the travel direction by calculating the smartphone’s orientation, using its embedded geomagnetic sensor. We also decided to simultaneously calculate the distance from the indoor map data. The procedure by which the travel direction and distance are calculated is shown in Figure 3.

Figure 3

Flow of direction finding and distance calculation.

First, the geographic coordinates are obtained from the visible light ID sent from an LED light. Second, the geomagnetism beneath the LED light is obtained from the geomagnetic sensors. Geomagnetic sensors show the absolute orientation by detecting the direction of the earth’s magnetic field lines, but as mentioned previously, we know that the magnetic field can be distorted in indoor spaces with rebars, etc. Therefore, we obtain the geomagnetic data beneath the LED light in advance and make corrections using these values. Third, we obtain the next LED lights’ geographic coordinates from the route to the destination and calculate the angle of the travel direction and the distance.

3. Development of indoor navigation system

In this section, we present the design and implementation method for the indoor navigation system.

3.1 Indoor navigation system design

We designed the indoor navigation system using the component technology mentioned in section 2. As shown in Figure 4, the system is composed of LED lights, a smartphone with integrated receiver and headphones. For obtaining positional information and calculating the route retrieval, both necessary for the indoor navigation system, we use a cloud environment from Panasonic that is referred to as the positional information base. The visible light ID is sent from the LED light using visible light communication and is received by the receiver. Then, the smartphone obtains the visible light ID from the receiver using Bluetooth. The smartphone receives the positional information, etc., from the positional information base through Wi-Fi, and the positional information and the guidance content are combined in audio files using a speech synthesizer system (Table 3) and sent to the headphones.

Figure 4

Indoor navigation system.

Table 3 Voice guidance list

The overall system sequence is shown in Figure 5. First, the user long-presses the smartphone screen, using the touch screen, to indicate the start of the voice navigation The smartphone requests the positional information for the whole floor surface from the positional information base and on receipt informs the user of the destinations with their corresponding numbers. Second, the user inputs his destination number using the speech recognition function, the smartphone requests the route from the positional information base and, on receipt, informs the user of the entire route. Third, the user starts to travel, and when the system receives a visible light ID from an LED lamp while the user is travelling, it calculates, in the sequence shown in Figure 3, the travel direction and the distance and informs the user by combining this in speech as shown in Table 3. If the user has strayed from the route, the user is again informed of any route changes. This is repeated until the destination is reached, upon which the user ends the guidance by pressing the smartphone screen for a long time.

Figure 5

Sequence diagram of system.

3.2 Implementation of indoor navigation system

We implemented the system based on the design described in section 3.1. We used Eclipse as a highly compatible integrated development environment as it needs to run on the Android OS, as shown in Table 4, and we developed the application using the Android Development Tool and JDK6. In the verification test, we used a Panasonic P-07C smartphone. We further prepared an additional smartphone for the test, created a visible light ID data frame as a substitute transmitter according to this study’s specifications and transmitted using Bluetooth.

Table 4 Development environment

4. The verification test and result

In this section, we present the verification test and document the verification log analysis results and the interviews with the test subjects.

4.1 The verification test

First, we mounted LED lights transmitting visible light IDs on the ceiling. As shown in Figure 6, we set the illumination range height to be at 1400mm from the floor and 1076mm across, and we enabled receipt of the visible light ID within this range. Moreover, we measured, at a height of 1400mm from the floor beneath the LED light, the azimuth based on the true north using a magnetic compass.

Figure 6

Accuracy of visible light communication.

Then, we measured the geomagnetism beneath the LED lights and identified the geomagnetic distortion ranging from 40° to −157° (Figure 7). We decided to provide guidance by calculating the travel direction with these values as the correction values.

Figure 7

Geomagnetic sensor value.

Then, we asked visually impaired people to try the system, and while obtaining logs from the visible light ID receiver and the geomagnetic sensor, we also conducted interviews with the subjects. For the verification test, we gained the cooperation of the Osaka Municipal Association for the Welfare of the Visually Impaired and conducted the test with six visually impaired people. Their attributes and the test equipment, etc., are shown in Table 5.

Table 5 Demonstration experiment

The subjects had a smartphone with an integrated receiver around their neck and headphones on their ears. To make sure it was not difficult to hear external voices while travelling, we used open headphones with pieces of sponge attached to the headband so that the ears did not get covered (Figure 8).

Figure 8

Test prototype machine.

The test course, as shown in Figure 9, started beyond the reception area (icon ‘S’) and a as goal, had to reach the multipurpose toilets (icon ‘G’). From the starting point, the course went straight ahead for 25m and then made a sharp left turn. Then, it then turned right after running approximately 20m straight ahead, with the multipurpose toilets 5m on the right side.

Figure 9

The test course.

We conducted the test for one person at a time, with the users walking on their own from operation to destination. We first explained and then executed the procedure using the indoor navigation system, until the destination was reached.

While the subject was travelling, the sequence shown in Figure 3 was followed, based on the visible light ID and the geomagnetic values obtained from beneath the LED lights, and the subject was informed of the travel direction and distance. If the subject strayed from the route while travelling, the route was recalculated and the subject was re-informed.

4.2 Result of verification test

For the travel direction logs, we obtained the visible light ID and the geomagnetic values beneath the LED lights for each subject. The results, as shown in the graph of Figure 10, were identified the subject’s position within range of 1–2m and all corrected in the range from 66° to −59°. Essentially, the correction results on the graph are 0°, and the geomagnetic position values do not vary. However, when the subjects travelled with the smartphone suspended from a strap around their neck, making it swing more than anticipated, the position for which the geomagnetic value was obtained and the corrected value diverged, and hence errors occurred. There were further errors for each LED light position; with the smallest error range being for the position in which the visible light ID ‘031103’ was received, they ranged from 10.54° to 9.47°. We also established that there was an error variation for each subject. The person with the most stable error was subject F, with errors ranging from 13.89° to −16.62°. On the other hand, subject E had the highest variation ranging from 66° to −37°.

Figure 10

Result of geomagnetic correction.

Test results showed that three people with low vision reached the goal. Three blind people arrived at the goal by checking the position of the wall along the route with their cane a few times.

4.3 Interviews

After the test, we conducted interviews with all subjects.

These covered various issues, but for this article, we will limit the discussion to the guidance functions relating to position, travel direction, and distance (Table 6). People with low vision expressed opinions about the overall route and distance, while blind people voiced opinions about the travel direction and the need for a more accurate match with the timing of the guidance.

Table 6 Result of interview

5. Discussion

In this section, we will discuss the indoor navigation system, particularly the positional information and the travel direction functions. We will also discuss the results of the interviews after the test.

5.1 The positional information and the travel direction

We reduced the error range through geomagnetic corrections and provided more guidance accuracy for the travel direction. However, apart from the LED light ‘031103’, the lights had errors greater than equal to ±15°, and we could not consistently provide correct travel direction guidance. As previously mentioned, the reason lies, in addition to spurious noise, in the fact that the subjects travelled with the smartphones suspended from a strap around their necks, and that with the swinging, the obtained geomagnetic sensor values changed. It is conceivable that the smartphones swung excessively at the spot where the visible light ID ‘030204’ (which had a particularly large error) was received, because the subjects were close to where they made a few successive turns. To confirm this inference, the authors, after the test, travelled the course with the smartphone held still in their hands. As shown in the results in Figure 11, the geomagnetic errors settled in the 8° to −29° range. Realistically, it would be difficult to travel while keeping a smartphone still and holding a cane. We believe this can be dealt with by fixing an optical sensor to, for example, a belt, in a manner that would not obscure the sensor from visible light, and by keeping the smart phone in a pocket and transmitting using Bluetooth.

Figure 11

Result of holding the smartphone.

In addition, although this was not an issue with this experiment, there is the possibility of situations occurring where the magnetic field changes and the correction does not function because of floor alterations or the presence of magnetic materials in the actual space. To solve this problem, geomagnetic values need to be obtained on these occasions, which is a hurdle in actually applying the system practically.

5.2 The results of the interviews

Because an error in the direction equivalent to a one-hour difference in the clock position is a big problem for visually impaired people and especially the blind, we concluded that a higher directional accuracy is needed. In addition, because there is an individual variation in the guidance methods for the travel direction, a significant improvement could be made by employing a configuration that allowed the user to choose, apart from clock positions, modes for guidance using, for example, eight directions or angles.

On the other hand, as some visually impaired people walk fast, the timing of the spoken navigation needs to be matched as closely as possible to the travelling speed. We need to consider a configuration where guidance is given not at a bend, but approximately 1–4m before the bend. Moreover, a feedback sound is played when an LED light is correctly passed, but, because blind people usually feel uneasy about whether they are walking a route accurately, we also want to consider a configuration that provides a continuous feedback sound while travelling.

6. Conclusion

We developed and tested an indoor navigation system for visually impaired people using visible light communication that makes use of LED lights and a geomagnetic sensor integrated in a universally used smartphone. To support travel for visually impaired people, accurate guidance for the positional information and travel direction are needed, and we have confirmed that the positional and directional accuracy improves through visible light communication and geomagnetic value correction. Therefore, we have concluded that our approach will be effectual system for the visually impaired people. An issue to consider in the future is the need to establish an azimuth accuracy detection method.