Tools

Two robots were used for this project: FortyTwo, a Nomad 200 and Charm, a Nomad Scout. All we have to know for now is that both run the Linux operating system and use a frame grabber which is able to take images at a resolution of 640x480 pixels. These robots are described in more detail in sections 3.2.1 and 3.3.1, respectively.

The only inputs used in the experiments performed are a sequences of images grabbed by the CCD camera and the internal odometry of the robot. Internal odometry is only used during a learning stage. So the robot performs its task by using visual sensors and only visual sensors. Furthermore, sonars and infrareds have been used in a safety procedure in order not to damage the robot.

In this chapter I am going to talk about the vision process used to transform the sequence of images into consistent outputs for the tasks. Figure 2.1 describes the mechanism used for both corridor following and ego-motion prediction.

Figure 2.1: Overview of the mechanisms used. The images grabbed are processed to extract features which are used as inputs for a neural network. The neural network then predicts the result, which can be either a steering angle or a distance estimation.

The video camera continually grabs pictures. These pictures are processed in order to extract features. Then a neural network predicts the output using the features of the picture and possibly the difference between the current picture and the last one.

For the corridor following, the requested output is the direction to follow given by the steering angle. The neural network uses only the current image for this part of the project.

For the ego-motion estimation, the requested output is the distance between the two frames whose features were provided to the neural network.

The following sections describes in more detail which features were extracted from the frames and which neural network architectures were used.