Week 6 (29/3 – 04/4) – Improvement of gradient grids and creation of sobel filter grids

Although being fully functional, the work developed in previous weeks needed improvement, as the program used too many computational resources and becoming increasingly slow with the addition of new features.

In that line, the code already developed was reformed to become faster and more efficient. The cube list markers were replaced with occupancy grid maps, allowing to define the probability of each grid square to be an obstacle.

The grids are basically defined by its height, width, and resolution, and allow its placement in a specific frame, in this case, the moving_axis frame, as previously decided. The occupancy maps provide better and faster visualization, being much easier to see areas with higher densities or gradients, as visible in video 1.

Also to decrease the computational resources used, instead of counting how many points were in each square, which requested N x M x np (number of points in the point cloud) iterations, now, for each point cloud point, the program calculates instead in which square it belongs. This requires np iterations, resulting in a very significant decrease in the number of iterations.

Video 1 shows that the gradient direction seems to have little significance in road limits detection, unlike the gradient magnitude and its components, where darker lines are clearly visible. It’s also worth noting that the algorithm was developed to mark positive and negative gradients as an obstacle, meeting the initial goals.

Vid. 1: Improved density and gradient occupancy grids. (Daniela Rato, 2019)

Despite the good results obtained from gradient filters, testing other solutions is important to select the best one. In that line, a new grid was created, calculating edges with a Sobel filter (fig. 1) applied to the gradient matrix. The obtained results are visible in videos 2 and 3, that show the algorithm running with different parameters.

The Sobel filter is a good edge detecting filter, and by the analysis of the video, it produces good results in edge detection in the point cloud data, although there are more defined lines when it comes to cars and buildings.

Fig. 1: Sobel filter – horizontal and vertical directions. (https://blog.saush.com/2011/04/20/edge-detection-with-the-sobel-operator-in-ruby/)

Vid. 2: Sobel filter occupancy grid. (Daniela Rato, 2019)
Vid. 3: Sobel filter occupancy grid with different parameters (Daniela Rato, 2019)

Edit: Also, Prewitt and Kirsh’s filters were tested in point cloud density edge detection. Both filters show good results detecting road limits, but still identify road as an obstacle in some cases, depending on road conditions, car velocity, and inclination. Video 4 shows the results obtained in recorded rosbag around University of Aveiro.

Vid. 4: Prewitt and Kirsh edge detection applied to an occupancy grid map (Daniela Rato, 2019)

Week 5 (21/3 – 28/3) – Creation of density and gradient grids

In the first part of the week, a method to evaluate the point cloud density inside a cuboid with a predefined size was developed. This allowed creating a grid of cuboids in which each one had its own correspondent density. The purpose was to divide the space of interest in small enough portions for later evaluate density changes.

The first necessary procedure was to transform the PointCloud to the moving_axis frame of the car, and then use the PCL CropBox tool to crop the PointCloud in the defined cuboid volume and count the numbers of points in the output cloud.

Vid. 1 shows a dynamic use of this method. For better visual outcome, each square of the grid has a color that matches a range of densities, which allows already to some understanding of what are the low and high density zones.

Vid. 1. Density grid demonstration (Daniela Rato, 2019)

After the development of the density grid, followed the implementation of similar gradient grids. To each cuboid, the vertical and horizontal gradient were calculated (Gx and Gy, in vid. 2), as well as the gradient magnitude and direction.

In vid. 2 there is a representation of the four gradient grids with the color bars created (fig. 1).

Vid. 2. Gradient grids demonstration (Daniela Rato, 2019)
Fig. 1. Colobar correspondent to the gradient visualization.

Week 4 (15/3 – 21/3) – Medium road plane and total accumulated cloud

One of the main challenges presented in this dissertation is how to discover something that doesn’t exist. It was relatively easy to find where the biggest concentration of points was (corresponding to positive road curbs) but, as observed in fig. 1, when talking about negative road curbs there will be a “shadow” in the point cloud, that is, an “empty space”. So how will we identify the absence of points?

Fig. 1. Positve vs negative road curbs

After discussing what would be the best solution to solve this problem, the decision was to calculate the medium road plane, and divide the space above it into small cuboids, find the point cloud density in each one and calculate the “negative” point cloud, that is, removing points in the high density zones and inserting points in low density zones, resulting in a cloud with only points in low (or zero) density zones.

Concatenating this point cloud with the one given by the road_reconstruction_node ( the one who identifies positive obstacles) we will have a point cloud where points represent obstacles, which means known areas where the vehicle can not go because there are obstacles in that zone.

In that line, the first thing to do was to calculate the medium road plane. For that, a ransac technique was applied to the raw cloud, resulting in a cloud with the inlier points and the respective plane coefficients. That cloud is represented by the black points in vid. 1. After that, a marker with a planar shape was created and adapted to the plane coefficients calculated before, as it shows in the following video. This allowed to define the search area for the spatial partition feature.

Vid. 1. Demonstration of the developed plane fitting feature in a rosbag record (Daniela Rato, 2019)

The fact that the total accumulated point cloud of a certain path may also provide interesting information was also discussed, not also in the line of this work but also in 3D road reconstruction through point clouds. So a feature that generates a point cloud was created, to which it will constantly be added new points corresponding to laser readings obtained from the car movement. This cloud is periodically saved in a .pcd format that can be visualized in the pcl_viewer.

Fig. 2 shows the comparison between the resulting point cloud and the corresponding real circuit and the reliability of the obtained results.

Week 3 (8/3 – 14/3) – Simulation tools

One of the goals of this work is to complement the simulation tools already created to contemplate the study of negative obstacles and inverted road curbs.

Simulation tools are very important to visualize and understand the problem before field analysis and test different algorithms.

In the line of this work there are two simulators in progress: a Matlab simulator that is based of mathematical equations to simulate laser beams and its intersections with a defined plane, and a Gazebo simulator that has a 3D model of the AtlasCar2 and has features that allows the definition of different types of LIDAR and also 3D models to simulate different types of roads. The last one is implemented in ROS and has the ultimate goal of simulating the movement of the car along the road and create a point cloud as if the real world car was actually moving.

Gazebo simulator
Matlab simulator

Both this tools are still in progress with the final goal of simulating different types of situations in different road conditions.

Week 2 (1/3 – 7/3) – The understanding of previous work

After the study of different works and approaches to road detection using LIDAR, it was time to understand the work already done in the AtlasCar2 in previous years.

In the following figure can be seen a play of a rosbag that recorded a trip around University of Aveiro. The white points in the image represent the raw point cloud obtained directly from the LIDAR readings, and the colored points represent a point cloud where the algorithm filters are already applied.

This algorithm consists on the application of a voxel filter to decrease the point cloud density, followed by the elimination of points that have few neighbors in a predefined radius, in this case, less than 15 neighbors in a 0.2m radius. This allows to identify the zones with higher density that correspond to well defined perpendicular road curbs.

Where this algorithm fails is when there are negative obstacles that form a shadow and represent a zone where there is no points in the point cloud.

It is then concluded that the elimination of points from the raw point cloud is not the best way to go, or at least if not combined with the parallel use of other algorithms that analyze not only the high density zones as well as the low or no density zones.

The Journey Begins

In today’s world there is a constant search for speed, autonomy and efficiency. People constantly search for solutions and technologies that can upgrade their lifestyle and save them time. In that line, the development autonomous vehicles are growing at high speed and are one of the most talked topics of the present.

One of the main fields of autonomous driving is the perception of road and to determine the navigable limits of the road, that is, the limits where it is safe for the vehicle to move. Therefore, arises the theme of this dissertation that aims to find a solution for that problem with a use of a LIDAR sensor mounted on the front of a car close to the ground. LIDAR sensors measure the distance to the closest object and with them it is possible to withdraw a point cloud with three dimensional points corresponding to the intersection of laser beams with the objects ahead.

There are three main goals to be achieved in the end of this dissertation. The first one is to parameterize and improve the existing solution of road curb detection and obstacles at road level. The second one is to study and develop a solution for low curbs with accumulated point clouds in LIDAR. And the last one and to test and integrate the developed solution in the real environment aboard AtlasCar2.

This dissertation is one among many done in the AtlasCar2 and pretends to complement and improve the previous work. The majority of the work will be done aboard the car and in LAR (Laboratory of Automation and Robotics) and will be oriented by professor Vitor Santos.