Wikipedia: “A self-driving car, also known as an autonomous vehicle (AV or auto), driverless car, or robo-car is a vehicle that is capable of sensing its environment and moving safely with little or no human input.

Self-driving cars combine a variety of sensors to perceive their surroundings, such as radar, lidar, sonar, GPS, odometry, and inertial measurement units. Advanced control systems interpret sensory information to identify appropriate navigation paths, as well as obstacles and relevant signage.”

Self-driving cars came to be one of the most interesting areas of Artificial Intelligence. With the high growth of companies like Elon Musk’s Tesla, which promotes Electrical Powered vehicles as the future of human civilization, it also opens a huge spot for turning those vehicles into AI-powered machines in order to make our day-to-day life easier.

If you are a car enthusiast, you must know of the evolution of the driving assistance systems of modern cars, starting as a gimmick just a few years ago, now are standard to almost every car. Handling things like a reaction to speed bumps, reading speed signs on the highways, steering you into the right column, keeping a safe distance from the car in front of you, self-parking, etc… are just a few examples of what these new cars are capable of.

There are tons of companies that work on creating self-driving car systems, that are not traditionally car companies. Examples of companies like this are NVIDIA, Comma AI, Waymo, Lyft, etc.

Now, with a growing industry like this, it means that there will be a huge flow of money, which means this is important. As we mentioned before, Autonomous Vehicles systems are one of the most popular areas of AI, since they are a combination of different AI techniques and fields.

For example, in the core of each of these systems, there is Computer Vision which reads data from the cameras and sends to the system of the car, there is Reinforcement Learning meaning that the car acts like an agent in the environment it drives, there are Recommended Systems, which assist you driving (safe distance, lane change), and plus, on top, everything is powered by Deep Learning.

Dropping all of these names means that that flow of money is justified, since you are combining the hottest trends of Computer Science and the Automobile industry.

The AV systems industry is expected to expand to over $250 billion by 2025 in North America, and the engineering positions in this industry to over $400k per year according to Forbes.

Now, this means, that this is important, and if you want to make the top money as an AI engineer, this is the industry to be a part of.


In this article, we are going to try to learn, how the process of creating a self-driving car system goes, by using a very good example from a company named Valohai. So it will be a tutorial on how can we can create the system for self-driving cars with Python, which means it will be easy to understand for everyone.

Valohai: “Valohai is the only machine learning platform built for private installations with the company’s intellectual property’s safety at the core.

Boost your data scientists’ productivity by letting them concentrate on the model building while Valohai automates your MLOps. Build an automatic regulatory compliant audit-trail from experiment to inference, with Valohai’s automatic version control.

Valohai integrates with any framework from TensorFlow to DL4J, any toolset from Jupyter notebooks to GitLab, and any environment from Azure to AWS. It’s built on top of an open API that ensures you can integrate it with anything custom within your enterprise.

MLOps (machine learning operations) is a practice that aims to make developing and maintaining production machine learning seamless and efficient. While MLOps is relatively nascent, the data science community generally agrees that it’s an umbrella term for best practices and guiding principles around machine learning – not a single technical solution.”



How to build a self-driving car system using Python:

Valohai’s example gives us two routes. We can use the prebuilt dataset gathered by them, or we can gather our own dataset, by using their driving simulator. In the interest of time and resources, we are going to use the prebuilt dataset.

First of all, we will download the dataset, following the link here. Then we are going to download the simulator by clicking on the link with the name of your operating system: Linux, Windows, Mac.

Then you want to clone the Valohai project from their repository:

When you are done with this step. This is what the project structure should look like.


Project structure for self-driving car system

Image 1: Project structure for self-driving car system


The next step is to install the requirements. To do that, open the terminal, go to your project, and run “pip install -r requirements-drive.txt”.

A little side note, if you don’t want the libraries from the requirements folder to install on your system, you can create a virtual environment first, activate it, and install them inside the environment. To do that, just run “virtualenv venv” in the terminal (venv is the name, it can be whatever you want it to be).

After that, you want to activate it. If you are on Linux just run “venv/bin/activate”, if you are on Windows, copy the absolute path to the activate.bat file and run it in the terminal “D:\selfdriving-example\venv\Scripts\activate.bat”.

After this, you can run “pip install -r requirements-drive.txt”.

After the installation is done, we are going to unzip the prebuilt dataset.


We can see that there is a .csv file with information about:

  • Image file from the LEFT camera
  • Image file from the CENTER camera
  • Image file from the RIGHT camera
  • A new row in driving_log.csv file with:
  • The orientation of the steering wheel (-1 to 1)
  • Status of throttle pedal (0 to 1)
  • Status of the brake pedal (0 to 1)
  • Speed of the car (0 to 30)


There is the IMG folder, with the images from the camera.

Now, in order to run this, we must create an account to Valohai, which is free, and then provide a link to our GitHub repository (fork the example repo from Valohai or just provide their link)

After this go to Valohai, and login/create a profile. And create a new project. As shown in the image below.


Valohai project for self-driving car system using Python

Image 2: Valohai project


Then follow the steps on the picture below, to reference your GitHub repository from the Valohai project.


How to Add Add GitHub to Valohai

Image 3: How to Add Add GitHub to Valohai


After that click on the Executions tab and create new execution, as shown in the image below.


How to Create Execution for Self-Driving Car System

Image 4: How to Create Execution for Self-Driving Car System


Dataset URL

Image 5: Dataset URL


On Image 5, you will have to provide the data, but in this case, you don’t have to since everything is set in the .yaml file of the project.

After this, we just execute the configuration.


Executing the configuration

Image 6: Executing the configuration


After the execution is done, you will download the output and put it into the project directory on your local PC.


How to Download the output

Image 7: How to Download the output


After you added the model.h5 into the project directory, open your simulator, and click on the AUTONOMOUS MODE button, and go to the terminal and run “python model.h5”.


Run self-driving car simulator

Image 8: Run self-driving car simulator



Self-Driving Car Simulator running

Image 9: Self-Driving Car Simulator running


In Image 9, we can see the simulator running, you can drive the car using the arrows on your keyboard, or the buttons w, a, s & d, but we need to see how the Autonomous driving looks, we can play the game later.

After this just run “python model.h5”, and the car should drive.

Here is an example video of how it should look.


If you want to use your own data, you will need to record it first by running the simulator in Training mode. It will generate IMG and .csv files like in the .tar file with the prebuilt data. After this, you need to add your data to drive or upload it to your Valohai project. Then you will need to edit your .yaml file to know where to check for data in the training environment, and that should be it.

NOTE: You don’t need to use Valohai to train the data, but keep in mind you need a powerful machine to do it, and Valohai has that.



So, we made our first self-driving simulation. Not really the hardest, since most of it was already made, but as I said, you can send your own training data, and compare the results.

As I said, it is a very interesting industry combining many different AI fields and techniques, which as we already concluded, leads to high salaries. So let this be a motivation for you to dive into this field, and start learning.

You can find Valohai’s blog here. The blog is based on Manajit Pal’s tutorial.

If you are interested in learning Artificial Intelligence or Computer Science for free, check our previous articles:

Like with every post we do, we encourage you to continue learning, trying, and creating.

Facebook Comments