Self-Driving Cars Will Look At You To Learn What To Do

Illustration for article titled Self-Driving Cars Will Look At You To Learn What To Do

Self-driving cars spend a lot of time looking at their surroundings to know how they should respond to the road. But autonomous cars will likely spend some time looking at you to work out how they should behave, too.


A team of researchers from Cornell University has been developing a new system which trains a camera on the driver in order to understand what they’re up to, predicting actions from three seconds out. In the first instance, the system could be used to supplement driver aids that already appear on cars, but in the future it could be used to help train autonomous cars by providing them with a rich seam of extra data.

The team has built a Recurrent Neural Network which takes in a number of data sources to predict what the person behind the wheel might do. It assembles measurements of speed, GPS positions, and head orientation of the driver to assign a probability to whether or not the car will change lanes in the next few seconds.

Illustration for article titled Self-Driving Cars Will Look At You To Learn What To Do

After collecting data from 10 different drivers who covered 1,000 miles of freeway and city over two months, the team annotated the footage in order to train their neural network. With 700 events—300 lane changes, 130 turns, and almost 300 randomly chosen examples of driving straight—chewed through, it was then let loose on live data to guess what might happen. The team write in a paper published on arXiv that the software “can anticipate maneuvers 3.5 seconds before they occur in realtime.”

That’s pretty impressive—but then, lane changes on freeways are probably some of the easiest to predict. It will be interesting to see what the team can actually do with the data for current and forthcoming cars, and how they can tweak it to predict more subtle actions on busy city streets.

[arXiv via arXiv Blog]

Image by HomeArt/Shutterstock



The English Guy


So Prius’ AI will learn to hypermile annoyingly in the fast lane while directing Siri to yell smug abuse at pickups.

BMWs will auto park diagonally and lane change without blinkers.

Pickups will identify the presence of lift kits and Got Dirt? stickers and know to never go off road.

Corvettes will detect groups of schoolgirls, slow down and rev sexily.

Buick Regals will drive through farmers’ markets.

I, for one, loathe our robot overlords.

We were a big enough bunch of douchebags to begin with. For The Maker’s sake, don't copy us!