Drivers and pedestrians are expected to follow the road rules they learned in school but in reality, that is often not the case. Pedestrians cross against a red light if the road is clear. Drivers lose patience in slow traffic and cut into the next lane when they see an opening.

While today’s autonomous driving systems have made great progress in avoiding accidents by using sensors and cameras they are still a long way from being able to accurately anticipate what an unpredictable human being might do. Experts say that technical limitation will slow the widespread adoption of autonomous driving in urban and city environments.

Now a number of startups are trying to solve the problem by building human behavior data sets that help self-driving car systems predict and understand human behavior more accurately.

“A human behavior data set will be a great aid in the current development phase of self-driving cars,” said Li-Ta Hsu, assistant professor at the Hong Kong Polytechnic University, whose research work includes autonomous driving. “People spend [their first] 18 years learning social rules to be able to pass a driving test. Current autonomous cars don’t have that level of social understanding.”

An autonomous car is driven on a test track for automated driving in Berlin, Germany, September 19, 2019. (Picture: Reuters)

Berlin-based Phantasma Lab, started by two entrepreneurs who became friends after they met at a talent investor program in the German city, is developing a human behavior data set using virtual simulations based on mathematical rules and is now in talks with Chinese self-driving car makers about adopting the technology.

The company was founded in 2018 by software engineer Maria Meier and simulation expert Ramakrishna Nanjundaiah after they finished a three-month program run by talent investor organization Entrepreneur First (EF) in Berlin. Phantasma Lab raised 1 million British pounds (US$1.2 million) in a fundraising round last year led by EF and is in talks with investors to raise more.

“There are so many individuals out there who could be a company founder, and should be a founder,” said Alice Bentinck, the co-founder of EF. “What we do is unlock that talent pool.”

The company has already started generating revenue from selling the data set to self-driving car companies. “This is a fundamental problem. [They] need a lot of this data and can’t get it from the real world,” said Nanjundaiah, the 31-year-old chief executive officer of the company.

Meier, 30, who is chief technology officer, points out that better sensors and cameras will not solve the problem. “Sensors will help the car recognize what was there, but not what to do after that,” she said.

The Berlin company sees China as one of its major markets and is in talks with several potential customers. “I get the feeling China wants to try out new technologies and is very future-oriented so Chinese companies may be more open to change than what I see in Europe,” Meier said.

Traffic during rush hour in Nanning, Guangxi province on September 18, 2019. Technical limitations will slow the widespread adoption of autonomous driving in urban and city environments. (Picture: Bloomberg)

Chinese autonomous driving companies like Baidu, and Auto X are investing heavily to push the development of self-driving cars in the country. Last month search giant Baidu launched robotaxi services in Changsha, the capital of Hunan province, with the vehicles restricted to a 50-kilometer-long open road trial area.

Meier represented Phantasma in the 2019 edition of the She Loves Tech global startup competition for women, taking home first prize. She became interested in technology after her father gave her an old computer and she began building a website to publish stories about her pets.

“Sometimes women are a bit underestimated. I’ve had experiences where a lot of people, mostly men, try to explain technical things to me and I have to tell them I already know because I’m a software engineer,” Meier said.

Phantasma’s competitors include US-based startup Perceptive Automata which uses behavioral science techniques to characterize the way human drivers understand the states of the minds of other humans, and then trains its deep learning models to acquire that same ability. Japanese car giant Toyota, whose AI venture capital unit is an investor in the US company, this year introduced the Guardian automated safety system which takes over from drivers if a life-threatening situation occurs on the road.

Still, the day you can hop into a driverless car and sleep soundly until it arrives safely at your destination is a long way off, according to experts. A further challenge for autonomous carmakers is that each city will have different levels of complexity when it comes to road rules and human behavior.

“What factors do you consider if you want to cross a street in Hong Kong? Is the road broad or narrow; are you familiar with the road and what if there is a policeman present,” said Hsu. “But it will not be exactly the same for a city in Germany or Japan. I don’t think it will be an easy problem to fix.”

Meier and Nanjundaiah agree it will be a slow road ahead to achieve that capability. “The time when you can deploy a self-driving car in any part of the world that can drive by itself – that is going to be at least a decade away, ” Nanjundaiah said.