Nvidia simulator and Safety Force Field and other news from GTC

Topic: 
Tags: 

This week I am at the Nvidia GPU Technology Conference, which has become a significant conference for machine learning, robots and robocars.

Here is my writeup on a couple of significant announcements from Nvidia -- a new simulation platform and a "safety force field" minder for robocar software, along with radar localization and Volvo parking projects.

News from Nvidia GTC

Comments

Mobileye told VentureBeat that close to a million cars are funneling mapping data back to Mobileye’s cloud platform, in addition to 20,000 aftermarket units. Jan 2019

If the average driver drove a 1000 unique miles per year, that is a billion miles of mapped motorways.

But what good is it just sitting there?

Does the BMW and GM blockchain interest in the Mobility Open Blockchain Initiative (MOBI) save Intel SDV program in time.

Or does Toyota change things?

Speaking of Mobileye's biggest advantage, Tong Lifeng believes that it is the core algorithm of machine vision. There are more than 800 algorithm teams in Israel.

In Israel, there is a 100 L4 self-driving team. source EEAsia

Only 100 on the SDV AV team?

Wonder how many at Aurora or Waymo

Always useful to see headcount comparison, even if skewed.

Toyota is 1900+ in TRI group.

Aurora now has about 200 employees per reCode

No breakthroughs here but "intersection detection" could become part of a presentation by the Vision Zero ADAS AEB / LKA improvement.

ICRI-IACV
Intelligent Driving Laboratory: Emphasis on learning from traffic accidents, simulation and safety verification.

The Intelligent Driving Laboratory was established in early 2018. It is dedicated to the research of key technologies of automatic driving, and the deep integration of automatic driving technology with intelligent transportation, intelligent city and other future directions, and the end-to-end technology research that promotes each other.

Wu Xiangbin, Director of Intel China Research Institutes Intelligent Driving Laboratory

According to Wu Xiangbin, director of the lab, the current research in the laboratory emphasizes learning about driving accidents, and through automatic deep accident analysis, automatic scene reconstruction and key scene library generation, and inheriting with the autopilot vehicle simulation tool, accelerate the performance simulation iteration and safety verification of the autopilot algorithm, and finally draw inferences.

In the aspect of road simulation for automatic driving, the laboratory has built random test for routine scene, key scene test and adaptability test for high-risk area. The test variables include extreme weather, lighting conditions, environmental visibility, road geometry characteristics and other combination conditions. In addition, Intel has sponsored an open source autopilot simulator.

In the aspect of vehicle-road coordination, traffic efficiency and safety are improved through intelligent real-time and full-view video analysis of key traffic scenarios. At the same time, Wu Xiangbin said that at present, the incidence of traffic accidents at intersections accounts for 50% of the worlds traffic accidents, so the laboratory will start from the intersection to study the construction of intelligent traffic intersections.

At the same time, the Intel Intelligent Network Lianhe Auto University Cooperative Research Center was officially launched in November 2018. As a connecting center, the Intelligent Driving Laboratory will cooperate with six research teams including Tsinghua University, the Institute of Automation of Chinese Academy of Sciences and Tongji University in the next three to five years to promote the large-scale practical deployment of automatic driving. This centre will study security, open data sets, human-machine interface/regulation and policy, advanced algorithms and architectures, networked vehicles and intelligent infrastructure.

Mobileye will have "24 million cars
mapping and sending data by 2022".

In addition, more possible as "number of OEM contracts in pipeline" (from today's Investor Day), as 24 million does not include new OEM pipeline.

Note that at OS kickoff this week in UK, interviews state several million already. So in 2 1/2 years, minimally adding an additional 22 million vehicles with REM harvesting capability for mapping from current contracts.

Shashua officially announces Ford with Nissan, BMW, and VW.

Also, Intel going "all in" in TaaS as a "Full Service Provider" for global RoboTaxi market. Intel building a "full stack MaaS" global solution.

Some slides as well.

Investor Day video
["https://youtu.be/V55ZhZJ2FFM"]

REM contracts with OEMs- over
24M vehicles by 2023, not 2022.
source: press kit pdf

The riddle is solved.

StradVision, which possesses deep learning-based camera image detection technology per Hwang Jae-ho

All the information can be referenced back to May 2018 when the radar projects started. Very detailed background information if you look

Use both headline for details
Hyundai Mobis Aims to Develop All Autonomous Driving Sensors by 2020

Hyundai Mobis Invests in an AI Sensor Startup for Developing Deep Learning Cameras9

This may explain why Intel EyeC radar group started hiring in August 2018.

Is EyeC radar a 4D imaging RADAR like Arbe ?

Does the talent wars cause the design of mmwave 4D imaging RADAR to linger like 5G smartphone modems.

The riddle is solved.

StradVision, which possesses deep learning-based camera image detection technology per Hwang Jae-ho

All the information can be referenced back to May 2018 when the radar projects started. Very detailed background information if you look

Use both headline for details
Hyundai Mobis Aims to Develop All Autonomous Driving Sensors by 2020

Hyundai Mobis Invests in an AI Sensor Startup for Developing Deep Learning Cameras9

This may explain why Intel EyeC radar group started hiring in August 2018.

More inclined to believe Daimler and not BMW, and Bosch loses out.

If a combined camera and radar suite
is key to success as Mobis states, Intel motivation for developing EyeC Radar could either be economic or performance optimization.

"can implement optimal performance for autonomous driving only by securing all of the three technologies (perception, decision and control)"

StradVision has won two orders from a Tier-1 company to supply StradVision’s object detection software for a premium German automotive manufacturer

Mobileye design or program wins in 2023-2024 will take a punch given likely losing design wins to Hyundai Mobis.

April 11. 2019
"How to Run Millions of Self Driving Car Simulations on GCP"
on YouTube

CIFUS objected to Navinfo investing in HERE, so Toyota TRI-AD's
Autonomous Mapping Platform AMP ambition to open source an HD map is great fodder for conspiracists.

For that matter, how and when Waymo, the German auto industry, GM, Apple, Intel+Mobileye, or Tesla react is hard to predict.

Quote about TRI-AD in interview in March 2019

"It is well known that Toyota is developing an open source autonomous driving HD map with the intention of grabbing 20 billion Markets"

SLAMcore or Perceptin.io founder Shaoshan Liu have been in VIO and V-SLAM space for some time. Is Intel Realsense r&d also in agreement with possible disruptive capability.

If the OEMs knew, where is the press on this story?

markets

"The majority of modern visual SLAM systems are based on tracking a set of points through successive camera frames and using these to triangulate their 3D position; while simultaneously using estimated point locations to calculate the camera pose that could have observed them. If computing position without first knowing location initially seems unlikely, finding both simultaneously looks like an impossible chicken and egg problem. In fact, by observing a sufficient number of points it is possible to solve for both structure and motion concurrently. By carefully combining measurements of points made over multiple frames it is even possible to retrieve accurate position and scene information with just a single camera."

v-slam still needs gnss to initialize with so Toyota TRI-AD needs tech, as well as OTA tech. Not sure if Mobileye 8 Connect has GNSS tech inside, though it has some type of wireless/OTA tech to expedite REM collection transfer. Intel has both GNSS and modems in in-house portfolio though. Not sure how much of 8 Connect is Intel inside though.

I wonder how many domain experts from the auto industry work in Israel for Intel.

/ The Intel Capital president said the semiconductor giant will continue competing with Nvidia in Israel as it makes strides in autonomous vehicle technology led by Mobileye.

"We are well ahead of everyone else, and though we may not have fully autonomous cars until at last 2021, we will get there first,” Brooks said.

From 2019 Intel Capital Global Summit on April 1, 2019.

"Bloomberg quotes Erez Dagan, executive VP of strategy for Intel's Mobileye unit, as having said at conference in Detroit that Ford has signed on to join Mobileye's Road Experience Management, or REM, platform."

SAE World Congress reporters could pry out more from Intel this week.

VW and Ford could be ironing out another partnership beyond vans and trucks, or even Argo.ai.

Mobileye China presentation in recent EEAsia article shows Ford as well, so it could be a China arrangement.

Comments from Argo.ai or Civil Maps may not be proper here so do not expect any. Volkswagen and Ford are still in talks per Ford CEO Hackett yesterday.

The CES 2019 slide number #7 could be a new VW project or a reference to Isreal project.

Today's Ford CTO Ken Washington interview of Mobileye at SAE is available as a video replay.

With the re-alignment today of Ford
With Jim Farley as president, new businesses, technology & strategy starting May 1, petsonally do not expect much until then for legal reasons.

Will EyeC engineering samples rollout coincide with EyeQ6

EyeQ6 2022 production estimate
unveil / fab prod / auto prod modyr
Eyeq4 Mar,2015 fp Q22016 ap Q1 2018
Eyeq5 Jan,2018 fp Jan 2019 ap Q1 2020
Eyeq6 est Q42019 est Q32020 estQ12022

Does EyeQ5 go product H1 of next year? Previous Post is typo error.
Bit of stretch so I doubt it.

H2 likely given SDK and on-die Atom.

"Toyota's budget and technology resources are inexhaustible, and it resolutely refuses to adopt Mobileye's technology. Toyota has always claimed that it could achieve better results independently than those of Mobileye, and therefore has no need to tie itself to the Israeli company's closed system. Since Toyota has close ties with a number of Japanese tier-1 suppliers, it can mobilize the awesome development resources of the entire Japanese industry for its needs."

Shie Mannor leads Technion in new
"TECHNION AND INTEL JOINT CENTER FOR ARTIFICIAL INTELLIGENCE founded on
Oct 9, 2018" but on November 7, Mannor joined FORD as well to lead a new SAIPS division funded with $12.5m designing a decision making system for Ford SDV. Mannor joined SAIPS IN August 2018.

Ford bought SAIPS IN 2016
yes 2016.

Welcome to academia.

Toyota, Cortica, Perceptive Automata. SLAMcore, Renesas .......

Toyota SDV Lexus demo in late 2019, and 2020 rollout of pilot is on it's way.

Intel has a true competitor.

Intel needs to show a breakthrough in the SDV space.

Though Intel+Mobileye is not standing still, cannot imagine what a breakthrough would be in localization and mapping and data collection outside of RSS adoption.

2019-Apr-12
ZF coPILOT debut today, available 2021. Needs 2 winters of testing?

Not a breakthrough. but free addition of Intel APB added to AEB will be bundled with tricam.

Intel Newsroom

Search Newsroom...
shashua eyeq5 2x1
Using Autonomous Vehicle Technology to Make Roads Safer Today
Technologies Developed for Fully Autonomous Vehicles Can Improve the Advanced Driver Assistance Systems Already in Wide Use
Editorial
January 8, 2019

Amnon ShashuaBy Professor Amnon Shashua

Safety has always been our North Star. We view it as a moral imperative to pursue a future with autonomous vehicles (AV), but to not wait for it when we have the technology to help save more lives today.

We fundamentally also believe that everything we do must scale, and we constantly search for the best ways to match our technology to market needs. Founded on the idea that we could use computer vision technology to help save lives on the road, Mobileye became a pioneer in advanced driver assistance systems (ADAS). These capabilities are now scaling up to become the building blocks for a fully autonomous vehicle.

More: Intel at CES 2019 | Autonomous Driving at Intel | Mobileye News

The same is also true in reverse. New technologies developed specifically for AVs are enabling greater scale of advanced driving assistance systems and bringing a new level of safety to roads.

AV Technology Raises ADAS to the Next Level

There are five commonly accepted levels of vehicular autonomy. (Zero is no autonomy.) ADAS systems fall into levels 1 and 2, while levels 3 to 5 are degrees of autonomy ranging from autonomy in some circumstances to full autonomy with no human intervention.

While level 1 and 2 cars can be bought today, cars with varying degrees of autonomy are still in development. We know self-driving cars are technically possible. But the true challenge to get them out of the lab and onto the roads lies in answering more complex questions, like those around safety assurance and societal acceptance. To that end, we have been innovating around the more difficult enablers of AV technology such as mapping and safety.

This technology envelope that we’ve designed around the AV will take ADAS to the next level.

At Mobileye, we developed Road Experience Management™ (REM™) technology to crowdsource the maps needed for AVs – what we call the global Roadbook™. We are now harnessing those maps to improve the accuracy of ADAS features. An example of this is the work that Volkswagen and Mobileye are continuing in their efforts to materialize a L2+ proposition combining the front camera and Roadbook technologies, and leveraging the previously announced data harvesting asset. The ongoing development activity is targeting a broad operational envelope L2+ product addressing mass market deployment.

Amnon Shashua Mobileye CES 2019 3
Professor Amnon Shashua, Intel senior vice president and preside
Professor Amnon Shashua, Intel senior vice president and preside
Professor Amnon Shashua, Intel senior vice president and preside
Professor Amnon Shashua, Intel senior vice president and preside
Professor Amnon Shashua, Intel senior vice president and preside
Professor Amnon Shashua, Intel senior vice president and preside
Professor Amnon Shashua, Intel senior vice president and preside
Professor Amnon Shashua, Intel senior vice president and preside
Professor Amnon Shashua, Intel senior vice president and preside

» Download all images (ZIP, 28 MB)
We also developed the technology-neutral Responsibility-Sensitive Safety (RSS) mathematical approach to safer AV decision-making, which is gaining traction as industry and governments alike have announced plans to adopt RSS for their AV programs and help us work toward development of an industry standard for AV safety. For example, China ITS Alliance – the standards body under the China Ministry of Transportation – has approved a proposal to use RSS as the framework for its forthcoming AV safety standard; Valeo adopted RSS for its AV program and agreed to collaborate on industry standards; and Baidu announced a successful open-source implementation of RSS in Project Apollo.

"Today, we are taking RSS technology back into our ADAS lab and proposing its use as a proactive augment to automatic emergency braking (AEB). We call this automatic preventative braking (APB). Using formulas to determine the moment when the vehicle enters a dangerous situation, APB would help the vehicle return to a safer position by applying small, barely noticeable preventative braking instead of sudden braking to prevent a collision.

If APB were installed in every vehicle using an affordable forward-facing camera, we believe this technology can eliminate a substantial proportion of front-to-rear crashes resulting from wrong driving decision-making. And if we add surround camera sensing and the map into the equation so that preventative braking can be applied in more situations, we can hope to eliminate nearly all collisions of this nature."

Please just don't post press releases.

The amount of time to develop and test APB for L2 or L2+ ADAS and incorporate technology could affect EyeQ6 feature set completion deadline.

"Autonomous Vehicle platform software team supporting Mobileye’s EyeQ and FPGA solutions"
San Jose group.

EyeC is most likely a ultra-wide band 79 GHz Radar (see Yole)

" new 79 GhZ ... used for Simultaneous Localization And Mapping (SLAM) providing accurate distance information to detected objects in real time. It would be helpful to complement geo-localization technologies for autonomous driving especially in urban canyon condition where GNSS technologies show some accuracy issue. Another advantage of 79 GHz Radar is the mitigation of interference issues that could happen when the streets will be loaded with Radars embedded in the cars."

Mobileye hiring focus on "sensors" and "peripheral devices".

"integration of bleeding edge sensors and other peripheral devices into the hardware & software platform" JR0100377

Since EyeC for redundacy, and thus future breakthrough of totally separate systems, I cannot find any current breakthrough possible. RSS in ADAS L2 is all I see.

RSS live fleet testing predates May 2018 per Intel press. Announced in October of 2017, and research begun in mid 2016 ("we have been working on RSS 2 1/2 years"), so hard to determine how much BMW saw early on given May 2017 tie-up. Vision Zero APB appears Dec 2018, and suggested in ADAS in Jan 2019. Not sure what a breakthrough could be.

typo mistake JUL 2016 for BMW

BMW had to have advanced insight of RSS one would think.

RSS live fleet testing predates May 2018 per Intel press. Announced in October of 2017, and research begun in mid 2016 ("we have been working on RSS 2 1/2 years"), so hard to determine how much BMW saw early on given Jul 2016 tie-up. Vision Zero APB appears Dec 2018, and suggested in ADAS in Jan 2019. Not sure what a breakthrough could be.

Both Toyota and SoftBank's start of Monet Technologies, and Honda now on board and another pending investment in Uber could propel Waymo to new partnerships.

Google purchase of HERE instead of a HERE IPO ?

Google Maps platform w/REM is a thought but since Intel chose AWS to host REM, end of that discussion not to mention ADAS not involved. As the map companies all jockey to be relevant, survival seems pinned to SDV. Why not tether existence to both safety and SDV but not sure how.

Nothing in Japan SIP-adus conference notes about high-definition digital road maps of help

Sony to Delay Automotive 7.42MP Sensor Production to 2020 Nikkei reports
IMX324 had Intel input so will delay affect Intel SoC debut or production

2019 SDV arena being dominated by mapping news and suggests an industry in flux over the localization platform. Has sensor fusion become an issue? Open-source data has limits. Did ZF stumble across issues or is the flying drone world entering into the field?

“ZF AD Environment”

Airbus provides its unique high precise Ground Control Points (GCPs),
serves as independent data source to improve and validate accuracy, based on aerial and space borne approach, to complement ZF semantic cards and will be integrated as foundation layers into the “ZF AD Environment” –

the “ZF AD Environment” – an enhanced HD maps solution ZF will
present soon – where all needed information for autonomous driving will be implemented in a cloud based system.

With ZF well along with “ZF AD Environment” and COPilot, and surely Nvidia Mapstream, how does Bosch respond?

Nicolas Peter said at the Shanghai Autoshow No Plans to Develop Compact Vehicle With Rival pouring cold water on rumors that BMW was about to deepen its alliance with Daimler.

“We have no plans to develop a smaller car together with a German competitor,” Peter said at the Shanghai Autoshow

Apple's Lidar leak ambitions almost sound reactionary [seeking "revolutionary design" and form factor worries in 2019].

Radar imaging satellites like TerraSAR-X are able to acquire images having very high absolute geo-location accuracy, due the availability of precise orbit information. By using multiple stereo images in a radargrammetric process, so-called Ground Control Points (GCPs) can be extracted. GCPs are precisely measured land marks given the exact position on the earth. These GCPs are derived from pole-like structures along the road e.g. street lights, signs or traffic lights, since these objects are having a high backscatter in the radar image and therefore being easily identifiable in multiple images. By using a stack of multiple TerraSAR-X images, a dense point cloud of GCPs having an accuracy of less than 10 centimeters can be automatically extracted.

However, in order to make use of this high positional accuracy for the use case of autonomous driving, the link between landmarks like street lights identified from mobile mapping data and the coordinates of the respective GCP needs to be established. The goal of this project is to find and implement an algorithm for the automatic matching of 3D point clouds from GCPs extracted by radar space geodesy and in-situ LIDAR mobile mapping data derived from a car acquisition. A precise matching process would enable the generation of an accurate data basis as indispensable basis for highly automated and autonomous driving.

No where near a breakthrough.

Carmera and ZF / Airbus PoC's now seem rolled out too early for press.

[/"https://arxiv.org/pdf/1812.07179.pdf"/]

"Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving."

Intel ME CEO at March 2019 MIT talk

re RSS "so we basically remove the decisions outside of the statistical machine-learning domain"

Jerusalem
/Second, it has lots and lots of pedestrians. All the ultra-orthodox there and they don't respect the road, so they'll move in and out. So moving in narrow streets. So right now, we are 90% of the type of scenarios that we want to be in. We have another about four or five months to finish that. Then the car would be handling the most difficult scenarios a car could ever handle. So some of those scenarios would be even scary for a human driver. If you come and rent a car and drive in those areas that we drive, at some point, you'll simply stop the car and get out of the car. So it's really scary.

... Level 4 can include also, in case you see someone like a policeman waving a traffic sign or something like that, you stop, and then a teleoperator makes the decision. Some are building teleoperations where you have one teleoperator per 10 vehicles. Because the teleoperator is not supposed to drive the vehicle, or avoid accidents...Teleoperator, when the car stops, asking for guidance, then the teleoperator gets into action.

So in a level 4, you have this flexibility to handle these edge cases through a teleoperation. Level 5, I think will become much, much later. I think level 4 will take us at least a decade of maturing, practicing, going from one teleoperator for 10 vehicles to one to 100 vehicles at the end of the maturity cycle. And then you can let go of teleoperators at all for level 5. Yeah?

then the jump is going to be to level 4 or 5. And level 4 or 5 is going to start purely robo-taxi, not passenger cars, because of cost. With robo-taxis, you can build a business model of ride hailing. And we did all the math, all the business calculations, because we are going to do this in Tel Aviv as a commercial business. You can have a system that costs tens of thousands of dollars on top of the cost of the car and still make a very profitable business of ride hailing because there's no driver in the loop

AMNON SHASHUA: Sorry. When I mentioned--the purpose of the teleoperator is not to avoid accidents. With all the stuff that I talked about in terms of validating the perception, validating the driving policy, the autonomous car should never cause an accident. The teleoperator gets engaged when the car stops and is confused of what to do. There are multiple choices, and the human operator tells the car what choice to take, or tell the car, stay there and I'll send some help to take you off the road./

Hail FLA w/SB 932 passes, thunderstorm clouds brewing.
redundancy required for an ASIL-D system. Up to date maps for FLA a task.

Foretellix will integrate an open-source version of Intel RSS Responsibility-Sensitive Safety into its software

Among others, Ford will meet with representatives from cybersecurity for autonomous vehicles company Karamba Security Ltd. and automotive chipmaker Mobileye.

Late last year, Ford Motor invested $12.5 million in its Israeli subsidiary, SAIPS AC Ltd., to establish a new unit that will focus on designing a decision-making system for autonomous vehicles.

Pages

Add new comment