For those in Silicon Valley, I will be giving a talk at the monthly autonomous vehicle enthusiast meetup. Some time ago I did my general talk, but this one will get into the meat on some of the big myths and issues. With luck we'll get some good debate going.
For many decades I've had an ongoing debate with my friend David Brin over the ideas in his book The Transparent Society where he ponders what happens when cameras and surveillance technology become so cheap it's impossible to stop them from being everywhere.
I was just outside Weiser Idaho, a small town on the Snake river, for the 2017 Eclipse, which was an excellent, if short, spectacle which reawakened U.S. interests in total eclipses. They are, as I wrote earlier, the most spectacular natural phenomenon you can see on the Earth, but due to their random pattern it's been a long time since one has covered so much of the world's richest country.
For me, it was my sixth total eclipse, but the first I could drive to. I began this journey in Mexico in 1991, with the super-eclipse of that year, which also was the last to visit the United States (it was visible on the big island of Hawai`i.) Since then I have flown around the world to the Curacao area, to the Black Sea, to the Marshall Islands (more photos) and French Polynesia to see other total eclipses. And I will continue to do so starting with 2 years from now in Argentina.
I recommend before you read that you enjoy my Gallery of 2017 Eclipse Photos in HD resolution. When going through them I recommend you click the "i" button so you can read the descriptions; they do not show in the slide show.
I did not photograph my first eclipse (nor should anybody) but every photographer, seeing such a spectacle, hopes to capture it. We can't, because in addition to being the most spectacular natural event, it's also the one with the greatest dynamic range. In one small field you have brilliant jets of fire coming off the sun, its hot inner atmosphere, its giant glowing outer atmosphere and a dimly lit dark sky in which you can see stars. And then there is the unlit side of the moon which appears to be the blackest thing you have ever seen. While you can capture all these light values with a big bracket, no display device can come close to showing that 24 stop range. Only the human eye and visual system can perceive it.
Some day though, they will make reasonable display devices that can do this, but even then it will be tough. For the eclipse covers just a few degrees of sky, but in reality it's a full 360 experience, with eerie light in all directions and the temporary light of twilight in every direction. Still, we try.
In the future, when there is a retinal resolution VR headset with 24 bits of HDR light level ability, we might be able to show people an eclipse without going to one. Though you should still go.
That's why these photographs are so different. Every exposure reveals a different aspect of the eclipse. Short exposures show the prominences and the "chromosphere" -- the inner atmosphere of the sun visible only at the start and end of the eclipse. Longer exposures reveal more of the giant corona. The fingers of the outer corona involve 2 or 4 second exposures! The most interesting parts happen at 2nd and 3rd contact (the start and end) and also have many aspects. About 1/60th of a second shows the amazing diamond ring by letting the tiny sliver of sun blow out the sensor to make the diamond, as it does to the eye.
One thing that saddens and frustrates me is that all of this is only visible in a band less than 100 miles wide where the eclipse is total. Outside that, for thousands of miles, one can see (with eye protection) a "partial eclipse." They both get called an eclipse but the difference is night and day. Yet I think the naming makes people not understand the difference. They think a "90% partial eclipse" is perhaps 90% as interesting as a total eclipse. Nothing could be more wrong. There are really three different things:
I constantly meet people who think they saw "the eclipse" when to me and all others who have seen one, only the total eclipse is the eclipse. While the 98% partial is interesting, nobody should ever see that, because if you are that close to the band of totality, you would be nuts not to make the effort to go that extra distance. In a total eclipse, you see all that the partial has to offer, and even a few partial effects not seen except at 99.9%
As such, I propose we rename the partial eclipse, calling it something like a "grazing transit of the moon." An eclipse technically is a transit of the moon over the sun, but my main goal is to use a different term for the partial and total so that people don't get confused. To tell people in the partial zone "you saw a transit, hope it was interesting" while telling people in the total zone, "You saw a solar eclipse, wasn't that the most amazing thing you've ever seen?"
This was the first eclipse I have ever driven to, and because of that, I went a bit overboard, able to bring all sorts of gear. I had to stop myself and scale back, but I still brought 2 telescopes, 4 cameras, one long lens, 5 tripods and more.
Last night, as they were towing our plane from the gate in Miami there was a very unusual bump -- turns out they put the tow bar on wrong and damaged the landing gear. It became clear in time that we would not fly that night (FA timeout loomed.) I've seen this a lot, so I was on the phone immediately to book another flight, but I would still need a hotel voucher for the night, as would most other folks on the flight, even if they took the same flight the next day after the repair.
E-mail is facing a decline. This is something I lament, and I plan to write more about that general problem, but today I want to point out something that is true, but usually not recognized. Namely that E-mail today is often secure in transit, and we can make better use of that and improve it.
The right way to secure any messaging service is end-to-end. That means that only the endpoints -- ie. your mail client -- have the keys and encrypt or decrypt the message. It's impossible, if the crypto works, for anybody along the path, including the operators of the mail servers as well as the pipes, to decode anything but the target address of your message.
We could have built an end-to-end secure E-mail system. I even proposed just how to do it over a decade ago and I still think we should do what I proposed and more. But we didn't.
Along the way, though, we have mostly secured the individual links an E-mail follows. Most mail servers use encrypted SMTP over TLS when exchanging mail. The major web-mail programs like Gmail use encrypted HTTPS web sessions for reading it. The IMAP and POP servers generally support encrypted connections with clients. My own server supports only IMAPS and never IMAP or POP, and there are others like that.
What this means is that if I send a message to you on Gmail, while my SMTP proxy and Google can read that message, nobody tapping the wire can. Governments and possibly attackers can get into those servers and read that E-mail, but it's not an easy thing to do. This is not perfect, but it's actually pretty useful, and could be more useful.
We're all shocked at the idea of a growing neo-Nazi movement, at the horrible attack in Virginia and the lack of condemnation by the President. It's making us forget that the neo-Nazi radical right are trolls with many parallels to online trolls. And the only thing to do is not to feed the trolls, and definitely don't attack the civil rights that they make use of.
A protest march has 3 main functions:
I will be heading to western Idaho this weekend to watch my sixth total Eclipse. That makes me a mid-grade eclipse chaser, so let me tell you some important things you need to know, which are not in some of the other eclipse guides out there. For good general sites look at places like NASA's Eclipse Guide which has nice maps or this map.
The difference between a total solar eclipse and a partial one -- even a 99% partial one -- is literally and metaphorically night and day. It's like the difference between sex and holding hands. They are really two different things with a similar sounding name. And a lunar eclipse is again something vastly different. This does not mean a high-partial eclipse is not an interesting thing, but the total eclipse is by far the most spectacular natural phenomenon visible on this planet. Beyond the Grand Canyon, Yosemite, Norway, etc. So if you can get to totality, get there. Do not think you are seeing the eclipse if you don't get into the zone of totality.
Many people seek to get close to the centerline of the eclipse. This provides the longest eclipse for your area. You will only lose a modest number of seconds if you are within 15 miles of the centerline, so you don't have to get exactly there, and in fact it may be too crowded there.
On the other hand there are those who deliberately get close to the edge, giving up 30-40% of their eclipse time in order to see more "edge effects." Near the edge, the edge effects are longer and a bit more spectacular. In particular the diamond ring will be a fair bit longer, and you may see more prominences and chromosphere for longer. If this is your first eclipse, I am not sure you want to get too close to the edge. But try any of the map web sites that will tell you your duration, and get somewhere that has within 30-40 seconds of the centerline time.
You've been hearing endless talk about eclipse glasses and how well made they are. Eclipse glasses are only for the boring partial phase. They give you a way to track the progress of the moon while waiting for the main event. Once totality is over, everybody packs up and does not even bother to watch the 2nd half of the partial eclipse, that's how boring the partial part is.
But don't be one of those people who, told about the danger of eclipses, does not watch totality with your bare eyes. In fact, use binoculars in addition to your naked eyes, and perhaps a short look through a telescope -- but not during the diamond rings or any partial phase.
Update: There is a nice large sunspot group that should still be there on Eclipse day, making the partial phase more interesting to those with good eyesight.
In totality you are looking not at the sun, but its amazing atmosphere -- the "corona" -- full of streamers, and many times the size of the sun or moon. You may also see jets of fire coming off the sun, and at the start and end of totality you will see the hot red inner atmosphere of the sun, known as the chromosphere.
If you are crazy enough to be outside the total zone but close to it, you sadly still can't look with your bare eyes at any part of the eclipse.
An eclipse is most glorious in the sky but a lot of other things happen around it. As it gets very close to total you will see the nature of the sunlight change and become quite eerie. Shadows of trees will turn into collections of crescents. About 20-60 seconds before and after totality, if you have a white sheet on the ground, you will see ripples of light waving, like on the bottom of a giant swimming pool. And the shadow. You will see it approach. If you are up on a mountain or in a plane this will be more obvious. It is going at 1,000 to 2,000 miles per hour.
Almost all robocars use maps to drive. Not the basic maps you find in your phone navigation app, but more detailed maps that help them understand where they are on the road, and where they should go. These maps will include full details of all lane geometries, positions and meaning of all road signs and traffic signals, and also details like the texture of the road or the 3-D shape of objects around it. They may also include potholes, parking spaces and more.
The maps perform two functions. By holding a representation of the road texture or surrounding 3D objects, they let the car figure out exactly where it is on the map without much use of GPS. A car scans the world around it, and looks in the maps to find a location that matches that scan. GPS and other tools help it not have to search the whole world, making this quick and easy.
Google, for example, uses a 2D map of the texture of the road as seen by LIDAR. (The use of LIDAR means the image is the same night and day.) In this map you see the location of things like curbs and lane markers but also all the defects in those lane markers and the road surface itself. Every crack and repair is visible. Just as you, a human being, will know where you are by recognizing things around you, a robocar does the same thing.
Some providers measure things about the 3D world around them. By noting where poles, signs, trees, curbs, buildings and more are, you can also figure out where you are. Road texture is very accurate but fails if the road is covered with fresh snow. (3D objects also change shape in heavy snow.)
Once you find out where you are (the problem called "localization") you want a map to tell you where the lanes are so you can drive them. That's a more traditional computer map, though much more detailed than the typical navigation app map.
Some teams hope to get a car to drive without a map. That is possible for simpler tasks like following a road edge or a lane. There you just look for a generic idea of what lane markings or road edges should look like, find them and figure out what the lanes look like and how to stay in the one you want to drive in. This is a way to get a car up and running fast. It is what humans do, most of the time.
Most teams try to do more than driving without a map because software good enough to do that is also software good enough to make a map. To drive without a map you must understand the geometry of the road and where you are on it. You must understand even more, like what to do at intersections or off-ramps.
Creating maps is effectively the act of saying, "I will remember what previous cars to drive on this road learned about it, and make use of that the next time a car drives it."
Put this way it seems crazy not to build and use maps, even with the challenges listed below. Perhaps some day the technology will be so good that it can't be helped by remembering, but that is not this day.
There are many strong advantages of having the map:
One intermediate goal might be to have cars that can drive with only a navigation map, but use more detailed maps in "problem" areas. This is pretty similar, except in database size, with automatic map generation with human input only on the problem areas. If your non-map driving is trustworthy, such that it knows not to try problem areas, you could follow the lower cost approach of "don't map it until somebody's car pulled over because it could not handle an area."
There are two or three components of the maps people are building, in order to perform the functions above. At the most basic level is something not too far above the navigation maps found in phones. That's a vector map, except with lane level detail. Such maps know how many lanes there are, and usually what lanes connect to what lanes. For example, they will indicate that to turn right, you can use either of the right two lanes at some intersections.
Ten years ago, I asked Cell companies to let me have more than one phone on the same number. Recently I noticed the ability to almost do that with Google Project FI cell service.
Something I do from time to time is a road trip in a rental car. And while car rental companies much prefer the business customer who rents a big car at a high price, then just drives it to their meeting and back to the airport, they are not averse to the less profitable road trip business.
So here are some things they could do to make it better for that sort of customer.
I've written before about overbooking and how it's good for passengers as well as for the airlines. If we have a service (airline seats, rental cars, hotel rooms) where the seller knows it's extremely likely that with 100 available slots, 20 will not show up, we can have two results:
Earlier I noted that Nidi Kalra of Rand spoke at the AVS about Rand's research suggesting that purely road testing robocars is an almost impossible task, because it would take hundreds of millions to a billion miles of driving to prove that a robocar is 10% better than human drivers.
(If the car is 10x better than humans, it doesn't take that long, but that's not where the first cars will be.)
This study has often been cited as saying that it's next to impossible to test robocars. The authors don't say that -- their claim is that road testing will not be enough, and will take too long to really work -- but commenters and press have taken it further to the belief that we'll never be able to test.
The mistake is that while it could take a billion miles to prove a vehicle is 10% safer than human drivers, that is not the goal. Rather, the goal is to decide that it's unlikely it is much worse than that number. It may seem like "better than X" and "not worse than X" are the same thing, but they are not. The difference is where you give the benefit of the doubt.
Consider how we deal with new drivers. We give them a very basic test and hand them a licence. We presume, because they are human teens, that they will have a safety record similar to other human teens. Such a record is worse than the level for experienced drivers, and in fact one could argue it's not at all safe enough, but we know of no way to turn people into experienced drivers without going through the risky phase.
If a human driver starts showing evidence of poor skills or judgments -- lots of tickets, and in particular multiple accidents, we pull their licence. It actually takes a really bad record for that to happen. By my calculations the average human takes around 20 years to have an accident that gets reported to insurance, and 40-50 years to have one that gets reported to police. (Most people never have an injury accident, and a large fraction never have any reported or claimed accident.)
Today's news is preliminary, but a U.S. house committee panel passed some new federal regulations which suggest sweeping change in the US regulatory approach to robocars.
I have recently managed to dig up some old documents from the earliest days of car regulation. Here is a report from NHTSA on the state of affairs near the turn of the 20th century.
In recent years, we've seen much excitement about the idea of carriages and coaches with the addition of "motors" which can propel the carriage without relying entirely on the normal use of horses or other beasts of burden. These "Horseless carriages," sometimes also known as "auto mobile" are generating major excitement, and prototypes have been generated by men such as Karl Benz and Armand Peugeot, along with the Duryea brothers, Ransom Olds and others in the the USA. The potential for these carriages has resulted in many safety questions and many have asked if and how NHTSA will regulate safety of these carriages when they are common.
Previously, NHTSA released a set of 4, and later 5 levels to classify and lay out the future progression of this technology.
Level zero is just the existing rider on horseback.
Level one is the traditional horse drawn carriage or coach, as has been used for many years.
A level 2 carriage has a motor to assist the horses. The motor may do the work where the horses trot
along side, but at any time the horses may need to take over on short notice.
In a level 3 carriage, sometimes the horses will provide the power, but it is allowed to switch over entirely to the "motor," with the
horses stepping onto a platform or otherwise being raised to avoid working them. If the carriage approaches an area it can't handle, or the motor has problems,
the horses should be ready, with about 10-20 seconds notice, to step back on the ground and start pulling. In some systems the horse(s) can be in a hoist which can raise or lower them from the trail.
A Level 4 carriage is one which can be pulled entirely by a motor in certain types of terrain or types of weather -- an operating domain --
but may need a horse at other times. There is no need for a sudden switch to the horses, which should be pulled in a trailer so
they can be hitched up for travel outside the operating domain.
The recently added fifth level is much further in the future, and involves a "horseless" carriage that can be auto mobile in all situations, with no need for any horse at all. (It should carry a horse for off-road use or to handle breakdowns, but this is voluntary.)
In San Francisco, I'm just back from the annual Automated Vehicle Symposium, co-hosted by the AUVSI (a commercial unmanned vehicle organization) and the Transportation Research Board, a government/academic research organization. It's an odd mix of business and research, but also the oldest self-driving car conference. I've been at every one, from the tiny one with perhaps 100-200 people to this one with 1,400 that fills a large ballroom.
Tuesday morning did not offer too many surprises. The first was an announcement by Toyota Research Institute of a $100M venture fund. Toyota committed $1B to this group a couple of years ago, but surprisingly Gil Pratt (who ran the DARPA Robotics Challenge for humanoid-like robots) has been somewhat a man of mixed views, with less optimistic forecasts.
Different about this VC fund will be the use of DARPA like "calls." The fund will declare, "Toyota would really like to see startups solving problem X" and then startups will apply, and a couple will be funded. It will be interesting to see how that pans out.
At CES, Nissan showed off their plan to have a remote control room to help robocars get out of sticky situations they can't understand like unusual construction zones or police directing traffic. Here, they showed it as further along and suggested it will go into operation soon.
This idea has been around for a while (Nissan based it on some NASA research) and at Starship, it has always been our plan for our delivery robots. Others are building such centers as well. The key question is how often robocars need to use the human assistance, and how you make sure that unmanned vehicles stay in regions where they can get a data connection through which to get help. As long as interventions are rare, the cost is quite reasonable for a larger fleet.
This answers the question that Rod Brooks (of Rethink Robotics and iRobot) recently asked, pondering how robocars will handle his street in Cambridge, where strange things like trucks blocking the road to do deliveries, are frequently found.
It's a pretty good bet that almost all our urban spaces will have data connectivity in the 2020s. If any street doesn't have solid data, and has frequent bizarre problems of any type, yet is really important for traversal by unmanned vehicles -- an unlikely trifecta -- it's quite reasonable for vehicle operators to install local connectivity (with wifi, for example) on that street if they can't wait for the mobile data companies to do it. Otherwise, don't go down such streets in empty cars unless you are doing a pickup/drop-off on the street.
Karl Iagenemma of nuTonomy told the story of moving their cars from Singapore, where driving is very regulated and done on the left, to Boston where it is chaotic and done on the right.
I've written a few times that perhaps the biggest unsolved problem in robocars is how to know we have made them safe enough. While most people think of that in terms of government certification, the truth is that the teams building the cars are very focused on this, and know more about it than any regulator, but they still don't know enough. The challenge is going to be convincing your board of directors that the car is safe enough to release, for if it is not, it could ruin the company that releases it, at least if it's a big company with a reputation.
While very few details have come out, Reuters reports that new proposed congressional bills on self-driving cars will reverse many of the provisions I critiqued in the NHTSA regulations last year.
Plans are underway to ask for a legal mandate to install radio communications devices in all new cars, starting around 2020. These radios would do "vehicle to vehicle" (v2v) and vehicle to infrastructure communication using a wifi-derived protocol called DSRC.
Waymo (Google) has announced a pilot project in Phoenix offering a full ride service, with daily use, in their new minivans. Members of the public can sign up -- the link is sure to be overwhelmed with applicants, but it has videos and more details -- and some families are already participating. There's also a Waymo Blog post. I was in Phoenix this morning as it turns out, but to tell real estate developers about robocars, not for this.
There are several things notable about Waymo's pilot:
There has been much speculation recently because of some departures from Apple's car team that they had given up. In fact, last week they applied for self-driving car test plates for California. I never thought they had left the game.
Not everybody loves video calls, but there are times when they are great. I like them with family, and I try to insist on them when negotiating, because body language is important. So I've watched as we've increased the quality and ease of use.
The ultimate goals would be "retinal" resolution -- where the resolution surpasses your eye -- along with high dynamic range, stereo, light field, telepresence mobility and VR/AR with headset image removal. Eventually we'll be able to make a video call or telepresence experience so good it's a little hard to tell from actually being there. This will affect how much we fly for business meetings, travel inside towns, life for bedridden and low mobility people and more.
Here's a proposal for how to provide that very high or retinal resolution without needing hundreds of megabits of high quality bandwidth.
Many people have observed that the human eye is high resolution on in the center of attention, known as the fovea centralis. If you make a display that's sharp where a person is looking, and blurry out at the edges, the eye won't notice -- until of course it quickly moves to another section of the image and the brain will show you the tunnel vision.
Decades ago, people designing flight simulators combined "gaze tracking," where you spot in real time where a person is looking with the foveal concept so that the simulator only rendered the scene in high resolution where the pilot's eyes were. In those days in particular, rendering a whole immersive scene at high resolution wasn't possible. Even today it's a bit expensive. The trick is you have to be fast -- when the eye darts to a new location, you have to render it at high-res within milliseconds, or we notice. Of course, to an outside viewer, such a system looks crazy, and with today's technology, it's still challenging to make it work.
With a video call, it's even more challenging. If a person moves their eyes (or in AR/VR their head) and you need to get a high resolution stream of the new point of attention, it can take a long time -- perhaps hundreds of milliseconds -- to send that signal to the remote camera, have it adjust the feed, and then get that new feed back to you. There is no way the user will not see their new target as blurry for way too long. While it would still be workable, it will not be comfortable or seem real. For VR video conferencing it's even an issue for people turning their head. For now, to get a high resolution remote VR experience would require sending probably a half-sphere of full resolution video. The delay is probably tolerable if the person wants to turn their head enough to look behind them.
One opposite approach being taken for low bandwidth video is the use of "avatars" -- animated cartoons of the other speaker which are driven by motion capture on the other end. You've seen characters in movies like Sméagol, the blue Na'vi of the movie Avatar and perhaps the young Jeff Bridges (acted by old Jeff Bridges) in Tron: Legacy. Cartoon avatars are preferred because of what we call the Uncanny Valley -- people notice flaws in attempts at total realism and just ignore them in cartoonish renderings. But we are now able to do moderately decent realistic renderings, and this is slowly improving.
My thought is to combine foveal video with animated avatars for brief moments after saccades and then gently blend them towards the true image when it arrives. Here's how.
The animated rendering will, today, both be slightly wrong, and also suffer from the uncanny valley problem. My hope is that if this is short lived enough, it will be less noticeable, or not be that bothersome. It will be possible to trade off how long it takes to blend the generated video over to the real video. The longer you take, the less jarring any error correction will be, but the longer the image is "uncanny."
While there are 100 million photoreceptors in the whole eye, but only about a million nerve fibers going out. It would still be expensive to deliver this full resolution in the attention spot and most likely next spots, but it's much less bandwidth than sending the whole scene. Even if full resolution is not delivered, much better resolution can be offered.
You can also do this in stereo to provide 3D. Another interesting approach was done at CMU called pseudo 3D. I recommend you check out the video. This system captures the background and moves the flat head against it as the viewer moves their head. The result looks surprisingly good.
