I predict and propose many things online. It's nice to note when they become real. In this article from 2006, I describe the value of a robot that could crawl along power lines laying fiber and TIL that Facebook has indeed built such a robot. This week they have released it with the odd name of Bombyx.
Internet economics, technology and issues
The instinct of many transportation planners is to make "smart infrastructure," and to try to make plans for it going out 30 years. That's impossible, nobody knows what smart will mean in 5 years. The internet solve this problem, and grew by making the infrastructure as stupid as possible, and it revolutionized the world. The internet teaches lessons for how all infrastructure planning must go in the future -- keep the physical as simple as possible, do everything in the virtual, software layer.
There are many tools now being used to replace physical conferences and meetings -- not just Zoom. And no one system is complete, or even best-of-breed in all the various functions it provides. It's time for these tools to develop a way to interoperate, so people can build an event mixing and matching tools, but allowing attendees to flow smoothly between the tools without needing to create different accounts, re-authenticate or have a large learning curve.
Nick Denton was a sleazebag. I knew that within one minute of meeting him, as he described the new web site he was planning, called "Valleywag." He was proud he had learned the name of Larry Page's girlfriend and he could break that story, as if who Larry was dating was worthy news of some kind.
A new service called Red Carpet was announced, which will offer first-run movies in the homes of the very wealthy. You need a $15,000 DRM box and movie rentals are $1,500 to $3,000 per rental. That price is not a typo.
So I wrote an article pondering why that is, and why this could not be done at a price that ordinary people could afford, similar to the price of going to the movies.
Most of the world was wowed by the Google Duplex demo, where their system was able to cold-call a hairdresser and make an appointment with her, with the hairdresser unaware she was talking to an AI. The system included human speech mannerisms and the ability to respond to the random phrases the hairdresser through back.
A huge opportunity awaits a young social media company that is poised to take advantage of the fall of Facebook (and Twitter). Is somebody out there ready to carry the ball and make it happen. It probably has to be somebody already with most of this done, or even operating.
You, by definition, read blog posts. But the era of lots of individual personal web sites seems to be on the wane. It used to be everybody had a "home page" and many had one that updated frequently (a blog) but I, and many other bloggers, have noticed a change of late. It can be seen in the "referer" summaries you get from your web server that show who is making popular links to your site.
I have many things to discuss on the problem of "fake news" (which is to say, deliberately constructed false reports aimed to be spread to deceive) and the way it spreads through social media. This hot topic, seen as one of the largest threats to democracy to ever arise -- especially when combined with automated microtargeting of political propaganda -- is causing people to clamour for solutions.
Having written yesterday about a decision to sell Bitcoin, I want to re-examine two posts I made in the past which are now even more apropos.
My sell decision was (at least temporarily) wise as it dropped to $9300 quickly. I don't think it will necessarily never see $11K again. It is a speculative value with no fundamentals behind it to help judge the right price range.
E-mail is facing a decline. This is something I lament, and I plan to write more about that general problem, but today I want to point out something that is true, but usually not recognized. Namely that E-mail today is often secure in transit, and we can make better use of that and improve it.
The right way to secure any messaging service is end-to-end. That means that only the endpoints -- ie. your mail client -- have the keys and encrypt or decrypt the message. It's impossible, if the crypto works, for anybody along the path, including the operators of the mail servers as well as the pipes, to decode anything but the target address of your message.
We could have built an end-to-end secure E-mail system. I even proposed just how to do it over a decade ago and I still think we should do what I proposed and more. But we didn't.
Along the way, though, we have mostly secured the individual links an E-mail follows. Most mail servers use encrypted SMTP over TLS when exchanging mail. The major web-mail programs like Gmail use encrypted HTTPS web sessions for reading it. The IMAP and POP servers generally support encrypted connections with clients. My own server supports only IMAPS and never IMAP or POP, and there are others like that.
What this means is that if I send a message to you on Gmail, while my SMTP proxy and Google can read that message, nobody tapping the wire can. Governments and possibly attackers can get into those servers and read that E-mail, but it's not an easy thing to do. This is not perfect, but it's actually pretty useful, and could be more useful.
Not everybody loves video calls, but there are times when they are great. I like them with family, and I try to insist on them when negotiating, because body language is important. So I've watched as we've increased the quality and ease of use.
The ultimate goals would be "retinal" resolution -- where the resolution surpasses your eye -- along with high dynamic range, stereo, light field, telepresence mobility and VR/AR with headset image removal. Eventually we'll be able to make a video call or telepresence experience so good it's a little hard to tell from actually being there. This will affect how much we fly for business meetings, travel inside towns, life for bedridden and low mobility people and more.
Here's a proposal for how to provide that very high or retinal resolution without needing hundreds of megabits of high quality bandwidth.
Many people have observed that the human eye is high resolution on in the center of attention, known as the fovea centralis. If you make a display that's sharp where a person is looking, and blurry out at the edges, the eye won't notice -- until of course it quickly moves to another section of the image and the brain will show you the tunnel vision.
Decades ago, people designing flight simulators combined "gaze tracking," where you spot in real time where a person is looking with the foveal concept so that the simulator only rendered the scene in high resolution where the pilot's eyes were. In those days in particular, rendering a whole immersive scene at high resolution wasn't possible. Even today it's a bit expensive. The trick is you have to be fast -- when the eye darts to a new location, you have to render it at high-res within milliseconds, or we notice. Of course, to an outside viewer, such a system looks crazy, and with today's technology, it's still challenging to make it work.
With a video call, it's even more challenging. If a person moves their eyes (or in AR/VR their head) and you need to get a high resolution stream of the new point of attention, it can take a long time -- perhaps hundreds of milliseconds -- to send that signal to the remote camera, have it adjust the feed, and then get that new feed back to you. There is no way the user will not see their new target as blurry for way too long. While it would still be workable, it will not be comfortable or seem real. For VR video conferencing it's even an issue for people turning their head. For now, to get a high resolution remote VR experience would require sending probably a half-sphere of full resolution video. The delay is probably tolerable if the person wants to turn their head enough to look behind them.
One opposite approach being taken for low bandwidth video is the use of "avatars" -- animated cartoons of the other speaker which are driven by motion capture on the other end. You've seen characters in movies like Sméagol, the blue Na'vi of the movie Avatar and perhaps the young Jeff Bridges (acted by old Jeff Bridges) in Tron: Legacy. Cartoon avatars are preferred because of what we call the Uncanny Valley -- people notice flaws in attempts at total realism and just ignore them in cartoonish renderings. But we are now able to do moderately decent realistic renderings, and this is slowly improving.
My thought is to combine foveal video with animated avatars for brief moments after saccades and then gently blend them towards the true image when it arrives. Here's how.
- The remote camera will send video with increasing resolution towards the foveal attention point. It will also be scanning the entire scene and making a capture of all motion of the face and body, probably with the use of 3D scanning techniques like time-of-flight or structured light. It will also be, in background bandwidth, updating the static model of the people in the scene and the room.
- Upon a saccade, the viewer's display will immediately (within milliseconds) combine the blurry image of the new target with the motion capture data, along with the face model data received, and render a generated view of the new target. It will transmit the new target to the remote.
- The remote, when receiving the new target, will now switch the primary video stream to a foveal density video of it.
- When the new video stream starts arriving, the viewer's display will attempt to blend them, creating a plausible transition between the rendered scene and the real scene, gradually correcting any differences between them until the video is 100% real
- In addition, both systems will be making predictions about what the likely target of next attention is. We tend to focus our eyes on certain places, notably the mouth and eyes, so there are some places that are more likely to be looked at next. Some portion of the spare bandwidth would be allocated to also sending those at higher resolution -- either full resolution if possible, or with better resolution to improve the quality of the animated rendering.
The animated rendering will, today, both be slightly wrong, and also suffer from the uncanny valley problem. My hope is that if this is short lived enough, it will be less noticeable, or not be that bothersome. It will be possible to trade off how long it takes to blend the generated video over to the real video. The longer you take, the less jarring any error correction will be, but the longer the image is "uncanny."
While there are 100 million photoreceptors in the whole eye, but only about a million nerve fibers going out. It would still be expensive to deliver this full resolution in the attention spot and most likely next spots, but it's much less bandwidth than sending the whole scene. Even if full resolution is not delivered, much better resolution can be offered.
Stereo and simulated 3D
You can also do this in stereo to provide 3D. Another interesting approach was done at CMU called pseudo 3D. I recommend you check out the video. This system captures the background and moves the flat head against it as the viewer moves their head. The result looks surprisingly good.
Here's the situation: You're in a place with no bandwidth or limited bandwidth. It's just the place that you need to download an app, because the good apps, at least, can do more things locally and not make as much use of the network. But you can't get to the app store. The archetype of this situation is being on a plane with wifi and video offerings over the wifi. You get on board and you connect and it says you needed to download the app before you took off and got disconnected.