Time for the fourth screen -- the always on wall computer

In media today, it's common to talk about three screens: Desktop, mobile and TV. Many people watch TV on the first two now, and tools like Google TV and the old WebTV try to bring interactive, internet style content to the TV. People like to call the desktop the "lean forward" screen where you use a keyboard and have lots of interactivity, while the TV is the "lean back" couch-potato screen. The tablet is also distinguishing itself a bit from the small screen normally found in mobile.

More and more people also find great value in having an always-on screen where they can go to quickly ask questions or do tasks like E-mail.

I forecast we will soon see the development of a "fourth screen" which is a mostly-always-on wall panel meant to be used with almost no interaction at all. It's not a thing to stare at like the TV (though it could turn into one) nor a thing to do interactive web sessions on. The goal is to have minimal UI and be a little bit psychic about what to show.

One could start by showing stuff that's always of use. The current weather forecast, for example, and selected unusual headlines. Whether each member of the household has new mail, and if it makes sense from a privacy standpoint, possibly summaries of that mail. Likewise the most recent status from feeds on twitter or Facebook or other streams. One could easily fill a screen with these things so you need a particularly good filter to find what's relevant. Upcoming calendar events (with warnings) also make sense.

Some things would show only when important. For example, when getting ready to go out, I almost always want to see the traffic map. Or rather, I want to see it if it has traffic jams on it, no need to show it when it's green -- if it's not showing I know all is good. I may not need to see the weather if it's forecast sunny either. Or if it's raining right now. But if it's clear now and going to rain later I want to see that. Many city transit systems have a site that tracks when the next bus or train will come to my stop -- I want to see that, and perhaps at morning commute time even get an audio alert if something unusual is up or if I need to leave right now to catch the street car. A view from the security camera at the door should only show if somebody is at the door.

There are so many things I want to see that we will need some UI for the less popular ones. But it should be a simple UI, with no need to find a remote (though if I have a remote -- any remote -- it should be able to use it.) Speech commands would be good to temporarily see other screens and modes. A webcam (and eventually Kinect style sensor) for gestural UI would be nice, letting me swipe or wave to get other screens. The wall screen would be quite different if there are multiple people in the house. Ideally it should figure out who is looking at it (with some face recognition, or detection of the bluetooth signatures of the phone in my pocket, or recognition of the character of my voice if I am talking. There are a few tricks that can be used. The bluetooth one probably would work to say which people are in the house and who has left, at any rate, as well as who is close and who is far. If there is only one person it might be OK to show the headlines of E-mails, otherwise just the counts of new mails for household members would be shown.

The screen, if able to identify when it is being looked at and by whom, could arrange to not show you things you already know. For example, if you saw that the weather will be sunny in the morning, there is no need to show you the weather again unless it is changing. Indeed one could sort of view "staring" as a UI -- the longer you look at the screen, the more it shows you. With eye-tracking one could also consider a gaze based user interface -- you start with the main screen but if you gaze at spots on the left and right you sweep to new screens.

It should also figure when nobody is around and go blank to save power. But it should be a low-power LED backlit display only taking a small number of watts.

It could have a touchscreen like a tablet of course, and then be able to do all a tablet does. Which is fine -- expensive at first, but fine. It might also just have a touch "edge" which you can swipe left and right or press spots to change modes. It will always revert to "main screen mode" after a while. And it might also like to revert to digital photo frame mode, for this is a good function.

These screens could be standalone units mounted on walls with access to the local Wifi. They might also just be a flat-panel with Wifi which has everything painted by a server elsewhere in the home, which would make it cheaper to have several. Since almost all PCs these days can display on two monitors, the 4th screen could also be plugged into a PC if there is one near the wall it's mounted on. It also makes a lot of sense for digital TV set top boxes, like the Google TV, to have a 2nd video port which can be used for 4th screen display. Extra flat-panels are so cheap these days that it makes sense to have an additional screen with any TV. Rather than overlay certain types of things on top of the TV, it makes sense to use the extra monitor for stuff. For example, the playback cursor of a video (showing how far we are and how much time is left) might make sense on a small monitor (in big type) rather than overlaying on the main display when you ask for it. When nobody is watching TV, the extra display and operate in full 4th-screen mode. The TV can do this as well, as can any PC in screen saver, but this is consumptive of power.

We've seen basic 4th screens show up in a few places, like fridge doors or new smart kitchens. What's changing now is that the LCD panels are hitting $100 in price and the electronics for a controller can be even cheaper than that, if make in quantity. And all this is cheaper than a typical digital picture frame, at least at the resolutions of those cheap panels.

What else do you want to be able to glance at on your 4th screen?

Comments

Location and a "mortal peril" indicator for each family member.

http://www.useit.com/alertbox/20021209.html

I was trying to stay away from the spooky, privacy-invading apps but yes, I can see parents and others wanting this on their 4th screen. I do like the Harry Potter idea. There's no question that famous SF shows like Star Trek have sometimes inspired developers to build fictional technologies. It would indeed be strange if they didn't start getting inspired by Harry Potter, since that's the most popular fantasy of this generation (perhaps of any generation.)

I've long thought that telecommuting would really benefit from an always-on window that, critically, was frosted for most of the time. No-one wants to think that they're being watched continuously, but it is useful to see whether someone is in their seat or not; therefore, you could have a screen that was frosted and showed the general shapes of your colleagues. As you approached the screen, their images would become sharper, but they would also see you approaching.

Twenty years ago when I was designing large software projects, I wanted the entire wall to be a screen so I could see more code at the same time.

Randy

As a physician, the thought of walking into a patient's room - and suddenly one of the walls dissolves to show all the patient's x-rays, EKGs, and other relevant data to the encounter is tantalizing.

If the patient could see the Wall too - suddenly, just as useful as the stethoscope might be the laserpointer - as the clinician points out all of the salient clinical facts related to the patient's current condition.

It seems like this could be a very straightforward calculation as to what year the cost per pixel to cover an average wall might reach say $1,000. Anyone know where to find cost per pixel data from the last 50 years? (Seems like a very Kurzweilian type of forecast...)

Isn't this the whole point of Google Glass? Rather than having to be in front of a particular wall, the information travels with you. The privacy issue is resolved also, as only you can see your email summaries and other personal info.

Ordinary smartphones seem to be headed in this direction as well. At least some of what you mention is in Google Now. iOS 7 gives a summary of your day (calendar, weather, messages) right on the lock screen. You do still have to pick up the device and swipe once to see it, but again, the information is always with you.

While I wrote this before Google Glass (or Google Now which is a big part of Glass) you still have to ask Glass for most info. (It tells you about incoming texts etc.) It doesn't know you are "looking at it."

And people will not wear glass all the time around the house.

Add new comment