My 4-camera 4K eclipse video and about traffic from the Eclipse

The Eclipse of 2017 caused dire traffic warnings, even from myself. Since a total eclipse is the most amazing thing you will see, and one was coming to a rich country where almost everybody owns a car, and hundreds of millions live within a day’s drive — I wondered how we would not have horrendous traffic. (You can see my main Eclipse report and gallery here or see all my Eclipse articles.)

Also look out below for a new 4K video I made from having 4 different video cameras running around the eclipse. I have started you 3 minutes in for the short-attention-span world, but you might also enjoy the 3 minutes leading up as the excitement builds. Even on an HD display, be sure to click through to Youtube to watch it full screen.

As described, the 4 cameras are two 4K cell phones facing forward and back, plus an HD video from a 1200mm superzoom camera and snippets of 4K video and stills from the main telescope and Sony A7rII.

The big places for predicted bad traffic were central Oregon, because it was the place with the best weather that was closest to everybody from Seattle to Los Angeles, and areas of South Carolina which were closest for the whole eastern seaboard. At a popular Eclipse site, they had a detailed analysis of potential traffic but in many cases, it was quite wrong.

The central Oregon spine around the tiny town of Madras did get really bad traffic, as in reports of 4 to 6 hours to get out. That was not unexpected, since the area does not have very many roads, and is close to Washington and relatively close to California. At the same time, a lot of traffic diverted to the Salem area, which got a nice clear sky forecast. It has an interstate and many other roads. Planning ahead, Madras was the best choice because the weather is much more unpredictable west of the Cascades. But once the forecast became clear, many people from Seattle, Portland and California should have shifted to the more populated areas with the larger roads.

I decided, since it was only 2 hours more driving to Weiser (on the Oregon/Idaho border) but much less traffic, to go to the Snake River valley. It was the right choice — there was almost no traffic leaving Weiser. In fact, Weiser did not get overwhelmed with people as had been expected, disappointing the businesses. Many thought that a large fraction of Boise would have tried to get up to that area, but they didn’t. We actually wandered a bit and ended up over the river in a school field in Annex, Oregon.

There was no problem finding space, even for free.

This is a pattern we’ve seen many times now — dire predictions of terrible traffic, then almost nothing. It turns out the predictions work too well. The famous Carmageddon#History) in Los Angeles never materialized — even with a major link cut, traffic was lighter than normal.

This is, in turn a tragedy. It seems a lot of people did not go see the eclipse because they were scared of bad traffic. What a great shame.

4K Video

At my sight I had 4 cameras recording video. I set up two cell phones, both able to do 4K, looking at our group from in front and behind. The one behind I put in portrait mode, almost capturing the sun, to show that view, while the one in front showed us looking at the eclipse and also the shadow approaching on the hills.  read more »

Computational photography will turn the photo world upside-down

The camera industry is about to come crashing down thanks to the rise of computational photography.

Many have predicted this for some time, and even wondered why it hasn’t happened. While many people take most of their photos with their cell phones, at this point, if you want to do serious photography, in spite of what it says on giant Apple billboards, you carry a dedicated camera, and the more you want from that camera, the bigger the lens on the front of it is.

That’s because of some basic physics. No matter how big your sensor is, the bigger the lens, the more light that will come in for each pixel. That means less noise, more ability to get enough light in dark situations, faster shutter speeds for moving subjects and more.

For serious photographers, it also means making artistic use of what some might consider a defect of larger lenses — only a narrow range of distances is in focus. “Shallow depth of field” lets photographers isolate and highlight their subjects, and give depth and dimensionality to photos that need it.

So why is it all about to change?

Traditional photography has always been about capturing a single frame. A frozen moment in time. The more light you gather, the better you can do that. But that’s not the way the eye works. Our eyes are constantly scanning a dynamic scene in real time, assembling our image of the world in our brains. We combine information captured at different times to get more out of a scene than our eyes as cameras can extract in a single “frame” (if they had frames.)

Computational photography adds smart digital algorithms not just to single frames, but to quickly shot sequences of them, or frames from multiple different lenses. It uses those to learn more about the image than any one frame or lens could pull out.  read more »

Photo gallery from 2017 total solar eclipse

I was just outside Weiser Idaho, a small town on the Snake river, for the 2017 Eclipse, which was an excellent, if short, spectacle which reawakened U.S. interests in total eclipses. They are, as I wrote earlier, the most spectacular natural phenomenon you can see on the Earth, but due to their random pattern it’s been a long time since one has covered so much of the world’s richest country.

For me, it was my sixth total eclipse, but the first I could drive to. I began this journey in Mexico in 1991, with the super-eclipse of that year, which also was the last to visit the United States (it was visible on the big island of Hawai`i.) Since then I have flown around the world to the Curacao area, to the Black Sea, to the Marshall Islands (more photos) and French Polynesia to see other total eclipses. And I will continue to do so starting with 2 years from now in Argentina.

See the gallery

I recommend before you read that you enjoy my Gallery of 2017 Eclipse Photos in HD resolution. When going through them I recommend you click the “i” button so you can read the descriptions; they do not show in the slide show.

HDR from main camera

Why it’s impossible (today) to photograph

I did not photograph my first eclipse (nor should anybody) but every photographer, seeing such a spectacle, hopes to capture it. We can’t, because in addition to being the most spectacular natural event, it’s also the one with the greatest dynamic range. In one small field you have brilliant jets of fire coming off the sun, its hot inner atmosphere, its giant glowing outer atmosphere and a dimly lit dark sky in which you can see stars. And then there is the unlit side of the moon which appears to be the blackest thing you have ever seen. While you can capture all these light values with a big bracket, no display device can come close to showing that 24 stop range. Only the human eye and visual system can perceive it.

Some day though, they will make reasonable display devices that can do this, but even then it will be tough. For the eclipse covers just a few degrees of sky, but in reality it’s a full 360 experience, with eerie light in all directions and the temporary light of twilight in every direction. Still, we try.

In the future, when there is a retinal resolution VR headset with 24 bits of HDR light level ability, we might be able to show people an eclipse without going to one. Though you should still go.

Moment of 3rd contact

That’s why these photographs are so different. Every exposure reveals a different aspect of the eclipse. Short exposures show the prominences and the “chromosphere” — the inner atmosphere of the sun visible only at the start and end of the eclipse. Longer exposures reveal more of the giant corona. The fingers of the outer corona involve 2 or 4 second exposures! The most interesting parts happen at 2nd and 3rd contact (the start and end) and also have many aspects. About 1/60th of a second shows the amazing diamond ring by letting the tiny sliver of sun blow out the sensor to make the diamond, as it does to the eye.

Time to rename the partial eclipse

One thing that saddens and frustrates me is that all of this is only visible in a band less than 100 miles wide where the eclipse is total. Outside that, for thousands of miles, one can see (with eye protection) a “partial eclipse.” They both get called an eclipse but the difference is night and day. Yet I think the naming makes people not understand the difference. They think a “90% partial eclipse” is perhaps 90% as interesting as a total eclipse. Nothing could be more wrong. There are really three different things:

  1. The total eclipse, the most amazing thing you will ever see.
  2. The >98% partial eclipse (and annular eclipse) which are definitely an interesting event, but still just a tiny shadow of what a total eclipse is.
  3. The ordinary partial eclipse, which is a fun and educational curiosity.

I constantly meet people who think they saw “the eclipse” when to me and all others who have seen one, only the total eclipse is the eclipse. While the 98% partial is interesting, nobody should ever see that, because if you are that close to the band of totality, you would be nuts not to make the effort to go that extra distance. In a total eclipse, you see all that the partial has to offer, and even a few partial effects not seen except at 99.9%

A wider angle HDR with deep corona

As such, I propose we rename the partial eclipse, calling it something like a “grazing transit of the moon.” An eclipse technically is a transit of the moon over the sun, but my main goal is to use a different term for the partial and total so that people don’t get confused. To tell people in the partial zone “you saw a transit, hope it was interesting” while telling people in the total zone, “You saw a solar eclipse, wasn’t that the most amazing thing you’ve ever seen?”

Automating the photography

This was the first eclipse I have ever driven to, and because of that, I went a bit overboard, able to bring all sorts of gear. I had to stop myself and scale back, but I still brought 2 telescopes, 4 cameras, one long lens, 5 tripods and more.  read more »

Digitizing your papers, literally, for the future, with 4K video

I have so much paper that I’ve been on a slow quest to scan things. So I have high speed scanners and other tools, but it remains a great deal of work to get it done, especially reliably enough that you would throw away the scanned papers. I have done around 10 posts on digitizing and gathered them under that tag.

Recently, I was asked by a friend who could not figure out what to do with the papers of a deceased parent. Scanning them on your own or in scanning shops is time consuming and expensive, so a new thought came to me.

Set up a scanning table by mounting a camera that shoots 4K video looking down on the table. I have tripods that have an arm that extends out but there are many ways to mount it. Light the table brightly, and bring your papers. Then start the 4K video and start slapping the pages down (or pulling them off) as fast as you can.

There is no software today that can turn that video into a well scanned document. But there will be. Truth is, we could write it today, but nobody has. If you scan this way, you’re making the bet that somebody will. Even if nobody does, you can still go into the video and find any page and pull it out by hand, it will just be a lot of work, and you would only do this for single pages, not for whole documents. You are literally saving the document “for the future” because you are depending on future technology to easily extract it.  read more »

What colour is the dress? It's both.

Perhaps by now you are sick of the dress that 3/4 people see as “white and gold” and 1/4 people see as “dark blue and black.” If you haven’t seen it, it’s easy to find. What’s amazing is to see how violent the arguments can get between people because the two ways we see it are so hugely different. “How can you see that as white????” people shout. They really shout.

There are a few explanations out there, but let me add my own:

  • The real dress, the one you can buy, is indeed blue and black. That’s well documented.
  • The real photo of the dress everybody is looking at, is light blue and medium gold, because of unusual lighting and colour balance.

That’s the key point. The dress and photo are different. Anybody who saw the dress in almost any lighting would say it was blue and black. But people say very different things about the photo.

To explain, here are sampled colour swatches from the photo, on white and dark backgrounds.

You can see that the colours in the photo are indeed a light blue and a medium to dark gold. Whatever the dress really is, that’s what the photo colours are.

We see things in strange light all the time. Indoors, under incandescent light bulbs, or at sunset, everything is really, really yellow-red. Take a photo at these times with your camera set to “sunshine” light and you will see what the real colours look like. But your brain sees the colours very similarly to how they are in the day. Our brains are trained to correct for the lighting and try to see the “true” (under sunlight) colours. Sunlight isn’t really white but it’s our reference for white.

Some people see the photo and this part of their brain kicks in, and does the correction, letting them see what the dress looks like in more neutral light. We all do this most of the time, but this photo shows a time when only some of us can do it.

For the white/gold folks, their brains are not doing the real correction. We (I am one of them) see something closer to the actual colour of the photo. Though not quite — we see the light blue as whiter and the gold as a little lighter too. We’re making a different correction, and it seems going a bit the other direction. Our correction is false, the blue/black folks are doing a better job at the correction. It’s a bit unusual that the the results are so far apart. The blue/blacks see something close to the real dress, and the white/golds see something closer to the actual photo. Hard to say if “their kind” are better or worse than my kind because of it.

For the white/gold folks, our brains must be imagining the light is a bit blueish. We do like to find the white in a scene to help us figure out what colour the light is. In this case we’re getting tricked. There are many other situations where we get our colour correction wrong, and I will bet you can find other situations where the white/golds see the sunlit colour, and the black/blues see something closer to the photograph.

Camera mounting -- beyond the tripoid screw and dovetail plate

For many decades, cameras have come with a machine screw socket (1/4”-20) in the bottom to mount them on a tripod. This is slow to use and easy to get loose, so most photographers prefer to use a quick-release plate system. You screw a plate on the camera, and your tripod head has a clamp to hold those plates. The plates are ideally custom made so they grip an edge on the camera to be sure they can’t twist.

There are different kinds of plates, but in the middle to high end, most people have settled on a metal dovetail plate first made by Arca Swiss. It’s very common with ball-heads, but still rare on pan-heads and lower end tripods, which use an array of different plate styles, including rectangles and hexagons.

The plates have issues — the add weight to your camera and something with protruding or semi-sharp edges on the bottom. They sometimes block doors on the bottom of the camera. If they are not custom, they can twist, and if they are custom they can be quite expensive. They often have tripod holes but those must be off-center.

Arca style dovetails are quite sturdy, but must be metal. With only the 2 sides clamped they can slide to help you position the camera. It is hard, but not impossible to make them snap in, so they usually are screwed and unscrewed which takes time and work and often involves a knob which can get in the way of other things. They are 38mm wide, and normally the dovetails are parallel to the sensor plane, though for strength the plates on big lenses are sometimes perpendicular, which is not an issue for most ball heads.

It’s time the camera vendors accepted that the tripod screw is a legacy part and move to some sort of quick release system standardized and built right into the cameras. The dovetail can probably be improved on if you’re going to start from scratch, and I’m in favour of that, but for now it is almost universal among serious photographers so I will discuss how to use that.

I have seen a few products like this — for example the E-mount to EOS adapter I bought includes a tripod wedge which has both a screw and ARCA dovetails. (Considering the huge difference in weight between my mirrorless cameras and old Canon glass, this mount is a good idea.)

The screens

Many cameras are deep enough that a 38mm wide dovetail (with tripod hole) could be built into the base of the camera. You would have to open the clamp fully to insert unless you wanted the dovetails to run the entire length, which you don’t, but I think most photographers would accept that to have something flush. It would expand the size of the camera slightly, perhaps, but much less than putting on a plate does — and everybody with high end cameras puts on a plate.

Today, though, many cameras have flip-up screens. They are certainly very handy. As people want their screens as big as possible, this can be an issue as the screen goes down flush with the bottom. If there’s a clamp on the bottom, it can block your screen from getting out. One idea would be to design clamps that taper away at the back, or to accept the screen won’t go down all the way.

The smaller cameras

A lot of new cameras are not 38mm deep, though. Putting plates on them is even worse as they stick out a lot. While again, a new design would help solve this problem, one option would be to standardize on a narrower dovetail, and make clamps that have an adapter that can slide in, seat securely so it won’t pop when the pressure is applied, and hold the narrower plate. That or have a clamp with a great deal of travel but that tends to take a lot of time to adjust. (I will note that there are 2 larger classes of dovetails used for heavy telescopes, known as the Vixen and the Losmandy “D”. Some vixen clamps are actually able to grab an arca plate, even though they are not as deep because of the valley often formed with the dovetail and the top of the plate.

It’s also possible to have a 2 level clamp that can grab a smaller plate but there must be a height gap, which may or may not work.

Narrower plates would be used only on smaller and lighter cameras, where not as much strength is needed. However, here again it might be time to design something new.

A locking pin

For some time, camcorders have established a pattern of having a small hole forward of the tripod screw for a locking pin. This allows a much sturdier mount that can’t twist with no need to grab edges of the camera body. Still cameras could do well to establish pin positions — perhaps one one forward, and one to the side. All they have to do is have small indentations for these pins, which typically come spring-loaded on the plates so you can still use them if the hole is not there. (The camcorder pin is placed forward of the tripod hole, but often “forward” is in the direction of the rails.)

For small cameras, it would be necessary to put the dovetail rails perpendicular to the sensor, and they would be very short. That’s OK because those cameras are small and light. The clamps screws would need to be flush with the top of the clamp. (This is sometimes true but not always.)

The presence of a pin would allow small, generic clamps to sturdily hold many cameras. For larger cameras, bigger plates would be available. The cost and size of plates would go down considerably.

The tripod leg screw

The world also standardized on using a bigger machine screw — 3/8”-16 thread — to connect tripod legs to tripod heads. This is a stronger screw, but could also use improvement. The fact that it takes time to switch tripod heads is not that big a deal for most photographers, but the biggest problem is there is no way, other than friction, to lock it, and many is the time that I have turned my tripod head loose from my legs. Here, some sort of clamp or retractable pin would be good, but frankly another clamp (quick release or not) might make sense, and it could become a standard for heavier duty cameras as well.

Something entirely new

I would leave it to a professional mechanical engineer to design something new, but I think a great system would scale to different sizes, so that one can have variants of it for small, light devices, and variants for big, heavy gear, with a way that the larger clamps could easily adapt to hold some of the smaller sizes. I would also design it to be backwards compatible if practical — it is probably easy to leave a 1/4-20 hole in the center, and it may even be possible in the larger sizes to have dovetails that can be gripped by such clamps.

Future proofing video with high-res stills

On Saturday I wrote about how we’re now capturing the world so completely that people of the future will be able to wander around it in accurate VR. Let’s go further and see how we might shoot the video resolutions of the future, today.

Almost everybody has a 1080p HD camera with them — almost all phones and pocket cameras do this. HD looks great but the future’s video displays will do 4K, 8K and full eye-resolution VR, and so our video today will look blurry the way old NTSC video looks blurry to us. In a bizarre twist, in the middle of the 20th century, everything was shot on film at a resolution comparable to HD. But from the 70s to 90s our TV shows were shot on NTSC tape, and thus dropped in resolution. That’s why you can watch Star Trek in high-def but not “The Wire.”

I predict that complex software in the future will be able to do a very good job of increasing the resolution of video. One way it will do this is through making full 3-D models of things in the scene using data from the video and elsewhere, and re-rendering at higher resolution. Another way it will do this is to take advantage of the “sub-pixel” resolution techniques you can do with video. One video frame only has the pixels it has, but as the camera moves or things move in a shot, we get multiple frames that tell us more information. If the camera moves half a pixel, you suddenly have a lot more detail. Over lots of frames you can gather even more.

This will already happen with today’s videos, but what if we help them out? For example, if you have still photographs of the things in the video, this will allow clever software to fill in more detail. At first, it will look strange, but eventually the uncanny valley will be crossed and it will just look sharp. Today I suspect most people shooting video on still cameras also shoot some stills, so this will help, but there’s not quite enough information if things are moving quickly, or new sides of objects are exposed. A still of your friend can help render them in high-res in a video, but not if they turn around. For that the software just has to guess.

We might improve this process by designing video systems that capture high-res still frames as often as they can and embed them to the video. Storage is cheap, so why not?

I typical digital video/still camera has 16 to 20 million pixels today. When it shoots 1080p HD video, it combines those pixels together, so that there are 6 to 10 still pixels going into every video pixel. Ideally this is done by hardware right in the imaging chip, but it can also be done to a lesser extent in software. A few cameras already shoot 4K, and this will become common in the next couple of years. In this case, they may just use the pixels one for one, since it’s not so easy to map a 16 megapixel 3:2 still array into a 16:9 8 megapixel 4K image. You can’t just combine 2 pixels per pixel.

Most still cameras won’t shoot a full-resolution video (ie. a 6K or 8K video) for several reasons:

  • As designed, you simply can’t pull that much data off the chip per unit time. It’s a huge amount of data. Even with today’s cheap storage, it’s also a lot to store.
  • Still camera systems tend to compress jpegs, but you want a video compression algorithm to record a video even if you can afford the storage for that.
  • Nobody has displays to display 6K or 8K video, and only a few people have 4K displays — though this will change — so demand is not high enough to justify these costs
  • When you combine pixels, you get less noise and can shoot in lower light. That’s why your camera can make a decent night-time video without blurring, but it can’t shoot a decent still in that lighting.

What is possible is a sensor which is able to record video (at the desired 30fps or 60fps rate) and also pull off full-resolution stills at some lower frame rate, as long as the scene is bright enough. That frame rate might be something like 5 or even 10 fps as cameras get better. In addition, hardware compression would combine the stills and the video frames to eliminate the great redundancy, though only to a limited extent because our purpose is to save information for the future.

Thus, if we hand the software of the future an HD video along with 3 to 5 frames/second of 16megapixel stills, I am comfortable it will be able to make a very decent 4K video from it most of the time, and often a decent 6K or 8K video. As noted, a lot of that can happen even without the stills, but they will just improve the situation. Those situations where it can’t — fast changing objects — are also situations where video gets blurred and we are tolerant of lower resolution.

It’s a bit harder if you are already shooting 4K. To do this well, we might like a 38 megapixel still sensor, with 4 pixels for every pixel in the video. That’s the cutting edge in high-end consumer gear today, and will get easier to buy, but we now run into the limitations of our lenses. Most lenses can’t deliver 38 million pixels — not even many of the high-end professional photographer lenses can do that. So it might not deliver that complete 8K experience, but it will get a lot closer than you can from an “ordinary” 4K video.

If you haven’t seen 8K video, it’s amazing. Sharp has been showing their one-of-a-kind 8K video display at CES for a few years. It looks much more realistic than 3D videos of lower resolution. 8K video can subtend over 100 degrees of viewing angle at one pixel per minute of arc, which is about the resolution of the sensors in your eye. (Not quite, as your eye also does sub-pixel tricks!) At 60 degrees — which is more than any TV is set up to subtend — it’s the full resolution of your eyes, and provides an actual limit on what we’re likely to want in a display.

And we could be shooting video for that future display today, before the technology to shoot that video natively exists.

Near-perfect virtual reality of recent times and tourism

Recently I tried Facebook/Oculus Rift Crescent Bay prototype. It has more resolution (I will guess 1280 x 1600 per eye or similar) and runs at 90 frames/second. It also has better head tracking, so you can walk around a small space with some realism — but only a very small space. Still, it was much more impressive than the DK2 and a sign of where things are going. I could still see a faint screen door, they were annoyed that I could see it.

We still have a lot of resolution gain left to go. The human eye sees about a minute of arc, which means about 5,000 pixels for a 90 degree field of view. Since we have some ability for sub-pixel resolution, it might be suggested that 10,000 pixels of width is needed to reproduce the world. But that’s not that many Moore’s law generations from where we are today. The graphics rendering problem is harder, though with high frame rates, if you can track the eyes, you need only render full resolution where the fovea of the eye is. This actually gives a boost to onto-the-eye systems like a contact lens projector or the rumoured Magic Leap technology which may project with lasers onto the retina, as they need actually render far fewer pixels. (Get really clever, and realize the optic nerve only has about 600,000 neurons, and in theory you can get full real-world resolution with half a megapixel if you do it right.)

Walking around Rome, I realized something else — we are now digitizing our world, at least the popular outdoor spaces, at a very high resolution. That’s because millions of tourists are taking billions of pictures every day of everything from every angle, in every lighting. Software of the future will be able to produce very accurate 3D representations of all these spaces, both with real data and reasonably interpolated data. They will use our photographs today and the better photographs tomorrow to produce a highly accurate version of our world today.

This means that anybody in the future will be able to take a highly realistic walk around the early 21st century version of almost everything. Even many interiors will be captured in smaller numbers of photos. Only things that are normally covered or hidden will not be recorded, but in most cases it should be possible to figure out what was there. This will be trivial for fairly permanent things, like the ruins in Rome, but even possible for things that changed from day to day in our highly photographed world. A bit of AI will be able to turn the people in photos into 3-D animated models that can move within these VRs.

It will also be possible to extend this VR back into the past. The 20th century, before the advent of the digital camera, was not nearly so photographed, but it was still photographed quite a lot. For persistent things, the combination of modern (and future) recordings with older, less frequent and lower resolution recordings should still allow the creation of a fairly accurate model. The further back in time we go, the more interpolation and eventually artistic interpretation you will need, but very realistic seeming experiences will be possible. Even some of the 19th century should be doable, at least in some areas.

This is a good thing, because as I have written, the world’s tourist destinations are unable to bear the brunt of the rising middle class. As the Chinese, Indians and other nations get richer and begin to tour the world, their greater numbers will overcrowd those destinations even more than the waves of Americans, Germans and Japanese that already mobbed them in the 20th century. Indeed, with walking chairs (successors of the BigDog Robot) every spot will be accessible to everybody of any level of physical ability.

VR offers one answer to this. In VR, people will visit such places and get the views and the sounds — and perhaps even the smells. They will get a view captured at the perfect time in the perfect light, perhaps while the location is closed for digitization and thus empty of crowds. It might be, in many ways, a superior experience. That experience might satisfy people, though some might find themselves more driven to visit the real thing.

In the future, everybody will have had a chance to visit all the world’s great sites in VR while they are young. In fact, doing so might take no more than a few weekends, changing the nature of tourism greatly. This doesn’t alter the demand for the other half of tourism — true experience of the culture, eating the food, interacting with the locals and making friends. But so much commercial tourism — people being herded in tour groups to major sites and museums, then eating at tour-group restaurants — can be replaced.

I expect VR to reproduce the sights and sounds and a few other things. Special rooms could also reproduce winds and even some movement (for example, the feeling of being on a ship.) Right now, walking is harder to reproduce. With the OR Crescent Bay you could only walk 2-3 feet, but one could imagine warehouse size spaces or even outdoor stadia where large amounts of real walking might be possible if the simulated surface is also flat. Simulating walking over rough surfaces and stairs offers real challenges. I have tried systems where you walk inside a sphere but they don’t yet quite do it for me. I’ve also seen a system where you are held in place and move your feet in slippery socks on a smooth surface. Fun, but not quite there. Your body knows when it is staying in one place, at least for now. Touching other things in a realistic way would require a very involved robotic system — not impossible, but quite difficult.

Also interesting will be immersive augmented reality. There are a few ways I know of that people are developing

  • With a VR headset, bring in the real world with cameras, modify it and present that view to the screens, so they are seeing the world through the headset. This provides a complete image, but the real world is reduced significantly in quality, at least for now, and latency must be extremely low.
  • With a semi-transparent screen, show the augmentation with the real world behind it. This is very difficult outdoors, and you can’t really stop bright items from the background mixing with your augmentation. Focus depth is an issue here (and is with most other systems.) In some plans, the screens have LCDs that can go opaque to block the background where an augmentation is being placed.
  • CastAR has you place retroreflective cloth in your environment, and it can present objects on that cloth. They do not blend with the existing reality, but replace it where the cloth is.
  • Projecting into the eye with lasers from glasses, or on a contact lens can be brighter than the outside world, but again you can’t really paint over the bright objects in your environment.

Getting back to Rome, my goal would be to create an augmented reality that let you walk around ancient Rome, seeing the buildings as they were. The people around you would be converted to Romans, and the modern roads and buildings would be turned into areas you can’t enter (since we don’t want to see the cars, and turning them into fast chariots would look silly.) There have been attempts to create a virtual walk through ancient Rome, but being able to do it in the real location would be very cool.

Shuttle fly-by most photographed event in history?

A follow-up thought about yesterday’s shuttle fly-by and panorama. I was musing, might this be perhaps the most photographed single thing in human history to date?

Here’s the reasoning. Today there are more cameras and more photographers than ever, and people use them all the time in a way that continues to grow. To be a candidate for a most-photographed event, you would need to be recent, and you would need to take place in front of a ton of people, ideally with notice. It seemed like just about everybody in Sacramento, the Bay Area and LA was out for this and holding up a phone or camera.

Of course, many objects are more photographed, like the Golden Gate Bridge the shuttle flew over, but I’m talking here of the event rather than the object. Of course this is an event which moved over the course of thousands of miles.

Other candidates:

  • The other shuttle fly-overs done over New York and Washington — also with large populations
  • Total eclipses of the sun which go over highly populated areas. The 2009 eclipse went over Shanghai, Varanasi and many other hugely populated areas but was clouded out for many. Nobody has yet to make a photo of an eclipse that looks like an eclipse, of course — I’ve seen them all, including many of the clever HDRs and overlays — but that doesn’t stop people from trying.
  • The 1999 eclipse did go over a number of large European cities, but this was before the everybody-is-photographing era
  • Most lunar eclipses are seen by as much as half the world, though they are hard to photograph with consumer camera gear, and only a fraction of people go out to watch and photograph them, but they could easily be a winner.

Prior to the digital era, a possible winner might be the moon landing. Back in 1969, every family had a camera, though usage wasn’t nearly what it is today. However, I remember the TV giving lessons on how to photograph a TV screen. Everybody was shooting their TV for the launches and the walk on the moon. Terrible pictures (much like early camera phone pictures) but people took them to be a part of the event. I recall taking one myself though I have no idea where it is.

Of course there may be objective ways to measure this today, by tracking the number of photos on photo sharing and social sites, and extrapolating the winner. If the shuttle is the winner for now, it won’t last long. Photography is going to grow even more.

I should also note that remote photography, like we did for Apollo, is clearly much larger, in the form of recording video. For those giant events viewed by billions — World Cup, Olympics, Oscars etc. — huge numbers of people are recording them, at least temporarily.

Panorama of Shuttle fly-by at Moffett Field

Today marked the last trip through the air for the space shuttle, as the Endeavour was carried to LA to be installed in a museum. The trip included fly-overs of the Golden Gate bridge and many other landmarks in SF and LA, and also a low pass over Nasa Ames at Moffett Field, where I work at Singularity University. A special ceremony was done on the tarmac, and I went to get a panoramic photo. We all figured the plane would come along the airstrip, but they surprised us, having it fly a bit to the west so it suddenly appeared from behind the skeleton of Hangar One, the old dirigible hangar. That turned out to be bad for my photography, as I didn’t get much advance notice, and the shot of the crowd I had done a few minutes before had everybody expectantly looking along the runway, and not towards the west where the plane and shuttle appear in my photo.

However, it did make for a very dramatic arrival. So while different parts of this shot are at slightly different times, it does capture the scene of Moffett field and the crowd awaiting the shuttle, and its arrival. I do however have a nice hi-res photo for you to enjoy as well as the panoramic shot of the Endeavour shuttle fly-by.

New panoramas of Israel, and of course a proposal for peace

I’m back from our fun “Singuarlity Week” in Tel Aviv, where we did a 2 day and 1 day Singularity University program. We judged a contest for two scholarships by Israelis for SU, and I spoke to groups like Garage Geeks, Israeli Defcon, GizaVC’s monthly gathering and even went into the west bank to address the Palestinian IT Society and announce a scholarship contest for SU.

Of course I did more photography, though the weather did not cooperate. However, you will see six new panoramas on my Israel Panorama Page and my Additional Israeli panoramas. My favourite is the shot of the western wall during a brief period of sun in a rainstorm.

In Ramallah, the telecom minister for the Palestinian Authority asked us, jokingly, “how can this technology end the occupation?” But I wanted to come up with a serious answer. Everybody who goes to the middle east tries to come up with a solution or at least some sort of understanding. Israelis get a bit sick of it, annoyed that outsiders just don’t understand the incredible depth and nuance of the problem. Outsiders imagine the Israelis and Palestinians are so deep in their conflict that they are like fish who no longer see the water.

In spite of those warnings, here’s my humble proposal for how to use new media technology to help.

Take classrooms of Israelis and classrooms of Palestinians and give them a mandatory school assignment. Their assignment is to be paired with an online buddy from the “other side.” Students would be paired based on a matching algorithm, considering things like their backgrounds, language skills or languages and subjects they want to learn. The other student, with whom they would interact over online media and video-conferencing (like Skype or Google Hangouts,) would become a study partner and the students would collaborate on projects suitable to them. They might also help one another learn a language, like English, Arabic or Hebrew. Students would be encouraged to add their counterpart to their social networking circles.

Both students would also be challenged to write an essay attempting to see the world from the point of view of the other. They will not be asked to agree with it, but simply to be able to write from that point of view. And their counterpart must agree at the end that it mostly does reflect their point of view. Students would be graded on this.

It would be important not to have this be a “forced friendship.” The students would be told it was not demanded they forget their preconceptions; not demanded they agree with everything their counterpart says. In fact, they would be encouraged to avoid conflict, to not immediately contradict statements they think are false. That the goal is not to convince their counterpart of things but to understand and help them understand. And in particular, projects should be set up where the students naturally work together viewing the teachers as the common enemy.

At the end of the year, a meeting would be arranged. For example, west bank students would be thrilled at a chance to visit the beach or some amusement park. A meeting on the west bank border on neutral ground might make sense too, though parents would be paranoid about safety and many would veto trips by their children into the west bank.

Would this bring peace? Hardly on its own. But it would improve things if every student at least knew somebody from outside their world, and had tried to understand their viewpoint even without necessarily agreeing with it. And some of the relationships would last, and the social networks would grow. Soon each student would have at least one person in their network from outside their formerly insular world. This would start with some schools, but ideally it would be something for every student to do. And it could even be expanded to include online pen-pals from other countries. With some students it would fail, particularly older ones whose views are already set. Alas, for younger ones, finding a common language might be difficult. Few Israelis learn Arabic, more Palestinians learn Hebrew and all eventually want to learn English. Somebody has to provide computers and networking to the poorer students, but it seems the cost of this is small compared to the benefit.

A foveal digital camera sensor

Earlier I wrote about desires for the next generation of DSLR camera and a number of readers wrote back that they wanted to be able to swap the sensor in their camera, most notably so they could put in a B&W sensor with no colour filter mask on it. This would give you better B&W photos and triple your light gathering ability, though for now only astronomers are keen enough on this to justify filterless cameras.

I’m not sure how easy it would be to make a sensor that could be swapped, due to a number of problems — dust, connectivity and more. In fact I wonder if an idea I wrote about earlier — lenses with integrated sensors might have a better chance of being the future.

Here’s another step in that direction — a “foveal” digital camera that has tiny sensors in the middle of the frame and larger ones out at the edges. Such sensors have been built for a variety of purposes in the past, but might they have application for serious photography?

For example, the 5d Mark II I use has 22 million 6.4 micron sensors. Being that large, they are low noise compared to the smaller sensors found in P&S cameras. But the full frame requires very large, very heavy, very expensive lenses. Getting top quality over the large image circle is difficult and you pay a lot for it.

Imagine that this camera has another array, perhaps of around 16 million pixels of 1.6 micron size in the center. This allows it to shoot a 16MP picture in the small crop zone or a 22MP picture on the full frame. (It also allows it to shoot a huge 252 megapixel image that is sharp in the center but interpolated around the edges.) The central region would have transistors that could combine all the wells of a particular colour in the 4x4 array that maps to one large pixel. This is common in the video modes on DSLR cameras, and helps produce pixels that are much lower noise than the tiny pixels are on their own, but not as good as the 16x larger big pixels, though the green pixels, which make up half the area, would probably do decently well.

As a result, this camera would not be as good in low light, and the central region would be no better in low light than today’s quality P&S cameras. But that’s actually getting pretty good, and the results at higher light levels are excellent.

The win is that you would be able to use a 100mm/f2 lens with the field of view of a 400mm lens for a 16MP picture. It would not be quite as good as a real 400mm f/2.8L Canon lens of course. But it could compare decently — and that 400mm lens is immense, heavy and costs $10,000 — far more than the camera body. On the other hand a decent 100mm f/2.8 lens aimed at the smaller image circle would cost a few hundred dollars at most, and do a very good job. A professional wildlife or sports photographer might still seek the $10K lens but a lot of photographers would be much happier to carry the small one, and not just for the saved cost. You would not get the very shallow depth of field of the 400mm f/2.8 — it would be about double with a small sensor 100mm f/2 — but many would consider that a plus in this situation, not a minus.

You could also use 3.2 or 2.1 micron sensors for better low-noise and less of a crop (or focal length multiplier as it is incorrectly called sometimes.)

One other benefit is that, if your lens can deliver it, and particularly when you have decent lighting, you would get superb resolution in the center of your full frame photos, as the smaller pixels are combined. You would get better colour accuracy, without as many bayer interpolation artifacts, as you would truly sense each colour in every pixel, and much better contrast in general. You would be making use of the fact that your lens is sharper in the center. Jpeg outputs would probably never do the 250 megapixel interpolated image, but the raw output could record all the pixels if it is not necessary to combine the wells to improve signal/noise.

Botswana / Falls panorama gallery up

I have put up a new gallery of panoramic photos from my trip earlier this year to Botswana (with short stays in South Africa and Zimbabwe.) There are some interesting animal and scenic shots, and also some technically difficult shots such as Victoria Falls from a helicopter. (I also have some new shots of Niagara falls from a fixed wing plane which is even harder.)

In the case of the helicopter, which is still moving as it was just a regular tour helicopter, the challenge is to shoot very fast and still not make mistakes in coverage. I took several panos but only a few turned out. Victoria Falls can really only be viewed from the air — on the ground the viewing spots during high water season are in so much mist that it’s actually raining hard all around you, and in any event you can’t see the whole falls. One lesson is to try not to be greedy and get a 200m pano. Stick to 50 to 100mm at most.

On this trip I took along a 100-400mm lens, and it was my first time shooting with such a long lens routinely. I knew intellectually about the much smaller depth of field at 400mm, but in spite of this I still screwed up a number of panoramas, since I normally set focus at one fixed distance for the whole pano. Stopping down 400mm only helps a little bit. Wildlife will not sit still for you, creating extra challenges. I already showed you this elephant shot but I am also quite fond of this sunset on the Okavango delta. While this shot may not appear to have wildlife, the sun is beaming through giant spiderwebs which are the work of “social spiders” which live in nests, all building the same web. I recommend zooming in on the scene in the center. I also have some nice regular photos of this which will be up later.

I am still a bit torn about the gallery of ordinary aspect ratio photos. I could put them up on my photo site easily enough, but I’ve noticed photos get a lot more commentary and possibly viewing when done on Google+/Picasa. This is a sign of a disturbing trend away from the distributed web, where people and companies had their own web sites and got pagerank and subscribers, to the centralized AOL style model of one big site (be it Facebook or Google Plus) which is attractive because of its social synergies.

What do I want in a 5d Mark 3 (next generation digital SLR)

I shoot with the Canon 5d Mark II. While officially not a pro camera, the reality is that a large fraction of professional photographers use this camera rather than the Eos-1D cameras which are faster but much bulkier and in some ways even inferior to the 5D. But it’s been out a long time now, and everybody is wondering when its successor will come and what features it will have.

Each increment in the DSLR world has been quite dramatic over the last decade. There’s always been a big increase in resolution with the new generation, but now at 22 megapixels there’s less call for that. While there are lenses that deliver more than 22 megapixels sharply, they are usually quite expensive, and while nobody would turn down 50mp for free, there just wouldn’t be nearly as much benefit from it than the last doubling. Here’s a look at features that might come, or at least be wished for.

Better Pixels

More pixels may not be important, but everybody wants better pixels.

  • Low noise / higher ISO: The 5D2 astounded us with ISO 3200 shots that aren’t very noisy. Unlike megapixels, there is almost no limit to how high we would like ISO to go at low noise levels. Let’s hope we see 12,500 or more at low noise, plus even 50,000 noisy. Due to physics, smaller pixels have higher noise, so this is another reason not to increase the megapixel count.
  • 3 colour: The value of full 3-colour samples at every pixel has been overstated in the past. The reason is that Bayer interpolation is actually quite good, and almost every photographer would rather have 18 million bayer pixels over 6 million full RGB pixels. It’s not even a contest. As we start maxing out our megapixels to match our lenses, this is one way to get more out of a picture. But if it means smaller pixels, it causes noise. The Foveon approach which stacked the 3 pixels would be OK here — finally. But I don’t expect this to be very likely.
  • Higher dynamic range: How about 16 bits per pixel, or even 24? HDR photography is cool but difficult. But nobody doesn’t want more range, if only for the ability to change exposure decisions after the fact and bring out those shadows or highlights. Automatic HDR in the camera would be nice but it’s no substitute for try high-range pixels.

Video & Audio

Due to the high quality video in the 5D2, many professional videographers now use it. Last week Canon announced new high-end video cameras aimed at that market, so they may not focus on improvements in this area. If they do, people might like to see things like 60 frame video, ability to focus while shooting, higher ISO, and 4K video.  read more »

Panoramic article in Photo Technique

A little self-plug. I have an article on an introduction to panoramic photographic technique the November issue of Photo Technique with a few panos in it. This is old world journalism, folks — you have to read it on paper at least for now.

In the meantime, I’m working on upcoming galleries of photos from Botswana, Eastern Europe and Burning Man for you. I have already placed two of my Botswana photos into my gallery of favourite panoramas. This includes a lovely group of elephants in Savuti and a sunset on the Okavango delta that is one of my new favourites.

We decided to go to Harvey’s pan in Savuti one afternoon and lucked upon a large breeding group of elephant just on their way there. I caught them in one of my first long lens panoramas. Long lens panos are fairly difficult due to the limited depth of field, but they get great detail on the baby elephant.

Much more to come!

Gallery of regular photos from Burning Man 2010

As I prepare for Burning Man 2011, I realized I had not put my gallery of regular sized photos up on the web.

Much earlier I announced my gallery of giant panoramas of 2010 which features my largest photos in a new pan-and-zoom fullscreen viewer, I had neglected to put up the regular sized photos.

So enjoy: Gallery of photos of Burning Man 2010

I still need to select and caption 2007 and 2009 some day.

Back from Botswana, I want better audio for my video

This blog has been silent the last month because I’ve been on an amazing trip to Botswana and a few other places. There will be full reports and lots of pictures later, but today’s idea comes from experiments in shooting HD video using my Canon 5D Mark II. As many people know, while the 5D is an SLR designed for stills, it also shoots better HD video than all but the most expensive pro video cameras, so I did a bit of experimenting

The internal mic in the camera is not very good, and picks up not just wind but every little noise on the camera, including the noises of the image stabilizer found in many longer lenses. I brought a higher quality mic that mounts on the camera, but it wasn’t always mounted because it gets a little in the way of both regular shooting and putting the camera away. When I used it, I got decent audio, but I also got audio of my companion and our guide rustling or shooting stills with their own cameras. To shoot a real video with audio I had to have everybody be silent. This is why much of the sound you see in nature documentaries is actually added later, and very often just created by Foley artists. I also forgot to turn on my external mic, which requires a small amount of power, a few times. That was just me being stupid — as the small battery lasts for 300 hours I could have just left it on the whole trip. (Another fault I had with the mic, the Sennheiser MKE 400, was that the foam wind sleeve kept coming off, and after a few times I finally lost it.)  read more »

Definition of pixels for the world's biggest photos

I shoot lots of large panoramas, and the arrival of various cheaper robotic mounts to shoot them, such as the Gigapan Epic Pro and the Merlin/Skywatcher (which I have) has resulted in a bit of a “mine’s bigger than yours” contest to take the biggest photo. Some would argue that the stitched version of the Sloane Digital Sky survey, which has been rated at a trillion pixels, is the winner, but most of the competition has been on the ground.

Many of these photos have got special web sites to display them such as Paris 26 gigapixels, the rest are usually found at the site where you can even view the gigapans sorted by size to see which ones claim to be the largest.

Most of these big ones are stitched with AutopanoPro, which is the software I use, or the Gigapan stitcher. The largest I have done so far is smaller, my 1.4 gigapixel shot of Burning Man 2010 which you will find on my page of my biggest panoramas which more commonly are in the 100mp to 500mp range.

The Paris one is pretty good, but some of the other contenders provide a misleading number, because as you zoom in, you find the panorama at its base is quite blurry. Some of these panoramas have even just been expanded with software interpolation, which is a complete cheat, and some have been shot at mixed focal length, where sections of the panorama are sharp but others are not. I myself have done this, for example in my Gigapixel San Francisco from the end of the Golden Gate I shot the city close up, but shot the sky and some of the water at 1/4 the resolution because there isn’t really any fine detail in the sky. I think this is partially acceptable, though having real landscape features not at full resolution should otherwise disqualify a panorama. However, the truth is that sections of sky perhaps should not count at all, and anybody can make their panorama larger by just including more sky all the way to the zenith if they choose to.

There is a difficult craft to making such large photos, and there are also aesthetic elements. To really count the pixels for the world’s largest photos, I think we should count “quality” pixels. As such, sky pixels are not generally quality pixels, and distant terrain lost in haze also does not provide quality pixels. The haze is not the technical fault of the photographer, but it is the artistic fault, at least if the goal is to provide a sharp photo to explore. You get rid of haze only through the hard work of being there at the right time, and in some cities you may never get a chance.

Some of the shots are done through less than ideal lenses, and many of them are done use tele-extenders. These extenders do get more detail but the truth is a 2x tele-extender does not provide 4 times as many quality pixels. A common lens today is a 400mm with a 2x extender to get 800mm. Fairly expensive, but a lot cheaper than a quality 800mm lens. I think using that big expensive glass should count for more in the race to the biggest, even though some might view it as unfair. (A lens that big and heavy costs a ton and also weighs a lot, making it harder to get a mount to hold it and to keep it stable.) One can get very long mirror “lens” setups that are inexpensive, but they don’t deliver the quality, and I don’t believe work done with them should score as high as work with higher quality lenses. (It may be the case that images from a long telescope, which tend to be poor, could be scaled down to match the quality of a shorter but more expensive lens, and this is how it should be done.)

Ideally we should seek an objective measure of this. I would propose:

  • There should be a sufficient number of high contrast edges in the image — sharp edges where the intensity goes from bright to dark in the space of just 1 or 2 pixels. If there are none of these, the image must be shrunk until there are.
  • The image can then be divided up into sections and the contrast range in each evaluated. If the segment is very low contrast, such as sky, it is not counted in the pixel count. Possibly each block will be given a score based on how sharp it is, so that background items which are hazy count for more than nothing, but not as much as good sharp sections.
  • I believe that to win a pano should not contain gross flaws. Examples of such flaws include stripes of brightness or shadow due to cloud movement, big stitching errors and checkerboard patterns due to bad overlap or stitching software. In general that means manual exposure rather than shots where the stitcher tries to fix mixed exposures unless it does it undetectably.

Some will argue with the last one in particular, since for some the goal is just to get as many useful pixels as possible for browsing around. Gigapixel panoramas after all are only good for zooming around in with a digital viewer. No monitor can display them and sometimes even printing them 12 feet high won’t show all their detail, and people rarely do that. (Though you can see my above San Francisco picture as the back wall of a bar in SF.) Still, I believe it should be a minimum bar than when you look at the picture at more normal sizes, or print it out a few feet in size, it still looks like an interesting, if extremely sharp, picture.

Ideally an objective formula can be produced for how much you have to shrink what is present to get a baseline. It’s very rare that any such panorama not contain a fair number of segments with high contrast edges and lines in them. For starters, one could just put in the requirement that the picture be shrunk until you have a frame that just about anybody would agree is sharp like an ordinary quality photo when viewed 1:1. Ideally lots of frames like that, all over the photo.

Under these criteria a number of the large shots on gigapan fall short. (Though not as short as you think. The zoom viewer lets you zoom in well past 1:1, so even sharp images are blurry when zoomed in fully. On my own site I set maximum zoom at 200%.)

These requirements are quite strict. Some of my own photos would have to be shrunk to meet these tests, but I believe the test should be hard.

Shoot Nikon? Please help review my article on choosing lenses for Nikon cameras

For many years I have had a popular article on what lenses to buy for a Canon DSLR. I shoot with Canon, but much of the advice is universal, so I am translating the article into Nikon.

If you shoot Nikon and are familiar with a variety of lenses for them, I would appreciate your comments. At the start of the article I indicate the main questions I would like people’s opinions on, such as moderately priced wide angle lenses, as well as regular zooms.

If you “got a Nikon camera and love to take photographs” please read the article on what lens to buy for your Nikon DSLR and leave comments here or send them by email to I’m also interested in lists of “what’s in your kit” today.

Burning Man 2010 Panoramas with new Flash Viewer

I have put up a page of panoramas from Burning Man 2010. This page includes my largest yet, a 1.2 billion pixel image of the whole of Black Rock City which you will find first on the page. I am particularly proud of it, I hope you find it as amazing as I do.

There are many others, including a nice one of the Man while they dance before the burn with the whole circle of people, a hi-res of the temple and the temple burn, and more.

However, what’s really new is I have put in a Flash-based panorama zoom viewer. This application lets you see my photos for the first time at their full resolution, even the gigapixel ones. You can pan around, zoom in and see everything. For many of them, I strongly recommend you click the button (or use right-click menu) to enter fullscreen mode, especially if you have a big monitor as I do. There you can pan around with the arrow keys and zoom in and out with your mouse wheel. There are other controls (and when not in fullscreen mode you can also use shift/ctrl or +/- for zooming.) A help page has full details.

Go into the gigapixel and shot and zoom around. You’ll be amazed what you find. I have also converted most of my super-size city photos of Black Rock City to the zoom viewer, they can be found at the page of Giant BRC photos as well as many of my favourites from the various years. I’m also working at converting some of my other photos, including the gallery of my largest images which I built recently. It takes time to build and upload these so it will be some while before the big ones are all converted. I may not do the smaller ones.

If you don’t have flash, it displays the older 1100 pixel high image, and you can still get to that via a link. If you have flashblock, you will need to enable flash for my photo site because it will detect you have no flash player and display the old one.

Get out the big monitor and it will feel like you’re standing on a tower in Black Rock City with a pair of binoculars. The gigapixel image is also up on gigapan.

Syndicate content