Can phones help older folks enjoy restaurant conversation again?

Even outdoors, a big crowd can make it impossible to hear the conversation. Even more in Sicily.

A recent New York Times article on going to restaurants over 50 didn't need to remind me of something almost all of us oldsters already know -- you can't hear people any more except in rare quiet restaurants. If I sit down with a group, at best I might manage a conversation with the people on my left and right, and if I am lucky, those opposite. And my voice will be sore.

So here's the answer, and app I want. An audio conferencing app for people in loud restaurants.

Everybody brings a smartphone and their earbuds, with microphone. If you want to be really fancy have a whole headset, but that looks a little geeky at a social event.

The phones sync up and do a local ad-hoc wifi (or bluetooth) mesh network. You all start the app and swipe an agreed code so people at the next table can't join or decrypt.

Then they share audio for the dinner. Get those earbuds that plug up your ears to block out the rest of the room. Talk in an ordinary voice.

As a fancy feature, you see the faces of the other people on your screen. You can mute out some, or turn them down to quiet, to let you have a conversation in a subgroup. You can freely have a conversation with somebody way at the other end of the table. You can make your conversation private so the others can't hear it even if they try, but that's probably rude. Anybody can but in and announce something to the whole table and revert to group conversation. Perhaps the tool is very smart and figures out what subgroups are talking to one another and how that switches, and adjusts volumes accordingly. Perhaps you can even use voice commands, and say people's names to start talking to them. (The phones will share names and photos so you know who is who -- another useful thing when you're getting older.)

The system could try to figure out the arrangement of people around the table. Or people could quickly touch and drag the faces to collaboratively do that layout. That could help the system decide what volumes to use.

That leaves the problem of the waiter. You need another microphone for the waiter. So somebody has to bring a spare phone. Or quickly unplug their earbuds and hand their phone to the waiter -- who, it being his or her job -- can hear you all without the buds. Or maybe restaurants get used to this and the waiters get the app on their phones, the system designed to let them speak to the group but not listen back.

Or, of course, people can all pull out their earphones when the waiter shows up.

Latency

The biggest issue will be latency. You can't really have much at all, unless people are willing to tolerate everybody's lips being out of sync. That precludes too much analysis of background noise with fancy techniques. Even the 10-20ms of phones and peer to peer wifi/bluetooth may be too much for people to like. It could be this would need to end up as a pure analog or barely digital radio product, which means it costs money and is harder for everybody to have one. Some phones add a lot of latency to their microphone. Apple phones are generally pretty good. Some Android phones are good, but others are terrible and might not be able to participate. (It's OK if the phone has bad playback latency as that only punishes the owner, while bad microphone latency punishes everybody listening to the owner.)

Of course, if we could just find a way to increase the video latency to match the audio latency it would all be in sync. :-)

Fancy features

Great audio will take some work. Each microphone will be picking up every voice around it, though if it gets close to the owner's mouth the problem is easier. You need the audio to feel natural to get an acceptable quality level. There are some tricks to mute which don't clip the start of every phrase.

Those who are young and don't need earbuds would like to use this without putting them on. One solution is for them to just put them on their neck, or clip on just a microphone. Restaurants could come with a supply of cheap microphones. They can be had for $2 from China -- you don't need top quality. Older folks could even just carry a bag of them to loan out. Of course, newer smartphones have started removing the headphone/microphone jack, requiring a different type of earbuds, though they also have gotten cheap.

It's also possible to do fancy things using arrays of microphones. With beamforming techniques, software with an array of microphones can isolate a voice from the din of the crowd without needing the person to wear a microphone. It causes some minor distortions. As such, it is possible, in theory, to put a set of phones on the table and use their internal microphones (with excellent clock sync) in this way. This lets somebody join the conversation and be heard by everybody by just putting their phone down on the table. They still need good ears to hear people.

And yes, this could even work on an individual basis. A person with decaying hearing could just ask the one person they are talking to do this. They could even use their own phone if the idiots who designed Bluetooth's headset profile had not decided it was only for low quality phone calls, and hard limited the quality of the microphone signal.

The system could allow nearly private conversations (ie. feed no audio of you to anybody but your subgroup) but for social reasons, that might not be a great idea. You are supposed to hear the conversations of your neighbours, and join in them at appropriate times. So it should let you hear the others at reduced volume, and allow you to mute them, but not them to block you, since they could not do that at a physical table. If you want to flirt with the cutie at the far end, you could do so, but expect that others might hear you. (In fact, your immediate neighbours will of course hear you.) And if you mute somebody, they should always be able to unmute themselves to get your attention.

Another bonus feature -- one I proposed long ago for all cell phones -- is for the system to give you feedback on how loudly you need to talk. Everybody talks too loud into their phone when in an open area, when in fact the phone is picking you up just fine. The phone can compare the audio levels of the background and the speaker (as it hears them) and if the speaker is being needlessly loud (as is our instinct in a loud room) it can issue a beep in the ears to remind them to quiet. A visual display can also help with this, if the person is watching. New AI tools can also help remove move of the background noise.

Getting started

Fortunately, you don't need to get the whole table on board to try this out. If a group goes out to a loud restaurant, a subset group of 4-6 people could put on earbuds and try this. They could now converse with the other members -- no matter where they are around the table -- and could also talk to their immediate neighbours, by pulling out the buds for a while.

Indeed, it can work with only 2 or 3 people, at least for a while, though you probably don't plan to talk to only 1 or 2 other people all night. (Though that's what you get without this.)

Comments

Some years ago I decided to test-drive hearing aids. Went to DishDash in Sunnyvale, which is both the best and loudest Middle Eastern restaurant around (it's always crowded, and the main room walls and ceiling echo badly.) They let me have a conversation across the table, even with just one.
There are cellphone hearing aid programs; you need to use a wired headset, because Bluetooth latency is too high, and I haven't seen a phone with a good way to connect dual mikes, so it's only mono, but that's enough to help high-frequency roll-off some.

Here's a radical suggestion (which should also work for restaurants): An office with “library rules”

Groups come to restaurants as much for conversation as food. So I can't see this applying. Put a bunch of tables together doing that and it's doomed. So fancy restaurants have booths and private rooms and sound deadening walls. But most don't.

That's a marvelous idea, particularly since my hearing is not what it once was. Nor is is just for dining out. It'd work well in situations where people who're separated need to chat. That'd include having being seated apart on a plane or in a crowd at an amusement park.

For now, people might try to make do with what are called "walkie talkie apps." You can find a list of the more popular ones here:

http://bestappsguru.com/best-walkie-talkie-apps-android-ios/

It is of course very easy to do this with push to talk -- but I think that would put a damper on regular conversation, so I would want to find a way to do it with just normal conversational flow, like a full duplex audioconferencing system. With the added challenge of the high background noise, and the fact it hears other members of the group through microphones other than their own. The closer you can get the microphone to your mouth, the better.

Just have everyone dial into the same conference call number. No new app needed. You could even use a free conference call service.

As a product manager, seems one could validate this idea very quickly, first with the conferencing feature via each user's carrier via the conference feature already baked into many phone apps.

Then if that idea had legs, perhaps try it with the mobile versions of Skype, Zoom, or GotoMeeting via the restaurant's IP or via each individual's carrier.

If this appears to be "a thing" then you can spend dev dollars on exploring NFC or Bluetooth approaches.

But the latency on that would be extreme. Ever tried to talk to somebody on a cell phone when they are in the same room and you are looking at them? It will drive you nuts. However, you can test at least the audio isolation.

Recent trends in restaurant decor have made them much noisier places. See the Atlantic article "How Restaurants Got So Loud" (unable to provide link, rejected as spam)

If you use enough cameras and mics to track the voices and ears of everyone at the table, plus whatever audio is coming at them from other tables, can't you broadcast appropriate cancelling waveforms for each ear of each person? I have no idea if audio cancelling can be broadcast at a distance even if you did know the final waveform you were trying to suppress.

I think the restaurant could reasonably *boost* the right voices to the right listeners, but that's kind of adding to the overall problem.

Would be a major research breakthrough. But no, you can't do the cancel waveforms like that -- you need to be right at the ear.

Add new comment