Submitted by brad on Sat, 2009-11-28 16:48.
Some recent searches have revealed unusual activity on twitter, and I wonder where it’s going. Narcissus searches on twitter reveal a variety of accounts tweeting links into my blog and sites, for reasons not clearly apparent.
For example, a week ago, a half dozen identical twitter accounts all tweeted my post about electric cars playing music. All the accounts had pictures of models as their icon, and the exact same set of twitter posts, which seem to be a random collection of blog and news URLs with a bit.ly pointer to the item, all posted via twitterfeed. These accounts seem to follow and be followed by about 500, presumably the same list.
Here are some of the accounts:
Then more recently I see another set of accounts which all follow about 20 people but are followed by about 200 to 500. They are all posting “from API” and again are just posting links, this time with tinyurl.com. The account names are odd, too.
These also seem to to have cute girls as icons. However, strangely, the many followers appear to be real, or at least some of them appear to be. Why are people following a spam robot? Are the followers people who were paid to do it, or are in some twitter-optimization scheme?
What I am curious about is the motive. Are they linking to real sites in the hope of gaining some sort of legitimacy in twitter indexing engines, so that later they can start linking to people who pay for it? (Twitter SEO?) Are they trying to form twitter equivalents of link farms? Are they just hoping that site authors will see the backlinks and look at them for some later purpose? (You would be amazed how many hits on a web server are there just to put a spammer in the “Referer” field, either to get you to look, or to show up in referer logs that some sites post to the web.)
Thoughts on what’s up?
Submitted by brad on Wed, 2009-10-28 22:03.
I just returned from Jeff Pulver’s “140 Characters” conference in L.A. which was about Twitter. I asked many people if they get Twitter — not if they understand how it’s useful, but why it is such a hot item, and whether it deserves to be, with billion dollar valuations and many talking about it as the most important platform.
Some suggested Twitter is not as big as it appears, with a larger churn than expected and some plateau appearing in new users. Others think it is still shooting for the moon.
The first value in twitter I found was as a broadcast SMS. While I would not text all my friends when I go to a restaurant or a club, having a way so that they will easily know that (and might join me) is valuable. Other services have tried to do things like this but Twitter is the one that succeeded in spite of not being aimed at any specific application like this.
This explains the secret of Twitter. By being simple (and forcing brevity) it was able to be universal. By being more universal it could more easily attain critical mass within groups of friends. While an app dedicated to some social or location based application might do it better, it needs to get a critical mass of friends using it to work. Once Twitter got that mass, it had a leg up at being that platform.
At first, people wondered if Twitter’s simplicity (and requirement for brevity) was a bug or a feature. It definitely seems to have worked as a feature. By keeping things short, Twitter makes is less scary to follow people. It’s hard for me to get new subscribers to this blog, because subscribing to the blog means you will see my moderately long posts every day or two, and that’s an investment in reading. To subscribe to somebody’s Twitter feed is no big commitment. Thus people can get a million followers there, when no blog has that. In addition, the brevity makes it a good match for the mobile phone, which is the primary way people use Twitter. (Though usually the smart phone, not the old SMS way.)
And yet it is hard not to be frustrated at Twitter for being so simple. There are so many things people do with Twitter that could be done better by some more specialized or complex tool. Yet it does not happen.
Twitter has made me revise slightly my two axes of social media — serial vs. browsed and reader-friendly vs. writer friendly. Twitter is generally serial, and I would say it is writer-friendly (it is easy to tweet) but not so reader friendly (the volume gets too high.)
However, Twitter, in its latest mode, is something different. It is “sampled.” In normal serial media, you usually consume all of it. You come in to read and the tool shows you all the new items in the stream. Your goal is to read them all, and the publishers tend to expect it. Most Twitter users now follow far too many people to read it all, so the best they can do is sample — they come it at various times of day and find out what their stalkees are up to right then. Of course, other media have also been sampled, including newspapers and message boards, just because people don’t have time, or because they go away for too long to catch up. On Twitter, however, going away for even a couple of hours will give you too many tweets to catch up on.
This makes Twitter an odd choice as a publishing tool. If I publish on this blog, I expect most of my RSS subscribers will see it, even if they check a week later. If I tweet something, only a small fraction of the followers will see it — only if they happen to read shortly after I write it, and sometimes not even then. Perhaps some who follow only a few will see it later, or those who specifically check on my postings. (You can’t. Mine are protected, which turns out to be a mistake on Twitter but there are nasty privacy results from not being protected.)
TV has an unusual history in this regard. In the early days, there were so few stations that many people watched, at one time or another, all the major shows. As TV grew to many channels, it became a sampled medium. You would channel surf, and stop at things that were interesting, and know that most of the stream was going by. When the Tivo arose, TV became a subscription medium, where you identify the programs you like, and you see only those, with perhaps some suggestions thrown in to sample from.
Online media, however, and social media in particular were not intended to be sampled. Sure, everybody would just skip over the high volume of their mailing lists and news feeds when coming back from a vacation, but this was the exception and not the rule.
The question is, will Twitter’s nature as a sampled medium be a bug or a feature? It seems like a bug but so did the simplicity. It makes it easy to get followers, which the narcissists and the PR flacks love, but many of the tweets get missed (unless they get picked up as a meme and re-tweeted) and nobody loves that.
On Protection: It is typical to tweet not just blog-like items but the personal story of your day. Where you went and when. This is fine as a thing to tell friends in the moment, but with a public twitter feed, it’s being recorded forever by many different players. The ephemeral aspects of your life become permanent. But if you do protect your feed, you can’t do a lot of things on twitter. What you write won’t be seen by others who search for hashtags. You can’t reply to people who don’t follow you. You’re an outsider. The only way to solve this would be to make Twitter really proprietary, blocking all the services that are republishing it, analysing it and indexing it. In this case, dedicated applications make more sense. For example, while location based apps need my location, they don’t need to record it for more than a short period. They can safely erase it, and still provide me a good app. They can only do this if they are proprietary, because if they give my location to other tools it is hard to stop them from recording it, and making it all public. There’s no good answer here.
Submitted by brad on Sun, 2009-06-07 16:29.
Twenty years ago (Monday) on June 8th, 1989, I did the public launch of ClariNet.com, my electronic newspaper business, which would
be delivered using USENET protocols (there was no HTTP yet) over the internet.
ClariNet was the first company created to use the internet as its platform for business, and as such this event has a claim at being the birth of the “dot-com” concept which so affected the world in the two intervening decades. There are other definitions and other contenders which I discuss in the article below.
In those days, the internet consisted of regional networks, who were mostly non-profit cooperatives, and the government funded “NSFNet” backbone which linked them up. That backbone had a no-commercial-use policy, but I found a way around it. In addition, a nascent commercial internet was arising with companies like UUNet and PSINet, and the seeds of internet-based business were growing. There was no web, of course. The internet’s community lived in e-Mail and USENET. Those, and FTP file transfer were the means of publishing. When Tim Berners-Lee would coin the term “the web” a few years later, he would call all these the web, and HTML/HTTP a new addition and glue connecting them.
I decided I should write a history of those early days, where the seeds of the company came from and what it was like before most of the world had even heard of the internet. It is a story of the origins and early perils and successes, and not so much of the boom times that came in the mid-90s. It also contains a few standalone anecdotes, such as the story of how I accidentally implemented a system so reliable, even those authorized to do so failed to shut it down (which I call “M5 reliability” after the Star Trek computer), stories of too-early eBook publishing and more.
There’s also a little bit about some of the other early internet and e-publishing businesses such as BBN, UUNet, Stargate, public access unix, Netcom, Comtex and the first Internet World trade show.
Extra, extra, read all about it: The history of ClariNet.com and the dawn of the dot-coms.
Submitted by brad on Sat, 2009-03-14 16:43.
As you may know, I allow anonymous comments on this blog. Generally, when a blog is small, you don’t want to do too much to discourage participation. Making people sign up for an account (particularly with email verification) is too much of a barrier when your comment volume is small. You can’t allow raw posting these days because of spammers — you need some sort of captcha or other proof-of-humanity — but in most cases moderate readership sites can allow fairly easy participation.
Once a site gets very popular, it probably wants to move to authenticated user posting only. In this case, once the comment forums are getting noisy, you want to raise the bar and discourage participation by people who are not serious. My sub blog on Battlestar Galactica has gotten quite popular of late, and is attracting 100 or more comments per post, even though it has only 1/10th the subscribers of the main blog. Almost all post using the anonymous mechanism which lets them fill in a name, but does nothing to verify it. Many still post under the default name of “Anonymous.”
Some sites let you login using external IDs, such as OpenID, or accounts at Google or Yahoo. On this site, you can log in using any ID from the drupal network, in theory.
However, drupal (which is the software running this site) and most other comment/board systems are not very good at providing an intermediate state, which I will call “casual comments.” Here’s what I would like to see:
- Unauthenticated posters may fill in parameters as they can now (like name, email, URL) and check a box to be remembered. They would get a long-term cookie set. The first post would indicate the user was new.
- Any future posts from that browser would use that remembered ID. In fact, they would need to delete the cookie or ask the site to do so in order to change the parameters.
- If they use the cookie, they could do things like edit their postings and several of the things that registered users can do.
- If they don’t pick a name, a random pseudonym would be assigned. The pseudonym would never be re-used.
- Even people who don’t ask to be remembered would get a random pseudonym. Again, such pseudonyms would not be re-used by other posters or registered users. They might get a new one every time they post. Possibly it could be tied to their IP, though not necessarily traceable back to it, but of course IPs change at many ISPs.
- If they lose the cookie (or move to another computer) they can’t post under that name, and must create a new one. If they want to post under the same name from many machines, create an account.
- The casual commenters don’t need to do more special things like create new threads, and can be quite limited in other ways.
In essence, a mini-account with no authorization or verification. These pseudonyms would be marked as unverified in postings. A posting count might be displayed. A mechanism should also exist to convert the pseudonym to a real account you can login from. Indeed, for many sites the day will come when they want to turn off casual commenting if it is getting abused, and thus many casual commenters will want to convert their cookies into accounts.
The main goal would be to remove confusion over who is posting in anonymous postings, and to stop impersonation, or accusations of impersonation, among casual posters.
I don’t think it should be too hard to make a module for drupal to modify the comment system like this if I knew drupal better.
Submitted by brad on Mon, 2009-02-02 23:05.
As I move to get more paper out of my life, one thing I’m throwing away with more confidence is manuals. It’s pretty frequent that I can do a search for product model numbers or other things on a manual, and find a place to download the PDF. Then I can toss the manual. I need to download the PDF, because the company might die and their web site might go away.
I would like to make this even easier. For starters, it would be nice if the UPC database (UPC are the bar codes found on all retail products) would also offer a link to getting all manuals and paper that come with a product. I would then be able to just photograph the bar codes of all my products with my phone or camera, and cause automatic download or escrow of all manuals. Perhaps a symbol next to the UPC could tell me this is guaranteed to work.
It would be even better if companies escrowed the manuals, which is to say paid a one-time fee to a trustable company which would promise to keep the documents online forever. This company must be backed by a very solid company itself, perhaps a consortium of all the major vendors with a pact that if any of them go other, the rest take up the slack of maintaining the site.
In fact, all free, public documents should have a code on them that can be turned into a URL where I can fetch the document, as PDF, HTML or even MSWord. Any attempt to scan such a document would pick up this code and know it doesn’t have to scan the rest unless it is marked up. For books, we sould key off the ISBN as well as the UPC. Eventually one of the newer, compact 2-D “barcodes” could be used to code a number to find the docs.
Of course, many products are now coming without manuals at all, and that’s largely fine with me.
Submitted by brad on Fri, 2009-01-16 16:10.
Some of you may know that I started a sub-blog for my thoughts on my favourite SF TV show, Battlestar Galactica. This sub-blog was dormant while the show was off the air, but it’s started up again with new analysis as the first new episode of the final 10 (or 12) episodes airs tonight. (I will be missing watching it near-live as I will be giving a talk tonight on Robocars at the Future Salon in Palo Alto.) Reports are that one big mystery — the last Cylon — is revealed tonight.
So if you watch Battlestar Galactica, you may want to subscribe to the feed for the Battlestar Galactica Analysys Bog right here on this site. And I’ll go out on a limb and promote my two top candidates for the mystery Cylon.
Some recent posts of note:
Submitted by brad on Mon, 2008-08-18 21:22.
NBC has had just a touch of coverage of Michael Phelps and his 8 gold medals, which in breaking Mark Spitz’s 7 from 1972 has him declared the greatest Olympic athlete, or even athlete of all time. And there’s no doubt he’s one of the greatest swimmers of all time and this is an incredible accomplishment. Couch potato that I am, I can hardly criticise him.
(We are of course watching the Olympics in HDTV using MythTV, but fast-forwarding over the major bulk of it. Endless beach volleyball, commercials and boring events whiz by. I can’t imagine watching without such a box. I would probably spend more time, which they would like, but be less satisfied and see fewer of the events I wish to.)
Phelps got 8 Gold but 3 of them were relays. He certainly contributed to those relays, may well have made the difference for the U.S. team and allowed it to win a gold it would not have won without him. So it seems fair to add them, no?
No. The problem is you can’t win relay gold unless you are lucky enough to be a citizen of one of a few powerhouse swimming nations, in particular the USA and Australia, along with a few others. Almost no matter how brilliant you are, if you don’t compete for one of these countries, you have no chance at those medals. So only a subset of the world’s population even gets to compete for the chance to win 7 or 8 medals at the games. This applies to almost all team medals, be they relay or otherwise. Perhaps the truly determined can emigrate to a contending country. A pretty tall order.
Phelps one 5 individual golds, and that is also the record, though it is shared by 3 others. He has more golds than anybody, though other athletes have more total medals.
Of course, swimming is one of the special sports in which there are enough similar events that it is possible to attain a total like this. There are many sports that don’t even have 7 events a single person could compete in. (They may have more events but they will be divided by sex, or weight class.)
Shooting has potential for a star. It used to even be mixed (men and women) until they split it. It has 9 male events, and one could in theory be master of them all.
Track and Field has 47 events split over men and women. However, it is so specialized in how muscles are trained that nobody expects sprinters to compete in long events or vice versa. Often the best sprinter does well in Long Jump or Triple Jump, allowing the potential of a giant medal run for somebody able to go from 100m to 400m in range. In theory there are 8 individual events 400m or shorter.
And there are a few other places. But the point is that to do what Phelps (or Spitz) did, you have to be in a small subset of sports, and be from a small set of countries. There have been truly “cross sport” athletes at the Olympics but in today’s world of specialized training, it’s rare. If anybody managed to win multiple golds over different sports and beat this record, then the title of greatest Olympian would be very deserving. One place I could see some crossover is between high-diving and Trampoline. While a new event, Trampoline seems to be like doing 20 vaults or high dives in a row. And not that it wasn’t exciting to watch him race.
More Burning Man packing…
Submitted by brad on Thu, 2008-05-29 15:20.
If you run a web site of reasonable popularity, you probably get invitations to sign up for ad networks from time to time. They want you to try them out, and will sometimes talk a great talk about how well they will do.
I always tell them “put your money where your mouth is — guarantee at least some basic minimum during the trial.”
Most of them shut up when I ask for that, indicating they don’t really believe their own message. I get enough that I
wrote a page outlining what I want, and why I want it — and why everybody should want it.
If you have a web site with ads, and definitely if you have an ad network, consider reading what I want before I’ll try your ad network.
Submitted by brad on Sat, 2008-01-12 21:48.
The last week saw some serious signs that Blu-Ray could win the high-def DVD war over HD-DVD. Many people have been waiting for somebody to win the war so that they don’t end up buying a player and a video collection in the format that loses. (Strangely, the few players that supported both formats tended to cost much more than two individual players.)
Now there’s a report that the new profile for Blu-ray will obsolete many old players. So even those who made the right bet and didn’t get a PS3 may be just as screwed.
Something amazes me that has amazed me since the days of the first Audio CD players in the 80s. The Audio CD redbook format was defined early, and it was a lot of work to get reasonable combined audio + data disks because of it. And long after burnable CDs became popular (and into DVDs) it’s been the case that many home players can’t read the disks at all until they are “finalized” and unable to take more data. There were many other problems. And that’s not itself the problem, as there will always be demands you don’t anticipate.
But it’s not as though these devices don’t have a readily available means by which to be given new programming. They have a drive in them, and it would have been easy to issue CDs or DVDs with signed new firmwares on them. Indeed, since the disks have always been vastly huge compared to the firmwares of the devices they played on, it’s usually been the case that a disk wishing to use a new format could probably include new firmware for every known player in a small part of the disk. That’s certainly true for blu-ray.
Now of course if a player doesn’t have enough memory or CPU or graphics power, you can’t update it to do things it simply isn’t capable of doing. But you should be able to always update it to understand at least the structure of new formats, and know what they can use and what they can’t. Of course, all updates must be signed by a highly protected manufacturers key, so that attackers can’t hack your firmware, and the user should have to confirm on their remote that they want to accept the update. And yes, if that key is compromised and people don’t insert a disk with a revocation command on it quickly enough, there can be trouble. But it’s better than having players that slow down progress in the business.
(And yes, I realize that many early CD players did not have rewritable firmware, since ROM was cheaper than EEPROM and flash didn’t come along for a while. But it would have been worth it, and there’s no excuse for not having safely flashable firmwware today in just about anything.)
And on another rant, I’ve always been amazed at the devices that do allow firmware flashing but don’t have a safety mechanism. There are many devices, some still made today, that can be turned into “bricks” if you flash buggy firmware, in that you can no longer flash new firmware. Every device should have, in unwritable storage, the most basic and well tested firmware reloader that can be invoked if the recently installed firmware has failed. Some devices have this but it’s taken a long time.
While I don’t really seek a game machine because it would suck up too much time, it may be time for a PS3 as a Blu-Ray player. They are not much more expensive than the standalone players, and of course do much more. If I wanted a game machine it would probably be a Wii. We found one this year for a gift for the nephews, but they got another one so I ended up selling it for a $100 profit on Craigslist, the prevailing market being what it was. Made an Egyptian boy very happy, as they are very hard to get over there.
Submitted by brad on Thu, 2007-12-27 18:34.
A story from the New York Times suggests it costs over $12,000/year to store a movie in digital form.
This number is entirely bogus, and based on old thinking, namely the assumptions of offline storage on DVDs and tapes.
Offline media do degrade, and you must copy them before they
have a chance to degrade, which takes people, though frankly it’s still should not be as
expensive as this. To do my calculations, I am going to assume a movie needs 100gb of storage with low-loss lossy compression. You can scale the numbers up if you like if you want to assume more, even at 1 TB it doesn’t change that much.
A film occupying 100gb of storage can go on about 20 dvds (or 11 dual layer,)
costing about $8. It can go on 4 independent sets of 20 DVDs for $32 in media. Ideally you could rack these in a DVD jukebox, but if they are just sleeved, then once a year a person could pull out the DVDs, put them in a reader which would test them. Any that tested fine would be re-sleeved, those that did not would flag for the others to be pulled, and then copied to new media. (Probably better media, like blu-ray.) There are algorithms to distribute the data so that a large number of the disks must fail in that year to actually lose something. Of course, you use different vaults around the world.
When approaching the point where failure rates go up for the media, you re-burn new copies even if the old ones still test fine.
This takes human time, though not all that much. Perhaps half an hour of actual human time swapping disks though much more real time to burn them, but you don’t do just one at a time.
However, even better is the new style of archival — online storage. Hard disks are 20 cents/gigabyte
and continuing to fall. NAS boxes are more expensive now but there is no reason they won’t drop to very reasonable prices, so that a NAS case adds perhaps 5 cents/gigabyte (ie. $100 for a 4x500gb drive box which lasts for 10-15 years.) (NAS boxes are small boxes that hold a collection of drives and allow access to them over ethernet. No computer is needed.) They also cost about 2 cents/gb/year for power if on all the time, and some small amount for space, though they would tend to sit in computer centers that already exist.
Those are today’s prices, which will just get cheaper, except for the power. Much cheaper. If a drive lasts an average of 4 years before failing and a NAS lasts 10 years, this works out to 7.5 cents/gigabyte/year. Of course you will store your files redundantly, in 4 different places (which is actually overkill) and so it’s 30 cents/gigabyte/year.
Which is still just $30 for a 100gb file, or $300 for a TB.
Online storage is live. You can regularly check the integrity, all the time. You can either leave it off and spin it up every few days (to not use power) or just leave it on all the time.
two or three of the 4 disks fails, computers can copy the data to fresh disks in the
network, and you are alive. Your disks should last 3 to 4 years but many will last
much longer. You need a computer system to control all this, but you only need one for the entire cloud of NAS boxes, or at most a few. Its cost is low.
The real cost is people. But companies like Google have solved the problem of running large server farms. They tolerate single drive failures. The computers copy the data to new drives right away, and technicans go by every few days to pull old ones and slot in fresh ones for the next need — not for the same file. This takes just a few minutes of the tech’s time. And there is no rush to their work. Fore each 100gb file, you should expect to have a replacement about once every 4 years (ie. the lifetime of an average drive.)
Now all this is at today’s price of $100 for a 500gb drive. But that’s dropping fast, faster than Moore’s law. The replacements will be 1TB and 2TB drives before long, and the cost will continue to fall. And this is with 4 copies of every file. You can actually get by with less using modern data distribution algorithms which can scatter a file of 100gb into 200 1gb pieces, for which almost half must be lost before the whole file is lost. Several data centers could burn down without losing any files if things are done right. I have not accounted for bandwidth here for replacements, which usually would be done in the same data center except in unusual circumstances.
The biggest cost is the people to set all this up. However, presuming big demand, the cost per gigabyte for those people should become modest.
Submitted by brad on Thu, 2007-11-15 16:55.
The strike by screenwriters in the Porn Writers Guild of America is wreaking
a less public havoc on the pornography industry. Porn writers, concerned
about declining revenue from broadcast TV, also seek a greater share of
revenue from the future growth areas of DVD and online sales.
“Online sales and DVD may one day be the prime sources of revenue in
our industry,” stated union spokesman Seymour Beaver. We want to be
sure we get our fair share of that for providing the writing that makes
this industry tick.
“It’s getting terrible,” reported one porn consumer who refused to
give his name. “I just saw Horny Nurses 14 and I have to tell you
it was just a reshash of the plots from Horny Nurses 9 and 11. It’s
like they didn’t even have a writer.”
“Fans are not going to put up with movies lacking in plot, character
and dialogue, and that’s what they’ll get if they don’t meet our
terms,” said Beaver. Beaver, who claims to have a copyright on
the line, “Oh yes, baby, do it just like that, oh yeah” says he
will not allow use of his lines without proper payment of residuals.
Some writers also fear that the move to online will result in
customers simply downloading individual scenes rather than
seeking movies with a cohesive story thread that makes you
care about the characters. “I saw one movie with 5 scenes,
and no character was in 2 of them,” complained one writer.
“What do people want? Movies where the actors just walk into
a room, strip and just go at it? Where they always start with
oral sex, then doggy, and then a money shot? Fans will walk
if that’s all they get,” according to PWGA member Dick Member.
“And don’t think about doing the lonely housewife and the pool-boy
again. I own that.”
An industry spokesman said they had not yet seen any decline
in revenues due to the strike, as they have about 2 million
already-written scripts on the shelves. In addition, Hot
Online Corporation spokesman Ivana Doit claimed their company
is experimenting with a computer program that creates scripts
through a secret algorithm. Scripts penned by the computer have
already brought in a million in sales, claims Doit, but she would
not indicate which films this applied to.
Submitted by brad on Sun, 2007-10-21 00:01.
After going through the VHS to digital process, which I lamented earlier I started wondering about the state of digitizing old vinyl albums and tapes is.
There are a few turntable/cd-writer combinations out there, but like most people today, I’m interested in the convenience of compressed digital audio which means I don’t want to burn to CDs at all, and nor would I want to burn to 70 minute CDs I have to change all the time just so I can compress later. But all this means I am probably not looking for audiophile quality, or I wouldn’t be making MP3s at all. (I might be making FLACs or sampling at a high rate, I suppose.)
What I would want is convenience and low price. Because if I have to spend $500 I probably would be better off buying my favourite 500 tracks at online music stores, which is much more convenient. (And of course, there is the argument over whether I should have to re-buy music I already own, but that’s another story. Some in the RIAA don’t even think I should be able to digitize my vinyl.)
For around $100 you can also get a “USB turntable.” I don’t have one yet, but the low end ones are very simple — a basic turntable with a USB sound chip in it. They just have you record into Audacity. Nothing very fancy. But I feel this is missing something.
Just as the VHS/DVD combo is able to make use of information like knowing the tape speed and length, detecting index marks and blank tape, so should our album recorder. It should have a simple sensor on the tone arm to see as it moves over the album (for example a disk on the axis of the arm with rings of very fine lines and an optical sensor.) It should be able to tell us when the album starts, when it ends, and also detect those 2-second long periods between tracks when the tone arm is suddenly moving inward much faster than it normally is. Because that’s a far better way to break the album into tracks than silence detection.
(Of course, you can also use CDDB/Freedb to get track lengths, but they are never perfect so the use of this, net data and silence
detection should get you perfect track splits.) It would also detect skips and repeats this way. read more »
Submitted by brad on Sun, 2007-06-24 20:50.
At Supernova 2007, several of us engaged Andrew Keen over his controversial book "The Cult of the Amateur." I will admit to not yet having read the book. Reviews in the blogosphere are scathing, but of course the book is entirely critical of the blogosphere so that's not too unexpected.
However, one of the things Keen said he worries about is what he calls the "scarcity of talent." He believes the existing "professional" media system did a good enough job at encouraging, discovering and promoting the talent that's out there, and so the world doesn't get more than slush with all the new online media. The amount of talent he felt, was very roughly constant.
I presented one interesting counter to this concept. I am from Canada. As you probably know, we excel at Hockey. Per capita certainly, and often on an absolute scale, Canada will beat any other nation in Hockey. This is only in part because of the professional leagues. We all play hockey when we are young, and this has no formal organization. The result is more talented players arise. The same is true for the USA in Baseball but not in Soccer, and so on.
This suggest that however much one might view YouTube as a vaster wasteland of terrible video, the existence of things like YouTube will eventually generate more and better videographers, and the world will be richer for it, at least if the world wants videographers. One could argue this just takes them away from something else but I doubt that accounts for all of it.
Submitted by brad on Thu, 2007-04-05 00:06.
I’ve written before about the problems with TV advertising. Recently I’ve been thinking more about the efficiency of various methods of advertising — to the target, not to the advertiser. Almost all studies of advertising concern how effectively advertising turns into leads or sales, but rarely are the interests of the target of the ad considered directly.
I think that has to change, because we’re getting more tools to avoid advertising and getting more resistant. I refuse to watch TV with ads, because at $1.20 per hour of advertising watched, it’s a horrible bargain. I would rather pay if I could, and do indeed buy the DVDs in many cases, but mostly my MythTV skips the ads for me. The more able I am to do this, the more my desires as a target must be addressed.
Advertising isn’t totally valueless to the target. In fact, Google feels one big reason for their success is that they deliver ads you might actually care to look at. There are other forms of advertising with the same mantra out there, and they tend to do well, such as movie trailers and Superbowl ads.
Consider a video ad lasting 30 seconds, with a $10 CPM. That means the advertiser pays one cent per viewer of the ad. The viewer spends 30 seconds. On the other hand, a box with 3 or 4 Google ads, as you might see on this page, is typically scanned in well under a second. These ads also earn (as a group) about a $10 CPM though they are paid per click. Google doesn’t publish numbers, but let’s assume a $10 CPM and a 1% click-through on the box. It’s actually higher than this.
In the 30 seconds a TV ad takes, I can peruse perhaps 50 boxes, bars or banners of web ads. That will expose me to over 100 product offers that in theory match my interests, compared to 1 for the video ad. The video ad will of course be far more convincing as it is getting so much attention, but in terms of worthwhile products offered to me per second, it’s terrible.
It isn’t quite this simple though, since I will click on one ad every every minute spent looking at ads (not every minute on the web) and perhaps spend another minute looking in detail at what the ad had to offer. That particular, very well targeted site, gains the wealth of attention the video ad demands, but far more efficiently.
I think this area is worth of more study in the industry, and I think it’s a less understood reason why Google is getting rich, and old media are running scared. In the future, people will tolerate advertising less and less unless it is clearer to them what value they are getting for it. Simply being able to get free programming is not the value we’re looking for, or if it is, we want a better deal — more programming in exchange for our valuable attention. But we want more than that better deal. We want to be advertised to efficiently, in a way that considers our needs and value. The companies that get that will win, the dinosaurs will find themselves in the movie “The Sixth Sense” — dead people, who don’t know they’re dead.
Submitted by brad on Mon, 2007-04-02 17:41.
The human voice is a pretty versatile instrument, and many skilled vocalists have been able to do convincing imitations of other sounds, and we’ve all heard “human beat box” artists work with a microphone to do great sounds.
That got me thinking, could we train a choir to work together to sound like anything, starting with violins, and perhaps even a piano or more?
The idea would be to get some vocalists to make lots of sounds, both pure tones and more complex ones, and break them apart with spectrum analysis. Do the same for the target sound — try to break it up into components that might be made by human vocal cords with appropriate spectrum analysis.
Then find a way to easily add the human sounds together to sound like the instrument. Each singer might focus on one of the harmonics or other tonal qualities of the instrument. Do it first in the computer, and then see if the people can do it together, without being distracted. Then work on doing the attack and decay and other artifacts of the start and end of notes.
If it all worked, it would be a fun gag for a choir to suddenly sound like a piano or violin playing a popular piece. Purer tones like a flute might be harder than complex tones. Percussion is obviously possible though it might need some amplification. Indeed, amplification to adjust the levels properly might help a lot but would be slightly more artificial than hearing this without any electronics. Who knows, perhaps a choir could even sound like an orchestra playing the opening to Beethoven’s 5th, something everybody knows well.
Submitted by brad on Thu, 2007-03-29 22:32.
If you’ve looked around, you probably noticed a high-def DVD player, be it HD-DVD or Blu-Ray, is expensive. Expect to pay $500 or so unless you get one bundled with a game console where they are subsidized.
Now they won’t follow this suggestion, but the reality is they didn’t need to make the move to these new DVD formats. Regular old DVD can actually handle pretty decent HDTV movies. Not as good as the new formats, but a lot better than plain DVD. I’ve seen videos with the latest codecs that pack a quite nice HD picture into 2.5 to 3 gigabytes for an hour. I’ve even seen it in less, down to 1.5 gigabytes (actually less that SD DVDs) at 720p 24 fps, though you do notice some problems. But it’s still way better than a standard DVD. Even so, a dual layer DVD can bring about 9 gb, and a double sided dual layer DVD gives you 18gb if you are willing to flip the disk over to get at special features or the 2nd half of a very long movie. Or of course just do 2-disk sets.
Now you might feel that the DVD industry would not want to make a new slew of regular DVD players with the fancier chips in them able to do these mp4 codecs when something clearly better is around the corner. And if they did do this, it would delay adoption of whatever high def DVD format they are backing in the format wars. But in fact, these disks could have been readily playable already, with no change, for the millions who watch DVDs on laptops and media center PCs. More than will have HD DVD or Blu-Ray for some time to come, even with the boost the Playstation 3 gives to Blu-Ray. read more »
Submitted by brad on Sat, 2007-03-17 19:33.
When I watch SF TV shows, I often try to imagine a backstory that might make the story even better and SF like. My current favourite show is Battlestar Galactica, which is one of those shows where a deep mystery is slowly revealed to the audience.
So based on my own thoughts, and other ideas inspired from newsgroups, I’ve jotted down a backstory to explain the results you see in the show. Of course, much of it probably won’t end up being true, but there are hints that some of it might.
In my Battlestar Galactica back-story I explain why
- Why everybody — even the so-called humans — is a Cylon
- Who the Final 5 are and what they are doing
- Why all this has happened before and is happening again
- How the Cylons were made, and where they got their biotech
Of course, ignore this if you don’t watch the show. It’s pure fanfic/speculation.
The show remains one of the great SF TV shows, though it has been bogging down of late. This timeline may be a plea to return the show to some good hard SF roots. Posthumanism and strife between humans and AIs are hot themes in modern SF, and BSG is most interesting if it’s set in our future with things to say about the relationship between man, machine and artificial biological intelligence.
Update: I have updated the article based on the season finale, which confirmed a number of my speculations though of course not all of them.
Submitted by brad on Thu, 2007-03-08 14:31.
I have written several times before about Peerflix — Now that I’ve started applying some tags as well as categories to my items you can now see all the Peerflix stories using that link — and the issues behind doing a P2P media trading/loaning system. Unlike my own ideas in this area, Peerflix took a selling approach. You sold and bought DVDs, initially for their own internal currency. It was 3 “Peerbux” for new releases, 2 for older ones, and 1 for bargain bin disks.
That system, however, was failing. You would often be stuck for months or more with an unpopular disk. Getting box sets was difficult. So in December they moved to pricing videos in real dollars. I found that interesting because it makes them, in a way, much closer to a specialty eBay. There are still a lot of differences from eBay — only unboxed disks are traded, they provide insurance for broken disks and most importantly, they set the price on disks.
One can trade DVDs on eBay fairy efficiently but it requires a lot of brain effort because you must put time into figuring good bid and ask prices for items of inconsequential price. Peerflix agreed that this is probably a poor idea, so they decided to set the prices. I don’t know how they set their initial prices, but it may have been by looking at eBay data or similar information. read more »
Submitted by brad on Sun, 2007-02-25 19:49.
I’ve been seeing a lot of press lately worrying that the internet won’t be able to handle the coming video revolution, that as more and more people try to get their TV via the internet, it will soon reach a traffic volume we don’t have capacity to handle. (Some of this came from a Google TV exec’s European talk, though Google has backtracked a bit on that.)
I don’t actually believe that, even given the premise behind that statement, which is traditional centralized download from sites like Youtube or MovieLink. I think we have the dark fiber and other technology already in place, with terabits over fiber in the lab, to make this happen.
However, the real thing that they’re missing is that we don’t have to have that much capacity. I’m on the board of Bittorrent Inc., which was created to commercialize the P2P file transfer technology developed by its founder, and Monday we’re launching a video store based on that technology. But in spite of the commercial interest I may have in this question, my answer remains the same.
The internet was meant to be a P2P network. Today, however, most people do download more than they upload, and have a connection which reflects this. But even with the reduced upload capacity of home broadband, there is still plenty of otherwise unused upstream sitting there ready. That’s what Bittorrent and some other P2P technologies do — they take this upstream bandwidth, which was not being used before, and use it to feed a desired file to other people wishing to download the file. It’s a trade, so you do it from others and they do it for you. It allows a user with an ordinary connection to publish a giant file where this would otherwise be impossible.
Yes, as the best technology for publishing large files on the cheap, it does get used by people wanting to infringe copyrights, but that’s because it’s the best, not because it inherently infringes. It also has a long history of working well for legitimate purposes and is one of the primary means of publishing new linux distros today, and will be doing hollywood major studio movies Feb 26.
Right now the clients connect with whoever they can connect with, but they favour other clients that send them lots of stuff. That makes a bias towards other clients to whom there is a good connection. While I don’t set the tech roadmap for the company, I have expectations that over time the protocol will become aware of network topology, so that it does an even better job of mostly peering with network neighbours. Customers of the same ISP, or students at the same school, for example. There is tons of bandwidth available on the internal networks of ISPs, and it’s cheap to provide there. More than enough for everybody to have a few megabits for a few hours a day to get their HDTV. In the future, an ideal network cloud would send each file just once over any external backbone link, or at most once every few days — becoming almost as efficient as multicasting.
(Indeed, we could also make great strides if we were to finally get multicasting deployed, as it does a great job of distributing the popular material that still makes up most of the traffic.)
So no, we’re not going to run out. Yes, a central site trying to broadcast the Academy Awards to 50 million homes won’t be able to work. And in fact, for cases like that, radio broadcasting and cable (or multicasting) continue to make the most sense. But if we turn up the upstream, there is more than enough bandwidth to go around within every local ISP network. Right now most people buy aDSL, but in fact it’s not out the question that we might see devices in this area move to being soft-switchable as to how much bandwidth they do up and and how much down, so that if upstream is needed, it can be had on demand. It doesn’t really matter to the ISP — in fact since most users don’t do upstream normally they have wasted capacity out to the network unless they also do hosting to make up for it.
There are some exceptions to this. In wireless ISP networks, there is no up and downstream, and that’s also true on some ethernets. For wireless users, it’s better to have a central cache just send the data, or to use multicasting. But for the wired users it’s all 2-way, and if the upstream isn’t used, it just sits there when it could be sending data to another customer on the same DSLAM.
So let’s not get too scared. And check out the early version of bittorrent’s new entertainment store and do a rental download (sadly only with Windows XP based DRM, sigh — I hope for the day we can convince the studios not to insist on this) of multiple Oscar winner “Little Miss Sunshine” and many others.
Submitted by brad on Thu, 2007-01-18 20:19.
(Note I have a simpler article for those just looking for advice on how to get their Widescreen TV to display properly.)
Very commonly today I see widescreen TVs being installed, both HDTV and normal. Flat panel TVs are a big win in public places since they don’t have the bulk and weight of the older ones, so this is no surprise, even in SDTV. And they are usually made widescreen, which is great.
Yet almost all the time, I see them configured so they take standard def TV programs, which are made for a 4:3 aspect ratio, and stretch them to fill the 16:9 screen. As a result everybody looks a bit fat. The last few hotel rooms I have stayed in have had widescreen TVs configured like this. Hotel TVs disable you from getting at the setup mode, offering a remote control which includes the special hotel menus and pay-per-view movie rentals. So you can’t change it. I’ve called down to the desk to get somebody to fix the TV and they often don’t know what I’m talking about, or if somebody comes it takes quite a while to get somebody who understands it.
This is probably because I routinely meet people who claim they want to set their TV this way. They just “don’t like” having the blank bars on either side of the 4:3 picture that you get on a widescreen TV. They say they would rather see a distorted picture than see those bars. Perhaps they feel cheated that they aren’t getting to use all of their screen. (Do they feel cheated with a letterbox movie on a 4:3 TV?)
It is presumably for those people that the TVs are set this way. For broadcast signals, a TV should be able to figure out the aspect ratio. NTSC broadcasts are all in 4:3, though some are letterboxed inside the 4:3 which may call for doing a “zoom” to expand the inner box to fill the screen, but never a “stretch” which makes everybody fat. HDTV broadcasts are all natively in widescreen, and just about all TVs will detect that and handle it. (All U.S. stations that are HD always broadcast in the same resolution, and “upconvert” their standard 4:3 programs to the HD resolution, placing black “pillarbox” bars on the left and right. Sometimes you will see a program made for SDTV letterbox on such a channel, and in that case a zoom is called for.)
The only purpose the “stretch” function has is for special video sources like DVD players. Today, almost all widescreen DVDs use the superior “anamorphic” widescreen method, where the full DVD frame is used, as it is for 4:3 or “full frame” DVDs. Because TVs have no way to tell DVD players what shape they are, and DVD players have no way to tell TVs whether the movie is widescreen or 4:3, you need to tell one or both of them about the arrangement. That’s a bit messy. If you tell a modern DVD player what shape TV you have, it will do OK because it knows what type of DVD it is. DVD players, presented with a widescreen movie and a 4:3 TV will letterbox the movie. However, if you have a DVD player that doesn’t know what type of TV it is connected to, and you play a DVD, you have to tell the TV to stretch or pillarbox. This is why the option to stretch is there in the first place.
However, now that it’s there, people are using it in really crazy ways. I would personally disable stretch mode when playing from a source known not to be a direct video input video player, but as I said people are actually asking for the image to be incorrectly stretched to avoid seeing the bars.
So what can we do to stop this, and to get the hotels and public TVs to be set right, aside from complaining? Would it make sense to create “cute” pillarbars perhaps with the image of an old CRT TV’s sides in them? Since HDTVs have tons of resolution, they could even draw the top and bottom at a slight cost of screen size, but not of resolution. Some TVs offer the option of gray, black and white pillars, but perhaps they can make pillars that somehow match the TV’s frame in a convincing way, and the frame could even be designed to blend with the pillars.
Would putting up fake drapes do the job? In the old days of the cinema, movies came in different widths sometimes, and the drapes would be drawn in to cover the left and right of the screen if the image was going to be 4:3 or something not as wide. They were presumably trying to deal with the psychological problem people have with pillarbars.
Or do we have to go so far as to offer physical drapes or slats which are pulled in by motors, or even manually? The whole point of flatscreen TVs is we don’t have a lot of room to do something like this, which is why it’s better if virtual. And of course it’s crazy to spend the money such things would cost, especially if motorized, to make people feel better about pillarbars.
I should also note that most TVs have a “zoom” mode, designed to take shows that end up both letterboxed and pillarbarred and zoom them to properly fit the screen. That’s a useful feature to have — but I also see it being used on 4:3 content to get rid of the pillarbars. In this case at least the image isn’t stretched, but it does crop off the top and bottom of the image. Some programs can tolerate this fine (most TV broadcasts expect significant overscan, meaning that the edges will
be behind the frame of the TV) but of course on others it’s just as crazy as stretching. I welcome other ideas.
Update: Is it getting worse, rather than better? I recently flew on Virgin America airlines, which has widescreen displays on the back of each seat. They offer you movies (for $8) and live satellite TV. The TV is stretched! No setting box to change it, though if you go to their “TV chat room” you will see it in proper aspect, at 1/3 the size. I presume the movies are widescreen at least.