OP
sygeek
Technomancer
The case for piracy
By Brett Elliott
When it comes to copyright theft and piracy, many people assume there's just one side - the side of truth, justice and copyright owners. Beyond that there are parasitical thieves. When most governments come to legislate on the matter, their response is usually one of listening to what big corporations and lobby groups say and nodding in agreement. For the general public, years of being bombarded by cross platform marketing campaigns have ingrained people with various "Piracy bad. Copyright good" slogans.
We've been deluged with the arguments against piracy for years. But what's the other side of the story? Could it possibly be that copyright infringers and pirates aren't always the bad guys? Are copyright owners their own worst enemy? Judge for yourself and tell us what you think.
Contempt for customers
We'll start with an area that many reading this can relate to. Commercial media's contempt for its audience. These are some examples which touched me and they may ring bells for you.
Gladiator, Channel 10. We're back from a commercial break. Rusty arrives back at home in Spain to find his wife and son raped and crucified. It's arguably the most touching scene of the whole movie. What better time for a giant cartoon helicopter to fly around the screen announcing, "Don't forget, Merrick and Rosso! The B-Team! Every Wednesday night at 7.30!"
I remember every syllable of that ad. Positioning ads like this is, Gruen has told us, is most effective as we're at our most vulnerable. But at the same time this was like the network raising its middle finger at the us and yelling, "Lap it up, suckers!" But is there a way to treat your audience with any more contempt?
I think Channel 7 managed it. Remember the TV show Lost? The first series had a huge buzz about it - largely from making huge waves in America, weeks beforehand. Many people downloaded the series from the US as it aired. I stuck with Channel 7 for some kind of local solidarity reasons. The anticipation coming up to the final, 24th episode revolved around the big reveal, "What's under the hatch?" Then, after watching religiously week after week, there was an unexplained six week hiatus. Six weeks! Again, I restrained myself from downloading the final episodes and stuck with 7. Finally, the show reappeared. Then, in the very first ad slot of the very first ad break there came the trailer, "Don't forget to keep watching the final episodes of Lost [as if!] when we show you what's under the hatch!"
Then they showed us what was under the hatch. Right there and then.
I won't tell you what I shrieked at the TV. But perhaps you can imagine. As spoilers go, that was huge. That was the last episode of Lost I watched... On Channel 7.
This happens all the time. Channel 11, the other day, came back from a five minute ad break to show the last ten seconds of a Simpsons episode. Ten seconds! But I think the 'abject contempt to its viewers award' must go to Channel 9.
I could regale you for ages with my Channel 9 rage. Yet I keep finding myself watching movies which are butchered by having five-minutes-on-five-minutes-off ads at the end. Using Tivo to buffer programs for an hour before watching - so that I can skip through the ads - is one way round it. Of course, this forces 9 to use in-program display ads to make up the revenue. Somehow I don't care. Because there are two areas where 9's actions are the scheduling equivalent of dropping a turd on my doorstep.
Sporting events
I remember my dad ringing up from the UK and remarking how excellent and exciting the Melbourne Commonwealth Games were. Discussion in the office had confirmed that I wasn't the only person who found 9's delayed and appalling coverage unwatchable. It's been the same for subsequent Commonwealth Games and the Olympics. If you could watch events Live on the internet, wouldn't you? There's no other legal way to watch most of them Live (if at all) in Australia.
Did you want to watch all of the matches in the Rugby World Cup? Must have sucked how 9 bought the rights and then DIDN'T SHOW THE MATCHES LIVE! For those who knew what they were doing, you could watch them free on the internet. What other option did they have?
The English Premiership
Nine's treatment of sport is a local problem. Globally, the big issue is English Soccer. The rights are managed by Sky TV (The UK's equivalent to Foxtel). To be fair, the money Sky pumped into the sport, plus the huge improvements in coverage, is one of the reasons this is the most popular league in world sport. But for those of us who had little money, we'd rather be in a position to actually watch a game on TV than know that only the moneyed people had access to the improved coverage. There was the option of traipsing down the pub, but that meant coming home most-likely drunk, reeking of cigarette smoke (before the ban) and still having spent money. But the real problem was this...
You'd ring up Sky. "Hi, I'd like to subscribe to Sky to watch the football please?"
"Certainly, which football do you want?"
"The Premiership football."
Certainly, it's available on this package, that package and the other package."
"No I just want the football. I don't want the US Soap channel, the African Animal Channel, the Infomercial Channel... etc etc etc."
Indeed, Sky spreads its Premiership games across several channels in several packages so you have to subscribe to all their other crap in order to get the few football matches that you want to pay for. The resulting monthly fee is well over a hundred dollars. Even to watch the odd pay per view game you have to pay for Sky and then pay for a package in order to pay for the pay per view.
Or... you can just watch it live on the internet. For free.
In Australia, it's a similar problem. But I'm not subscribing to Foxtel just to watch my team play the occasional game in the middle of the night. I'd gladly pay to watch the matches I want to see. But I can't. As a result, I hardly watch any matches anymore. But if there's a big one, then my one and only option is to watch it live on the internet. What else can I do?
The problem is such that there are large international communities all over the world, telling people where to watch games live on the web. Some websites even charge a fee to provide a high-quality online stream. The charges cover hosting costs and, once there are enough people connected, they accept no more customers for fear of dropping quality. So people are actually PAYING to watch these matches illegally when they could watch them illegally for free!
Overseas Content
This problem reappears in many other areas too. A major one concerns Japanese anime (cartoons). Ars Technica did an excellent investigation into this matter. It found that there were huge online communities sharing copyrighted content, but that money was not a reason for doing so. Typically, when a cartoon appeared in Japan, it would take a year for it to appear overseas. When it did appear it would be dubbed with dumb-ass American dialogue which obliterated many of the cultural references which made the cartoons popular in the first place.
One of the 'infringing' community websites then did what one would hope the rest of the media industry would do - it realised that there was an enormous demand for overseas content to be aired online immediately after publication and that people would happily pay for it.
The BBC
Recently, the BBC launched iPlayer in Australia. This gives you access to much of the BBC's vast television archives. To a degree, this has long been desired by overseas residents. But the dominating discussion was all about the BBC's failure to allow payment of an overseas licence fee to let international viewers watch Live BBC content.
I lived in Japan several years ago, and people from all nationalities said at the time they'd love to pay to watch live BBC TV. The demand is enormous but when I recently asked the BBC, they said it wasn't going to happen.
Sure they get huge sums for licensing internally-produced programs and series, but they may get even more by allowing online access to international paying customers. However, even if this did happen, there would be issues with the BBC covering international events. A good example is Formula 1. Many Australian F1 fans baulk at Channel Ten's coverage and are only too glad for the switchover to the BBC's outstanding race commentary. Not having to suffer ads or Mark-Webber-obsessed presenters who struggle to contain their disappointment at not talking about V8s or motorbikes is a constant bugbear for many. Those who know about the internet know how to stream the BBC's coverage Live so there are no ads or interruptions. You can't pay for that though. Many would if they could.
But if there's one prime example of the problems of surrounding the BBC, copyright infringement and international viewers it's a certain program with 350 million viewers worldwide...
Top Gear
I used to watch Top Gear. I can't now. There are several hundred thousand Australians who are in the same boat. SBS picked it up long ago and built a regular audience of over a million people. Then 9 bought the rights and quickly decimated the audience. It's around 400,000 now. How on earth did it manage to do that in car-crazy Australia?
First you need to know that the BBC sends out an international version of Top Gear to overseas licensees which has 15 minutes cut from each show - to allow for ads. Consequently, if you want to watch a full episode of Top Gear your only option is to download it illegally from the web, or wait ages for the DVDs to appear. Then there was the fact that SBS had a two-YEAR delay in showing episodes. Nonetheless, a million loyal fans watched it and I was one of them.
After switching network, Channel 9 bragged about fast-tracking UK episodes. All sounded good. But then it took the already-short international version and butchered it by cutting more content out to add even more ads. The following week, despite promises of a new episode it showed an ancient, years-old episode. Apparently, it was OK to say this was a new episode because it was new to Channel 9. Cue ten years' worth of old episodes appearing randomly interspersed with more--recent episodes and Blam! the audience walked. I'd happily pay to watch Top Gear. Channel 9 makes it unwatchable. My only option is to download it. I haven't watched it in years.
The Music Industry
Piracy has affected few industries more than music. Back in the early days of the internet, services like Napster, Kazaa and Audio Galaxy appeared which let you swap songs with other people online. At the time, there was no talk of copyright infringement, it was just something that geeky internet users did and it felt like a more-efficient way of swapping cassettes and CDs in the playground. Unfortunately, it was so efficient that the global and industrialised scale destroyed the traditional way in which music was produced and marketed. Quite rightly, the services were shut down. But the story doesn't end there.
The age of compressed music formats and MP3 music players had begun. Once the third-generation iPod hit the market, along with iTunes, compressed digital music became mainstream. What a great opportunity for the music industry: the customers wanted compressed music delivered online and it was cheap to do. But could the industry have screwed things up any more?
Rather than give customers what they wanted publishers threw every toy they had out of the pram and hit the litigation button. One example saw the recording industry sue a 12-year old girl and won $2000. From her point of view she was simply using a free service on the internet that all her friends were using and discussing. One wonders how happy the recording industry was with its $2000 payout. Over the years industry bodies have spent far more money suing people than they recouped through the courts.
One of the main reasons we all have anti-piracy slogans embedded in our brains is because the music industry chose to try and protect its existing market and revenue streams at all costs and marginalise and vilify those who didn't want to conform to the harsh new rules being set.
The Napster brand went legit, iTunes rose and Sony started offering its vast music catalogues online. But instead of selling the compressed music that the public wanted, the industry "sold" music riddled with Digital Rights Management (DRM) 'copyright protection' meaning that the music would only play back on certain devices under certain conditions. Music was also being sold using formats which wouldn't work on all music players and compressed to degrees that resulted in a loss of quality which turned-off enthusiasts. In short, despite selling the music, you didn't own what you'd bought. You were essentially "renting" the rights to the music. Shouldn't there have been intervention from the government?
After a while Sony got bored of the lack of traction which its appalling model had generated and turned off its entire system. This meant that everyone who had "bought" music from Sony couldn't play it on anything other than the old devices launched to go with it. People who had invested heavily in Sony's music were ignored.
Around this time Sony also came up with other ways to stop people listening to the music they had bought. A system appeared which inserted noise and interference when people tried to compress music from CDs. Consequently, if you only listened to MP3 music, you couldn't actually legally get an MP3 version of a song. Even if you had paid money for a CD. Sony even topped this by secretly putting computer software on its audio CDs which secretly installed licensing software on your computer if you tried to compress the music on it. Not only was this a gross breach of privacy, but the 'rootkit' that was installed was a major security threat. This was one occasion where Sony got hammered for its actions. Ultimately, though the publishers were treating their paying customers as potential criminals and the widespread resentment was palpable.
As time wore on, it became clear that the DRM on music was linked to the original hardware you had when it was bought. For many people, if you bought legitimate compressed music online, like I did, when you go to play it you get the following message...
I paid good money for those songs. Am I supposed to buy them again? Or can I download them illegally from the internet in clear conscience?
Things seem to be slowly changing with Apple offering DRM-free, higher-quality songs on iTunes now and with the industry recognising the importance of the online music store. Nonetheless, you're still forced to buy from one seller, using one format and at a quality which, these days, could be higher. The best sales model surely came from legally-spurious site, AllOfMp3.
This Russian based site allowed you to purchase almost any song in any format using any level of compression that you wanted and charged a low price for it. In other words, it recognised public demand and gave people exactly what they wanted.
But its licensing model was dodgy at best. It did pay royalties but at tiny Russian radio-play levels. Many of the songs were sold without permission from the copyright holders. It got sued by everyone for a staggering $1.65 trillion, but was eventually acquitted.
Outlandish lawsuits like this have become the norm for media publishers and their industry organisations. At no point did they realise that this was the most obvious business model to use - to give people what they want at a fair price.
Nowadays, the publishers seem to have moved on. They're still suing downloaders and crippling innovative internet-radio business models like Pandora, but the new popular model seems to be charging a subscription fee for on-demand access to entire music catalogues - iTunes' iCloud music service, Last.fm and Sony's Anubis are good examples. I've used the latter for months and it's excellent.
Movies
Heaps of movies are illegally downloaded these days, but unlike the music industry, the film industry is thriving. Theories abound as to the impact of downloading movies over the internet: there is evidence which suggests that those who download movies tend to be enthusiasts who spend more on movies in the first place (as is the case with music downloaders). Certainly the cinema trade is booming. My pet theory is that many downloaders download movies they aren't particularly fussed about seeing (not enough to pay for them anyway) or which are unavailable where they live. But the constant engagement with movies keeps them in the "film enthusiast" bracket and that makes them go to the cinema when something that they're particularly keen on appears.
Hysterical lawyers say otherwise. More on that below. Either way, movie downloading is a contentious business as are its consequences.
There is obviously a huge public craving for movies and video on demand but the only place that you can get many movies is illegally online. Legal services in Australia tend to, well, suck. Tivo has boasted for years about the thousands of movies you can pay for on demand. Most of them seem to have Marilyn Monroe or John Wayne in them. Selections aren't much better elsewhere. If you want to pay for good video on demand services the best you can do is pay for quasi-legal access to American sites like Netflix. Or download illegally. Either way, you're probably a criminal.
Fair Dealing
In Australia, America and other countries, there are laws which protect people from innocently using "copyrighted" media in non-commercial and various reasonable ways. But don't expect to find authorities standing up for your rights.
Youtube is a prime example. If you make a video of something, but in the background there's a song playing - from a nearby radio or whatever - it gets banned. Want to share your child's birthday party with friends and family? You'd better not play any recorded Happy Birthday song in the background - you'll get your account suspended.
Other bans stem from people making their own movie mashups or discussing clips from mainstream media.
It's difficult to imagine what lawyers and publishers have to gain by banning people from doing this and vilifying them for doing so. Youtube got sick of dealing with individual take-down requests and waved the white flag long ago. It just bans things automatically now.
Almost all of these infringements are actually allowable under Fair Use (America) and Fair Dealing (Australia) legislation. But to the publishers and establishment, it too-often seems, you're just a criminal.
Harsh Litigation
Most troubling of all is that prosecuting people for suspected copyright infringement has become an industry in its own right. Legal firms are buying the rights from publishers to sue people on their behalf. It's an evolution of the ambulance chasing lawyer. There's a straight-forward business model for it.
You send out letters to potential copyright infringers telling them that they have downloaded something illegally and will be sued for anything up to $150,000. They have the option to settle beforehand - typically for a few thousand dollars - just enough to save on hiring a lawyer to defend the case.
The threat is based on the fact that if you have downloaded a movie then you will also have uploaded it and distributed it to thousands of people. In reality, however, if you download something using bittorrent only a small fraction gets uploaded. It would take balls of steel and deep pockets to explain that one in court though.
In America the music and movie lobbies have pushed through a non-government accord which allows corporations to punish suspected copyright infringers without any trial or due legal process. The US government, it transpires, has few issues with this. It's not yet clear whether Australia will follow along similar lines.
Industry bodies are certainly wanting to enforce their will on Australian legislation, though, as the battle between iiNet and AFACT illustrates. AFACT has been defeated several times but hasn't given up. More recently, The Age uncovered a new Gold Coast operation which is planning to demand money from downloaders of porn. There are fears that this could be the thin end of the wedge for Australia.
However, the article also points out that similar UK operations were eventually denounced in the House of Lords as "straightforward legal blackmail." So not all governments are as compliant as the copyright industry might like.
Conclusion
Nowadays, copyright barely resembles what it was originally designed for i.e. to protect both parties: inventors and content creators on the one side and the public on the other. Corporate America and government compliance have written out public interests in many instances. The case of Mickey Mouse is illustrative.
Nonetheless, there's an air of inevitability about it all. Historically, how often have incumbent, monopolistic industries shrugged their shoulders and written off their entire business model to embark on a journey along a crowded new highway, with rules set by customers, that leads who-knows-where?
On a personal note, I suspect that once the world's internet infrastructure comes up to speed, we'll all be using on-demand subscription models and the notion of buying content to keep will feel archaic. Even so, more needs to be done to protect the public from ham-fisted copyright industries demanding payment for everything.
A great deal of copyright infringement does not stem from criminal behaviour. Much of it occurs simply because there is all-too-often no other way to legally access the content you want - even if you do want to pay for it.
It's worth remembering that there are many big losers because of piracy, but these have been well covered elsewhere. The video games industry, for example, is a major loser, but we'll deal with that another time. This article is one of few that deals with the flipside of the argument and so please remember that it is intends to describe and inform - not endorse any infringement. Has it changed your opinion on the matter or confirmed it? Let us know below.
We've been deluged with the arguments against piracy for years. But what's the other side of the story? Could it possibly be that copyright infringers and pirates aren't always the bad guys? Are copyright owners their own worst enemy? Judge for yourself and tell us what you think.
Contempt for customers
We'll start with an area that many reading this can relate to. Commercial media's contempt for its audience. These are some examples which touched me and they may ring bells for you.
Gladiator, Channel 10. We're back from a commercial break. Rusty arrives back at home in Spain to find his wife and son raped and crucified. It's arguably the most touching scene of the whole movie. What better time for a giant cartoon helicopter to fly around the screen announcing, "Don't forget, Merrick and Rosso! The B-Team! Every Wednesday night at 7.30!"
I remember every syllable of that ad. Positioning ads like this is, Gruen has told us, is most effective as we're at our most vulnerable. But at the same time this was like the network raising its middle finger at the us and yelling, "Lap it up, suckers!" But is there a way to treat your audience with any more contempt?
I think Channel 7 managed it. Remember the TV show Lost? The first series had a huge buzz about it - largely from making huge waves in America, weeks beforehand. Many people downloaded the series from the US as it aired. I stuck with Channel 7 for some kind of local solidarity reasons. The anticipation coming up to the final, 24th episode revolved around the big reveal, "What's under the hatch?" Then, after watching religiously week after week, there was an unexplained six week hiatus. Six weeks! Again, I restrained myself from downloading the final episodes and stuck with 7. Finally, the show reappeared. Then, in the very first ad slot of the very first ad break there came the trailer, "Don't forget to keep watching the final episodes of Lost [as if!] when we show you what's under the hatch!"
Then they showed us what was under the hatch. Right there and then.
I won't tell you what I shrieked at the TV. But perhaps you can imagine. As spoilers go, that was huge. That was the last episode of Lost I watched... On Channel 7.
This happens all the time. Channel 11, the other day, came back from a five minute ad break to show the last ten seconds of a Simpsons episode. Ten seconds! But I think the 'abject contempt to its viewers award' must go to Channel 9.
I could regale you for ages with my Channel 9 rage. Yet I keep finding myself watching movies which are butchered by having five-minutes-on-five-minutes-off ads at the end. Using Tivo to buffer programs for an hour before watching - so that I can skip through the ads - is one way round it. Of course, this forces 9 to use in-program display ads to make up the revenue. Somehow I don't care. Because there are two areas where 9's actions are the scheduling equivalent of dropping a turd on my doorstep.
Sporting events
I remember my dad ringing up from the UK and remarking how excellent and exciting the Melbourne Commonwealth Games were. Discussion in the office had confirmed that I wasn't the only person who found 9's delayed and appalling coverage unwatchable. It's been the same for subsequent Commonwealth Games and the Olympics. If you could watch events Live on the internet, wouldn't you? There's no other legal way to watch most of them Live (if at all) in Australia.
Did you want to watch all of the matches in the Rugby World Cup? Must have sucked how 9 bought the rights and then DIDN'T SHOW THE MATCHES LIVE! For those who knew what they were doing, you could watch them free on the internet. What other option did they have?
The English Premiership
Nine's treatment of sport is a local problem. Globally, the big issue is English Soccer. The rights are managed by Sky TV (The UK's equivalent to Foxtel). To be fair, the money Sky pumped into the sport, plus the huge improvements in coverage, is one of the reasons this is the most popular league in world sport. But for those of us who had little money, we'd rather be in a position to actually watch a game on TV than know that only the moneyed people had access to the improved coverage. There was the option of traipsing down the pub, but that meant coming home most-likely drunk, reeking of cigarette smoke (before the ban) and still having spent money. But the real problem was this...
You'd ring up Sky. "Hi, I'd like to subscribe to Sky to watch the football please?"
"Certainly, which football do you want?"
"The Premiership football."
Certainly, it's available on this package, that package and the other package."
"No I just want the football. I don't want the US Soap channel, the African Animal Channel, the Infomercial Channel... etc etc etc."
Indeed, Sky spreads its Premiership games across several channels in several packages so you have to subscribe to all their other crap in order to get the few football matches that you want to pay for. The resulting monthly fee is well over a hundred dollars. Even to watch the odd pay per view game you have to pay for Sky and then pay for a package in order to pay for the pay per view.
Or... you can just watch it live on the internet. For free.
In Australia, it's a similar problem. But I'm not subscribing to Foxtel just to watch my team play the occasional game in the middle of the night. I'd gladly pay to watch the matches I want to see. But I can't. As a result, I hardly watch any matches anymore. But if there's a big one, then my one and only option is to watch it live on the internet. What else can I do?
The problem is such that there are large international communities all over the world, telling people where to watch games live on the web. Some websites even charge a fee to provide a high-quality online stream. The charges cover hosting costs and, once there are enough people connected, they accept no more customers for fear of dropping quality. So people are actually PAYING to watch these matches illegally when they could watch them illegally for free!
Overseas Content
This problem reappears in many other areas too. A major one concerns Japanese anime (cartoons). Ars Technica did an excellent investigation into this matter. It found that there were huge online communities sharing copyrighted content, but that money was not a reason for doing so. Typically, when a cartoon appeared in Japan, it would take a year for it to appear overseas. When it did appear it would be dubbed with dumb-ass American dialogue which obliterated many of the cultural references which made the cartoons popular in the first place.
One of the 'infringing' community websites then did what one would hope the rest of the media industry would do - it realised that there was an enormous demand for overseas content to be aired online immediately after publication and that people would happily pay for it.
The BBC
Recently, the BBC launched iPlayer in Australia. This gives you access to much of the BBC's vast television archives. To a degree, this has long been desired by overseas residents. But the dominating discussion was all about the BBC's failure to allow payment of an overseas licence fee to let international viewers watch Live BBC content.
I lived in Japan several years ago, and people from all nationalities said at the time they'd love to pay to watch live BBC TV. The demand is enormous but when I recently asked the BBC, they said it wasn't going to happen.
Sure they get huge sums for licensing internally-produced programs and series, but they may get even more by allowing online access to international paying customers. However, even if this did happen, there would be issues with the BBC covering international events. A good example is Formula 1. Many Australian F1 fans baulk at Channel Ten's coverage and are only too glad for the switchover to the BBC's outstanding race commentary. Not having to suffer ads or Mark-Webber-obsessed presenters who struggle to contain their disappointment at not talking about V8s or motorbikes is a constant bugbear for many. Those who know about the internet know how to stream the BBC's coverage Live so there are no ads or interruptions. You can't pay for that though. Many would if they could.
But if there's one prime example of the problems of surrounding the BBC, copyright infringement and international viewers it's a certain program with 350 million viewers worldwide...
Top Gear
I used to watch Top Gear. I can't now. There are several hundred thousand Australians who are in the same boat. SBS picked it up long ago and built a regular audience of over a million people. Then 9 bought the rights and quickly decimated the audience. It's around 400,000 now. How on earth did it manage to do that in car-crazy Australia?
First you need to know that the BBC sends out an international version of Top Gear to overseas licensees which has 15 minutes cut from each show - to allow for ads. Consequently, if you want to watch a full episode of Top Gear your only option is to download it illegally from the web, or wait ages for the DVDs to appear. Then there was the fact that SBS had a two-YEAR delay in showing episodes. Nonetheless, a million loyal fans watched it and I was one of them.
After switching network, Channel 9 bragged about fast-tracking UK episodes. All sounded good. But then it took the already-short international version and butchered it by cutting more content out to add even more ads. The following week, despite promises of a new episode it showed an ancient, years-old episode. Apparently, it was OK to say this was a new episode because it was new to Channel 9. Cue ten years' worth of old episodes appearing randomly interspersed with more--recent episodes and Blam! the audience walked. I'd happily pay to watch Top Gear. Channel 9 makes it unwatchable. My only option is to download it. I haven't watched it in years.
The Music Industry
Piracy has affected few industries more than music. Back in the early days of the internet, services like Napster, Kazaa and Audio Galaxy appeared which let you swap songs with other people online. At the time, there was no talk of copyright infringement, it was just something that geeky internet users did and it felt like a more-efficient way of swapping cassettes and CDs in the playground. Unfortunately, it was so efficient that the global and industrialised scale destroyed the traditional way in which music was produced and marketed. Quite rightly, the services were shut down. But the story doesn't end there.
The age of compressed music formats and MP3 music players had begun. Once the third-generation iPod hit the market, along with iTunes, compressed digital music became mainstream. What a great opportunity for the music industry: the customers wanted compressed music delivered online and it was cheap to do. But could the industry have screwed things up any more?
Rather than give customers what they wanted publishers threw every toy they had out of the pram and hit the litigation button. One example saw the recording industry sue a 12-year old girl and won $2000. From her point of view she was simply using a free service on the internet that all her friends were using and discussing. One wonders how happy the recording industry was with its $2000 payout. Over the years industry bodies have spent far more money suing people than they recouped through the courts.
One of the main reasons we all have anti-piracy slogans embedded in our brains is because the music industry chose to try and protect its existing market and revenue streams at all costs and marginalise and vilify those who didn't want to conform to the harsh new rules being set.
The Napster brand went legit, iTunes rose and Sony started offering its vast music catalogues online. But instead of selling the compressed music that the public wanted, the industry "sold" music riddled with Digital Rights Management (DRM) 'copyright protection' meaning that the music would only play back on certain devices under certain conditions. Music was also being sold using formats which wouldn't work on all music players and compressed to degrees that resulted in a loss of quality which turned-off enthusiasts. In short, despite selling the music, you didn't own what you'd bought. You were essentially "renting" the rights to the music. Shouldn't there have been intervention from the government?
After a while Sony got bored of the lack of traction which its appalling model had generated and turned off its entire system. This meant that everyone who had "bought" music from Sony couldn't play it on anything other than the old devices launched to go with it. People who had invested heavily in Sony's music were ignored.
Around this time Sony also came up with other ways to stop people listening to the music they had bought. A system appeared which inserted noise and interference when people tried to compress music from CDs. Consequently, if you only listened to MP3 music, you couldn't actually legally get an MP3 version of a song. Even if you had paid money for a CD. Sony even topped this by secretly putting computer software on its audio CDs which secretly installed licensing software on your computer if you tried to compress the music on it. Not only was this a gross breach of privacy, but the 'rootkit' that was installed was a major security threat. This was one occasion where Sony got hammered for its actions. Ultimately, though the publishers were treating their paying customers as potential criminals and the widespread resentment was palpable.
As time wore on, it became clear that the DRM on music was linked to the original hardware you had when it was bought. For many people, if you bought legitimate compressed music online, like I did, when you go to play it you get the following message...
*www.abc.net.au/technology/images/general/general/musicfail.png
I paid good money for those songs. Am I supposed to buy them again? Or can I download them illegally from the internet in clear conscience?
Things seem to be slowly changing with Apple offering DRM-free, higher-quality songs on iTunes now and with the industry recognising the importance of the online music store. Nonetheless, you're still forced to buy from one seller, using one format and at a quality which, these days, could be higher. The best sales model surely came from legally-spurious site, AllOfMp3.
This Russian based site allowed you to purchase almost any song in any format using any level of compression that you wanted and charged a low price for it. In other words, it recognised public demand and gave people exactly what they wanted.
But its licensing model was dodgy at best. It did pay royalties but at tiny Russian radio-play levels. Many of the songs were sold without permission from the copyright holders. It got sued by everyone for a staggering $1.65 trillion, but was eventually acquitted.
Outlandish lawsuits like this have become the norm for media publishers and their industry organisations. At no point did they realise that this was the most obvious business model to use - to give people what they want at a fair price.
Nowadays, the publishers seem to have moved on. They're still suing downloaders and crippling innovative internet-radio business models like Pandora, but the new popular model seems to be charging a subscription fee for on-demand access to entire music catalogues - iTunes' iCloud music service, Last.fm and Sony's Anubis are good examples. I've used the latter for months and it's excellent.
Movies
Heaps of movies are illegally downloaded these days, but unlike the music industry, the film industry is thriving. Theories abound as to the impact of downloading movies over the internet: there is evidence which suggests that those who download movies tend to be enthusiasts who spend more on movies in the first place (as is the case with music downloaders). Certainly the cinema trade is booming. My pet theory is that many downloaders download movies they aren't particularly fussed about seeing (not enough to pay for them anyway) or which are unavailable where they live. But the constant engagement with movies keeps them in the "film enthusiast" bracket and that makes them go to the cinema when something that they're particularly keen on appears.
Hysterical lawyers say otherwise. More on that below. Either way, movie downloading is a contentious business as are its consequences.
There is obviously a huge public craving for movies and video on demand but the only place that you can get many movies is illegally online. Legal services in Australia tend to, well, suck. Tivo has boasted for years about the thousands of movies you can pay for on demand. Most of them seem to have Marilyn Monroe or John Wayne in them. Selections aren't much better elsewhere. If you want to pay for good video on demand services the best you can do is pay for quasi-legal access to American sites like Netflix. Or download illegally. Either way, you're probably a criminal.
*www.abc.net.au/technology/images/general/general/billboardcopyright.jpg
Pirates are terrorists. Tell the police. This image is used under fair dealing.
Pirates are terrorists. Tell the police. This image is used under fair dealing.
Fair Dealing
In Australia, America and other countries, there are laws which protect people from innocently using "copyrighted" media in non-commercial and various reasonable ways. But don't expect to find authorities standing up for your rights.
Youtube is a prime example. If you make a video of something, but in the background there's a song playing - from a nearby radio or whatever - it gets banned. Want to share your child's birthday party with friends and family? You'd better not play any recorded Happy Birthday song in the background - you'll get your account suspended.
Other bans stem from people making their own movie mashups or discussing clips from mainstream media.
It's difficult to imagine what lawyers and publishers have to gain by banning people from doing this and vilifying them for doing so. Youtube got sick of dealing with individual take-down requests and waved the white flag long ago. It just bans things automatically now.
Almost all of these infringements are actually allowable under Fair Use (America) and Fair Dealing (Australia) legislation. But to the publishers and establishment, it too-often seems, you're just a criminal.
Harsh Litigation
Most troubling of all is that prosecuting people for suspected copyright infringement has become an industry in its own right. Legal firms are buying the rights from publishers to sue people on their behalf. It's an evolution of the ambulance chasing lawyer. There's a straight-forward business model for it.
You send out letters to potential copyright infringers telling them that they have downloaded something illegally and will be sued for anything up to $150,000. They have the option to settle beforehand - typically for a few thousand dollars - just enough to save on hiring a lawyer to defend the case.
The threat is based on the fact that if you have downloaded a movie then you will also have uploaded it and distributed it to thousands of people. In reality, however, if you download something using bittorrent only a small fraction gets uploaded. It would take balls of steel and deep pockets to explain that one in court though.
In America the music and movie lobbies have pushed through a non-government accord which allows corporations to punish suspected copyright infringers without any trial or due legal process. The US government, it transpires, has few issues with this. It's not yet clear whether Australia will follow along similar lines.
Industry bodies are certainly wanting to enforce their will on Australian legislation, though, as the battle between iiNet and AFACT illustrates. AFACT has been defeated several times but hasn't given up. More recently, The Age uncovered a new Gold Coast operation which is planning to demand money from downloaders of porn. There are fears that this could be the thin end of the wedge for Australia.
However, the article also points out that similar UK operations were eventually denounced in the House of Lords as "straightforward legal blackmail." So not all governments are as compliant as the copyright industry might like.
Conclusion
Nowadays, copyright barely resembles what it was originally designed for i.e. to protect both parties: inventors and content creators on the one side and the public on the other. Corporate America and government compliance have written out public interests in many instances. The case of Mickey Mouse is illustrative.
Nonetheless, there's an air of inevitability about it all. Historically, how often have incumbent, monopolistic industries shrugged their shoulders and written off their entire business model to embark on a journey along a crowded new highway, with rules set by customers, that leads who-knows-where?
On a personal note, I suspect that once the world's internet infrastructure comes up to speed, we'll all be using on-demand subscription models and the notion of buying content to keep will feel archaic. Even so, more needs to be done to protect the public from ham-fisted copyright industries demanding payment for everything.
A great deal of copyright infringement does not stem from criminal behaviour. Much of it occurs simply because there is all-too-often no other way to legally access the content you want - even if you do want to pay for it.
It's worth remembering that there are many big losers because of piracy, but these have been well covered elsewhere. The video games industry, for example, is a major loser, but we'll deal with that another time. This article is one of few that deals with the flipside of the argument and so please remember that it is intends to describe and inform - not endorse any infringement. Has it changed your opinion on the matter or confirmed it? Let us know below.
OP
sygeek
Technomancer
How to hire an idiot
Wow, I remember how idealistic I was when I was about to bring on my first employee! After dealing with bad bosses over my career, after doing a whole lot of thinking about how I was going to be a great boss, and after doing a whole lot of reading about how to hire effective people, I was really looking forward to it. I was going to:
-- Hire people smarter than myself, who get things done!
-- Trust them to do their job, let them do their job and give them enough resources to do it!
-- Pay them WELL and offer great benefits! Work at home! Sure, why not?
-- Give people second chances! Don't throw out resumes because of lack of buzzwords! Or disjointed writing! Or lack of education! It's all about Smart People who Get Things Done, not interviews or resumes or formalities! Have an open mind!
Only problem was, I couldn't quite afford an employee yet. By then I had been working a couple years by myself, earning good profits in the $200K range but it was based on just one or two sales a year, and each sale took 6-12 months to finalize. With so few customers I could easily go a year without sales, I feared, so had to set aside my profits to cover that. And if I were going to hire someone, I'd really want six months or a year's payroll set aside for them as well. I just couldn't afford that yet.
But of course without more employees, I couldn't make more money to pay for them. I was really at full "capacity," spending around six full-time months to land a sale, then the next six months servicing that sale before starting over again. So I knew my first employee would have to be a salesperson, but I just couldn't afford it and couldn't see how I'd be able to at my current rate...
Bullshit
Serendipitously, I was approached a little while later by a former VP of my big competitor, at my industry's main exhibition where I had a small booth. He was a friggin' VP of a $100 million a year company! Well, their former VP, he said. Wow though, I was flattered. I demoed my product to him, explained my company, and his mouth dropped open. He started gushing about how incredible my product was (well, it was, I guess) and asked why the "****" wasn't I selling $100 million a year?! I said well, I'm sort of at capacity... and... errrr... I'm more of an engineer, and, uh.... I don't know why. I didn't want to tell him what I feared, that it was just this thing I made on my own, some of the code was crap, and things like that just don't sell for millions.
Then he told me "**** man, I could sell $10 million of this a year!" His informality in that professional business setting I thought was a little strange, but...
Was it possible somebody could really do that? I sure couldn't, but maybe I was on the wrong path? I wasn't a business expert, so what did I know? He was a friggin VP of business development for a $100 million company! He must know what he's saying, right?
We talked a little more and I couldn't believe when he asked to work for me for free! Well, on commission. But hey, that's money I wouldn't have made anyway. If he brings in a million bucks a year in profit, he's worth 10% of that, certainly!
We settled on 10% and I'd pay for travel and some other expenses. No problem... if he could make the sales he insisted he could, he'd be well worth it. And with very little risk to me for all the work he'd be doing!
Just to make sure I wasn't being bullshitted, I called the competitor to ask about him. They verified that yes, he was a former VP there. Awesome, this guy was for real. And if he was good enough for them, he's good enough for me! We soon signed a deal.
Alarm Bells
I mentioned to my uncle (an experienced big-ticket salesperson) about this new guy I was bringing on board, and he told me to be careful because "guys like that will do anything to make the sale and don't care if they leave you high and dry. I've seen it LOTS."
Whatever, old man! Because I had the deal structured to account for that: he didn't earn commission until payment was received! It simply wasn't in his interest to do just "anything" to make the sale, because if the product wasn't as promised, the customer wouldn't pay and he wouldn't get his commission! Beautiful scheme. And I'd have all pricing authority so he couldn't sell it at a loss, either! Ha ha, nothing could possibly go wrong with this.
So I set off on my promises of being a great boss. I let my new sales guy do his thing, trusted his judgement, didn't ask to be CC'd on things, gave him the resources he needed, just set him loose. $25K in stuff he said we absolutely needed -- slick brochures, sponsor some conference, ads in the trade journal, coffee mugs, pens with our logo -- I readily paid for. I wanted him and us of course to succeed.
And really I was pretty damned honored having someone with his experience -- a friggin VP of a $100 million company -- working for me, and for free! Wow!
Okay though, this one thing didn't make sense: I had told him our cost for a particular solution we were giving an estimate for. I did that so he could figure out his commission, which was based on gross profit. I then got cc'd on a mail where he turned around and told the customer our exact cost, and "that means there's a lot of wiggle room on the price. I'm sure Bill will come down on it."
WTF? Why would a salesperson tell the customer our cost?! I mean, isn't that just common sense? I asked him and he said something about "don't worry, they know we make a profit."
Well that didn't make sense -- that seemed pretty stupid actually -- but this guy was a friggin VP of a $100 million company! I was honored to learn business from him!
Then there was this other strange thing: a customer asked if they could see a demo, so he asked me to approve the travel cost. Just knowing how the sales process works, I told him I didn't like spending money on demos until we were sure they had money and were really ready to buy. So he emailed them (cc'ing me) "Do you have money? Are you ready to buy? We don't give demos unless you are."
Why in the world would you say that to a customer?! But... he was the VP of a $100 million company, after all! He must know these customers extremely well, and maybe it's... maybe some kind of inside joke thing? Just how executives talk to each other?! Wow, I had so much to learn!
Hmmmm, then there was this other thing that didn't make sense either: he sent me a sales forecast, and in the "absolutely certain" column he had $5 million in sales over the next three months alone (!) I mean, holy ****! But wait: that one company on there -- I could have sworn they told me just six months ago they didn't have anything in the budget, but maybe in a couple years? And suddenly now they're ready to buy? Just like that? I asked him and he assured me that yes, they have money now and are definitely buying from us. Definitely! 100% certain.
Awesome! I mean this guy was a friggin VP of a $100 million company!!! There was so much I would get to learn from him!!!! $5 million in sales in three months!!!!11!!!
Hold on. Then he sent me a proposal he had been working on over the past month, for final review and "second set of eyes." I had previously sent him all my boilerplate proposal and price quote templates to show him what's worked for me in the past. I figured he could just fill out with the customer's particulars like I had done over the past couple years, and save a lot of time. But no, he said he was going to write a totally new awesome proposal package guaranteed to win. That's what he used to do as VP of the friggin $100 million company, after all! I told him great, I can't wait to see!
I started reading this thing and my face dropped in horror. It was the writing of a grade schooler. I'm no professional writer either, but... it was absolutely awful. Simplistic writing, full of cliches, full of grammatical errors, and absolutely lacking in any structure. It was just random thoughts strung together, topics bouncing around from idea to idea from one sentence to the next. There was no exposition of the customer's problem and how we were going to solve it, it was just him gushing about how "great" our product is and how "lots" of people like it. It was dizzying to read because there was no logic behind it -- it was along the lines of "This product is great. You will like this product, guaranteed. It has feature A. Feature C is great because it's so easy to use! It has feature B. The other great thing about feature C is tons of people told us they love it. Tons. It has feature D." (New paragraph)... on and on for 15 pages.
Okay, how could a friggin VP of a $100 million company read something like that and think "That's it! Yea!"?
I didn't care whether he was an experienced VP or not, I had to ask him WTF he was thinking, hopefully without offending him (too much). "Ummm, it was... interesting," I carefully offered, "but I'm just curious: did you proofread this at all?"
"Oh sure, I ran it through spell check and had my wife check it out too" he proudly replied.
"Ok, well... uhhhh... hey, didn't you also used to write proposals at [former company]?"
"Yep! Well, not exactly... other people wrote them I guess, but I oversaw it."
"Okay..."
"So -- what do you think? Kick ass, huh? I think this is a shoe-in for us! I really do, I can feel it."
Here unfortunately I sort of lost it. $100 million VP or not, that document was ****. No, I'm not a writer either, and no, our customers aren't English teachers, but what the ****? I can't put my company name behind that! It was ****. I told him that. I asked him what the **** he was thinking, why would he even set out to write a proposal if he knew he couldn't write -- I mean, why bother? A whole month to do that?!
He apologized. He said he was trying to do well, and he really thought he could write well, but "apparently I can't, and I accept that."
------------
I left to cool down and think about it more. Okay, no problem. So what, the guy can't write. We can use my previous templates and I'd just modify them for each new customer. We're talking about $5 million coming down the pike, after all! I'd write them myself all day for that kind of money! Woo-hoo!!
I stayed up all night rewriting the proposal, and we moved forward.
Well, the three months came and went. No sales. The proposal I wrote? Turns out they had never asked for it and didn't have money but thanked us for sending it. Uhhhh...
And the other $4.5 million in sales we were getting that month? A couple others "suddenly lost their funding." Another "got delayed by other problems but they're buying next month." Another was "I don't know what happened... I'm trying to find out."
But any week now! Any week was going to be the first big order! Just have patience! I mean, this guy was the VP of a $100 million company, after all! Who was I to question him? I was just some programmer who found myself in sales only because I had to.
Six more months went by. Not a single sale. Okay, well, it's a long sales cycle. I always figured I might go a year without a sale, so give the guy a chance. VP of a friggin $100 million company working for me for free! Woo-hoo!!
Still, I got more and more concerned. Something wasn't right. I suggested we start working "together" on sales since we both wanted them, after all, so could he start cc'ing me and we'd brainstorm ideas with each of these prospects? He thought that was a great idea.
So he started cc'ing me. And Oh My God. This guy was awful! Holy ****. His "sales technique" for the first new prospect I sent him consisted of literally begging the customer to buy "because our company is about to go in the shitter." Huh?! WHY WOULD YOU TELL OUR CUSTOMERS THAT?! And use obscenities in that kind of correspondence?! To a CUSTOMER?! I demanded an answer.
"Well, it's true, isn't it? Believe me, I've been in this industry for 30 years and they can handle it. That's just how these people are," he explained.
Okay, friggin $100 million VP or not, I was calling bullshit. My company does not correspond to people like that, that's not how you sell this product to these customers, that's not how people respond positively, that's not how to build a business! Bullshit.
And everything over the past almost 12 months, all the other bullshit started to come together. Really I felt awful, awful at being conned somehow, awful at myself for not checking up on him, for not even interviewing him, for not watching him, for just setting him loose and trusting him without "verifying." Everything he had told me was bullshit, all his forecasts, everything looking back at our correspondence about who had money, who was buying, everything he promised. All bullshit.
I took him out to dinner and we had a heartfelt Scooby-Doo reveal moment (you know, at the end of the show when all the masks would come off and the mystery would be explained):
Aha. Turns out the guy was a High School dropout. Got into drugs, booze, crime, turned his life around and got his GED. Went to work at a utility as a lineman and worked his way up. He had great people skills, remembered everyone's names, and that's really how he made the connections to keep getting promoted. Delegated everything to subordinates. Retired from that near the top and worked as an industry consultant because he knew everyone in the business. Did some work for the competitor. They liked how he knew all the top people at all the top customers, and offered him generous employment. He really wanted to be a VP so they said sure, how about assistant VP of business development. ("Whatever, just set appointments for us," was actually probably more like it).
He got fired within the year, he admitted. He said it was a "personal disagreement" but I wouldn't doubt it was utter incompetence.
And nope, he'd never done sales in his life. His job used to be setting appointments, mingling with customers at conferences, and getting their sales team in the door to make the sale. But he himself didn't do sales. Had no clue what was involved, had no clue what process customers go through to make a purchase, had no clue about techniques like "consultative selling" or who you have to convince in a business or institution to close a sale. No clue. But gosh, he was eager and willing to learn and felt great about this opportunity I was giving him!
Well, at least he was honest. He wasn't trying to deceive me, and he really thought he could do it, he explained. No hard feelings. But I didn't need an entry level salesperson, I needed an experienced salesperson right now. I told him he had to go, and he understood.
Sadly, I could have found out all of that by simply asking him before offering him the deal. I just never did. I mean, he was a friggin' VP of a $100 million company, after all!
----------------
Every time I relate this experience, I get a lot of head nods. I guess it's pretty common among business owners and anybody involved in HR, to get employees who just don't turn out as promised. But damn, I didn't think it would happen to me. I mean, I was prepared! I read a lot of books! I knew all about bad employees and how to avoid them! I was smart, dammit!
Well, my company survived. I went back to basics with my old way of selling and soon landed another nice sale. Then my next hire was a salesperson again, but thankfully this time I knew to check up on him before the hire, and knew to have him explain his strategies and techniques in the interview to make sure he knew his stuff. Thankfully, he's turned out to be a really good guy and so far has been doing really well.
And unfortunately what I really learned from this is something I actually already knew from my first year of employment right out of college: business executives are sometimes just full of ****!
-- Hire people smarter than myself, who get things done!
-- Trust them to do their job, let them do their job and give them enough resources to do it!
-- Pay them WELL and offer great benefits! Work at home! Sure, why not?
-- Give people second chances! Don't throw out resumes because of lack of buzzwords! Or disjointed writing! Or lack of education! It's all about Smart People who Get Things Done, not interviews or resumes or formalities! Have an open mind!
Only problem was, I couldn't quite afford an employee yet. By then I had been working a couple years by myself, earning good profits in the $200K range but it was based on just one or two sales a year, and each sale took 6-12 months to finalize. With so few customers I could easily go a year without sales, I feared, so had to set aside my profits to cover that. And if I were going to hire someone, I'd really want six months or a year's payroll set aside for them as well. I just couldn't afford that yet.
But of course without more employees, I couldn't make more money to pay for them. I was really at full "capacity," spending around six full-time months to land a sale, then the next six months servicing that sale before starting over again. So I knew my first employee would have to be a salesperson, but I just couldn't afford it and couldn't see how I'd be able to at my current rate...
Bullshit
Serendipitously, I was approached a little while later by a former VP of my big competitor, at my industry's main exhibition where I had a small booth. He was a friggin' VP of a $100 million a year company! Well, their former VP, he said. Wow though, I was flattered. I demoed my product to him, explained my company, and his mouth dropped open. He started gushing about how incredible my product was (well, it was, I guess) and asked why the "****" wasn't I selling $100 million a year?! I said well, I'm sort of at capacity... and... errrr... I'm more of an engineer, and, uh.... I don't know why. I didn't want to tell him what I feared, that it was just this thing I made on my own, some of the code was crap, and things like that just don't sell for millions.
Then he told me "**** man, I could sell $10 million of this a year!" His informality in that professional business setting I thought was a little strange, but...
Was it possible somebody could really do that? I sure couldn't, but maybe I was on the wrong path? I wasn't a business expert, so what did I know? He was a friggin VP of business development for a $100 million company! He must know what he's saying, right?
We talked a little more and I couldn't believe when he asked to work for me for free! Well, on commission. But hey, that's money I wouldn't have made anyway. If he brings in a million bucks a year in profit, he's worth 10% of that, certainly!
We settled on 10% and I'd pay for travel and some other expenses. No problem... if he could make the sales he insisted he could, he'd be well worth it. And with very little risk to me for all the work he'd be doing!
Just to make sure I wasn't being bullshitted, I called the competitor to ask about him. They verified that yes, he was a former VP there. Awesome, this guy was for real. And if he was good enough for them, he's good enough for me! We soon signed a deal.
Alarm Bells
I mentioned to my uncle (an experienced big-ticket salesperson) about this new guy I was bringing on board, and he told me to be careful because "guys like that will do anything to make the sale and don't care if they leave you high and dry. I've seen it LOTS."
Whatever, old man! Because I had the deal structured to account for that: he didn't earn commission until payment was received! It simply wasn't in his interest to do just "anything" to make the sale, because if the product wasn't as promised, the customer wouldn't pay and he wouldn't get his commission! Beautiful scheme. And I'd have all pricing authority so he couldn't sell it at a loss, either! Ha ha, nothing could possibly go wrong with this.
So I set off on my promises of being a great boss. I let my new sales guy do his thing, trusted his judgement, didn't ask to be CC'd on things, gave him the resources he needed, just set him loose. $25K in stuff he said we absolutely needed -- slick brochures, sponsor some conference, ads in the trade journal, coffee mugs, pens with our logo -- I readily paid for. I wanted him and us of course to succeed.
And really I was pretty damned honored having someone with his experience -- a friggin VP of a $100 million company -- working for me, and for free! Wow!
Okay though, this one thing didn't make sense: I had told him our cost for a particular solution we were giving an estimate for. I did that so he could figure out his commission, which was based on gross profit. I then got cc'd on a mail where he turned around and told the customer our exact cost, and "that means there's a lot of wiggle room on the price. I'm sure Bill will come down on it."
WTF? Why would a salesperson tell the customer our cost?! I mean, isn't that just common sense? I asked him and he said something about "don't worry, they know we make a profit."
Well that didn't make sense -- that seemed pretty stupid actually -- but this guy was a friggin VP of a $100 million company! I was honored to learn business from him!
Then there was this other strange thing: a customer asked if they could see a demo, so he asked me to approve the travel cost. Just knowing how the sales process works, I told him I didn't like spending money on demos until we were sure they had money and were really ready to buy. So he emailed them (cc'ing me) "Do you have money? Are you ready to buy? We don't give demos unless you are."
Why in the world would you say that to a customer?! But... he was the VP of a $100 million company, after all! He must know these customers extremely well, and maybe it's... maybe some kind of inside joke thing? Just how executives talk to each other?! Wow, I had so much to learn!
Hmmmm, then there was this other thing that didn't make sense either: he sent me a sales forecast, and in the "absolutely certain" column he had $5 million in sales over the next three months alone (!) I mean, holy ****! But wait: that one company on there -- I could have sworn they told me just six months ago they didn't have anything in the budget, but maybe in a couple years? And suddenly now they're ready to buy? Just like that? I asked him and he assured me that yes, they have money now and are definitely buying from us. Definitely! 100% certain.
Awesome! I mean this guy was a friggin VP of a $100 million company!!! There was so much I would get to learn from him!!!! $5 million in sales in three months!!!!11!!!
Hold on. Then he sent me a proposal he had been working on over the past month, for final review and "second set of eyes." I had previously sent him all my boilerplate proposal and price quote templates to show him what's worked for me in the past. I figured he could just fill out with the customer's particulars like I had done over the past couple years, and save a lot of time. But no, he said he was going to write a totally new awesome proposal package guaranteed to win. That's what he used to do as VP of the friggin $100 million company, after all! I told him great, I can't wait to see!
I started reading this thing and my face dropped in horror. It was the writing of a grade schooler. I'm no professional writer either, but... it was absolutely awful. Simplistic writing, full of cliches, full of grammatical errors, and absolutely lacking in any structure. It was just random thoughts strung together, topics bouncing around from idea to idea from one sentence to the next. There was no exposition of the customer's problem and how we were going to solve it, it was just him gushing about how "great" our product is and how "lots" of people like it. It was dizzying to read because there was no logic behind it -- it was along the lines of "This product is great. You will like this product, guaranteed. It has feature A. Feature C is great because it's so easy to use! It has feature B. The other great thing about feature C is tons of people told us they love it. Tons. It has feature D." (New paragraph)... on and on for 15 pages.
Okay, how could a friggin VP of a $100 million company read something like that and think "That's it! Yea!"?
I didn't care whether he was an experienced VP or not, I had to ask him WTF he was thinking, hopefully without offending him (too much). "Ummm, it was... interesting," I carefully offered, "but I'm just curious: did you proofread this at all?"
"Oh sure, I ran it through spell check and had my wife check it out too" he proudly replied.
"Ok, well... uhhhh... hey, didn't you also used to write proposals at [former company]?"
"Yep! Well, not exactly... other people wrote them I guess, but I oversaw it."
"Okay..."
"So -- what do you think? Kick ass, huh? I think this is a shoe-in for us! I really do, I can feel it."
Here unfortunately I sort of lost it. $100 million VP or not, that document was ****. No, I'm not a writer either, and no, our customers aren't English teachers, but what the ****? I can't put my company name behind that! It was ****. I told him that. I asked him what the **** he was thinking, why would he even set out to write a proposal if he knew he couldn't write -- I mean, why bother? A whole month to do that?!
He apologized. He said he was trying to do well, and he really thought he could write well, but "apparently I can't, and I accept that."
------------
I left to cool down and think about it more. Okay, no problem. So what, the guy can't write. We can use my previous templates and I'd just modify them for each new customer. We're talking about $5 million coming down the pike, after all! I'd write them myself all day for that kind of money! Woo-hoo!!
I stayed up all night rewriting the proposal, and we moved forward.
Well, the three months came and went. No sales. The proposal I wrote? Turns out they had never asked for it and didn't have money but thanked us for sending it. Uhhhh...
And the other $4.5 million in sales we were getting that month? A couple others "suddenly lost their funding." Another "got delayed by other problems but they're buying next month." Another was "I don't know what happened... I'm trying to find out."
But any week now! Any week was going to be the first big order! Just have patience! I mean, this guy was the VP of a $100 million company, after all! Who was I to question him? I was just some programmer who found myself in sales only because I had to.
Six more months went by. Not a single sale. Okay, well, it's a long sales cycle. I always figured I might go a year without a sale, so give the guy a chance. VP of a friggin $100 million company working for me for free! Woo-hoo!!
Still, I got more and more concerned. Something wasn't right. I suggested we start working "together" on sales since we both wanted them, after all, so could he start cc'ing me and we'd brainstorm ideas with each of these prospects? He thought that was a great idea.
So he started cc'ing me. And Oh My God. This guy was awful! Holy ****. His "sales technique" for the first new prospect I sent him consisted of literally begging the customer to buy "because our company is about to go in the shitter." Huh?! WHY WOULD YOU TELL OUR CUSTOMERS THAT?! And use obscenities in that kind of correspondence?! To a CUSTOMER?! I demanded an answer.
"Well, it's true, isn't it? Believe me, I've been in this industry for 30 years and they can handle it. That's just how these people are," he explained.
Okay, friggin $100 million VP or not, I was calling bullshit. My company does not correspond to people like that, that's not how you sell this product to these customers, that's not how people respond positively, that's not how to build a business! Bullshit.
And everything over the past almost 12 months, all the other bullshit started to come together. Really I felt awful, awful at being conned somehow, awful at myself for not checking up on him, for not even interviewing him, for not watching him, for just setting him loose and trusting him without "verifying." Everything he had told me was bullshit, all his forecasts, everything looking back at our correspondence about who had money, who was buying, everything he promised. All bullshit.
I took him out to dinner and we had a heartfelt Scooby-Doo reveal moment (you know, at the end of the show when all the masks would come off and the mystery would be explained):
Aha. Turns out the guy was a High School dropout. Got into drugs, booze, crime, turned his life around and got his GED. Went to work at a utility as a lineman and worked his way up. He had great people skills, remembered everyone's names, and that's really how he made the connections to keep getting promoted. Delegated everything to subordinates. Retired from that near the top and worked as an industry consultant because he knew everyone in the business. Did some work for the competitor. They liked how he knew all the top people at all the top customers, and offered him generous employment. He really wanted to be a VP so they said sure, how about assistant VP of business development. ("Whatever, just set appointments for us," was actually probably more like it).
He got fired within the year, he admitted. He said it was a "personal disagreement" but I wouldn't doubt it was utter incompetence.
And nope, he'd never done sales in his life. His job used to be setting appointments, mingling with customers at conferences, and getting their sales team in the door to make the sale. But he himself didn't do sales. Had no clue what was involved, had no clue what process customers go through to make a purchase, had no clue about techniques like "consultative selling" or who you have to convince in a business or institution to close a sale. No clue. But gosh, he was eager and willing to learn and felt great about this opportunity I was giving him!
Well, at least he was honest. He wasn't trying to deceive me, and he really thought he could do it, he explained. No hard feelings. But I didn't need an entry level salesperson, I needed an experienced salesperson right now. I told him he had to go, and he understood.
Sadly, I could have found out all of that by simply asking him before offering him the deal. I just never did. I mean, he was a friggin' VP of a $100 million company, after all!
----------------
Every time I relate this experience, I get a lot of head nods. I guess it's pretty common among business owners and anybody involved in HR, to get employees who just don't turn out as promised. But damn, I didn't think it would happen to me. I mean, I was prepared! I read a lot of books! I knew all about bad employees and how to avoid them! I was smart, dammit!
Well, my company survived. I went back to basics with my old way of selling and soon landed another nice sale. Then my next hire was a salesperson again, but thankfully this time I knew to check up on him before the hire, and knew to have him explain his strategies and techniques in the interview to make sure he knew his stuff. Thankfully, he's turned out to be a really good guy and so far has been doing really well.
And unfortunately what I really learned from this is something I actually already knew from my first year of employment right out of college: business executives are sometimes just full of ****!
Nipun
Whompy Whomperson
nisargshah95
Your Ad here
Nice read. Keep your work up and keep more articles coming.
OP
sygeek
Technomancer
All Programming is Web Programming
By Jeff Atwood
Michael Braude decries the popularity of web programming:
I hate to have to be the one to break the bad news to Michael, but for an increasingly large percentage of users, the desktop application is already dead. Most desktop applications typical users need have been replaced by web applications for years now. And more are replaced every day, as web browsers evolve to become more robust, more capable, more powerful.
You hope everything doesn't "move to the web"? Wake the hell up! It's already happened!
Any student of computing history will tell you that the dominance of web applications is exactly what the principle of least power predicts:
stupidestsimplest thing that could possibly work. If that scares you -- if that's disturbing to you -- then I humbly submit that you have no business being a programmer.
Should all applications be web applications? Of course not. There will continue to be important exceptions and classes of software that have nothing to do with the web. But these are minority and specialty applications. Important niches, to be sure, but niches nonetheless.
If you want your software to be experienced by as many users as possible, there is absolutely no better route than a web app. The web is the most efficient, most pervasive, most immediate distribution network for software ever created. Any user with an internet connection and a browser, anywhere in the world, is two clicks away from interacting with the software you wrote. The audience and reach of even the crappiest web application is astonishing, and getting larger every day. That's why I coined Atwood's Law.
As a software developer, I am happiest writing software that gets used. What's the point of all this craftsmanship if your software ends up locked away in a binary executable, which has to be purchased and licensed and shipped and downloaded and installed and maintained and upgraded? With all those old, traditional barriers between programmers and users, it's a wonder the software industry managed to exist at all. But in the brave new world of web applications, those limitations fall away. There are no boundaries. Software can be everywhere.
Web programming is far from perfect. It's downright kludgy. It's true that any J. Random Coder can plop out a terrible web application, and 99% of web applications are absolute crap. But this also means the truly brilliant programmers are now getting their code in front of hundreds, thousands, maybe even millions of users that they would have had absolutely no hope of reaching pre-web. There's nothing sadder, for my money, than code that dies unknown and unloved. Recasting software into web applications empowers programmers to get their software in front of someone, somewhere. Even if it sucks.
If the audience and craftsmanship argument isn't enough to convince you, consider the business angle.
Let's put aside, for the moment, the absurd argument that web development is not challenging, and that it attracts sub-par software developers. Even if that was true, it's irrelevant.
I hate to have to be the one to break the bad news to Michael, but for an increasingly large percentage of users, the desktop application is already dead. Most desktop applications typical users need have been replaced by web applications for years now. And more are replaced every day, as web browsers evolve to become more robust, more capable, more powerful.
You hope everything doesn't "move to the web"? Wake the hell up! It's already happened!
Any student of computing history will tell you that the dominance of web applications is exactly what the principle of least power predicts:
The web is the very embodiment of doing the
Should all applications be web applications? Of course not. There will continue to be important exceptions and classes of software that have nothing to do with the web. But these are minority and specialty applications. Important niches, to be sure, but niches nonetheless.
If you want your software to be experienced by as many users as possible, there is absolutely no better route than a web app. The web is the most efficient, most pervasive, most immediate distribution network for software ever created. Any user with an internet connection and a browser, anywhere in the world, is two clicks away from interacting with the software you wrote. The audience and reach of even the crappiest web application is astonishing, and getting larger every day. That's why I coined Atwood's Law.
Writing Photoshop, Word, or Excel in JavaScript makes zero engineering sense, but it's inevitable. It will happen. In fact, it's already happening. Just look around you.
As a software developer, I am happiest writing software that gets used. What's the point of all this craftsmanship if your software ends up locked away in a binary executable, which has to be purchased and licensed and shipped and downloaded and installed and maintained and upgraded? With all those old, traditional barriers between programmers and users, it's a wonder the software industry managed to exist at all. But in the brave new world of web applications, those limitations fall away. There are no boundaries. Software can be everywhere.
Web programming is far from perfect. It's downright kludgy. It's true that any J. Random Coder can plop out a terrible web application, and 99% of web applications are absolute crap. But this also means the truly brilliant programmers are now getting their code in front of hundreds, thousands, maybe even millions of users that they would have had absolutely no hope of reaching pre-web. There's nothing sadder, for my money, than code that dies unknown and unloved. Recasting software into web applications empowers programmers to get their software in front of someone, somewhere. Even if it sucks.
If the audience and craftsmanship argument isn't enough to convince you, consider the business angle.
Pretty soon, all programming will be web programming. If you don't think that's a cause for celebration for the average working programmer, then maybe you should find another profession.
OP
sygeek
Technomancer
Invasion of Privacy.
UPDATE (1/12/2011):
I received an email from Steve regarding this post. He sincerely apologized for his actions and realized now that what he did was wrong and simply asked that I modify the post to protect the identities of his family. I felt that this was a fair request, considering that his family had nothing to do with what Steve did and it doesn’t jeopardize the impact of the article. So, if you’re wondering why you’re seeing all the “[withheld]“‘s, that’s why!
PS – Yes, I realize the names are still shown in the images, but they’re not indexed by Google. I figured I’d point this out before I had 20,000 comments informing me of it.
END OF UPDATE
DISCLAIMER:
This is ABSOLUTELY for informational purposes ONLY. attackvector.org nor I will be held responsible for how you choose to use the information that I post on my blog. This individual, though he is a douche for sending spam, is a real person with a real life. By misusing the information found here, you have the power to potentially destroy someones real life. There’s a fine line between a legal hack and a felony. Information gathering is not illegal so long as it’s obtained through legal means. Using the information, however, is quite another story.
UPDATE: Because of something that one of my readers brought up, I want to clarify. The email that I received was not the run of the mill malware/spambot/whatever style email. The email was coming from his email address, using his business’s name, and advertising his business. I would have never posted this had I had any doubt that this may not have actually been sent, by him, in some fashion.
END OF DISCLAIMER.
I use spammers and pedophiles as test subjects when I’m working on something. This is mostly because it’s unlikely that they would go to the authorities and point the finger at me, knowing that I could easily turn around and say something to the effect of, “Well, yes I did pwn his box.. but you should have seen all the child porn I found on it.” owned x 2.
I happened to receive a piece of spam at the exact moment as I was going to start a post about privacy and anonyminity on the internet. I will consider this to be a sign from God that this dude needed to be set straight. Okay, maybe not. I’m not sure what the bible says about spam.. but if I were God, it would be into the pits of hell for them. So, since I cannot cast people into eternal suffering in a firey pit, I will have to settle for second best. Pwnage!
Whats even better, none of what I’m about to do is illegal. It’s a serious, serious invasion of privacy, and you definitely don’t want it to happen to you, but all of it can be harvested through public record, social networks, forum posts, etc etc etc.
First, lets take a look at the email that I received.
Ok, so, his email address is steve@barteritemsfortrade.com.. he’s sending email through server299.com.. and his real IP address is 67.185.122.64. All we really need is his email address and his IP. Lets see what we can find.
Just put the IP address in the box and hit “search”. Here’s what we find.
So, we’re narrowing it down.. we now know that it’s Spokane, Washington. Now we’re going to take a look at his email address. First, obviously, just google the email address. This will bring up information for virtually anything that the person has ever used their email on. Forums, social networks, etc.
In this case, however, nothing came up on google. We must dig deeper. Enter, whois!
Biz two? Does that mean there is a Biz One and a Biz Three, perhaps? Also, he’s using a PO Box.. blah.
Jackpot! We now have a last name and a phone number. We also have an additional email address/domain.
Hmm.. a real address.. no PO box on this domain. Is that an office? A house? Is it his house? I can assume that ‘snicho’ is short for ‘steve nicholas’, and it’s the administrative contact, which means he owns the domain.. so the address has something to do with him.
Enter.. Google Maps.
*www.attackvector.org/pics/13930.png
(If you’re wonder why it says “140 west 30th” and not “139 west 30th”, it’s because I slid the camera down a bit and Google tried to be helpful by changing the address)
Well, it’s definitely not an office building, so at this point I’m going to assume that it’s his house until I find out differently. We can further verify this by googling his name + city + state.
*www.attackvector.org/pics/nameres.png
That address looks rather familiar… oh yeah, it’s the address that was associated with his domain. We can be virtually certain at this point that that is his real address and house. Lets see who else lives in the house with him – just google the phone number listed.
*www.attackvector.org/pics/phoneres.png
Ok, so, [withheld] has the same last name as Steve, so I think we can safely say that this is his wife.
We’ll come back to her later. Lets see what else we can find about Steve.. I’m really starting to feel like family at this point.
Back when I googled his name + city + state, I noticed that below the address result, there was a LinkedIn page.. lets check that out.
Ok, so there’s all sorts of useful information.. but I found another email address.. steve.nicholas@itex.net Not often do I meet someone with as many email addresses as me.. lol.
So, back up to the top, we google for steve.nicholas@itex.net.
Some interesting stuff, but nothing really useful for my purposes. Lets check out Facebook and see if he’s a social butterfly. I log in and “search for friends” and enter his email address(es). His account is registered with the itex.net email address.
He doesn’t have his Facebook stuff set to private, so he’s kind of letting it all hang out. Thanks, Steve!
*www.attackvector.org/pics/stevefb.png
Yawn. The only thing interesting there, is that we’ve now definitely verified that that address is correct and that his wife’s name is definitely [withheld]. Maybe her page is more interesting.. lets look
Note: Passwords.. by building a profile of someone, you begin to get a feel of who they really are. I’m willing to bet that at least one of Steve’s passwords has something to do with fishing, trout, or cutthroats (type of trout – according to his facebook page).
[withheld]‘s Facebook:
Here’s something to take a mental note of. Women are generally more open about their personal lives and love to share with others. In one paragraph, we learn that she teaches at Salk Middle School, they’ve been married for 27 years, they have 2 daughters, [withheld] and [withheld], [withheld] is married to [withheld (both first & last name)] (note – this probably means that [withheld] is no longer [withheld] Nicholas, she’s probably [withheld (both first & last name)]). [withheld] lives in Vegas.
How ever would we find out more information about [withheld] and [withheld]? Oh yeah, friends lists. If the parents have Facebook, the kids most certainly have Facebook.. and barring any family drama, they’ll all be on each others friends lists. And, of course, I’m right.. found [withheld], [withheld], and [withheld].
Also, going through her wall posts gave up some information. They’re new grandparents.. their grandaughter [withheld] was born on March 15th.. this was [withheld] and [withheld]‘s daughter.
Now, lets see what Intelius says about [withheld] (note – I skipped Steve on Intelius because his entry is all screwed up.)
*www.attackvector.org/pics/intelius.png
Now we have ages, too. It’s interesting that there’s a “Ralph Steve Nicholas” listed, who has the same age as the other two Steve’s listed. Could Steve’s real name be Ralph??
Ok, anyway, lets see what I can find out about their house. Just about every county in the country allows you to view property tax records on the internet. I googled “spokane washington property tax records”. What you’re looking for is like, the assessor’s home page then just punch in the address and you can find a wealth of information.
What this record tells us, is that [withheld] actually owns the home.. Steve isn’t even listed. She’s also the sole person listed paying the property taxes. Interesting.. I wonder why?
Also, further down on the report, there’s two documents. A quit claim deed, and a statutory warranty deed. A warranty deed is issued in some states when a house is sold. It protects the buyer from having third parties come after them for unpaid debts and whatever. So, it appears as though they bought the house in 2001 for $110,000? Seems awfully low.
Now, lets look at the quit claim deed. First thing I notice. R Steve Nicholas is listed as “Husband of Grantee” I think Steve’s real name is Ralph. lol.
This is interesting.. quit claim deeds are used after a divorce to switch the owner of a property from one party to another at the county level. But they’re still married. The other times that I’ve seen quit claim deeds used is when people encounter serious financial trouble and need to file bankruptcy. They file independently and deed the house to their spouse.
Lets find out!
I am not going to tell you what service I use to obtain this information because I don’t want it to get abused and taken away. Also, I don’t think everyone should have access to it. SO.
Ok, so they did a joint bankruptcy in ’91 and it was discharged in ’93. I also have a list of their creditors.. no wonder they filed bankruptcy. Ouch.
One other piece of information that this offers, is previous addresses and the last 4 digits of their social security numbers. Keep in mind, a lot of people use the last 4 digits of their social for pin numbers.. because most pin numbers are limited to 4 digits. Stupid.
UPDATE: I’ve decided to X out the social security numbers because this post is starting to receive a ton of traffic and I’m not sure I want everyone visiting it to have this information. My intention of this article is not to make it easy to steal this guys identity.. it’s to point out a vulnerability. If you really want to find his social security number, lets just say.. it’s available via the internet.
Here’s something to really think about.. I was able to obtain all of the information in this post for 16 cents and by just using an email and IP address from a piece of spam.
Family members, ages, schools, anniversary dates, marriage lengths, hobbies, interests, phone numbers, addresses, property records, property taxes, pictures of their house, pictures of them, pictures of their children and grandchildren, deeds on their house, bankruptcies, employment history, previous addresses, previous creditors, and bits of social security numbers.
I’m pretty sure I’d be able to fake my way through one of those password reset forms.. you know, where you set up a “secret question” asking what your dogs name was, or where you went to school?
Beyond that, I’m fairly confident that at this point, if I were to call his bank and pretend to be him, I could easily pass when they asked me personal questions.
In closing.. you really need to pay close attention to what you’re posting on the internet. If I were a douche, I could ruin this guys life using this information. There are a lot of douches out there that are doing this type of stuff right now. Given an email address, phone number, or whatever, they build profiles on people which can be used to exploit them and steal identities.
The other thing that I’ve actually fallen victim to, is the speed of Google’s spiders and the fact that they index Craigslist. Lets say you run a business.. Catholic Charities R Us and in this post, you include an email address, phone number, something. Lets say you also make a post, days, weeks, whatever, later looking for whores, or something. Both of those posts will come up when Googling for your phone number.
Also, consider what you’re sending in this email. What if this guy had sent me an email trying to extort me, threaten me, whatever? I could turn this over to the authorities and they’d have their work cut out for them.
Not to try to scare people too much, but think about single women in the dating scene. They make a post somewhere with their email address and someone comes across it and is able to determine the same amount of information about them as what I did above? What if that person was more interested in something other than identity theft?
I think you get the idea.. essentially.. guard your personal information with your life. Never post your phone number on the internet (unless you’re using a proxy number, which is what I do), and make sure no personal information is associated with your email address before you go firing off emails to strangers.
I received an email from Steve regarding this post. He sincerely apologized for his actions and realized now that what he did was wrong and simply asked that I modify the post to protect the identities of his family. I felt that this was a fair request, considering that his family had nothing to do with what Steve did and it doesn’t jeopardize the impact of the article. So, if you’re wondering why you’re seeing all the “[withheld]“‘s, that’s why!
PS – Yes, I realize the names are still shown in the images, but they’re not indexed by Google. I figured I’d point this out before I had 20,000 comments informing me of it.
END OF UPDATE
DISCLAIMER:
This is ABSOLUTELY for informational purposes ONLY. attackvector.org nor I will be held responsible for how you choose to use the information that I post on my blog. This individual, though he is a douche for sending spam, is a real person with a real life. By misusing the information found here, you have the power to potentially destroy someones real life. There’s a fine line between a legal hack and a felony. Information gathering is not illegal so long as it’s obtained through legal means. Using the information, however, is quite another story.
UPDATE: Because of something that one of my readers brought up, I want to clarify. The email that I received was not the run of the mill malware/spambot/whatever style email. The email was coming from his email address, using his business’s name, and advertising his business. I would have never posted this had I had any doubt that this may not have actually been sent, by him, in some fashion.
END OF DISCLAIMER.
I use spammers and pedophiles as test subjects when I’m working on something. This is mostly because it’s unlikely that they would go to the authorities and point the finger at me, knowing that I could easily turn around and say something to the effect of, “Well, yes I did pwn his box.. but you should have seen all the child porn I found on it.” owned x 2.
I happened to receive a piece of spam at the exact moment as I was going to start a post about privacy and anonyminity on the internet. I will consider this to be a sign from God that this dude needed to be set straight. Okay, maybe not. I’m not sure what the bible says about spam.. but if I were God, it would be into the pits of hell for them. So, since I cannot cast people into eternal suffering in a firey pit, I will have to settle for second best. Pwnage!
Whats even better, none of what I’m about to do is illegal. It’s a serious, serious invasion of privacy, and you definitely don’t want it to happen to you, but all of it can be harvested through public record, social networks, forum posts, etc etc etc.
First, lets take a look at the email that I received.
Ok, so, his email address is steve@barteritemsfortrade.com.. he’s sending email through server299.com.. and his real IP address is 67.185.122.64. All we really need is his email address and his IP. Lets see what we can find.
Now we know that he’s connecting from Washington (wa.comcast.net). Lets see what Geo IP location says. I use this service, but there are many others. I’ve also written a few tools to do this as well, but we’re going to use what the average Joe has access to.
Just put the IP address in the box and hit “search”. Here’s what we find.
So, we’re narrowing it down.. we now know that it’s Spokane, Washington. Now we’re going to take a look at his email address. First, obviously, just google the email address. This will bring up information for virtually anything that the person has ever used their email on. Forums, social networks, etc.
In this case, however, nothing came up on google. We must dig deeper. Enter, whois!
Biz two? Does that mean there is a Biz One and a Biz Three, perhaps? Also, he’s using a PO Box.. blah.
Jackpot! We now have a last name and a phone number. We also have an additional email address/domain.
Hmm.. a real address.. no PO box on this domain. Is that an office? A house? Is it his house? I can assume that ‘snicho’ is short for ‘steve nicholas’, and it’s the administrative contact, which means he owns the domain.. so the address has something to do with him.
Enter.. Google Maps.
*www.attackvector.org/pics/13930.png
(If you’re wonder why it says “140 west 30th” and not “139 west 30th”, it’s because I slid the camera down a bit and Google tried to be helpful by changing the address)
Well, it’s definitely not an office building, so at this point I’m going to assume that it’s his house until I find out differently. We can further verify this by googling his name + city + state.
*www.attackvector.org/pics/nameres.png
That address looks rather familiar… oh yeah, it’s the address that was associated with his domain. We can be virtually certain at this point that that is his real address and house. Lets see who else lives in the house with him – just google the phone number listed.
*www.attackvector.org/pics/phoneres.png
Ok, so, [withheld] has the same last name as Steve, so I think we can safely say that this is his wife.
We’ll come back to her later. Lets see what else we can find about Steve.. I’m really starting to feel like family at this point.
Back when I googled his name + city + state, I noticed that below the address result, there was a LinkedIn page.. lets check that out.
Ok, so there’s all sorts of useful information.. but I found another email address.. steve.nicholas@itex.net Not often do I meet someone with as many email addresses as me.. lol.
So, back up to the top, we google for steve.nicholas@itex.net.
Some interesting stuff, but nothing really useful for my purposes. Lets check out Facebook and see if he’s a social butterfly. I log in and “search for friends” and enter his email address(es). His account is registered with the itex.net email address.
He doesn’t have his Facebook stuff set to private, so he’s kind of letting it all hang out. Thanks, Steve!
*www.attackvector.org/pics/stevefb.png
Yawn. The only thing interesting there, is that we’ve now definitely verified that that address is correct and that his wife’s name is definitely [withheld]. Maybe her page is more interesting.. lets look
Note: Passwords.. by building a profile of someone, you begin to get a feel of who they really are. I’m willing to bet that at least one of Steve’s passwords has something to do with fishing, trout, or cutthroats (type of trout – according to his facebook page).
[withheld]‘s Facebook:
Here’s something to take a mental note of. Women are generally more open about their personal lives and love to share with others. In one paragraph, we learn that she teaches at Salk Middle School, they’ve been married for 27 years, they have 2 daughters, [withheld] and [withheld], [withheld] is married to [withheld (both first & last name)] (note – this probably means that [withheld] is no longer [withheld] Nicholas, she’s probably [withheld (both first & last name)]). [withheld] lives in Vegas.
How ever would we find out more information about [withheld] and [withheld]? Oh yeah, friends lists. If the parents have Facebook, the kids most certainly have Facebook.. and barring any family drama, they’ll all be on each others friends lists. And, of course, I’m right.. found [withheld], [withheld], and [withheld].
Also, going through her wall posts gave up some information. They’re new grandparents.. their grandaughter [withheld] was born on March 15th.. this was [withheld] and [withheld]‘s daughter.
Now, lets see what Intelius says about [withheld] (note – I skipped Steve on Intelius because his entry is all screwed up.)
*www.attackvector.org/pics/intelius.png
Now we have ages, too. It’s interesting that there’s a “Ralph Steve Nicholas” listed, who has the same age as the other two Steve’s listed. Could Steve’s real name be Ralph??
Ok, anyway, lets see what I can find out about their house. Just about every county in the country allows you to view property tax records on the internet. I googled “spokane washington property tax records”. What you’re looking for is like, the assessor’s home page then just punch in the address and you can find a wealth of information.
What this record tells us, is that [withheld] actually owns the home.. Steve isn’t even listed. She’s also the sole person listed paying the property taxes. Interesting.. I wonder why?
Also, further down on the report, there’s two documents. A quit claim deed, and a statutory warranty deed. A warranty deed is issued in some states when a house is sold. It protects the buyer from having third parties come after them for unpaid debts and whatever. So, it appears as though they bought the house in 2001 for $110,000? Seems awfully low.
Now, lets look at the quit claim deed. First thing I notice. R Steve Nicholas is listed as “Husband of Grantee” I think Steve’s real name is Ralph. lol.
This is interesting.. quit claim deeds are used after a divorce to switch the owner of a property from one party to another at the county level. But they’re still married. The other times that I’ve seen quit claim deeds used is when people encounter serious financial trouble and need to file bankruptcy. They file independently and deed the house to their spouse.
Lets find out!
I am not going to tell you what service I use to obtain this information because I don’t want it to get abused and taken away. Also, I don’t think everyone should have access to it. SO.
Ok, so they did a joint bankruptcy in ’91 and it was discharged in ’93. I also have a list of their creditors.. no wonder they filed bankruptcy. Ouch.
One other piece of information that this offers, is previous addresses and the last 4 digits of their social security numbers. Keep in mind, a lot of people use the last 4 digits of their social for pin numbers.. because most pin numbers are limited to 4 digits. Stupid.
UPDATE: I’ve decided to X out the social security numbers because this post is starting to receive a ton of traffic and I’m not sure I want everyone visiting it to have this information. My intention of this article is not to make it easy to steal this guys identity.. it’s to point out a vulnerability. If you really want to find his social security number, lets just say.. it’s available via the internet.
Here’s something to really think about.. I was able to obtain all of the information in this post for 16 cents and by just using an email and IP address from a piece of spam.
Family members, ages, schools, anniversary dates, marriage lengths, hobbies, interests, phone numbers, addresses, property records, property taxes, pictures of their house, pictures of them, pictures of their children and grandchildren, deeds on their house, bankruptcies, employment history, previous addresses, previous creditors, and bits of social security numbers.
I’m pretty sure I’d be able to fake my way through one of those password reset forms.. you know, where you set up a “secret question” asking what your dogs name was, or where you went to school?
Beyond that, I’m fairly confident that at this point, if I were to call his bank and pretend to be him, I could easily pass when they asked me personal questions.
In closing.. you really need to pay close attention to what you’re posting on the internet. If I were a douche, I could ruin this guys life using this information. There are a lot of douches out there that are doing this type of stuff right now. Given an email address, phone number, or whatever, they build profiles on people which can be used to exploit them and steal identities.
The other thing that I’ve actually fallen victim to, is the speed of Google’s spiders and the fact that they index Craigslist. Lets say you run a business.. Catholic Charities R Us and in this post, you include an email address, phone number, something. Lets say you also make a post, days, weeks, whatever, later looking for whores, or something. Both of those posts will come up when Googling for your phone number.
Also, consider what you’re sending in this email. What if this guy had sent me an email trying to extort me, threaten me, whatever? I could turn this over to the authorities and they’d have their work cut out for them.
Not to try to scare people too much, but think about single women in the dating scene. They make a post somewhere with their email address and someone comes across it and is able to determine the same amount of information about them as what I did above? What if that person was more interested in something other than identity theft?
I think you get the idea.. essentially.. guard your personal information with your life. Never post your phone number on the internet (unless you’re using a proxy number, which is what I do), and make sure no personal information is associated with your email address before you go firing off emails to strangers.
OP
sygeek
Technomancer
Saving a life is easy, but I didn’t
By Dan Shapiro
I was reading Hacker News a few weeks ago and I stumbled on a story: Amit Gupta needs you. It turns out that Amit is the thoroughly likeable founder of Photojojo. Amit had the double misfortune to:
a) have acute leukemia, and
b) be South Asian.
The problem with the first one is obvious. The problem with the second one is that the life-saving marrow transplant that Amit needs requires a donor with a similar genetic makeup, and South Asians are dramatically underrepresented in the registered donor pool.
I read the amazing page dedicated to finding Amit a donor, and thought back to 1995. I was in my second year of college and there was a blood drive. A representative from the National Marrow Donor Program was there near the cafeteria in the quad while I was donating. She explained the marrow registry and asked me to sign up to be considered for a match for a marrow transplant.
At the time, the only way to donate marrow was to basically have someone drill holes in your bones and drain your skeleton, which kind of terrified me. Nowadays, of course, most donations require nothing more than sitting still for a few hours with an IV watching television. But after a lot of introspection, I decided that it was a rare occurrence in this world that you actually get to save the life of a stranger, and if skeleton-draining was the price of that, then so be it. I was also reassured that most folks are never matched with anyone.
Back to Amit and the present, it was clear that my genetic makeup wasn’t going to be much help for him. But I went over to marrow.org and looked around. I learned that it’s ridiculously easy these days to get tested and not very hard to donate if you’re matched. Despite this, the need is skyrocketing. Half of the people who need marrow transplants can’t locate a donor.
Then I realized – crap, how the heck are they going to get a hold of me if there’s a hit? All they have for contact info is my college dorm address! I can’t help Amit, but maybe I could help someone else in need. So I fussed around with the website to update my contact data. I couldn’t figure out how to find my old record, so I made a mental note to try and call them some time, and gave up.
Allow me to digress one more time before I get to the point. Five months ago I sold my startup, Sparkbuy, to Google. There were mountains of paperwork, and one bit that didn’t get wrapped up nicely was mail forwarding. Not email forwarding, mind you, but good, old-fashioned, paper-cut-on-your-tongue-from-sealing-the-envelope mail. I submitted the change of address request, but for some reason, mail piled up in my old office. They nagged me about it every few weeks. I procrastinated. After many months I finally went and picked it up.
Today I was sorting through that mail.
Did you know that, when the marrow donation center finds a match, they try desperately to reach the potential donor? Even if that person has moved from their dorm room long ago, even if their contact information has changed, even if they’re in a different state, even if 16 years have passed? They try. They look all over for ways to reach that person.
Almost 5 months ago, they found a match, and sent me a letter to the only address they could find for me. To my old company.
Today I read it.
I called immediately, of course. They said that they’d contact the patient’s doctor right away. But they told me the odds were good that, since 5 months had passed, “they found another match, or that the patient… is no longer eligible.”
a) have acute leukemia, and
b) be South Asian.
The problem with the first one is obvious. The problem with the second one is that the life-saving marrow transplant that Amit needs requires a donor with a similar genetic makeup, and South Asians are dramatically underrepresented in the registered donor pool.
I read the amazing page dedicated to finding Amit a donor, and thought back to 1995. I was in my second year of college and there was a blood drive. A representative from the National Marrow Donor Program was there near the cafeteria in the quad while I was donating. She explained the marrow registry and asked me to sign up to be considered for a match for a marrow transplant.
At the time, the only way to donate marrow was to basically have someone drill holes in your bones and drain your skeleton, which kind of terrified me. Nowadays, of course, most donations require nothing more than sitting still for a few hours with an IV watching television. But after a lot of introspection, I decided that it was a rare occurrence in this world that you actually get to save the life of a stranger, and if skeleton-draining was the price of that, then so be it. I was also reassured that most folks are never matched with anyone.
Back to Amit and the present, it was clear that my genetic makeup wasn’t going to be much help for him. But I went over to marrow.org and looked around. I learned that it’s ridiculously easy these days to get tested and not very hard to donate if you’re matched. Despite this, the need is skyrocketing. Half of the people who need marrow transplants can’t locate a donor.
Then I realized – crap, how the heck are they going to get a hold of me if there’s a hit? All they have for contact info is my college dorm address! I can’t help Amit, but maybe I could help someone else in need. So I fussed around with the website to update my contact data. I couldn’t figure out how to find my old record, so I made a mental note to try and call them some time, and gave up.
Allow me to digress one more time before I get to the point. Five months ago I sold my startup, Sparkbuy, to Google. There were mountains of paperwork, and one bit that didn’t get wrapped up nicely was mail forwarding. Not email forwarding, mind you, but good, old-fashioned, paper-cut-on-your-tongue-from-sealing-the-envelope mail. I submitted the change of address request, but for some reason, mail piled up in my old office. They nagged me about it every few weeks. I procrastinated. After many months I finally went and picked it up.
Today I was sorting through that mail.
Did you know that, when the marrow donation center finds a match, they try desperately to reach the potential donor? Even if that person has moved from their dorm room long ago, even if their contact information has changed, even if they’re in a different state, even if 16 years have passed? They try. They look all over for ways to reach that person.
Almost 5 months ago, they found a match, and sent me a letter to the only address they could find for me. To my old company.
Today I read it.
I called immediately, of course. They said that they’d contact the patient’s doctor right away. But they told me the odds were good that, since 5 months had passed, “they found another match, or that the patient… is no longer eligible.”
OP
sygeek
Technomancer
The Reason Android is Laggy
By Andrew Munn
By Andrew Munn
Follow up to “Android graphics true facts”, or The Reason Android is Laggy
Yesterday +Dianne Hackborn posted to Google+ an article that dismissed the common accusation that Android is laggy because UI rendering wasn’t hardware accelerated until Honeycomb:
*plus.google.com/105051985738280261832/posts/2FXDCz8x93s
It’s an insightful post that illuminates many of the complex issues with smooth Android rendering. Unfortunately, it doesn’t answer the fundamental question asked by both technical and nontechnical android users:
Why is Android laggy, while iOS, Windows Phone 7, QNX, and WebOS are fluid?
This post will attempt to answer that question.
However before I jump in, a couple disclaimers. First, I am a 3rd year undergraduate software engineering student. I interned on the Android team, and +Romain Guy who was responsible for much of the hardware acceleration work in Honeycomb, reviewed some of my code, but I was not on the framework team and I never read the Android rendering source code. I do not have any authoritative Android knowledge and I cannot guarantee what I say here is necessarily 100% accurate, but I have done my best to do my homework.
Second, I’m interning with the Windows Phone team starting in January, so it’s possible that this post will be unconsciously biased against Android, but if you ask any of my friends, it’s really hard to shut me up about Android. I have more Android t-shirts than days of the week and I’d rather give away my Macbook than my Nexus S. The Googlplex is like a second home - I’ve slept there on more than a few occasions to the dismay of startled janitors (and if you ever get a chance to visit, the banana french toast at Big Table Cafe is to die for). If anything, I’m probably biased in Android’s favor.
Finally, any opinions expressed in this article are solely my own and do not represent those of any past or future employers.
With that out of the way, lets dive right in.
Dianne starts off her post with a surprising revelation:
“Looking at drawing inside of a window, you don’t necessarily need to do this in hardware to achieve full 60fps rendering. This depends very much on the number of pixels in your display and the speed of your CPU. For example, Nexus S has no trouble doing 60fps rendering of all the normal stuff you see in the Android UI like scrolling lists on its 800x480 screen.”
Hun? How can this be the case? Anybody who’s used a Nexus S knows it slows down in all but the simplest of ListViews. And forget any semblance of decent performance if a background task is occurring, like installing an app or updating the UI from disk. On the other hand, iOS is 100% smooth even when installing apps. But we know Dianne isn’t lying about the potential CPU performance, so what’s going on?
The Root Cause
It’s not GC pauses. It’s not because Android runs bytecode and iOS runs native code. It’s because on iOS all UI rendering occurs in a dedicated UI thread with real-time priority. On the other hand, Android follows the traditional PC model of rendering occurring on the main thread with normal priority.
This is a not an abstract or academic difference. You can see it for yourself. Grab your closest iPad or iPhone and open Safari. Start loading a complex web page like Facebook. Half way through loading, put your finger on the screen and move it around. All rendering instantly stops. The website will literally never load until you remove your finger. This is because the UI thread is intercepting all events and rendering the UI at real-time priority.
If you repeat this exercise on Android, you’ll notice that the browser will attempt to both animate the page and render the HTML, and do an ‘ok’ job at both. On Android, this a case where an efficient dual core processor really helps, which is why the Galaxy S II is famous for its smoothness.
On iOS when an app is installing from the app store and you put your finger on the screen, the installation instantly pauses until all rendering is finished. Android tries to do both at the same priority, so the frame rate suffers. Once you notice this happening, you’ll see it everywhere on an Android phone. Why is scrolling in the Movies app slow? Because movie cover thumbnails are dynamically added to the movie list as you scroll down, while on iOS they are lazily added after all scrolling stops.
Other Reasons
The fundamental reason Android is laggy is UI rendering threading and priority, but it’s not the only reason. First, hardware acceleration, despite Dianna’s reservations, does help. My Nexus S has never been snappier since upgrading to ICS. Hardware acceleration makes a huge difference in apps like the home screen and Android market. Offloading rendering to the GPU also increases battery life, because GPUs are fixed-function hardware, so they operate at a lower power envelope.
Second, contrary to what I claimed earlier, garbage collection is still a problem, even with the work on concurrent GC in Dalvik. For example, if you’ve ever used the photo gallery app in Honeycomb or ICS you may wonder why the frame rate is low. It turns out the frame rate is capped at 30 FPS because without the cap, swiping through photos proceeds at 60 FPS most of the time, but occasionally a GC pause causes a noticeable “hiccup”. Capping the frame rate at 30 fixes the hiccup problem at the expense of buttery smooth animations at all times.
Third, there are the hardware problems that Dianne discussed. The Tegra 2, despite Nvidia’s grandiose marketing claims, is hurt by low memory bandwidth and no NEON instruction set support (NEON instructions are the ARM equivalent of Intel’s SSE, which allow for faster matrix math on CPUs). Honeycomb tablets would be better off with a different GPU, even if it was theoretically less powerful in some respects than the Tegra 2. For example, the Samsung Hummingbird in the Nexus S or Apple A4. It’s telling that the fastest released Honeycomb tablet, the Galaxy Tab 7.7, is running the Exynos CPU from the Galaxy S II.
Fourth, Android has a ways to go toward more efficient UI compositing. On iOS, each UI view is rendered separately and stored in memory, so many animations only require the GPU to recomposite UI views. GPUs are extremely good at this. Unfortunately, on Android, the UI hierarchy is flattened before rendering, so animations require every animating section of the screen to be redrawn.
Fifth, the Dalvik VM is not as mature as a desktop class JVM. Java is notorious for terrible GUI performance on desktop. However, many of the issues don’t carry over to the Dalvik implementation. Swing was terrible because it was a cross platform layer on top of native APIs. It is interesting to note that Windows Phone 7’s core UI is built in native code, even though the original plan was to base it entirely on Silverlight. Microsoft ultimately decided that to get the kind of UI performance required, the code would have to be native. It’s easy to see the difference between native and bytecode on Windows Phone 7, because third party apps are written in Silverlight and have inferior performance (NoDo and Mango have alleviated this problem and the Silverlight UIs are generally very smooth now).
Thankfully, each of the five issues listed above is solvable without radical changes to Android. Hardware acceleration will be on all Android phones running ICS, Dalvik continues to improve GC efficiency, the Tegra 2 is finally obsolete, there are existing workarounds for the UI compositing problems, and Dalvik becomes a faster VM with every release. I recently asked +Jason Kincaid of +TechCrunch if his Galaxy Nexus was smooth, and he had this to say:
“In general I've found ICS on the Galaxy Nexus to be quite smooth. There are occasional stutters — the one place where I can consistently get jitters on the Galaxy Nexus is when I hit the multitasking button, where it often will pause for a quarter second. That said, I find that the iPhone 4S also jitters more than I had expected, especially when I go to access the systemwide search (where you swipe left from the home screen).”
So there you go, the Android lag problem is mostly solved, right? Not so fast.
Going Forward
Android UI will never be completely smooth because of the design constraints I discussed at the beginning:
Work on Android started before the release of the iPhone, and at the time Android was designed to be a competitor to the Blackberry. The original Android prototype wasn’t a touch screen device. Android’s rendering trade-offs make sense for a keyboard and trackball device. When the iPhone came out, the Android team rushed to release a competitor product, but unfortunately it was too late to rewrite the UI framework.
This is the same reason why Windows Mobile 6.5, Blackberry OS, and Symbian have terrible touch screen performance. Like Android, they were not designed to prioritise UI rendering. Since the iPhone’s release, RIM, Microsoft, and Nokia have abandoned their mobile OS’s and started from scratch. Android is the only mobile OS left that existed pre-iPhone.
So, why doesn’t the Android team rewrite the rendering framework? I’ll let Romain Guy explain:
“...a lot of the work we have to do today is because of certain choices made years ago... ...having the UI thread handle animations is the biggest problem. We are working on other solutions to try to improve this (schedule drawing on vsync instead of block on vsync after drawing, possible use a separate rendering thread, etc.) An easy solution would of course to create a new UI toolkit but there are many downsides to this also.”
Romain doesn’t elaborate on what the downsides are, but it’s not difficult to speculate:
When the topic of Android comes up with both technical and nontechnical friends, I hear over and over that Android is laggy and slow. The reality is that Android can open apps and render web pages as fast or faster than iOS, but perception is everything. Fixing the UI lag will go a long way to repairing Android’s image.
Beyond the perception issue, lag is a violation of one of Google’s core philosophies. Google believes that things should be fast. That’s a driving philosophy behind Google Search, Gmail, and Chrome. It’s why Google created SPDY to improve on HTTP. It’s why Google builds tools to help websites optimize their site. It’s why Google runs it’s own CDN. It’s why Google Maps is rendered in WebGL. It’s why buffering on Youtube is something most of us remember, but rarely see anymore.
But perhaps the most salient reason why UI lag in Android is unacceptable comes from the field of Human-Computer Interaction (HCI). Modern touch screens imply an affordance language of 1 to 1 mapping between your finger and animations on the screen. This is why the iOS over-scroll (elastic band) effect is so cool, fun, and intuitive. And this is why the touch screens on Virgin America Flights are so frustrating: they are incredibly laggy, unresponsive, and imprecise.
A laggy UI breaks the core affordance language of a touch screen. The device no longer feels natural. It loses the magic. The user is pulled out of their interaction and must implicitly acknowledge they are using an imperfect computer simulation. I often get “lost” in an iPad, but I cringe when a Xoom stutters between home screens. The 200 million users of Android deserve better.
And I know they will have it eventually. The Android team is one of the most dedicated and talented development teams in the world. With stars like +Dianne Hackborn and +Romain Guy around, the Android rendering framework is in good hands.
I hope this post has reduced confusion surrounding Android lag. With some luck, Android 5.0 will bring the buttery-smooth Android we’ve all dreamed about since we first held an HTC G1. In the mean time, I’ll be in Redmond working my butt off trying to get a beautiful and smooth mobile OS some of the recognition it deserves.
Credits
Parts of this post was inspired by this reddit comment by ddtro who explained the UI thread and real-time issue:
ddtron comments on Facts and fiction about Android graphics rendering & UI smoothness
This explanation of Android versus iOS UI compositing on Hacker News by Corun was illuminating:
Android's graphics problems are not due to a lack of hardware acceleration. They... | Hacker News
Information about Android’s historical roots taken from In the Plex by +Steven Levy and Steve Jobs by Walter Isaacson
Yesterday +Dianne Hackborn posted to Google+ an article that dismissed the common accusation that Android is laggy because UI rendering wasn’t hardware accelerated until Honeycomb:
*plus.google.com/105051985738280261832/posts/2FXDCz8x93s
It’s an insightful post that illuminates many of the complex issues with smooth Android rendering. Unfortunately, it doesn’t answer the fundamental question asked by both technical and nontechnical android users:
Why is Android laggy, while iOS, Windows Phone 7, QNX, and WebOS are fluid?
This post will attempt to answer that question.
However before I jump in, a couple disclaimers. First, I am a 3rd year undergraduate software engineering student. I interned on the Android team, and +Romain Guy who was responsible for much of the hardware acceleration work in Honeycomb, reviewed some of my code, but I was not on the framework team and I never read the Android rendering source code. I do not have any authoritative Android knowledge and I cannot guarantee what I say here is necessarily 100% accurate, but I have done my best to do my homework.
Second, I’m interning with the Windows Phone team starting in January, so it’s possible that this post will be unconsciously biased against Android, but if you ask any of my friends, it’s really hard to shut me up about Android. I have more Android t-shirts than days of the week and I’d rather give away my Macbook than my Nexus S. The Googlplex is like a second home - I’ve slept there on more than a few occasions to the dismay of startled janitors (and if you ever get a chance to visit, the banana french toast at Big Table Cafe is to die for). If anything, I’m probably biased in Android’s favor.
Finally, any opinions expressed in this article are solely my own and do not represent those of any past or future employers.
With that out of the way, lets dive right in.
Dianne starts off her post with a surprising revelation:
“Looking at drawing inside of a window, you don’t necessarily need to do this in hardware to achieve full 60fps rendering. This depends very much on the number of pixels in your display and the speed of your CPU. For example, Nexus S has no trouble doing 60fps rendering of all the normal stuff you see in the Android UI like scrolling lists on its 800x480 screen.”
Hun? How can this be the case? Anybody who’s used a Nexus S knows it slows down in all but the simplest of ListViews. And forget any semblance of decent performance if a background task is occurring, like installing an app or updating the UI from disk. On the other hand, iOS is 100% smooth even when installing apps. But we know Dianne isn’t lying about the potential CPU performance, so what’s going on?
The Root Cause
It’s not GC pauses. It’s not because Android runs bytecode and iOS runs native code. It’s because on iOS all UI rendering occurs in a dedicated UI thread with real-time priority. On the other hand, Android follows the traditional PC model of rendering occurring on the main thread with normal priority.
This is a not an abstract or academic difference. You can see it for yourself. Grab your closest iPad or iPhone and open Safari. Start loading a complex web page like Facebook. Half way through loading, put your finger on the screen and move it around. All rendering instantly stops. The website will literally never load until you remove your finger. This is because the UI thread is intercepting all events and rendering the UI at real-time priority.
If you repeat this exercise on Android, you’ll notice that the browser will attempt to both animate the page and render the HTML, and do an ‘ok’ job at both. On Android, this a case where an efficient dual core processor really helps, which is why the Galaxy S II is famous for its smoothness.
On iOS when an app is installing from the app store and you put your finger on the screen, the installation instantly pauses until all rendering is finished. Android tries to do both at the same priority, so the frame rate suffers. Once you notice this happening, you’ll see it everywhere on an Android phone. Why is scrolling in the Movies app slow? Because movie cover thumbnails are dynamically added to the movie list as you scroll down, while on iOS they are lazily added after all scrolling stops.
Other Reasons
The fundamental reason Android is laggy is UI rendering threading and priority, but it’s not the only reason. First, hardware acceleration, despite Dianna’s reservations, does help. My Nexus S has never been snappier since upgrading to ICS. Hardware acceleration makes a huge difference in apps like the home screen and Android market. Offloading rendering to the GPU also increases battery life, because GPUs are fixed-function hardware, so they operate at a lower power envelope.
Second, contrary to what I claimed earlier, garbage collection is still a problem, even with the work on concurrent GC in Dalvik. For example, if you’ve ever used the photo gallery app in Honeycomb or ICS you may wonder why the frame rate is low. It turns out the frame rate is capped at 30 FPS because without the cap, swiping through photos proceeds at 60 FPS most of the time, but occasionally a GC pause causes a noticeable “hiccup”. Capping the frame rate at 30 fixes the hiccup problem at the expense of buttery smooth animations at all times.
Third, there are the hardware problems that Dianne discussed. The Tegra 2, despite Nvidia’s grandiose marketing claims, is hurt by low memory bandwidth and no NEON instruction set support (NEON instructions are the ARM equivalent of Intel’s SSE, which allow for faster matrix math on CPUs). Honeycomb tablets would be better off with a different GPU, even if it was theoretically less powerful in some respects than the Tegra 2. For example, the Samsung Hummingbird in the Nexus S or Apple A4. It’s telling that the fastest released Honeycomb tablet, the Galaxy Tab 7.7, is running the Exynos CPU from the Galaxy S II.
Fourth, Android has a ways to go toward more efficient UI compositing. On iOS, each UI view is rendered separately and stored in memory, so many animations only require the GPU to recomposite UI views. GPUs are extremely good at this. Unfortunately, on Android, the UI hierarchy is flattened before rendering, so animations require every animating section of the screen to be redrawn.
Fifth, the Dalvik VM is not as mature as a desktop class JVM. Java is notorious for terrible GUI performance on desktop. However, many of the issues don’t carry over to the Dalvik implementation. Swing was terrible because it was a cross platform layer on top of native APIs. It is interesting to note that Windows Phone 7’s core UI is built in native code, even though the original plan was to base it entirely on Silverlight. Microsoft ultimately decided that to get the kind of UI performance required, the code would have to be native. It’s easy to see the difference between native and bytecode on Windows Phone 7, because third party apps are written in Silverlight and have inferior performance (NoDo and Mango have alleviated this problem and the Silverlight UIs are generally very smooth now).
Thankfully, each of the five issues listed above is solvable without radical changes to Android. Hardware acceleration will be on all Android phones running ICS, Dalvik continues to improve GC efficiency, the Tegra 2 is finally obsolete, there are existing workarounds for the UI compositing problems, and Dalvik becomes a faster VM with every release. I recently asked +Jason Kincaid of +TechCrunch if his Galaxy Nexus was smooth, and he had this to say:
“In general I've found ICS on the Galaxy Nexus to be quite smooth. There are occasional stutters — the one place where I can consistently get jitters on the Galaxy Nexus is when I hit the multitasking button, where it often will pause for a quarter second. That said, I find that the iPhone 4S also jitters more than I had expected, especially when I go to access the systemwide search (where you swipe left from the home screen).”
So there you go, the Android lag problem is mostly solved, right? Not so fast.
Going Forward
Android UI will never be completely smooth because of the design constraints I discussed at the beginning:
- UI rendering occurs on the main thread of an app
- UI rendering has normal priority
Work on Android started before the release of the iPhone, and at the time Android was designed to be a competitor to the Blackberry. The original Android prototype wasn’t a touch screen device. Android’s rendering trade-offs make sense for a keyboard and trackball device. When the iPhone came out, the Android team rushed to release a competitor product, but unfortunately it was too late to rewrite the UI framework.
This is the same reason why Windows Mobile 6.5, Blackberry OS, and Symbian have terrible touch screen performance. Like Android, they were not designed to prioritise UI rendering. Since the iPhone’s release, RIM, Microsoft, and Nokia have abandoned their mobile OS’s and started from scratch. Android is the only mobile OS left that existed pre-iPhone.
So, why doesn’t the Android team rewrite the rendering framework? I’ll let Romain Guy explain:
“...a lot of the work we have to do today is because of certain choices made years ago... ...having the UI thread handle animations is the biggest problem. We are working on other solutions to try to improve this (schedule drawing on vsync instead of block on vsync after drawing, possible use a separate rendering thread, etc.) An easy solution would of course to create a new UI toolkit but there are many downsides to this also.”
Romain doesn’t elaborate on what the downsides are, but it’s not difficult to speculate:
- All Apps would have to be re-written to support the new framework
- Android would need a legacy support mode for old apps
- Work on other Android features would be stalled while the new framework is developed
When the topic of Android comes up with both technical and nontechnical friends, I hear over and over that Android is laggy and slow. The reality is that Android can open apps and render web pages as fast or faster than iOS, but perception is everything. Fixing the UI lag will go a long way to repairing Android’s image.
Beyond the perception issue, lag is a violation of one of Google’s core philosophies. Google believes that things should be fast. That’s a driving philosophy behind Google Search, Gmail, and Chrome. It’s why Google created SPDY to improve on HTTP. It’s why Google builds tools to help websites optimize their site. It’s why Google runs it’s own CDN. It’s why Google Maps is rendered in WebGL. It’s why buffering on Youtube is something most of us remember, but rarely see anymore.
But perhaps the most salient reason why UI lag in Android is unacceptable comes from the field of Human-Computer Interaction (HCI). Modern touch screens imply an affordance language of 1 to 1 mapping between your finger and animations on the screen. This is why the iOS over-scroll (elastic band) effect is so cool, fun, and intuitive. And this is why the touch screens on Virgin America Flights are so frustrating: they are incredibly laggy, unresponsive, and imprecise.
A laggy UI breaks the core affordance language of a touch screen. The device no longer feels natural. It loses the magic. The user is pulled out of their interaction and must implicitly acknowledge they are using an imperfect computer simulation. I often get “lost” in an iPad, but I cringe when a Xoom stutters between home screens. The 200 million users of Android deserve better.
And I know they will have it eventually. The Android team is one of the most dedicated and talented development teams in the world. With stars like +Dianne Hackborn and +Romain Guy around, the Android rendering framework is in good hands.
I hope this post has reduced confusion surrounding Android lag. With some luck, Android 5.0 will bring the buttery-smooth Android we’ve all dreamed about since we first held an HTC G1. In the mean time, I’ll be in Redmond working my butt off trying to get a beautiful and smooth mobile OS some of the recognition it deserves.
Credits
Parts of this post was inspired by this reddit comment by ddtro who explained the UI thread and real-time issue:
ddtron comments on Facts and fiction about Android graphics rendering & UI smoothness
This explanation of Android versus iOS UI compositing on Hacker News by Corun was illuminating:
Android's graphics problems are not due to a lack of hardware acceleration. They... | Hacker News
Information about Android’s historical roots taken from In the Plex by +Steven Levy and Steve Jobs by Walter Isaacson
OP
sygeek
Technomancer
The Dos And Don'ts Of Time Travel
By Jim Behrle
*www.theawl.com/wp-content/uploads/2011/10/TimeTravel_short-e1320084224895.jpg
So you’ve hooked electrodes and power couplings to an old-fashioned carousel in an abandoned amusement park on the outskirts of town. Or you’ve outfitted a Harley-Davidson with a flux capacitor—a classic. Or, my personal favorite, you’re using depleted uranium to turn the underused freight elevator in your building into a time-ship. As a soon-to-be time traveler, the last thing you want is somebody telling you “Do this!” and “Don’t do that!” You're about to become a pirate on the open waves of the ocean of time. Good for you! It's sure to be a wonderful adventure. One no doubt filled with romance, knowledge and treasure. But here, humbly, are a few things to keep in mind.
DO go forward in time first. No matter how stable you think your time machine is, your first jump should always be into the future. It’s a mistake to visit President Lincoln on your maiden voyage. The past is loud, smelly and dangerous. And without at least one pit stop in the future, the road backwards is a million times more difficult. Imagine getting one good jump out of your device and then getting stuck in, say, 1861. You’d have to live out the rest of your life in the dark past. They didn’t even have a sun until the 1840s. Great, if you are some kind of wild history nerd. But you have no resources. You probably don’t have the right kind of money. Clothes, forget it. Even Civil War reenactors are flushed out within seconds in the past. It’s best, no matter how flushed with megalomaniacal power the creation of a time machine has made you, that you go first into the future to get all the latest updates and then start thinking about venturing into the past. The Future is Your Friend. Think of it as a great big safe house for time travelers filled with strangers who may not be thrilled to help you, but probably will point you in the right direction. After all, time traveling is no big deal there. You remember how cool you felt when you suffered under the illusion that you were the only one you knew who had the new iPhone? In the future, iPhones aren’t very cool. And time machines are a commonplace of everyday life. Like a blender or a teleporter. They’ll know how to hook you up and get you ready for your journey back in time.
DO be wary of the past. In fact, it’s probably best to avoid Going Back In Time your first few trips out. As enticing an idea as it might be to track down the Buddha or watch Jesus die on the cross, let’s work up to those, okay? Aramaic isn’t exactly going to be falling off your tongue as a beginner. And you’ll find it’s the little things that will cause the misunderstandings that will get you nailed to a cross right next to your pal Jesus. They have plenty of trees to nail you to in the past; it’s no problem to add one more crazy-talking future freak to the crucifixion party.
There are some things in the past you simply cannot prepare yourself for. The smell. The weird diseases. Everyone’s voice seems really squeaky for some reason. And people are really short. Also, this is probably the most surprising thing, it’s practically a 24/7 grab-ass in the past. Man, woman, child. You will get used to it, but it's initially pretty strange.
*www.theawl.com/wp-content/uploads/2011/10/TimeTravel_SmellOfPast-e1320083234276.jpg
*www.theawl.com/wp-content/uploads/2011/10/TimeTravel_note-e1320083172580.jpg
DON'T be surprised that all your time-traveling friends are flakes. You’ll find that time travelers are world-class procrastinators. And why not, right? They’ve got all the time in the world and a million chances to get everything just right. It's not surprising that such people would develop a leisurely sense of pace. “Oh, you have been captured by a Mongol Army? OK, I will definitely get over there after a few weeks on the beaches of Atlantis.” That kind of thing. Time travelers, although they need you to watch their backs, do not need to help you right away.
DO get killed. DON’T Get Captured! Being killed in the past is better than being captured. You know how every episode of “Dr. Who” would be greatly sped up if the Doctor simply carried a gun and refused to be taken prisoner? Being taken hostage is a generally unpleasant experience. And the problem is that even if your time-traveling friends warn you over brunch that you should not go to Ancient Rome because on this trip you will be fed to lions, you won’t listen. Instead, you'll think, “Well, knowing that I will be more careful and make sure not to get taken prisoner.” Which will, through some overly-cautious sidesteps you make in response to this knowledge, probably lead right to your capture. And you could be captured for a while. And just because you later erase the past it doesn’t mean you will forget it, what with all the being chewed on by rats and beaten with medieval wifflebats.
Your time-traveling friends may eventually get around to helping you out of captivity, but, as discussed above, they’re most likely flakes. Who knows if they’ll even show up the same day you got captured, or if they’ll leave you in there to rot? “Oh, I thought you said August 1901! Not August 1701!” You certainly could rely on yourself to help yourself. By sending yourself back to one moment in the past 40 or 50 times you will have a pretty good posse of yourself there to handle most problems. Some time travelers are able to do this with regularity and effectiveness. How do you think this whole Occupy Wall Street thing started in the first place? But what tends to happen to time travelers over the years is that they grow more aloof—and less tied to their firmest of beliefs. When you time travel a lot, you start to see all sides of most arguments. You become a bit of a flake. And with all the time in the world, you rarely feel like doing the little things you promised yourself you would do. “I’ll get to that trap door eventually.” And then next thing you know, you’re 99 years old, on a beach getting busy with the King of France, and you have one of those Should Have Had a V-8 Head Smacking moments.
DON'T be too much of a perfectionist. Here’s something lots of time travelers do: get trapped in a situation, say at Ford's Theatre, and think, well, I’ll make myself come back here earlier in the day and make sure that I have a weapon taped under my seat. It might take you 30 or 40 tries to get things just right. But even then it’s questionable whether even having a weapon made things easier or harder in the first place. With all the power of time travel and infinite amounts of do-overs, time travelers tend to get a little bit paranoid about every little thing. They want everything to turn out just right, with no awkward moments or embarrassing scenarios. Remember: no one really knows you in the past. They’re not going to tweet all their friends if your toga falls off in front of Caesar or whatever. You don’t need to get everything exactly right. It’s just never going to happen. Even after a million tries you’re still not going to impress that lady or dude with the perfect line. They either like you or they don’t. There are lots of fish in the sea. (This is even more true when you consider that all the people in the past will now be within reach.)
*www.theawl.com/wp-content/uploads/2011/10/TimeTravel_Right-e1320083364929.jpg
DO feel free to be. Falling in love is OK. Don’t worry about knocking up people in the past or wonder if by impregnating someone you are changing the time-space continuum. It’s a mistake to think that you’re all that important to the flow of anything. Step on a butterfly in the past and maybe it gives the chance for another butterfly to land on a flower. Things tend to work out the same way, eventually. The Yankees won the World Series 40 times the first time through this current time narrative. Time travelers have all just compromised at 27 and left it at that. You know, whatever. I was originally shocked that time travelers had allowed many of the most heinous acts of human evil to go unchanged. I mean, imagine if Hitler had been stopped. Well, it has been imagined. Millions of times over. And it doesn’t mean that World War II and the Holocaust can't be averted. They just haven’t been yet. “Yet” is a very powerful concept to the time traveler, you’ll find. It has endless possibilities. Nothing is decided. And when they write the history books you’ll find even those are written in erasable ink.
DON’T worry about creating alternate universes or destroying the timeline. Really, don’t sweat it. No small thing you do—like, choosing the hashbrown casserole over grits at Ye Olde Historic Cracker Barrel—is going to set off a chain reaction that will unravel the present as we know it and threaten the very existence of everyone reading this article. That whole Gwyneth Paltrow misses a subway and opens up a wormhole which ruins her life thing is complete crap. Relax. You are here reading this. So, OK. If Einstein was wrong about the possibility of time travel in the first place (whether he was wrong or just flat-out lied about it for his own reasons, we may never know). He said that if you can’t travel faster than the speed of light then time travel is impossible. Well, roll over, Al. You apparently missed the whole neutrino thing. Alternative universes and broken timelines, well, let’s just say the science isn’t in. Yet. Get a hundred time travelers in a room together and you’ll have to listen to a lot of stories of how “in their experience” the past is this and not that. Getting time travelers to agree on anything is pretty pointless. They act even more entitled and righteous and professorial than elected officials.
*www.theawl.com/wp-content/uploads/2011/10/TimeTravel_ScrewingUpPresent-e1320083507863.jpg
DO take precautions. In an emergency it’s really most important to keep cool. It’s a good idea to keep an apartment in a neutral place during a peaceful time. You’ll find time travel to be exhausting, and you will need a place you know you can chillax. Time travel is also pretty addictive, so you'll need to find a way to allow yourself enough of a rest. Food in the past is mostly disgusting and will make you pretty sick at first. There is no good coffee practically anywhere. And, if you’re a drinker, you may be prone to drunk time-dialing. Always know where you are. Always leave yourself a note. Have you ever woken up someplace and not known how you got there? Multiply that by any place in time, any where in the world. Protect yourself at all times. You never know if someone you meet is Jack the Ripper, so just assume they are. You don’t have to live your life as a time traveler in secret. There are plenty of people in the past who get that time travelers exist and will be interested in your travels. And there are many others who will want to use all the information you have for their own benefit.
DO make money. If you’re low on money, the best way to get more is to gamble. Knowing how sports events, gladiator fights or dice will fall is a big benefit. Don’t forget to sometimes lose; you'll attract less attention that way. Think of it as the price of doing business in the past. You can also rob banks. And, if you want, give the money back down the road. They’ll never know. You can travel to Macy’s Herald Square location on Christmas and take, say, $50,000 cash from the safe. And bring it back down the road when you’re flush. Take out stock in some crappy company like Google and then sell it just before it goes belly-up. But, like, if you’re going back in time to commit armed robbery and you leave behind a giant trail of dead bodies, the more cleaning up you’re going to have to do. If you’re going back in time to be a mass-murderer, you’re wasting your precious adventure time. Time flies, literally, when you’re time traveling. And you’ll never get to do all the things you want to do if you are wasting it cleaning up after your poor decisions.
DON'T bring a friend. It’s tough to bring people with you, even ones you completely trust. Never mind that Dr. Who and Companions thing he has going. He is not the time traveler to model yourself after. I mean, celery pinned to your lapel? He attracts way too much attention. And he has seemingly infinite lives to play with as he infinitely renews himself in new hot young actor bodies. You, on the other hand, can die lots of times and be saved by your pals, but you will always have just the same one non-actor body. You will continue to get old and frail and fat while the Doctor will transform into another hot young actor. Why hasn’t Dr. Who turned into a woman? Because there would be no show; most women are too smart to get themselves into the stupid predicaments that the Doctor does. You may think it will be impressive to some friend of yours to bring them back in time to meet, I don’t know, Napoleon? But you’ll find that adding pals to your traveling party increases the danger of someone doing something stupid. Like getting drunk and taking off with your time machine. Time travel tends to be kind of a solo thing. Let your friends get their own time machines and have their own adventures. Which you can discuss over brunches in 1920's Paris.
DO be serene about what you can change and what you can't. Nothing that is done cannot be undone. And the world, the past and the future is waiting for you. It’s OK to feel nervous and a little overanxious. The past and even the future will ultimately be a little disappointing in some ways. And breathtaking in others. Try to enjoy yourself—but quietly, without drawing too much attention to yourself. Some people, like me, might put our poo in plastic, go to the zoo and chuck it at the monkeys. But that’s only if you really like trouble. And most people can do without trouble entirely. Time travelers are around us all the time, seeing their favorite movies in theaters and maybe just riding the Q train for kicks. If you get in trouble in the past, they might even lend you a hand. Most Americans want to meet Lincoln, for some reason. Possibly the hat. If you see him, say hello. You could warn him about the play, but who hasn’t. He’s as stubborn as any time traveler. And some people prefer to let history ride.
DON'T go looking for yourself. Also, be careful about visiting yourself in the past. You’ll find arguing with who you used to be to be an incredibly unpleasant experience. Trust me, you won’t want to listen to your time-traveling ass. You, Old You that is, may see Future You and think it's important to stay the course so you can become a time traveler (alter it and you might go into stamp collecting instead). You can’t talk yourself out of dating certain people for the most part: the Past You will resent the Future You for interfering. It might even make Past You want to date Person You Shouldn't Date even more. And let's face it: Some people are just attracted to terrible people. Just because you can travel through time doesn’t mean you can control it. Some things just have to happen. Some mistakes need to be made. When the team you’re not rooting for is about to score the winning touchdown, you don’t jump on the field and tackle them. That would just make things worse. Time Travelers don’t have a Hippocratic oath, and “harm” is pretty relative, but the old adage holds true for time travelers and is generally just a good policy to have: “Don’t **** with what you don’t understand.” There’s a certain zen quality to letting things happen. And to figuring things out for yourself. Enjoy the time it takes you, and where time takes you!
*www.theawl.com/wp-content/uploads/2011/10/TimeTravel_BewareThePast-e1320083421567.jpg
OP
sygeek
Technomancer
Famous Last Words by Bosses I've Had
Sorry to say, none of these were made up...
Me, 124 Monday lunches in a row: We need an adequate disaster recovery plan.
Boss: We do. We back up every day.
Me: What happens when we try to restore one of those backups?
Boss: I don't know. Why?
Me: Where's the test plan?
Boss: Jerry will make sure Fred's program works.
Me: Where's the "Expected Results" section on the test plan?
Boss: What?
Me: I don't have access to the production server.
Boss: I already emailed you your password.
Me: I know, but I don't know my login.
Boss: What's a login?
Me: That doesn't make any sense. Have the auditors approved it?
Boss: No, but we can't have everything.
Boss: I'm really upset that no one has updated me on Project 127.
Me: I cc'd you on all 9 Project 127 emails I sent this week.
Boss: I haven't had time to get caught up on my email.
Me: You've been invited to a meeting with 3 department heads to hash out their differences on Project 249.
Boss: I hate meetings.
Boss: Why haven't you started the Accounts Receivable project yet?
Me: Because management has not yet decided whether customer credit limits should be per division or companywide.
Boss: What difference does that make?
Boss: We have hundreds of past due orders.
Me: No, we have 22 past due orders.
Boss: I'm not going to argue with you.
Me: Good, because you'd lose.
Me: We're meeting with the customer at 8:00 a.m. tomorrow.
Boss: I hate mornings.
Me: The server crashed. IT Services is working to bring it back up.
Boss: Don't confuse me with all these technical details.
Me: The customer didn't receive that information because that product is not on our computer.
Boss: Give me a list of all products not on our computer.
Boss: Why haven't you started Project 193 yet?
Me: Because the customer has not yet committed to the specs.
Boss: What difference does that make?
Me: The program was written with 3 SQL selects inside a loop. It ran OK when we had 500 parts. Now that we have 10,000 parts, it runs real slow.
Boss: I don't understand.
Boss: What are you working on?
Me: Project 432, which you said was my top priority. Remember?
Boss: No.
Boss: Why aren't you working on Project 387?
Me: Because you said not to work on anything else until Project 432 was complete. Remember?
Boss: No.
Boss: I'm giving you only enhancements. I'm outsourcing all of the bug fixes.
Me: But this is a bug fix. It says so right here on the ticket.
Boss: Oh, I didn't have time to read the ticket.
Boss: Amazon is threatening to shut us down because we ship too many orders late. How do we fix this?
Me: Ship every order on time.
Boss: No, I meant, "How do we fix this with software?"
Boss, on December 31: Write a program to close every work order so we make our year end numbers.
Boss, on January 3: Why is the database so screwed up?
Boss: You did great this year. I'm giving you a 2% increase.
Me: I hate you. I quit.
Boss: Then I'll give you a 4% increase.
Me: I still hate you. I still quit.
Me, 124 Monday lunches in a row: We need an adequate disaster recovery plan.
Boss: We do. We back up every day.
Me: What happens when we try to restore one of those backups?
Boss: I don't know. Why?
Me: Where's the test plan?
Boss: Jerry will make sure Fred's program works.
Me: Where's the "Expected Results" section on the test plan?
Boss: What?
Me: I don't have access to the production server.
Boss: I already emailed you your password.
Me: I know, but I don't know my login.
Boss: What's a login?
Me: That doesn't make any sense. Have the auditors approved it?
Boss: No, but we can't have everything.
Boss: I'm really upset that no one has updated me on Project 127.
Me: I cc'd you on all 9 Project 127 emails I sent this week.
Boss: I haven't had time to get caught up on my email.
Me: You've been invited to a meeting with 3 department heads to hash out their differences on Project 249.
Boss: I hate meetings.
Boss: Why haven't you started the Accounts Receivable project yet?
Me: Because management has not yet decided whether customer credit limits should be per division or companywide.
Boss: What difference does that make?
Boss: We have hundreds of past due orders.
Me: No, we have 22 past due orders.
Boss: I'm not going to argue with you.
Me: Good, because you'd lose.
Me: We're meeting with the customer at 8:00 a.m. tomorrow.
Boss: I hate mornings.
Me: The server crashed. IT Services is working to bring it back up.
Boss: Don't confuse me with all these technical details.
Me: The customer didn't receive that information because that product is not on our computer.
Boss: Give me a list of all products not on our computer.
Boss: Why haven't you started Project 193 yet?
Me: Because the customer has not yet committed to the specs.
Boss: What difference does that make?
Me: The program was written with 3 SQL selects inside a loop. It ran OK when we had 500 parts. Now that we have 10,000 parts, it runs real slow.
Boss: I don't understand.
Boss: What are you working on?
Me: Project 432, which you said was my top priority. Remember?
Boss: No.
Boss: Why aren't you working on Project 387?
Me: Because you said not to work on anything else until Project 432 was complete. Remember?
Boss: No.
Boss: I'm giving you only enhancements. I'm outsourcing all of the bug fixes.
Me: But this is a bug fix. It says so right here on the ticket.
Boss: Oh, I didn't have time to read the ticket.
Boss: Amazon is threatening to shut us down because we ship too many orders late. How do we fix this?
Me: Ship every order on time.
Boss: No, I meant, "How do we fix this with software?"
Boss, on December 31: Write a program to close every work order so we make our year end numbers.
Boss, on January 3: Why is the database so screwed up?
Boss: You did great this year. I'm giving you a 2% increase.
Me: I hate you. I quit.
Boss: Then I'll give you a 4% increase.
Me: I still hate you. I still quit.
OP
sygeek
Technomancer
My Favorite Strange Number: Ω (classic repost)
By Mark C. Chu-Carroll
By Mark C. Chu-Carroll
I'm away on vacation this week, taking my kids to Disney World. Since I'm not likely to have time to write while I'm away, I'm taking the opportunity to re-run an old classic series of posts on numbers, which were first posted in the summer of 2006. These posts are mildly revised.
Ω is my own personal favorite transcendental number. Ω isn't really a specific number, but rather a family of related numbers with bizarre properties. It's the one real transcendental number that I know of that comes from the theory of computation, that is important, and that expresses meaningful fundamental mathematical properties. It's also deeply non-computable; meaning that not only is it non-computable, but even computing meta-information about it is non-computable. And yet, it's almost computable. It's just all around awfully cool.
So. What is it Ω?
It's sometimes called the halting probability. The idea of it is that it encodes the probability that a given infinitely long random bit string contains a prefix that represents a halting program.
It's a strange notion, and I'm going to take a few paragraphs to try to elaborate on what that means, before I go into detail about how the number is generated, and what sorts of bizarre properties it has.
Remember that in the theory of computation, one of the most fundamental results is the non-computability of the halting problem. The halting problem is the question "Given a program P and input I, if I run P on I, will it ever stop?" You cannot write a program that reads an arbitrary P and I and gives you the answer to the halting problem. It's impossible. And what's more, the statement that the halting problem is not computable is actually equivalent to the fundamental statement of Gödel's incompleteness theorem.
Ω is something related to the halting problem, but stranger. The fundamental question of Ω is: if you hand me a string of 0s and 1s, and I'm allowed to look at it one bit at a time, what's the probability that eventually the part that I've seen will be a program that eventually stops?
When you look at this definition, your reaction should be "Huh? Who cares?"
The catch is that this number - this probability - is a number which is easy to define; it's not computable; it's completely uncompressible; it's normal.
Let's take a moment and look at those properties:
Non-computable: no program can compute Ω. In fact, beyond a certain value N (which is non-computable!), you cannot compute the value of any bit of Ω.
Uncompressible: there is no way to represent Ω in a non-infinite way; in fact, there is no way to represent any substring of Ω in less bits than there are in that substring.
Normal: the digits of Ω are completely random and unpatterned; the value of and digit in Ω is equally likely to be a zero or a one; any selected pair of digits is equally likely to be any of the 4 possibilities 00, 01, 10, 100; and so on.
So, now we know a little bit about why Ω is so strange, but we still haven't really defined it precisely. What is Ω? How does it get these bizarre properties?
Well, as I said at the beginning, Ω isn't a single number; it's a family of numbers. The value of an Ω is based on two things: an effective (that is, turing equivalent) computing device; and a prefix-free encoding of programs for that computing device as strings of bits.
(The prefix-free bit is important, and it's also probably not familiar to most people, so let's take a moment to define it. A prefix-free encoding is an encoding where for any given string which is valid in the encoding, no prefix of that string is a valid string in the encoding. If you're familiar with data compression, Huffman codes are a common example of a prefix-free encoding.)
So let's assume we have a computing device, which we'll call φ. We'll write the result of running φ on a program encoding as the binary number p as &phi(p). And let's assume that we've set up φ so that it only accepts programs in a prefix-free encoding, which we'll call ε; and the set of programs coded in ε, we'll call Ε; and we'll write φ(p)↓ to mean φ(p) halts. Then we can define Ω as:
Ωφ,ε = Σp ∈ Ε|p↓ 2-len(p)
So: for each program in our prefix-free encoding, if it halts, we add 2-length(p) to Ω.
Ω is an incredibly odd number. As I said before, it's random, uncompressible, and has a fully normal distribution of digits. But where it gets fascinatingly strange is when you start considering its computability properties.
Ω is definable. We can (and have) provided a specific, precise definition of it. We've even described a procedure by which you can conceptually generate it. But despite that, it's deeply uncomputable. There are procedures where we can compute a finite prefix of it. But there's a limit: there's a point beyond which we cannot compute any digits of it. And there is no way to compute that point. So, for example, there's a very interesting paper where the authors computed the value of Ω for a Minsky machine; they were able to compute 84 bits of it; but the last 20 are unreliable, because it's uncertain whether or not they're actually beyond the limit, and so they may be wrong. But we can't tell!
What does Ω mean? It's actually something quite meaningful. It's a number that encodes some of the very deepest information about what it's possible to compute. It gives a way to measure the probability of computability. In a very real sense, it represents the overall nature and structure of computability in terms of a discrete probability.
Ω is actually even the basis of a halting oracle - that is, if you knew the value of Ω, then you could easily write a program which solves the halting problem!
Ω is also an amazingly dense container of information - as an infinitely long number and so thoroughly non-compressible, it contains an unmeasurable quantity of information. And we can't even figure out what most of it is!
Ω is my own personal favorite transcendental number. Ω isn't really a specific number, but rather a family of related numbers with bizarre properties. It's the one real transcendental number that I know of that comes from the theory of computation, that is important, and that expresses meaningful fundamental mathematical properties. It's also deeply non-computable; meaning that not only is it non-computable, but even computing meta-information about it is non-computable. And yet, it's almost computable. It's just all around awfully cool.
So. What is it Ω?
It's sometimes called the halting probability. The idea of it is that it encodes the probability that a given infinitely long random bit string contains a prefix that represents a halting program.
It's a strange notion, and I'm going to take a few paragraphs to try to elaborate on what that means, before I go into detail about how the number is generated, and what sorts of bizarre properties it has.
Remember that in the theory of computation, one of the most fundamental results is the non-computability of the halting problem. The halting problem is the question "Given a program P and input I, if I run P on I, will it ever stop?" You cannot write a program that reads an arbitrary P and I and gives you the answer to the halting problem. It's impossible. And what's more, the statement that the halting problem is not computable is actually equivalent to the fundamental statement of Gödel's incompleteness theorem.
Ω is something related to the halting problem, but stranger. The fundamental question of Ω is: if you hand me a string of 0s and 1s, and I'm allowed to look at it one bit at a time, what's the probability that eventually the part that I've seen will be a program that eventually stops?
When you look at this definition, your reaction should be "Huh? Who cares?"
The catch is that this number - this probability - is a number which is easy to define; it's not computable; it's completely uncompressible; it's normal.
Let's take a moment and look at those properties:
Non-computable: no program can compute Ω. In fact, beyond a certain value N (which is non-computable!), you cannot compute the value of any bit of Ω.
Uncompressible: there is no way to represent Ω in a non-infinite way; in fact, there is no way to represent any substring of Ω in less bits than there are in that substring.
Normal: the digits of Ω are completely random and unpatterned; the value of and digit in Ω is equally likely to be a zero or a one; any selected pair of digits is equally likely to be any of the 4 possibilities 00, 01, 10, 100; and so on.
So, now we know a little bit about why Ω is so strange, but we still haven't really defined it precisely. What is Ω? How does it get these bizarre properties?
Well, as I said at the beginning, Ω isn't a single number; it's a family of numbers. The value of an Ω is based on two things: an effective (that is, turing equivalent) computing device; and a prefix-free encoding of programs for that computing device as strings of bits.
(The prefix-free bit is important, and it's also probably not familiar to most people, so let's take a moment to define it. A prefix-free encoding is an encoding where for any given string which is valid in the encoding, no prefix of that string is a valid string in the encoding. If you're familiar with data compression, Huffman codes are a common example of a prefix-free encoding.)
So let's assume we have a computing device, which we'll call φ. We'll write the result of running φ on a program encoding as the binary number p as &phi(p). And let's assume that we've set up φ so that it only accepts programs in a prefix-free encoding, which we'll call ε; and the set of programs coded in ε, we'll call Ε; and we'll write φ(p)↓ to mean φ(p) halts. Then we can define Ω as:
Ωφ,ε = Σp ∈ Ε|p↓ 2-len(p)
So: for each program in our prefix-free encoding, if it halts, we add 2-length(p) to Ω.
Ω is an incredibly odd number. As I said before, it's random, uncompressible, and has a fully normal distribution of digits. But where it gets fascinatingly strange is when you start considering its computability properties.
Ω is definable. We can (and have) provided a specific, precise definition of it. We've even described a procedure by which you can conceptually generate it. But despite that, it's deeply uncomputable. There are procedures where we can compute a finite prefix of it. But there's a limit: there's a point beyond which we cannot compute any digits of it. And there is no way to compute that point. So, for example, there's a very interesting paper where the authors computed the value of Ω for a Minsky machine; they were able to compute 84 bits of it; but the last 20 are unreliable, because it's uncertain whether or not they're actually beyond the limit, and so they may be wrong. But we can't tell!
What does Ω mean? It's actually something quite meaningful. It's a number that encodes some of the very deepest information about what it's possible to compute. It gives a way to measure the probability of computability. In a very real sense, it represents the overall nature and structure of computability in terms of a discrete probability.
Ω is actually even the basis of a halting oracle - that is, if you knew the value of Ω, then you could easily write a program which solves the halting problem!
Ω is also an amazingly dense container of information - as an infinitely long number and so thoroughly non-compressible, it contains an unmeasurable quantity of information. And we can't even figure out what most of it is!
OP
sygeek
Technomancer
Cars Kill Cities
Cars Kill Cities
OK, I’m finally getting a chance to make another post. I have temporarily relocated to Mountain View, CA and have been up to my eyeballs in work, both ‘real’ work and research work. It’s nice to get back to this blog.
Cars do not belong in cities. A standard American sedan can comfortably hold 4+ adults w/ luggage, can travel in excess of 100 miles per hour, and can travel 300+ miles at a time without stopping to refuel. These are all great things if you are traveling long distances between cities. If you are going by yourself to pickup your dry cleaning, then cars are insanely over-engineered for the task. It’s like hammering in a nail with a diesel-powered pile driver. To achieve all these feats (high capacity, high speed, and long range driving), cars must be large and powered by fossil fuels. So when you get a few hundred (or thousand) cars squeezed onto narrow city streets, you are left with snarled traffic and stifling smog.
Even if you ignore the pollution, cars simply take up too much space. Next time you are stuck in traffic behind what seems like a million cars, try to imagine if all those cars where replaced by pedestrians or bike riders. Suddenly, the congestion is gone.
But why am I complaining about traffic? Traffic only affects those stuck in it, right? Once all cars go electric, essentially eliminating inter-city air pollution, then there will be no more problems for pedestrians, right? Wrong!! Probably the biggest problem with cars in cities is that they require huge amounts of land for storage (a.k.a. parking). Here is a photo of Midtown Atlanta between 5th street and 12th street. This is one of the densest and most pedestrian-friendly ares in the entire state of Georgia. The red blocks indicate parcels of land that are 100% dedicated to car storage.
Dedicating all this land to car storage basically reduces the density by about half, doubles the average distance between locations, and reduces walkability. Throw in the 16-lane interstate and the 45+ mph traffic on most of these streets, it becomes exceedingly hard to believe that this is one of the most walkable areas in the entire state. Such is life for pedestrians in a car-dominated city.
It wasn’t always this way. Atlanta, like all cities, used to be walkable and people actually lived IN the city instead of commuting 50 miles every day. But as more people moved away from the city, the more Atlanta had to become like a suburb, being retrofitted to handle all the automobile infrastructure required by a million 40 hour-a-week temporary citizens. The result of this retrofit is a wasteland of asphalt and isolated neighborhoods, a slow decimation that has rolled along since the innovation of the automobile.
Contrary to how it may sound, I do not want to rid the earth of cars. I just want to use them smarter. Do you really need a 2-ton vehicle to pickup your dry-cleaning? Probably not. Although I do see the appeal in loading a family of 6 into an SUV and traveling to Florida for vacation. That is a totally reasonable use of an automobile. What I really want is clean, walkable, safe, affordable, and family-friendly cities and towns. In a strange way, I kind of want to live in Mayberry.
In the next post*, I promise to discuss a few ideas that may get us a little closer to this goal.
*The thread will be updated when the next post is up
Cars do not belong in cities. A standard American sedan can comfortably hold 4+ adults w/ luggage, can travel in excess of 100 miles per hour, and can travel 300+ miles at a time without stopping to refuel. These are all great things if you are traveling long distances between cities. If you are going by yourself to pickup your dry cleaning, then cars are insanely over-engineered for the task. It’s like hammering in a nail with a diesel-powered pile driver. To achieve all these feats (high capacity, high speed, and long range driving), cars must be large and powered by fossil fuels. So when you get a few hundred (or thousand) cars squeezed onto narrow city streets, you are left with snarled traffic and stifling smog.
Even if you ignore the pollution, cars simply take up too much space. Next time you are stuck in traffic behind what seems like a million cars, try to imagine if all those cars where replaced by pedestrians or bike riders. Suddenly, the congestion is gone.
*i.imgur.com/WmUbb.jpg
But why am I complaining about traffic? Traffic only affects those stuck in it, right? Once all cars go electric, essentially eliminating inter-city air pollution, then there will be no more problems for pedestrians, right? Wrong!! Probably the biggest problem with cars in cities is that they require huge amounts of land for storage (a.k.a. parking). Here is a photo of Midtown Atlanta between 5th street and 12th street. This is one of the densest and most pedestrian-friendly ares in the entire state of Georgia. The red blocks indicate parcels of land that are 100% dedicated to car storage.
*i.imgur.com/Uu4Qs.jpg
Dedicating all this land to car storage basically reduces the density by about half, doubles the average distance between locations, and reduces walkability. Throw in the 16-lane interstate and the 45+ mph traffic on most of these streets, it becomes exceedingly hard to believe that this is one of the most walkable areas in the entire state. Such is life for pedestrians in a car-dominated city.
It wasn’t always this way. Atlanta, like all cities, used to be walkable and people actually lived IN the city instead of commuting 50 miles every day. But as more people moved away from the city, the more Atlanta had to become like a suburb, being retrofitted to handle all the automobile infrastructure required by a million 40 hour-a-week temporary citizens. The result of this retrofit is a wasteland of asphalt and isolated neighborhoods, a slow decimation that has rolled along since the innovation of the automobile.
Contrary to how it may sound, I do not want to rid the earth of cars. I just want to use them smarter. Do you really need a 2-ton vehicle to pickup your dry-cleaning? Probably not. Although I do see the appeal in loading a family of 6 into an SUV and traveling to Florida for vacation. That is a totally reasonable use of an automobile. What I really want is clean, walkable, safe, affordable, and family-friendly cities and towns. In a strange way, I kind of want to live in Mayberry.
In the next post*, I promise to discuss a few ideas that may get us a little closer to this goal.
*The thread will be updated when the next post is up
nisargshah95
Your Ad here
Re: Cars Kill Cities
Long time since your last post dude. Looking for some more articles...
Long time since your last post dude. Looking for some more articles...sekhar.mld
Broken In
The articles are good.
I read few.
It is always good to put some good facts and ideas into your brain, makes it more active.
I read few.
It is always good to put some good facts and ideas into your brain, makes it more active.