Category — tech
July 29, 2013 3 Comments
ECHELON is a code word for an automated global interception system operated by the intelligence agencies of the United States, the United Kingdom, Canada, Australia, and New Zealand, and led by the National Security Agency (NSA). I’ve seen estimates that ECHELON intercepts as 3 billion communications every day, including phone calls, e-mail messages, Internet downloads, satellite transmissions, and so on. The system gathers all of these transmissions indiscriminately, then sorts and distills the information through artificial intelligence programs.
Bruce Schneier, Secrets and Lies,2004, 2nd ed.
July 15, 2013 No Comments
Drew Crawford, in a long but well-researched essay on mobile app performance:
July 10, 2013 No Comments
A recent episode of the Planet Money podcast profiled Thomas Peterffy, one of the first people to experiment and be successful with high-frequency trading. They told the story of how he was doing algorithmic trading before any of the stock exchanges supported electronic trading, and before NASDAQ even existed. So how did he do it? That’s the fascinating part.
He made his money building a system that was able to assign a fair market price to stock options. He then compared these values to what the options were actually trading for, and arbitraged the difference. Back in the late 1970s when he first started, he would print out the numbers and bring them to the trading floor in a huge binder. When the stock exchange banned him from bringing the binder, he stuffed the papers into every pocket his suit had.
Then Peterffy got himself a system called Quotron, a computerized service that delivered stock prices to brokers (it was a replacement for the widely-used ticker tape system). If he’d used the system the way it was intended, he would’ve read the quotes as they came in on the Quotron, manually input them into his algorithm, run the numbers, and cashed in. But that wouldn’t have been that much better than just using ticker tape, and the fact that he had a computerized system meant the data was in there somewhere, in digital form. If he could figure out how to retrieve it he could pipe it into his system and save a crucial, time-consuming step.
Nowadays if we wanted to do something similar, we might look into whether the Quotron had an API, and if it did we’d query that for the information. If it didn’t have an API, well, we might look for another system that did.
But Quotron had no such ability. So he did what any hacker worth his salt would do. He broke out his oscilloscope, cut the wires on the Quotron, reverse-engineered the data signal, and patched it into his system. And you think screen-scraping is hard?
When NASDAQ, the first all-electronic stock exchange, came online, he was faced with a similar system. Brokers could trade directly on the exchange via computer. This was no doubt a huge breakthrough, but there was still no way his system could make the trades automatically. So, again, he busted out his oscilloscope and patched his way into NASDAQ.
We developers could learn from Peterffy. The ease of software engineering has made most of us too complacent. When Twitter’s API terms change, we complain about it for a few days, and then change our business models to suit the new rules. But the real innovation, the real interesting stuff, the way we’ll make $5.4 billion like Peterffy did, is by bending the rules and building systems that give us a leg up on the competition, or, better yet, improve people’s lives.
To be sure there are lots of hackers on the fringes of legality doing very interesting things, but the rest of us are somehow content to toe the line. We shouldn’t do anything that’s illegal, but we should get close. Innovation comes out of spurning the status quo, not complying with it. It’s time for people who know how to build things to bend the rules a little, and see what comes out the other side.
(The podcast was based on Peterffy’s story as told in the book Automate This: How Algorithms Came to Rule Our World.)
September 13, 2012 2 Comments
One of the hardest things for any software designer to do is to decide not to implement a feature. Many software projects have been delayed or even derailed by feature creep, or the tendency to widen the scope of a project during development. But in many cases, features that seem like “must-haves” during development can be deferred to later phases of development, or cut completely.
Today I just ran into another example, also from Apple. In Xcode, you can switch from a header file to its corresponding implementation file (and back) using the keyboard shortcut Command-Control-Arrow (any arrow). This is a really nice way of navigating back and forth while you’re creating new instance variables and methods for your classes. However, when you navigate in this way, the project browser at the left doesn’t update its highlight to indicate that you’re viewing a different file. Is this a bug? Probably not. It’s probably just the designers of Xcode deciding to rein in feature creep so that they can actually ship the product.
It’s so damn tempting to want to make sure every little bug is fixed and every little corner case is accounted for before you release your software. But, as they say, perfect is the enemy of the good. It’s crucial to know when something is good enough so you can ship it as soon as possible. With cut, copy, and paste, Apple finally introduced the feature into its third version of the iPhone’s operating system. By then they had already sold millions of phones to customers who decided they could live without that crucial feature.
March 14, 2012 No Comments
March 1, 2012 No Comments
I woke up today to this provocative article in The Guardian about how graphic designers are ruining the web. Naughton’s main argument seems to be that graphic design adds unnecessary bulk to websites, wasting bandwidth. Naughton is absolutely right that page sizes have increased over the last two decades of the web’s existence. He is also right that this is a problem.
However, he describes the problem as a “waste of bandwidth.” Last I checked, “bandwidth” is an infinite resource (unless maybe you extrapolate bandwidth to barrels of oil). The bigger problem is that more elements on a page (and bigger individual elements) will slow down page load times and potentially be frustrating for the user. If Naughton is saying that people who make websites should work to reduce the number and size of the elements on their pages, I completely agree.
But it does not then follow that websites also need to be ugly (he uses Norvig.com as an example of an underdesigned site that is compelling for its content if not its look and feel). Highly-designed websites need not be bulky. Just because the BBC News page sends 165 resources on every request to its homepage, doesn’t mean all designed sites do. NPR.org is a lean and mean website, requiring roughly 50% fewer requests than the BBC News. Yet I would say it offers a bit more of a user-friendly way to access information than Norvig’s site.
I’ll agree that some underdesigned sites are excellent because they are underdesigned: Craigslist.org and (the original) Google.com. But if Apple has taught us anything over the past decade, it is that things can be designed without being complicated and bulky. And that is the direction I’d like to see the web going in. That way we get to have our cake and eat it too.
- Graphic designers are ruining the web (guardian.co.uk)
February 19, 2012 8 Comments
My friend Michael McWatters tweeted his frustration today that there is no way change your Twitter password on their mobile site. I’ve butted up against this issue in the past, and the fact that you can’t even switch between the mobile and full site on their is immensely annoying (in fact, there isn’t even a footer on the site!).
With smartphone penetration growing ever higher, it’s increasingly important for companies not just to build mobile sites, but to build them well. Mobile sites can no longer play second fiddle to their desktop brethren. Over the past few months I’ve become increasingly sensitive to, and bugged by, the degree to which so many mobile sites are so badly implemented. With that in mind, here are my 5 “worst” practices of the mobile web.
- Don’t give users the choice of using the full site – not letting users choose to use the full site on their mobile device is presumptuous at best, and crippling at worst. Just because the screen is small doesn’t mean you don’t want to be able to access all of a site’s features in a pinch. On the iPhone anyway, browsing a full website is often very tolerable and should at least be an option for users. This is related to #2, which is…
- Don’t cripple your mobile site – while it may be true than on a dumb phone you likely do not need or want to access all of a site’s features on the go, on a smartphone you often do. A mobile site no longer needs to be a list of the 10 most visited pages on a site. Let’s start building mobile sites that allow access to some more advanced features like changing your password.
- Show an interstitial ad for your mobile app – have you ever clicked on a link on your phone only to be brought to an interstitial ad for a site’s mobile app, instead of the article you were trying to read? And of those times, how many times have you gone immediately to download the app instead of just closing out the ad and trying to read the article you were interested in?
- Don’t redirect from your mobile domain to the full site on a desktop browser – many sites with mobile domains will redirect you to it using browser detection. But many of those do not do the reverse redirect (i.e., visiting the mobile site on a desktop browser doesn’t redirect back to the full version). Being forced to view a mobile site on a desktop browser is torture.
- Redirect to your mobile domain, but not the specific page – all this redirecting has its place, but it’s so easy to get it wrong. On many occasions I have clicked on a link on my phone, gotten redirected to a mobile domain, and instead of it going to the article I was trying to read, I get placed on the homepage of the mobile site. So frustrating!
The mobile web is certainly in its infancy, but that’s no excuse for giving users such broken experiences. It’s 2011 and it’s imperative now that mobile sites are just as beautiful, simple, and elegant as the devices used to navigate them. If you have to choose between offering a mobile site that suffers from any of the worst practices listed above, and having no mobile site at all, choose the latter.
August 29, 2011 No Comments
My family and I haven’t watched “TV” in weeks. Granted, we don’t have cable (we use rabbit ears and a digital-to-analog converter box), but that’s not really the reason we haven’t been watching. The real reason is that Netflix instant streaming has changed our lives.
With the sheer volume of quality content that Netflix has (as well as other online video sites like Hulu), we are now at the point where we don’t really need to watch actual television. We are getting close to a point I like to call “TV Zero.”
By “TV Zero,” I don’t mean turning off all your screens and moving to Montana. I simply mean disconnecting from television as we know it (scheduled programs grouped into broadcast networks). I truly believe that, no matter how much the cable companies and networks drag their feet over the next few years, it’s just a matter of time before all programming formerly available on cable or over-the-air broadcast will be available on the internet. The experience is so much better.
For one thing, video over the internet is truly demand-based. I can watch any episode I want, at any time I want. For another thing, finding content is far easier, and has far more potential, than the current model that cable tv uses. Netflix can recommend shows I may never have heard of, based on what it already knows about my consumption habits. The array of content available is also more vast–services like Netflix can offer back catalogs of content providers with much lower incremental cost than, say, a cable company. In fact if you think about it, it’s kind of shocking that after 15 years of the “commercial internet” we’re still only in the early stages of this.
And then there’s all the recent buzz about Apple making a “smart tv.” If the rumors are true (and I believe they are , for the good reasons outlined here), the acceleration of our culture toward “TV zero” could increase tremendously. The potential for disruption and innovation in this space is huge, and in my opinion inevitable, and there’s no company in a better position to lead this change than Apple. But if Apple won’t do it, then someone else will. (Amazon? Google?)
One thing is certain, though: the cable companies will not go down without a lot of kicking and screaming. Unless someone in their ranks realizes the inevitability of this change, and figures out a way to profit madly from it.
- Where Is The U.S. Watching Online Video & How Often? [Infographic] (searchenginewatch.com)
- Legal this time? Startup offers local TV on the ‘Net, with a twist (arstechnica.com)
- Will streaming TV online lead to the death of the big media players? (guardian.co.uk)
- How To Cut The Cable And Watch TV Without A Monthly Fee (lockergnome.com)
- Cord Cutting Will Slow, But Continue to Grow (gigaom.com)
April 18, 2011 2 Comments
On Wednesday at the Launch Conference, travel search engine Hipmunk presented a new mobile version of their web app. But that’s not what I want to talk about. I want to talk about Hipmunk’s general approach to solving the problem of airfare search, and how it might be applied to other problems.
The genius of Hipmunk is in their “agony” algorithm, grounded in the key insight that when people search for airfares, price and departure time are rarely the only considerations. What people really want to know is how agonizing the trip will be, measured as a combination of price, duration, and number of layovers. So Hipmunk sorts your search results by this “agony” score (in descending order of course). Simple. Brilliant. There’s so much agony in the world; what else could this model be applied to?
The first thing that jumps to mind is turn-by-turn directions. Most navigation apps provide routes that optimize for distance or time, and in some cases by real-time traffic patterns. But there are a lot of other factors than can contribute to one’s agony while driving. For instance, given the choice, I’d much rather drive a scenic route than an interstate, but likely only if the scenic route isn’t orders of magnitude more time-consuming. Or maybe I’d like to drive a route with better food options than Shoney’s and Roy Rogers. Transit directions could also benefit from applying this model. I’d much rather take a trip that involved a transfer if the two subways were less crowded than the one, provided the trip duration wasn’t significantly longer.
Another great application of the “agony” model would be a site that helped you decide whether or not buy something online or at a nearby store. The algorithm could factor in a combination of item cost, shipping cost, shipping duration and return policy of the online option, and compare it to the item cost and travel distance to a local store that carries the item, as well as the real-time availability of that item in the store’s inventory (Milo.com is working on this last problem).
Sorting by “agony” factor is a powerful idea, and one that is quickly letting Hipmunk soar to the top of the travel search business. What other problems could you apply this model to?
February 25, 2011 No Comments