Monthly Archives: April 2009

The death of the Internet poll

This was long predicted and actually happened a long time ago but now the mainstream media is starting to figure it out.

Time magazine just found out that 4chan’s moot is the world’s most influential person.  One wonders if Time really understands how comprehensively they were hacked.  They went as far as to write about Will this stop them from using polls on the net?  I doubt it.  Polls seem to be the highest form of interactive news out there (sad indeed).

About once a week a silly poll surfaces on reddit (or some other community).  The latest is the Des Moines Register asking about gay marriage.  The community then mobs the poll.  This is the power of the slashdot effect.  It doesn’t require clever hacking like Anonymous did for the Time poll.

Some systems try to improve the value of voting by having some kind of registration.  This prevents trivial spoofing but is still very weak.  Large active online communities like reddit and 4chan can utterly swamp any voting system.  The more embarrassing the outcome for the poll sponsor the more likely that someone will mob it.

Ultimately this all comes back to the fact that identity is a remarkably vague concept.  The only reason “real” voting works is because it requires multiple factors and most importantly physical presence.

Anderson’s law

Stephen posted a thoughtful comment to my complaint.  I spent some time reading a few more of his posts.  I’ll re-iterate: he is clearly a craftsman.

I totally agree that marketing is an essential part of building a successful product and I really do like Stephen’s approach.

Looking deeper, my negative impression was in no small part a reaction to slideshare.net.  Why does this site exist?  It has executed on many of the formulae Stephen advocates in order to build a successful community and from that a successful business.  The result is an deep resemblance to YouTube.  And yet, at it’s core Slideshare is a PowerPoint to Flash converter.  Why isn’t this just an export plugin to PowerPoint combined with any existing content publishing site. The answer of course, is in Stephen’s post about value centered design.  Nobody can make money selling productivity software let alone a plugin.  The alternative is the advertising supported social networking pattern.

LinkedIn and iLike might have more value at their core but both have succumbed to a variation of the Zawinski’s Law of Software Envelopment: the Law of technological mundanity.  I begrudgingly use LinkedIn (and Facebook) as nothing more than a glorified address book.  Although friends of mine created iLike I must admit that iDon’tLike.  LinkedIn was created in the dawn of social networking and iLike has evolved into the same space out of necessity.

I really like what Stephen says about value centered design diagram.  I think it really conveys the necessary compromise between business goals and user goals.  It explains very well why so often the products available in the market leave end-users frustrated.  End-users often don’t think about the business needs.  Through Stephen’s eyes I can see how every product will eventually have (or need) a rich business ecosystem built around it.

I’d like to propose a draft of Anderson’s Law:

Every internet business must expand to create the full ecosystem of experiences.  Those which cannot so expand are replaced by ones which can.

I’m not happy with the phrase “full ecosystem of experiences” so I’m hoping that Stephen can sum it up better than I.

It’s interesting that free software can address simple problems (like sharing or converting PowerPoint slideshows) without needing to create a full internet business to support itself.  However, most free software suffers from the lack of good marketing.  Only when the software is truly great is word of mouth sufficient.

When all is said and done I am an engineer.  What I value are applications that, well, do something.  Perhaps I’m old school and stuck on “productivity apps” from the past.  With that in mind I came up with a few “real world” applications that I think would benefit from Stephen’s treatment.

The first is mint.com.  Overcoming my tin-foil-hat paranoia wasn’t easy but I created an account.  I’ve been pleasantly surprised so far but I know they could do a better job of seducing me.

The second is jumpchart.com.  I discovered it somewhat randomly and while it looks pretty slick, it’s definitely not a seductive application.  I’d like to believe that there’s some real value hidden inside.

The third is instacalc.com.  I found this in a list of most useful apps of 2008.  I’ll be honest, I didn’t find it that useful and there simply may not be enough meat on the bone here.

I’m looking forward to more conversation with Stephen.  I am much the richer for having discovered his post and then his follow-up.  Thanks!

Dangerous at any size

URL shortening services are all the rage. This is no doubt driven by increased use of Twitter and the need to deal with its 140 character limit.

This has of course generated a gold rush of investment in URL shortening services. It’s hard to see how these investments can really pay off.

As people use these services more and more the online community is waking up to their problems:

  • Single point of failure
  • Perfect way to mask SPAM
  • Delegation of trust
  • Delegation of control

The potential for abuse and mischief and outright malign intent is spectacular.  How long before someone implements an URL shortening service with a worm or SPAM Trojan?

The response has been predictably lame.  Now there are un-shortening libraries and even web services.  What a classic case of the solution being worse than the cure.  How do users deploy such solutions without falling afoul of exactly the same risks?

Jeff Attwood thinks Google should become the benevolent provider.  Hah!  The absolute last thing anyone should want is to put more trust and dependency in Google.  There’s no small irony in the fact that it is Google’s predilection for URLs that match the page title that has contributed to this problem.

The solution isn’t in the browser either.  Greasemonkey scripts that try to resolve the URLs are still subject to the same mischief as the user and represent yet another attack surface for the malign shortener.

The new hotness is the rev=”canonical” idea.  The concept is simple enough.  When you discover an URL on a page it self-describes a shortened version of itself, deemed canonical.  It sounds great but it isn’t.  Imagine if I shortened all the URLs on this page and specified URLs on my site (or even a shortener I choose to use).  I’m effectively claiming to represent the canonical URL for someone else’s content.  This goes from delegation of control to inversion of control.  How does the recipient know that server declaring the relationship is authoritative?  If the original URL is relative then it’s a safer bet but still not guaranteed.  Anything else and we’ve simply moved the problem around.  Perhaps I’m reacting to the word canonical.  I think it would have made more sense to simply say rel=”shorter” and not pretend that it is authoritative.  Maybe I’m missing something here, feel free to correct me.

Applications whose implementation details impose a limit on URLs should have an internal URL shortening service.  If Twitter wants to optimize the size of their packets then they can and should do so internally.  They don’t even need to buy an expensive 5 letter domain.  Instead of http://stup.id/ahk1n they can store !ahk1n.  The user edits URLs normally, the editor converts them when posting and the view converts them back.  Twitter users can continue to trust Twitter as much (or as little) and the solution doesn’t increase any of the risks listed above.  There’s reason to believe that flickr is working along these lines.

Instead of having rev=”canonical” on each link why not have a shortening service on each domain (or even URL space).  The service would convert URLs to and from long and short form in a way that truly is canonical.  The service could be declared once on the page.  A page that links to other domains could, at its discretion, discover the shortened URLs and use those.  Users would trust them no more or less than the full URL but considerably more than http://tinyurl.com/dkkweg.

The Art & Science of Seductive Interactions: So Right and Yet So Wrong

I came across this compelling, visual and manipulative presentation today.

If you’re building yet another social community site then this presentation is for you.  Stephen Anderson makes a good case for the use of human psychology in making your site attractive, viral, sticky and popular.  There’s no doubt that he knows what he’s doing.

What irks me about this presentation is how value is deprecated.  This isn’t about software that is valuable to the user.  It’s about teasing, cajoling, entertaining and seducing the user will help you meet your business goals.  Business goals of what?  Selling advertising to an arbitrarily segmented set of users?  Making the experience of buying shoes more fun?

When I think about the amount of investment and creativity that goes into sites that are ultimately specialized versions of some existing application I see a sad future for civilisation.

Even the slideshare site used to host the slideshow is an example of this.  They have dutifully implemented many of the seductive schemes he writes about.  The result is the slideshow equivalent of YouTube.  Why?  Why isn’t sharing a slideshow just a little widget and an export utility in Powerpoint or Keynote?  What about Why isn’t it an authoring tool for a standards based html slideshow?

It would be great to see Stephen Anderson’s skills in action on a more worthy purpose.  I think there is true value in scripting the user experience so that users aren’t overwhelmed, so they can immediately see the value your application provides and gradually learn how to use its full power.

To all the VCs out there: if the primary value add of a business is it’s slick marketing then please, look elsewhere.

Looking ahead to Hallowe’en

My friend John Ludwig loves to create amazing effects for Hallowe’en. He uses a lot of technology and has a lot of fun putting on a show. It’s the 2nd biggest tag on his site.

My wife not only considers Hallowe’en a national holiday, she thinks it’s the best holiday of the year.

It’s not really my thing but I just came across the ideal food for that evening. Stick dry pasta through hot dogs and then boil. Mixing colours and experimenting with angles and patterns can make for some truly disgusting creations.

Looking for the Fit / Fitnesse approach to GUI testing

I’ve been learning quite a bit about test frameworks, BDD and TDD lately. I saw Cucumber and Webrat in action at a startup led by a colleague and was very impressed. For HTML applications it’s definitely the way to go.

Several of my colleagues have a distrust of test languages like Cucumber and RSpec. Their point is that it’s better to write code in a real language. I understand their point but generally disagree. The value of Cucumber is that the language is deliberately very abstract, lacking in any kind of state and most definitely not Turing complete. This makes it easier to talk about, easier to re-use and prevents any crufty code sneaking into a spec.

Now I’m looking for BDD test frameworks for desktop GUI applications. So far I’ve been disappointed. This article tries to make the case that Fitnesse is applicable here but if anything it makes me 100% sure that FIT doesn’t fit. The sample table isn’t a set of declarations, it isn’t particularly human readable. It’s just a crappy programming language in disguise. If this is how people use BDD then I agree with my colleagues, I’d rather just write the tests in code.

The truth about running shoes

I’ve read articles like this one before.

When I played Ultimate I found that I was much better off running in my cleats (no padding) than running shoes.

I even bought a pair of shoes based on what I learned.

The results are ultimately up to you. Unpadded shoes won’t let you run further, harder or better. They’ll help you stay in touch with your body’s feedback and help you detect minor injuries and bad mechanics before you really hurt yourself.

Why blogging really matters

I noticed on Rick’s blog that the FTC is starting to look at whether blogger recommendations should incur liability.  This is a very dangerous direction to go.  Why does blogging need to be called out as a particular form of speech?  Ultimately I think what matters is the same as for dead tree journalism: transparency and ethics.  My concern however, is that increased regulation will start us down a road that treats every blog the same as a newspaper.

Blogging is important because it decentralizes and undermines the power that “the media” have held for a long time.  For the very same reasons that the Soviet Union controlled printing presses and photocopiers so must we go out of our way to stifle individual publishing rights.  Dave Winer has understood this for a long time.  I’m actually surprised he missed this one. 

What I write in this blog is not important.  It’s a nice way for me to compose a few thoughts and flatter myself that my readers, both of them, are interested.  What is important is that I am just one of thousands of citizen journalists exercising our first amendment rights.

Creating regulatory burdens on blogging will create a world of haves and have nots.  Just as the capital to run a printing press divided the people from the power, so can the regulatory hassles imposed by the FTC.

We don’t need new regulations or laws here.  We just need to enforce the ones we have.  I don’t envy the FTC the balancing act they must perform between 1st amendment rights and the desire to protect the people from misleading advertising.  I think it’s a mistake to believe that they can regulate the problem away.  Ironically the solution is actually to encourage more bloggers.  The blogosphere has always been better and faster at catching deceptive practices anyway.

Undertipping

At lunch today we had a less than perfect experience.  Four of us from work went to one of our favorite restaurants.  The waiter was new.  My friend Dave had a coupon for $10 off.  He carefully confirmed its validity with our waiter.  Our food took unusually long to come and when it did, it was the wrong order.  The waiter had heard salad special instead of salmon special (not once but twice).  Neither Dave nor I wanted to wait for our original order so we ate the salad.  it was OK.  Then it took unreasonably long to get the cheque.  Then it turned out that the coupon only worked if you bought two beverages.  So we paid for two, it was still worth it.  Then Dave’s coupon was incorrectly applied, twice.  I think our aggravation was compounded not only by the cascading problems but by the lack of the-customer-is-always-right gestures.

I tend to tip quite well, typically 20% for normal service.  After the debacle at lunch we were discussing how much to tip.  One of us was fuming and didn’t want to tip at all.  We ended up tipping 10% after the coupon had been subtracted from the total.  On one hand I felt bad because I know how much waiters depend on tips but on the other hand, I really felt that even 15% would be too high.  I’m rethinking how much I will tip in the future.  The maximum will still be 20% but I’m going to save that for truly exceptional service.  At the low end, below 10% is for willfully bad service.  In between there are ample opportunities for me to exercise my mental math.

Michael’s rules of component software

Why do we componentize software?  Clemens Szyperski‘s book makes no bones about it:

“Software components enable practical reuse of software ‘parts’ and amortization of investments over multiple applications.  There are other units of reuse, such as source code libraries, designs or architectures.  Therefore to be specific, software components are binary units of independent production, acquisition and deployment that interact to form a functioning system.  Insisting on independence and binary form is essential to allow for multiple independent vendors and robust integration”

Clemens’ book goes on to examine the different component systems like CORBA, DCOM, JavaBeans and more.

I’d say that in most people’s minds, the goal of component software is to approximate the Lego experience.  We want to assemble building blocks to make something.  Of course there’s a difference, we also think of ourselves as Lego block creators able to whip up a new component at will.  This is probably the first mistake.  Designing components is 10 times harder than using them.  I’ll extend that to say that designing component frameworks is 10 times harder than designing components.

My approach to software engineering is that for most problems the highest priority is maintainability of the code.  As such my radar is always watching for complexity that has little to do with the problem and a lot to do with the plumbing.  In my experience, most component frameworks look simple at first but hide a ton of complexity that is all about the plumbing.  It’s relatively easy to use a COM component, not hard to build one but things get messy very soon after that.  COM has a lot of rules which is another way of saying that COM has a lot of patterns and anti-patterns that are easy to get wrong.

When people confuse object oriented languages for component architectures then things get even worse.  If the size of the rule book is an indication of the potential for problems then C++ surely takes the cake.  No C++ developer ever truly knows all the rules and there is often dissent about the rules in the first place.  C++ works best when the development team has access to all the source, can debug through every class and someone is thinking about the whole.

The C++ problem is actually at the root of why I think most component architectures are flawed.  Many of them have roots in object oriented programming environments like C++.  Most of them have the word “object” in their name somewhere.  Designers wanted to extend object oriented programming concepts and beliefs into a Lego compositional model of software.

Remember when Windows DLLs were considered components?

Anyone who has written code for a while knows two things: we all pine for a Lego composition model and it doesn’t exist yet.

Object oriented programming, modularization, encapsulation are all code organization conventions intended to make large code projects easier to manage.  The shared theme is a good one, we can do great things if we can abstract messy details away.

Component software is at its worst when developers start to believe they can upgrade components in place without affecting dependent applications or components.  I wish I could explain in formal terms what experience has shown me over and over again.  The more complex the interactions between the components or the more complex the change the more likely it is that it will have unexpected consequences.  Consumers of components consume not just the published interface but the real-life behaviour.  Every bug and timing characteristic is a part of the contract between components and every security fix, upgrade and improvement runs the risk of changing that.

  • I’ve been bloodied and humbled by my own efforts in this space.  I’ve worked on frameworks, components and applications and learned some hard lessons along the way.  The outcome of this is what I call Michael’s rules of component software:

The goal of components is to allow isolated and stable modules of software that were created independently to be composed to create a solution.

Components are:

  • Created independently
  • Testable on their own
  • As useful as they are stable

I believe that there are only three good ways to compose components:

  1. Files. When two components communicate via a file then the file format becomes the interface.  It’s super easy to test and debug and it’s very hard to introduce time sensitive behaviour.  Obviously a lot depends on the file format.  Simple text files are better.  Sending code (even when disguised as xml) via files is cheating.
  2. Command lines. Sometimes files aren’t quite enough.  You need the “answer” now or you find yourself adding complexity to the file format just to parameterize what is clearly a request.  Command lines allow you to extend the file model, they preserve the process boundary and again, they are pretty easy to test in isolation.  The standard unix command set represents a very stable framework of components upon which many applications have been built.
  3. REST over HTTP.  When multiple clients are accessing the same shared resource, command lines just won’t do.  This happens most often over a network but can also happen on the same machine.  REST is the expression of a lot of real world experience about network and server components.  You have to get very formal about how state moves around.  Although REST doesn’t require HTTP there are rarely good reasons to invent a different protocol and plenty of problems when you do.

There are exceptions to this rule.  There are legacy protocols like SMTP that have been sufficiently baked that there’s no point in “fixing” them.  There are new components like memcached that warrant dedicated protocols instead of HTTP (although I believe that memcached is still RESTful).  The simple approach is to be very deliberate when departing from the above rules.

Updated July8th 2009

I think it’s time to add XMPP to the list as #4 but with a lot of a caution.  The scenarios where XMPP is truly needed are likely to be quite complicated and more likely to result in distributed code than truly separate components.