Polish

What is polish? Polish is taking things to their logical conclusion. It is tying up all lose ends, fully exploring all possibilities. It is solving the essence of the problem and documenting exactly how to reproduce the solution.

Polish is about simplicity, elegance, and removing clutter. It is removing the pieces of stone from the statue within the block of marble. Polish is exceeding expectations, going far above the norm. Polish is felt in a well designed product that just works, that fits in your hands comfortably. In the real world, these are items of just the right heft, perfect shape, and the simplest thing to get the job done. In the virtual world, they are novels that not only illustrate an idea, but put it into practice live before you imagination so that you feel as if you have lived it. Or they are mathematical models so eloquent they make you cry out that you did not see it first yourself!

Now there are those who will tell you that you should just do something, get anything in front of people so that you don't waste a lot of time creating something perfect that nobody wants. I would tend to agree, but with one qualification: what you put in front of them must be usable. If it is a video game, it better shoot. If it is a web app, it should allow you to do one thing well, and do it completely. If it is a mathematical proof it should be the simplest version of it that is still comprehensible.

How do we deal with complexity? With simplicity. Crush every problem down to the very root of the issue. Address this most fundamental problem in the simplest yet still helpful way, then iterate. Make sure at each stage you offer a complete solution to the simplest statement of that particular problem.

Technology allows us to economically pursue polish as never before in history. Now that one man can create ideas and products that are used by billions, no amount of effort can be wasted on perfecting the right idea. Your idea must compete with literally every other on earth in the past and present. Increasingly the only way to do this is with polish.

Communication of ideas between humans is hard and this will not change. With polish, rise to the challenge and transfer the essence of what the you have created into the hearts and minds of others. While all this means much more work for those who produce, it is an amazing time to be alive. We are to be witnesses and participants to the most complete, the most usable products and ideas from any time in history. Create things, and create them well.

The Appeal of Authority

Why does man crave authority? Why does man crave certainty? These two questions seem to be the restatement of a single human desire: security. We all crave security in various forms. Few can feel any long term happiness without it. Our longing for the known can drive reasonable men to very unreasonable actions. I have witnessed countless times otherwise sane persons throw all intellectual integrity to the wind for the simple illusion of security found through claims of certainty.

Want to influence men? Want to persuade others? Deal in certainty. Sell security as your commodity. Provide confidence in claims, guarantee your promises, and smile without flinching. How many times have we seen hucksters one job ahead of the curve, one promotion away from all their failed promises.? No one seems to remember long. Those around them simply eat up the certainty like candy and care less that the claims didn't pan out than that they were made with certainty and authority.

The selection bias provides us with our historical winners. Make enough authoritative claims and a few will turn out to be true. If you are lucky, you may even go down in history as a great visionary.

This works for blogging, the workplace, and especially in personal relationships. Go forth, provide authority, and make snap decisions with absolute confidence! You will be materially better off, and who can fathom the cost or currency of the soul?

Presence in the Cloud: It's not just for Buddy Lists Anymore!

I have been a huge fan of IM applications for a long time. There is just something wonderful about a dynamic list of agents and activities all flashing away down there on my taskbar. Presence is one of the cooler things to come out of the IM wave. But what happens when more than your friends start advertising their presence?

Imaging a grid of virtual instances, all advertising their current state via presence. They let you know load, health, activity, and heck they could even tell you they are bored. Now imagine another set of agents that are watching these node statuses. When a node is bored it gets spun down. When a component reports ill health, a ticket is issued for maintenance (hey IT guy, replace drive 145 in array 6).

What components do we currently write that would benefit from presence?

Scaling the Beast (Time and Hardware)

These are interesting times. Computer power keeps increasing while at the same time falling in price. Bandwidth continues to expand. As a result of all this horsepower languages continue to proliferate. I have in the last few years spent a lot of time with ruby and python, which are two very powerful and yet in relative terms, slow running. The great thing is that it no longer matters.

But this is old news. Now it feels like we are heading for an even bigger shift. The virtualization and commoditization of data centers is in full swing. Companies are offering full hosting stacks as a full blown service. Check out AWS, Mosso, 3Terra, and App Engine for just a taste of what is coming. These offerings include not only pay as you go hosting and hardware services from a very small scale up (some even start at free!) but are also starting to offer powerful software services such as data (not just file) storage, message queueing, and content delivory.

Software components running on the grid, storing their data on the grid, being accessed from other components on the grid. All running on this virtualized platform. No OS patches, no external security concerns, brain dead deployment. Point and click (or invoke your batch script) and send your component to the sky, spin up instances on demand, and revel in an ever approaching complete abstraction from almost all of the scaling, cost, and security issues that plague web developers today (twitter anyone?).

Because of this up coming shift away from hardware instances and to a component or service based computing model, many language concerns are slowly becoming obsolete. As virtual machines continue to gain popularity with developers, OSes and compile time dependencies are slowly being replaced by platforms. These platforms behave almost the same regardless of the underlying system. They all have the ability to make and receive network calls to provide and consume services. As a result, more and more services are being accessed via the network using simple RESTful or other largely text based apis. This frees the burden on languages of writing, porting, and maintaining each service for every single language out there. As a result, standard libraries can shrink, and services grow.

CouchDB, Solar, SimpleDB, Google's Database API are all fine examples of this from a data perspective. The Stomp messaging protocal to talk to JMS services, XMPP for instant messaging, presence, and a host of other interesting applications. IMAP, POP3, SMTP and many other internet protocals that have been around for eons. All these protocals allow components to expose and consume services without having to worry about many of the complexities that have been associated with component based architectures. Not only does this let you only write one api that can be consumed by any language, but also skirts the memory management issues of making calls to libraries in process. Issues like who cleans up the stack? Who allocates and frees which memory? All thorny issues that for the most part simply get in the way and soak up brain cycles. As a secondary effect, it sets up each component to scale out. Many bad habits that developers can fall into just aren't available to a loosely based component architecture. There are firmer parameters that when followed allow for near infinite scaling (disregarding cost issues for the moment). These parameters encourage good programming practices (horizontal scaling, loosely coupled, separately versioned components)

As more and more functionality moves out of in process libraries and into network components, developers will be freed to make language decisions not on libraries but on the syntax of the language itself. This should be good for everyone. There is no one size fits all language, and developers all have different personalities. The ability to choose platform based on components rather than virtual machine, language, or OS is something I am personally really looking forward to.

Stationary Optical Drive

It seems stupid that we spin disks mechanically several thousand times a minute. Lots of work, lots to go wrong. Generates heat, noise, and has hard limitations on how fast we can read data based on how fast we can spin a physical object.

What if instead we read data of a stationary platter with a stationary reader. We direct the reader using a similar method to a CRT. Basically we have two electromagnets, one for the x and one for the y axis. This shines the light onto the desired position of the platter, which then reflects onto a reflective bowl with a focal point of a sensor.

       +---- Sensor
       v
-------S-------   <-- Platter
\             / <-- reflective bowl
 \-----|-----/  
       ^
       +---- read laser and CRT like aiming device

We would probably need to sustain data rates of over 1GB/s and storage comparable to existing DVDs to be interesting.

Brains, Networking, or Luck?

I have been through several interviews in my life where I just couldn't believe I didn't get an offer. I was smart, quick witted, and nailed every question. Then you wait two weeks and get a two sentence email in your inbox saying they are going a different direction, thanks for playing.

Rejection never feels good, and you often wonder what happened, what you could have done to influence the outcome. Truth be told, it often has more to do with how your interviewers were feeling about themselves during that fateful half hour. Are they late on a big project? Did they eat that huge burrito with extra hot sauce again? Did they just get a raise and think they can accomplish anything with just a few more team members?

In an interview you basically have the span of an hour to evaluate how competent and amazing of a person the candidate is. Will this person make my life and the company's future significantly better? That's a tall order in that short of time. In fact, it's really bordering on impossible. Unfortunately given the reality of the situation, what usually happens is more or less luck. While the smart and the connected certainly have an edge up in the odds, all in all I would be surprised if luck did not have a significant if not the majority role in most decisions.

A key point to remember is that when you are evaluated for a job or really anything else in life, you are not being evaluated or judged. Your name (possibly), a few run-on sentences from your resume, and a couple flashes of memory recalled from your blip in that person's life are what is really being judged. This is both comforting and frustrating. Remember, very few things in life are an actual rejection of you. They are almost invariably a rejection or acceptance of a few bits about you that may or may not be true, and a whole pile of near random conditions.

Harder still is to keep all this in mind when hiring. All I can really come up with is to make sure you have multiple interviews on different days before deciding on an offer. At least this way you can level out some of the curve of what you ate for breakfast.

This all reminds me of three very different world views out there. The one where the "man" controls the world (think Marx), the one where the mob controls the world (think Rand's Atlas Shrugged), and the one where randomness rules them all (think Taleb's The Black Swan). I have held a variation of all three views at different stages in life, which one are you?