Memory Leak in ActiveMessaging

ActiveMessaging is great. It allows you to easily hook up to ActiveMQ to offload all your batch processing needs. Only problem, is it eats memory like crazy. Just hookup a simple queue with a publisher and consumer, write a few hundred thousand tickets, and watch the consumer eat all your available memory (will quickly eat a couple hundred megs and go on to use more than a GB).

In the gateway.rb, there is a dispatch method that routes the message to the appropriate processor:

def dispatch(message)
@@guard.synchronize {
begin
  prepare_application
  _dispatch(message)
rescue Object => exc
  ActiveMessaging.logger.error "Dispatch exception: #{exc}"
  ActiveMessaging.logger.error exc.backtrace.join("\n\t")
  raise exc
ensure
  reset_application
end
}
end

If you comment out the prepare_application and reset_application the memory consumption stops. You can chew through millions of tickets and stay at a steady usage. Only problem is that now, ActiveRecord will not keep its MySQL connection fresh, aka you will get a MySQL::Error: Mysql has gone away

These methods seem to wedge deep in rails' dispatch foo. Somewhere in there, it is likely doing validation on the connection. So, the trick will probably be to override the process!(message) method of the base processor class, and rescue MySQL::Error and call ActiveRecord::Base.verify_active_connections! and retry.

I will update this once I can validate it to see if this fixes the stale connection issue and if I run into any other issues, or if any kind commenter leaves the answer.

Entrepreneurs, Love, and Signals

Intelligences gets things done by planning, acting, and then making continual adjustments. In order to be successful they must be able to evaluate their current course against where they were planning on going. If things are off, they adjust, change directions, speed, etc. to get back on the right path. As the old tired quote goes, “How do you get to the top of a mountain? Just make sure each step leads you higher”.

Now we get signals along the way that indicate how we are doing. A simple signal would be a measurement. Are we higher or lower than last step? This can then be compared with the desired goal if it is itself measurable (I want to make it to 5000 ft). In this way we can make adjustments with each step. We can even make sophisticated decisions like should I go down 10 steps to the bridge to cross so I do not have to go down 1000 steps to get across a gorge.

Not all signals are equal. Some goals have very good signals (like in our example of height). Others have much more complex signals. A good example of a goal with complex signals is love. Love is many things to many people, but usually has something to do with prioritization of another over ones self, or at least over others compared to the one loved. But this notion of prioritization is very complex and has to do with hundreds of social conventions and traditions. Even amongst personality types we see differences in what is perceived as meeting this goal.

In fact almost all intra-personal relations are subject to these same complex signals. From friendship, to family, to gift giving, politics, and leadership. All enormously complex with a multitude of signals that very in their significance from person to person and even from time to time for the same individual.

Now a given intelligence is only capable of so much planning and signal processing. Like any other resource, planning, adjusting, and signal processing are subject to the limits of time, processing power, and material resources that can be used in goal attainment. This is a two way street. Interpersonal relations are most commonly a very intimate one-on-one type of goal. Even the best individuals a re by themselves capable of attaining goals with only a very few set of relations (relative to the total number of people in existence).

Now in any society of individuals inevitably a system of trade (market) comes into play . People produce and consume goods and services and through specialization come to use a common medium of exchange called a currency or more commonly money (This is obviously a very specious treatment of the subject. For those so inclined I highly recommend the books Human Action and Theory of Money and Credit by Von Mises, and Man, Economy, & State by Rothbard). Prices (the costs in money of specific goods) becomes a universal signal for making decisions relating to the market. Prices adjust relative to supply and demand for any specific good, which reflect the aggregate desires of individuals across a huge section of every day life (remember labor is just as much of a good as any other).

Entrepreneurs use these market signals (prices) to attain their goal (profit). In so doing they fulfill the desires of a great number of individuals in addition to their own. By creating a giving good and with an advanced distribution network as we have developed today, can impact the lives of millions. A humble farmer can now, with current technology, produce enough food for hundreds of individuals. A clothing designer can create fashion to be enjoyed by millions.

Markets effectively offer a very simple signal which masks the complexity of the underlying system of interpersonal signals. By using the simplified market signal, we are able to achieve a scale far greater than that which is available to us on a direct interpersonal level.

For the few interpersonal relations you have resources for, by all means work towards your goals. They are greatly rewarding. But if you want to maximize your goal obtainment, then use the market's signals and produce!

Meta and Politics

Sometimes we as humans are drawn to simple little systems, where the rules are clearly laid out, and the basic axioms are short. Things like Newtonian physics and Fibonacci sequences. When these simple systems have the ability to combine their rules in an open ended manner, the results can become quite complex, even though the ground rules are simple. These systems are happily adapted to computers which can handle insanely detailed application of these rules.

Reality however, is rarely one of these systems. Real systems tend to be much more complex. Intelligence for example, is certainly not readily describable in terms of a few basic axioms. Categorization, pattern matching, and allegory are all heavily dependent on the Meta. Anything dealing with general intelligence, communication, or flexible systems tends to be heavily imbued with Meta rules, that is, rules for describing rules.

When a system at its base is not a set of rules applied to the direct problem domain, but is instead made up of ground rules for the creation of rules depending on the current situation, potential for complexity soars. These meta rules allow the system to be very adaptable, and tends to fool our overly aggressive causal tendencies by producing counter-intuitive results. In fact, many times these Meta rules are completely hidden to the outside observer, masked by the generated rules for a given circumstance.

Many Meta systems are inherently recursive, not only allowing for several levels of rules, but also rules that depend and are defined in terms of themselves. This creates a situation in which all the rules can be explained, but the implications of those same rules is highly unknown (even when they are not hidden). The lines between Meta-rule and rule tend to blur.

Human intelligence appears to be highly meta, insanely complex, and capable of producing very unexpected results. Economics is the study of these systems in action, in parallel with billions of other similar systems all interacting at once. The resultant complexity astounds the mind. When so called "economists" are called upon to make predictions as to the future state of this massive interaction of complex systems (itself now a new monster system called the economy), one can only laugh, and shake one's head, as we watch the circus of those pretending to be the masters of meta fall down again and again.

So create your wrong headed doomed "Rescue" bills and regulations. Sell it to the public as a snake oil salesmen fleeces the poor and down trodden. And as disaster strikes yet again, sleep tight with the defense of "it couldn't be helped!", and "no one can blame us, we listened to the experts!".

Meaningful Phrases

I am trying to recall some of the moments in my past where I had or read a particular thought that produced or summed up a great deal of the way I think. Here is a sample of what came to mind.

People are fundamentally selfish. They act according to their most "perceived beneficial" action at all times. I was first exposed to this concept formally in C.S. Lewis' Mere Christianity. It was later spelled out in exhaustive detail by Von Mises' Human Action, and Murray Rothboard's Man, Economy, and State, and makes the basis of the Austrian School of Economics.

The Utopian vision of man vs the corrupt image of man. I first read about this in Thomas Sowell's book Basic Economics. Basically is man approaching some long away super society where he corrects his flaws and lives in harmony, or is where we are now basically where we have been all along and will always be. Basically realizing that you could split most political, moral, and social systems/thought with this one question. I tend to believe in the corrupt version.

The amount of information in a signal varies directly with how random it is. Put another way, the ability to compress information varies indirectly with how random it is. As an example, the particle movements of a wave crashing on the beach has an incredible amount of information in it. It would take incredible amounts of computer horse power to simulate it exactly. The main interest for me is that complexity is not in of itself useful or powerful. The more complex a system is not fundamentally better, and the reverse is more generally true.

Humans are rationalizing creatures rather than rational. The world is far more random than we think. This was an intuitive hunch for most of my life, with pointer's from Steven Pinker's books on the mind, but laid out beautifully by Taleb in The Black Swan, and Fooled by Randomness. People are fundamentally wired to see cause and effect in every situation and believe they know these causes, even in very complex systems. Humans are continually fooled by selection bias, and we share a universal inability to handle probabilistic thinking. This explains hero worship, our inability to learn from history, and explains the extreme epistemic hubris of most people. The apeal to me and the defining characteristic of the Austrian school of economics is its humility in regards to complex systems and its reluctance in determining cause and effect in history.

These are a few that came to mind this afternoon, what are yours?

Priming Surveys

Checking out frogmetrics.com and thinking it is really cool, but reminds me...

How many times have you seen a survey with the answer being from 1-10 or 1-5 stars? Every time I come across one of these I always wonder how in the world they aggregate these responses in a meaningful way. A very level person will answer most things close to the center (every thing is a 4-6) where a excitable personality tends to hit the extremes. They both likely meant the same thing by their feedback. The same thing goes with netflix movie rankings for example, and then they try and give me recommendations based off of what other users felt. Problem being is that the level heads and the excited are all mixed in together.

So is there a way you could get a sense of what personality they are to help classifier their answers? What if you asked a single question somewhere in the survey that was a "primer" question. Something that has a decent emotional response, like how would winning 100 dollars make you feel 1-10?

Named Parameters in Ruby

Ever forget whether the name or email parameter is first on a method like this?

signup(name,email)

Named parameters can help out so that you don't have to remember the order of parameters. As a fringe benefit the code can read a bit nicer and you can add in new optional parameters without breaking existing code.

Ruby as a language doesn't support named parameters to functions. But it does support converting named pairs into a hash if you provide an argument for it.

def signup(params)
  name=params[:name]
  email=params[:email]
  ...
end

This takes a little more work on the function declaration but it's not too bad. Now we can call the function like this:

signup(:name=>'Me', :email=>'me@net.com')

Suppose you wanted your name parameter to be optional and default to the email parameter. You can easily set default values for one or more of your expected parameters:

def signup(params)
  email=params[:email]
  name=params[:name]||email
  ...
end

With named parameters it often behooves you to do a bit more parameter checking.

def signup(params)
  email=params[:email] || raise("email parameter is required!")
  name=params[:name]||email
  ...
end

To make all parameters optional, set a default value for your parameter to {}.

def optional(params={})

My mind is out to get me

Maybe I am overly paranoid, but I think my mind is out to get me. Often times, I am woken up at night by my "I have to pee" agent of my mind. It gets me stumbling downstairs, only to realize, I didn't really have to go. Then it comes to light that I am actually really thirsty. Apparently I ignore my "gotta go" agent less than my "your thirsty" system, probably due to the consequences of ignoring each.

Do agents of your mind lie, cheat, and steal to get you (your agents actually) to do their bidding? Did my "thirsty" agent abuse my "gotta go" agent to do its bidding? Is your subconscious mind a scheduler that relies on agents telling the truth? If your agents start lying to your subconscious to affect behavior, who can you trust? Excuse me while I go lie down for a while, the world just got a little scarier for me.

Life recording and Infinite regress

What if you could record your life all the time? You would then start reviewing those recordings so you could improve at whatever you do. And then you would obviously want to improve your reviewing process (as its something you do) so you could become better at whatever you were doing in the first place. This could go on for a while.

Likely the law of diminishing returns would kick in and you would stop before too many meta layers. How do we as intelligent beings so effortlessly make these decisions? Just reading this probably makes you wonder what kind of idiot I am for even positing such an absurd situation. Maybe we are protected by a mechanism of boredom. Seems like this would be an effective deterrent to the infinite regress in many forms. How would you design a machine that gets bored? Or I suppose this is the same question as how do you build a machine that has interests?

Ruby Batch Processing: ActiveMQ and ActiveMessinging

Lets say you have some long running background tasks that are triggered by user actions. For example, you may want to allow users to upload a list of bookmark URLs that you will create thumbnails for.

Heres the sequence of events we are looking for.


user posts list -> rails controller -> create ticket(s) -> queue -> (offline) processor(s) do work


A message queue is a good solution for this type of setup if you want to have a number of processors that you can simply start more of to scale. ActiveMQ and ActiveMessaging using STOMP make a simple ruby/rails solution.

ActiveMQ
ActiveMessaging (A13g)

Create our processor

class ThumbnailProcessor < ApplicationProcessor
 # - using ActiveMQ STOMP extension prefetchSize 
 #   to only take 1 ticket at a time
 # - set ack to client or else prefetchSize won't 
 #   do any good
 subscribes_to :thumbnail, 
   { :activemq:prefetchSize=>1, :ack=>'client'}

 def on_message(msg)
   # stub for long running code that 
   # creates thumbnail from a url
   create_thumbnail(msg)
 end
end

Create our controller

class ThumbnailController < ApplicationController
 publishes_to :thumbnail

 def create
   # create a ticket for each url in list
   params[:urls].split.each do |url| 
     publish :thumbnail, url
   end
 end
end

Configure ActiveMessaging (config/messaging.rb)

ActiveMessaging::Gateway.define do |s|
 s.destination :thumbnail, '/queue/thumbnail'
end

Now spin up as many processors as you need (you can always start more later)

./script/poller start
./script/poller start
./script/poller start

and you are ready to roll!

Emergent Behavior and Software

Recently watched the interesting movie Idiocracy by Mike Judge. The basic premise is that intelligence reaches its pinnacle in evolutionary usefulness sometime during our recent history and that it is generally reduced in the next 500 years due to selective pressures to the point that the mean IQ is just about barely functional. This is an interesting idea based on the selfish gene theory and a rather non-blank slatish theory of mind that says most of what we posses in the mental department is god given. This allows natural selection to determine genetically what will be the dominant and useful traits for survival, reproduction, and ultimately the mean IQ of the general population.

Now there is another notion of selection that occurs not at the gene level, but at the idea level. This is often refered to as memes, either on a social/cultural level, or at a more basic story level. In this model, what survives is not the hardware but the software. If this model is true, then what will determine the daily lives of people will not be the quality of their hardware, but that of their software. What ideas, theories, algorithms will survive?

Nanny State or Idiocracy?

You decide.

Polish

What is polish? Polish is taking things to their logical conclusion. It is tying up all lose ends, fully exploring all possibilities. It is solving the essence of the problem and documenting exactly how to reproduce the solution.

Polish is about simplicity, elegance, and removing clutter. It is removing the pieces of stone from the statue within the block of marble. Polish is exceeding expectations, going far above the norm. Polish is felt in a well designed product that just works, that fits in your hands comfortably. In the real world, these are items of just the right heft, perfect shape, and the simplest thing to get the job done. In the virtual world, they are novels that not only illustrate an idea, but put it into practice live before you imagination so that you feel as if you have lived it. Or they are mathematical models so eloquent they make you cry out that you did not see it first yourself!

Now there are those who will tell you that you should just do something, get anything in front of people so that you don't waste a lot of time creating something perfect that nobody wants. I would tend to agree, but with one qualification: what you put in front of them must be usable. If it is a video game, it better shoot. If it is a web app, it should allow you to do one thing well, and do it completely. If it is a mathematical proof it should be the simplest version of it that is still comprehensible.

How do we deal with complexity? With simplicity. Crush every problem down to the very root of the issue. Address this most fundamental problem in the simplest yet still helpful way, then iterate. Make sure at each stage you offer a complete solution to the simplest statement of that particular problem.

Technology allows us to economically pursue polish as never before in history. Now that one man can create ideas and products that are used by billions, no amount of effort can be wasted on perfecting the right idea. Your idea must compete with literally every other on earth in the past and present. Increasingly the only way to do this is with polish.

Communication of ideas between humans is hard and this will not change. With polish, rise to the challenge and transfer the essence of what the you have created into the hearts and minds of others. While all this means much more work for those who produce, it is an amazing time to be alive. We are to be witnesses and participants to the most complete, the most usable products and ideas from any time in history. Create things, and create them well.

The Appeal of Authority

Why does man crave authority? Why does man crave certainty? These two questions seem to be the restatement of a single human desire: security. We all crave security in various forms. Few can feel any long term happiness without it. Our longing for the known can drive reasonable men to very unreasonable actions. I have witnessed countless times otherwise sane persons throw all intellectual integrity to the wind for the simple illusion of security found through claims of certainty.

Want to influence men? Want to persuade others? Deal in certainty. Sell security as your commodity. Provide confidence in claims, guarantee your promises, and smile without flinching. How many times have we seen hucksters one job ahead of the curve, one promotion away from all their failed promises.? No one seems to remember long. Those around them simply eat up the certainty like candy and care less that the claims didn't pan out than that they were made with certainty and authority.

The selection bias provides us with our historical winners. Make enough authoritative claims and a few will turn out to be true. If you are lucky, you may even go down in history as a great visionary.

This works for blogging, the workplace, and especially in personal relationships. Go forth, provide authority, and make snap decisions with absolute confidence! You will be materially better off, and who can fathom the cost or currency of the soul?

Presence in the Cloud: It's not just for Buddy Lists Anymore!

I have been a huge fan of IM applications for a long time. There is just something wonderful about a dynamic list of agents and activities all flashing away down there on my taskbar. Presence is one of the cooler things to come out of the IM wave. But what happens when more than your friends start advertising their presence?

Imaging a grid of virtual instances, all advertising their current state via presence. They let you know load, health, activity, and heck they could even tell you they are bored. Now imagine another set of agents that are watching these node statuses. When a node is bored it gets spun down. When a component reports ill health, a ticket is issued for maintenance (hey IT guy, replace drive 145 in array 6).

What components do we currently write that would benefit from presence?

Scaling the Beast (Time and Hardware)

These are interesting times. Computer power keeps increasing while at the same time falling in price. Bandwidth continues to expand. As a result of all this horsepower languages continue to proliferate. I have in the last few years spent a lot of time with ruby and python, which are two very powerful and yet in relative terms, slow running. The great thing is that it no longer matters.

But this is old news. Now it feels like we are heading for an even bigger shift. The virtualization and commoditization of data centers is in full swing. Companies are offering full hosting stacks as a full blown service. Check out AWS, Mosso, 3Terra, and App Engine for just a taste of what is coming. These offerings include not only pay as you go hosting and hardware services from a very small scale up (some even start at free!) but are also starting to offer powerful software services such as data (not just file) storage, message queueing, and content delivory.

Software components running on the grid, storing their data on the grid, being accessed from other components on the grid. All running on this virtualized platform. No OS patches, no external security concerns, brain dead deployment. Point and click (or invoke your batch script) and send your component to the sky, spin up instances on demand, and revel in an ever approaching complete abstraction from almost all of the scaling, cost, and security issues that plague web developers today (twitter anyone?).

Because of this up coming shift away from hardware instances and to a component or service based computing model, many language concerns are slowly becoming obsolete. As virtual machines continue to gain popularity with developers, OSes and compile time dependencies are slowly being replaced by platforms. These platforms behave almost the same regardless of the underlying system. They all have the ability to make and receive network calls to provide and consume services. As a result, more and more services are being accessed via the network using simple RESTful or other largely text based apis. This frees the burden on languages of writing, porting, and maintaining each service for every single language out there. As a result, standard libraries can shrink, and services grow.

CouchDB, Solar, SimpleDB, Google's Database API are all fine examples of this from a data perspective. The Stomp messaging protocal to talk to JMS services, XMPP for instant messaging, presence, and a host of other interesting applications. IMAP, POP3, SMTP and many other internet protocals that have been around for eons. All these protocals allow components to expose and consume services without having to worry about many of the complexities that have been associated with component based architectures. Not only does this let you only write one api that can be consumed by any language, but also skirts the memory management issues of making calls to libraries in process. Issues like who cleans up the stack? Who allocates and frees which memory? All thorny issues that for the most part simply get in the way and soak up brain cycles. As a secondary effect, it sets up each component to scale out. Many bad habits that developers can fall into just aren't available to a loosely based component architecture. There are firmer parameters that when followed allow for near infinite scaling (disregarding cost issues for the moment). These parameters encourage good programming practices (horizontal scaling, loosely coupled, separately versioned components)

As more and more functionality moves out of in process libraries and into network components, developers will be freed to make language decisions not on libraries but on the syntax of the language itself. This should be good for everyone. There is no one size fits all language, and developers all have different personalities. The ability to choose platform based on components rather than virtual machine, language, or OS is something I am personally really looking forward to.

Stationary Optical Drive

It seems stupid that we spin disks mechanically several thousand times a minute. Lots of work, lots to go wrong. Generates heat, noise, and has hard limitations on how fast we can read data based on how fast we can spin a physical object.

What if instead we read data of a stationary platter with a stationary reader. We direct the reader using a similar method to a CRT. Basically we have two electromagnets, one for the x and one for the y axis. This shines the light onto the desired position of the platter, which then reflects onto a reflective bowl with a focal point of a sensor.

       +---- Sensor
       v
-------S-------   <-- Platter
\             / <-- reflective bowl
 \-----|-----/  
       ^
       +---- read laser and CRT like aiming device

We would probably need to sustain data rates of over 1GB/s and storage comparable to existing DVDs to be interesting.

Brains, Networking, or Luck?

I have been through several interviews in my life where I just couldn't believe I didn't get an offer. I was smart, quick witted, and nailed every question. Then you wait two weeks and get a two sentence email in your inbox saying they are going a different direction, thanks for playing.

Rejection never feels good, and you often wonder what happened, what you could have done to influence the outcome. Truth be told, it often has more to do with how your interviewers were feeling about themselves during that fateful half hour. Are they late on a big project? Did they eat that huge burrito with extra hot sauce again? Did they just get a raise and think they can accomplish anything with just a few more team members?

In an interview you basically have the span of an hour to evaluate how competent and amazing of a person the candidate is. Will this person make my life and the company's future significantly better? That's a tall order in that short of time. In fact, it's really bordering on impossible. Unfortunately given the reality of the situation, what usually happens is more or less luck. While the smart and the connected certainly have an edge up in the odds, all in all I would be surprised if luck did not have a significant if not the majority role in most decisions.

A key point to remember is that when you are evaluated for a job or really anything else in life, you are not being evaluated or judged. Your name (possibly), a few run-on sentences from your resume, and a couple flashes of memory recalled from your blip in that person's life are what is really being judged. This is both comforting and frustrating. Remember, very few things in life are an actual rejection of you. They are almost invariably a rejection or acceptance of a few bits about you that may or may not be true, and a whole pile of near random conditions.

Harder still is to keep all this in mind when hiring. All I can really come up with is to make sure you have multiple interviews on different days before deciding on an offer. At least this way you can level out some of the curve of what you ate for breakfast.

This all reminds me of three very different world views out there. The one where the "man" controls the world (think Marx), the one where the mob controls the world (think Rand's Atlas Shrugged), and the one where randomness rules them all (think Taleb's The Black Swan). I have held a variation of all three views at different stages in life, which one are you?

Uncertainty, Arrogance, and Software

Why is software so hard? Why do projects routinely run over budget, over time, and require more people and hardware than forecast? Why is software development so far behind the other engineering disciplines when it comes to projects and budgeting? These are common questions for anybody working in or with the software industry.

Programmers will usually talk about complexity, changing requirements, mis-management, or pace of technology. While these are all real issues about why writing software is so hard and time consuming, it doesn't necessarily address why we are so often mistaken about the amount of effort a given project requires. Even being fully aware of the afore mentioned problems, most programmers will still give you a rosy eyed estimate that will be woefully off target (and almost always in the under estimating of the time and resources required for a given task).

I have been working at this for over 10 years now, and I am sad to say I don't feel any better at this than when I started. All I have learned is a healthy respect for the amount of time and effort those little boxes "encapsulate", and a near phobia of commitment to estimates of how long any given "piece" will take.

Central to the issue is our biological need to feel that we understand a situation in order that we might act. It seems a deep seeded reality that we are wired to generalize, categorize, and abstract messy reality so that we can make some forward progress in this crazy world. We are masters of this, and whisk away ugly detail with ease, boxing up thorny problems with the stroke of a dry erase marker creating orderly corrals where our shiny new abstractions live in words like "storage", "service", and "analysis".

When you are dealing with tigers jumping out of the woods at you or a fist speeding towards your face, this ability to take a messy situation and make it simple is a very useful trick. While pontificating about the underlying motives of the fist wielder or determining the exact species of tiger hurtling toward you may be all excellent strategies for handling the situation if time were no object, in our reality they just get you dead. In our reality, over estimating the amount of knowledge you posses in order to more quickly come to a decision can be a huge advantage when time critical situation are on the menu. In fact, this was the usual case until fairly recently in human history. It's no wonder that we are geared to handle these types of situations given the frequency and risk/reward of each type.

Unfortunately, this can leave us arrogant yet delusional, which can be a pretty nasty combo. Our tendency is to quickly feel like we understand a situation and are ready to act on it, while in reality we have very little understanding of the actual problem. This leads us to over estimation of our understanding of the factors involved which leads to an under estimation of the time involved (since we have to not only implement what didn't understand but also figure it out). This combined with the inconvenient tendency of reality to be more fractile than linear in nature means that things tend to get more complicated as you investigate them. The devil's in the details as they say.

For a much better account of our epistemic arrogance as it relates to many interesting (non-software) issues, check out Nassim Taleb's excellent book The Black Swan.

Google App Engine

If you haven't checked out google app engine yet you yeally need to. I wrote my first facebook app backed by app engine. The barrier to entry has now actually dropped to zero. It is amazing: marketing through facebook fiends is free, the host and database are now free, and all the tools required to write each piece are free. I look forward to seeing what comes out of this emerging platform.

Nuclear Power and the Cost of War

While doing a little BSing with a friend on IM about the costs and merit of war for oil, started wondering what it would look like if we instead spent this money on nuclear power plants. Now perhaps this isn't exactly your favored solution, but relax, I don't make policy, and this is just a thought experiment.

According to the ever popular Wikipedia we in the US consumed 29000 TWh of juice in 2005, equivalent to an (average) consumption rate of 3.3 TW. Likewise we are informed that the capital costs of a new plant are in the 5,000-6,000$ range per kilowatt, or lets say 5.5$/w capacity. Now, in our little war on Iraq we are likely spending around 4 trillion dollars (see past posts for references). A little math reveals that would buy us $4,000,000,000,000/$5.5 * 1w ~ 730GW of capacity. Given we consume around 3,300GW of energy, that would be about 730/3,300*100 = 22% of our needs. Now obviously we would have to pay to run the plants, guard them, and dispose of the wastes, I will lead it up to the reader to figure out those costs as I need some sleep now.

In summary, we could crank out a lot of nuclear power plants for the cost of this war for oil. Food for thought.

Objectivism Revisited

Rereading Atlas Shrugged for the second time, and was struck by this paragraph where Francisco is talking to Dagny:

"Look around you," he said, "A city is the frozen shape of human courage--the courage of those men who thought for the first time of every bolt, rivet and power generator that went to make it. The courage to say, not 'It seems to me,' but 'It is'--and to stake one's life on one's judgment. You're not alone. Those men exist. They have always existed. There was a time when human beings crouched in caves, at the mercy of any pestilence and any storm. Could men such as those on your Board of Directors have brought them out of the cave and up to this?"

- Page 510 in the first Plume printing 1999

I have been a pretty hard nosed skeptic for some time, and have often debated with people about what one can and cannot know (in the epistemological sense). Having agreed with Descartes in his Discourse on Method only as far as "I think therefore I am", suffice to say I don't think we can truly know much of anything.

Yet, we live in a seemingly real-time universe where we exist and function to some degree or another. Each thought, each decision leads to some action that we take in the praxeological sense. There is no way for the skeptic to hide from action indefinitely, and in fact, what little time there is really gone in the blink of an eye. For I can decide that I either have sufficient knowledge to start or continue an undertaking or I do not, but there is no escaping the decision to act. Either I build a bridge, create a piece of software, invent a new model of cognition or I do not. Either way, I have made my choice, and that choice results in inescapable action without the luxury of assured knowledge or truth.

I have never really dug in to see if I buy Objectivism or not, and while I certainly do agree with many of the resulting conclusions, I have been quite reluctant to accept the main pretexts on which the philosophy seems to be grounded. But if in this paragraph, and in the resulting thoughts it brings to mind, the underpinnings are to be found, then perhaps I need to reconsider what it means to be an Objectivist, and if I consider myself one.

Mises, Incentives, and Inflation

The Same Old Story

Governments have for ages been using inflationary monetary policies to hide the cost of wars, secretly tax the masses, and pass the blame to "profiteers" (congress and oil companies anyone?). With the advent of debt money, this has been pushed to ever greater extremes.

Here is Mises writing in 1912:

Who has any doubt that the belligerent peoples of Europe would have tired of war much more quickly if their governments had clearly and candidly laid before them at the time the account of their war expenditure? In no European country did the war party dare to impose taxation on the masses to any considerable extent for meeting the cost of the war. Even in England, the classical country of "sound money," the printing presses were set in motion. Inflation had the great advantage of evoking an appearance of economic prosperity and of increase of wealth, of falsifying calculations made in terms of money, and so of concealing the consumption of capital. Inflation gave rise to the pseudo-profits of the entrepreneur and capitalist which could be treated as income and have specially heavy taxes imposed upon them without the public at large -- or often even the actual taxpayers themselves -- seeing that portions of capital were thus being taxed away. Inflation made it possible to divert the fury of the people to "speculators" and "profiteers." Thus it proved itself an excellent psychological resource of the destructive and annihilist war policy.

Von Mises - The Theory of Money and Credit p. 254

Here was the story we were told without being told in 2003 by the War Party:

WASHINGTON (CNN) -- The White House is downplaying published reports of an estimated $50 billion to $60 billion price tag for a war with Iraq, saying it is "impossible" to estimate the cost at this time.

Now with official estimates topping $1 trillion dollars and more realistic projections over $4 trillion there can be no doubt that the masses would not stand to pay for this by means of direct taxation. When listening to the War Party beat their drums and recommend monetary policy, economics reminds us as always to look to the incentives. Yet most persistent of all, History softly thunders "there is nothing new under the sun".

Voxonomics and Austrian Economics

I strongly recommend Vox Day's voxonomics series, an excellent set of interviews and lectures on subjects relating to economics. I especially enjoyed his interview with Dr. Frank Shostak a noted scholar of Austrian economics. As the doctor is a bit hard to understand sometimes, I took the liberty of writing up a transcript. At the end they discuss suggested readings. I highly recommend Rothbard's Man, Economy, and State and The Mystery of Banking if you are lucky enough to find a copy. Also Mises' Human Action and The Theory of Money and Credit are excellent books. Mises is definitely the better author, but can be daunting and I tired of the endless references to old German economists I had never heard of or their theories which he spends a great deal of time refuting. Anyways, listen to the interview and enjoy an excellent overview of Austrian economics.

(0:39) Vox: Welcome to the third edition of Voxonomics. Today I'm speaking with Dr. Frank Shostak, Chief Economist of MF Global and an adjunct scholar at the Von Mises institute. He's a regular contributor to the Austrian Journal of Economics and he's doing some very interesting, if oxymoronic, work in developing a theory in applied Austrian economics. If you are interested in learning more about monetary theory, especially from an Austrian perspective, I encourage you to take a look at some of the links to Dr. Shostak's papers provided on the same post as this podcast.

How are you today?

(1:10) Dr: Good.

(1:12) Vox: I have some questions for you specifically relating to the Austrian economic theory. I was wondering if you could tell me what are some of the primary differences between Austrian economics and the Neo-Keynesian economics that dominates the government and media right now.

(1:27) Dr: Well, the most practical aspect of Austrian economics is basically the way that Austrian economics approaches the whole issue of the business cycle. That is the most important, the key actually to the whole framework.

Now for instance in the Austrian economics it pays attention to money supply and such while the mainstream economics today don't really look at money supply at all and if they look at money, they look at it in a misleading way. So for Austrian for instance, the whole issue of business cycle is the result of loose monetary policy of the central bank. In other words when central bank loosens monetary policy, which after a while, follows by monetary injections, this creates the foundation for so called bubble activities, which everybody talks about. Bubble activities in our framework are basically the result of various activities which spring up on the back of monetary printing. Without the monetary printing those activities would not be there.

(2:52) Vox: Right.

(2:53) Dr: Now how they actually emerge: when money is printed, what it does is basically divert the real savings, which is the foundation for real economic growth, from wealth generators to various parasitic activities or bubble activities. When I mention savings here, savings is the most important, the foundation for economic growth in our framework while in mainstream economics savings is completely neglected, or at the best they just will mention it occasionally. But for them savings, in particularly in Keynesian, is bad news because more savings means less spending. For them economic growth originates from consumer expenditure. In our case, economic growth, the heart of it, is the pool of real savings.

(3:52) Vox: So from an Austrian perspective, you're quite concerned about the fact that in the US economy the savings rate has actually been negative for some time.

(4:02) Dr: Well, put it this way, first of all, there is always an issue of whether we can measure savings at all. When we deal with savings we deal in the real terms and its not possible to add up potatoes and tomatoes as you know, so its not really possible to measure it. But in principle we can say qualitatively that there is a hell of a high chance that American savings is such, the pool of savings, its not very good.

(4:31) Vox: That touches on something else I want to get to. Is the relative disfavor in which the Austrian economics is held partly due to its distrust of empirical methodology? Because its a little bit difficult to even discuss economics if you're not framing it within an empirical measure.

(4:54) Dr: Well, I in particular have introduced what we call applied Austrian economics. We do actually measure things, we use for example money supply to assess the possible damage money supply will do to real economy. In other words, there are certain things that the mainstream economics tries to measure, that is actually all fallacy, cause it cannot measure it. But we acknowledge it, and when its due, we will use, so to speak, the real GDP, and we do use consumer price index, but we always acknowledge and say look, guys be careful, because real GDP as such is not really a real GDP, its mostly monetary turn over, it basically comprised of money supply. Consumer price index, beware, that logically speaking its not possible to construct so called total prices as such, because arithmetically it is not possible, but we do use such things also, and communicate also. So we are not against empirical analysis, in fact I am using extensive empirical analysis, but one needs to use it in the correct way, and we argue that the mainstream is abusing this, they are torturing the data, and they try to derive the theory out of the data rather than already having a certain theory, and applying this theory on the data, trying to make sense of the data.

(6:36) Vox: Well its quite clear considering the way in which the definition of the CPI has mutated in the last 20 years. You can't even compare what the reported rate of inflation was in 1980 to what it is today.

(6:50) Dr: CPI is not a measurement of inflation. If you go back to the 19th century for instance, 19th century economist, when you ask him what his definition of inflation is, he would tell you its changes in money supply, that's what inflation is, and if you go back a little bit more, they would tell you its about debasement of money, debasement of currency. Thats for them what inflation was, counterfeiting if you will. In 20th century things were twisted around. In stead of looking at what inflation is, people started to look at the symptoms of inflation.

(7:34) Vox: Right .

(7:35) Dr: Like in price changes, price of food goes up, and they blame the producers of food, etc. etc. While all this is not inflation, it could be symptoms of inflation.

(7:47) Vox: Lets get back to the cycles because, like you said, it's an important aspect of Austrian economics. Because they play such an important role in the economy, both in Austrian terms, and also in Neo-Keynesian terms, why are they so imperfectly understood? It seems the Neo-Keynesians have problems understanding when the cycles are taking place or what point we are at in the cycle, while Austrians seem to have some trouble gauging the extent of the peeks and troughs of the cycle.

(8:18) Dr: Well, first of all, the mainstream, the Keynesian, Neo-Keynesian, Neo-Classical, all of them, they explain cycles in terms of shocks. For them, business cycles are the result of some shock. So if you have shocks, then those shocks can trigger some kind of a cyclical movement. That's their theory. They don't really have a theory of business cycle as such.

Now as far as Austrians are concerned, at least for Mises, it is very clear what business cycle is. A business cycle is set in motion by the central bank, the existence of the central bank, that's what really sets in motion boom/bust cycles on account of their policies. In fact, prior to the industrial revolution, according to Mises and Rosenberg, there weren't such things as recurrent business cycles. There were famine, there were all sorts of things, but there wasn't a modern recurrent business cycle.

Today, this type of phenomena came about as the result of the introduction of central banking and modern banking as such, the fractional reserve lending and etc etc. So boom/bust cycles are purely monetary pumping, in other words, when the central bank loosens its stance it sets in motion, provides ammunition to, varies artificial forms of life by diverting real stuff, real resources, from wealth generators to false activities, or bubble activities. When the central bank reverses its stance, moves from a loose to tight stance, this sets in motion an economic bust. Obviously there are time lags, so it can happen, that while it's tightening, the effect from the previous loose policy still asserts its dominance on the economy but, after a while, the tighter stance takes over and then it's starting to bite and hit into various activities. And obviously because of the lag there are first activities which get effected, and then secondary, etc. etc.

The current bust we are observing right now, which everybody blames on the sub-prime mortgages, we Austrians say it has nothing to do with sub-prime mortgages. It has something to do with Mr. Greenspan's policies. Between 2001 to June 2004, Mr. Greenspan was pursuing an extremely loose monetary policy. He lowered interest rates from 6 percent in January 2001, to 1 percent by June 2003. As a result there was quite an acceleration of monetary printing by the fed. All this has given rise to various false activities, like sub-prime, like anything you want, there is so plenty of it, and obviously the stock market boom. Now in June 2004, Greenspan reversed his stance, gradually started to raise rates, from one and a quarter until five and a quarter we had by September of last year, when Bernanke was there. The period between June 2004 and September 2007, this is the period of tightening, that tight stance, which now is starting to bite, which hit first the sub-prime, they were just the first, and then you will have the other guys that come, the other sectors, etc. etc.

Now which kind of sectors will be effected is very difficult to say just like that, because it depends on magnitude of the bubble activities. The proportion of bubbles vis-a-vis genuine ones. If the proportion is not very large, then we may not have a serious recession. But if the proportion exceeds 50% lets say, then we could have serious trouble. And if the savings is not there, or not sufficient, then it will be very difficult to fund such a thing. You cannot fund bubble activities, which are not self funded. They cannot fund themselves. They are dependent on the wealth creators all the time. And if they are over 50% then wealth creators are in trouble.

(12:59) Vox: Is this your explanation for the problem that Japan has had getting out of its bust since 1990?

(13:07) Dr: Well, Japan never has gone out of the bust. Japan was in the so called suppression, mild depression if you want, deflation, since 1990 till recently and it still didn't recover actually. They actually created a massive bubble economy in the 80s as you know.

(13:30) Vox: Right.

(13:31) Dr: They tried to cool it off, they managed to cool it off, and in the mid 90s actually, they were advised by professor Friedman and various experts in America, that they should flood the economy with money on a massive scale and lower interest rates to nil, and thats what they did. They flooded the system with money on a massive scale, and nothing really happened. So they were in a suppressed type of depression if you want. They never showed the data correctly also, they never fired people just like that. They were in a depressed state until now even. They never allowed the liquidation of [zombie?] activities.

(14:26) Vox: Given what you're saying about the focus on the activities of the central banks, how do you distinguish Austrian theory from Friedman's monetary theory?

(14:36) Dr: Friedman got certain good things, we don't say that Friedman is bad actually, completely, we have some disagreement with Friedman of course. For Friedman first of all, he defines inflation as persistent changes in consumer prices.

(15:03) Vox: So he's mistaken a symptom.

(15:04) Dr: Right. So that's already wrong. Then secondly, he doesn't have any business cycle theory, it's more like empiricism, crude empiricism. He says money supply, (although we are more sympathetic to Friedman than other guys) creates shocks, and those shocks create other shocks. He did not have a theory of business cycles as such, but whatever he did have was related to monetary supply, but he never articulated it precisely like Austrians. Friedman was saying printing money could create economic growth. All the Keynesian and mainstream economists are saying you can print money and create economic growth. But after a while, Friedman would say, inflation would take over, and then all the short term economic growth would disappear. Now Austrians say this is all fallacy. Printing money cannot cause any economic growth. It can only reshuffle things. It can take from wealth producers and transfer resources to non-wealth producers. If printing money can create economic growth, then we can solve all the problems of wealth creation in the world today. Every third-world country knows how to print money and everything will be fine.

(16:35) Vox: Doesn't seem to be working so well for Zimbabwe.

(16:49) Dr: That's right. So Friedman was wrong. He saw that in the short term you could create economic growth. Obviously this is nonsense, we say untenable, it's not right. For instance for us, business cycle is when you take resources away from someone who produces wealth and give it to someone who doesn't produce wealth, who is useless, you just feed him. That really weakens the wealth producers. After a while when manifestation of this comes in terms of various symptoms which even the mainstream, central bank doesn't like when its starting to tighten, it hurts the various bubble activities, all the parasitic activities which shouldn't be there. If there is plenty of them of course, then they are a majority, they can dominate everything, and that's the issue.

(17:31) Vox: What is the issue today? I have talked to some people who are concerned about inflation because Bernanke is cutting again, and they are pumping up the money supply again, others are saying that it doesn't matter what he does because we are pushing on a string and heading for deflation. What's your take on the situation from your Austrian perspective?

(17:53) Dr: The key to whether we will move from inflation to deflation is what we call pool of real savings. In other words, imagine a company, a corporation, which has got ten activities. If eight activities are doing well and two are doing badly, two bubble activities, and eight good activities, then we are okay. The eight can carry the two bad ones. But if things are the other way around, lets say we have got only four good activities and six bad activities, then we are in trouble. We don't know at the moment whether we are in the state in which the pool of funding or pool of savings is expanding, or whether it's not. It's very critical, because if the pool of funding is stagnant or declining as we believe this was the case during the 1930s, then the chances for deflation are serious. What will happen then is bank lending will collapse, we are talking about commercial bank lending, and all the credit that was created through fractional reserve lending, the money created out of thin air will evaporate. Then you can have a large decline in money supply which will lead to so called deflation, thats a possibility. But if the pool of money is expanding, then Bernanke may be successful, and revive the struggling economy, to give it a push. At present, the jury is still out, we have to wait and see where we are right now.

(19:47) Vox: Isn't the fact that Bernanke is kicking around the idea of nationalizing the banking system an indication that he's not real confident about his ability to kick start it?

(19:59) Dr: That's absolutely right, what you're saying. But to me, Bernanke is also paranoid, because he is supposedly a great student of the great depression, like he studied a lot of it and has written a lot on the great depression, but he is a Friedmanite guy. He also has got a particular weird model called the financial accelerator model. This model that Bernanke uses, every small shock in the financial economy can create a massive shock in the real economy, so that's why he is so concerned about the various balances in the financial economy, that's why he was acting so aggressively so to speak.

But I basically believe that he is of the view that we might be heading for a great depression, that's the impression he gives me all the time at least. He believes, but doesn't say it loudly, that we may be heading for a great depression.

(21:03) Vox: Now Ludwig Von Mises makes for some heavy going for those who are interested in economics but may be new to it. Who's work would you recommend for an introduction into Austrian economics?

(21:14) Dr: Well, you're right, it's a very valid comment that Mises' writings are not always simple. It requires you to apply yourself. Incidentally it may be much easier than the mainstream economics because if you take mainstream economics it's all cluttered with algebraic formulas and geometrical type stuff, which makes it completely inaccessible to the layman. So the real difficulty is, I'm not sure if Mises is much harder than anybody else. But if somebody wants a good introductory book, unfortunately there are not many, I would still recommend some easy books by Mises, and of course Rothbard, Rothbard is the right man.

(22:14) Vox: What do you think about Taylor's introduction into Austrian economics?

(22:18) Dr: I think its good, its very good, but still its got a particular deep type of philosophical aspect, etc. and to say it gives you practical things, because most people want practical things, like most business men want something actionable, practical. So I believe the action stuff, in particular business cycle, money supply, such things, Rothbard is the right man. His booklet What Government Has Done to Our Money (http://www.mises.org/Money.asp) is really the best type of book that is available, a small book actually.

(23:00) Vox: Yeah, his book on the history of paper money is really good too.

(23:06) Dr: Yeah, yeah. Man, Economy, and State (http://www.mises.org/rothbard/mes.asp) if somebody has the care to read it, cause thats a great pretext to read it. And of course a very great book, the Mystery of Banking by Rothbard, is also a great book.

(23:27) Vox: One last question: what would you do if you were in Bernanke's situation now?

(23:32) Dr: If we have to accept the fact that the central bank exists and we have to operate it, I would say that Bernanke right now is in a very difficult situation. Most people are not aware, that he tries to inflate the economy now since September, inflate the market. Now if you noticed the interest rate spread has hardly budged, they are still widening, still very wide. The reason being because he didn't succeed in trying to raise money supply. That sounds very strange, paradoxical. The reason being because of the way the central bank really operates in terms of targeting interest rates. The Fed does not pump money just like that. First of all they have to adhere to a target. So Bernanke at present has a serious problem. He does understand that he would like to have more monetary liquidity, but he cannot pump, because the moment he will pump, it will upset the interest rate target. So the only way he would be able to freely pump is if the interest rate where to fall to zero, which is quite possible. Then he will have all the freedom to use the helicopter money. At the moment, the only damage he did is by manipulating interest rates, but he hasn't really caused massive damage from monetary pumping yet.

(24:09) Vox: So if you were in charge of the Fed, you would just lay everybody off and go home.

(25:14) Dr: Well, I wouldn't be doing like Mises said and say "I am resigning" or whatever, but put it this way: existence of the central bank is a useless institution. It causes only damage, let's accept it. But if we have to have it, then one thing that Murray Rothbard suggested in his writings, the congress must pass legislation that the Fed must not be allowed to buy assets. Because the way that the central bank is allowed to pump money is to buy assets. By buying treasury bonds in open market operations. This should be forbidden, and thats about it. Then you prevent the boom/bust cycle policies.

(26:01) Vox: Well thank you so much for your time Dr. Shostak, its been very enlightening.

(26:06) Dr: No, that's great.