Saturday, February 23, 2008

Edging towards Third

I don't think it is all that hard. I simply expect a government -- any one that thinks it is competent -- to setup and enforce a series of simple, reasonable rules for its people. The act of doing this correctly aught to make a fair and stable society. These rules govern the interaction and protect people from hazards. They allow us to come together as a society for mutual benefits. They are important.

So how annoyed should I be when a city can't even afford to cleanup the snow or patch the roads, but has enough money for the counselors to get perks?

How disappointed should I be when I see a province suffering from gas shortages? Water shortages? Electricity brown-outs? But instead they want to enforce policies on which types of dogs they like and which type that think are 'nasty'?

How frustrated should I be when I see a country that doesn't help its own stuck in foreign prisons? Instead it downloads it expenses on other so it can create 'tax breaks' for popularity? It spends its efforts policing hostile lands instead of fixing its own internal problems?

How irritated should one be, when all four -- four? -- layers of government are inept for every possible reason. When our infrastructure feels like it is circling the toilet bowl at an ever increasing speed. And when we are suffering the same instabilities as places with seriously questionable governmental practices?

In a crappy society, only the rich benefit. They do this by gumming up the works with rules or corruption. In a real sense corruption is just a constantly changing arbitrary set of unofficial rules usually involving financial compensation. Some bloated territorial 'ministry' with a million insane rules isn't that far off from the type of third-world booty capitalism that we are constantly promoting where everything is sealed with graft. Is there really that much of a difference if it's money or favors?

Not that humans being selfish and greedy should come as any surprise, our politicians and bureaucrats built their careers on exploiting their relationships. That is why they have control now. These people worked their way into power. But I am sick of them thinking we don't know, or that their half-baked answers are anything but. It is a huge lot of callous people who think they are fooling us, while they are busy protecting their own little and insignificant turfs. Petty people make petty problems.

You kinda wish these people would grow up and do the job they are paid to do. Is it really that hard to decide on civilized policies and make some rules to insure that the agenda gets pushed through? Is it so difficult to clean up and remove the old useless rules? To simplify the existing ones? I guess if your so busy worrying about your turf, or who will be offended or whom you owe or own, the interaction becomes more important than the underlying policy. Still you would hope that some of these people decide as a legacy to actually try to fix things for a change instead of just joining the fray.

So we have a wealthy, educated society that is fundamentally unhealthy because it is basically a 'slob'. We sit in our mess, never bothering to even try to clean it up, out of the shallow fear that that might just make us unpopular. Never really realizing that the people are just aching to hire or vote in people who can actually get the job done. We looking for people who can clean up the mess. People who can wash out all of the dirt and grim from the room and find ways to ensure it just doesn't fall back to its messy state again. People who aren't just interested in their playing their own power games.

Oddly, unlike most third world countries our shortages and financial crisis are entirely self-inflicted. We have become the masters of shooting ourselves in the foot. At the end of an economically high-time, when we should be best able to service our society, there is a huge unplowed snow bank on my street blocking traffic to sadly prove it.

Thursday, February 14, 2008

Lost in a Space

"Ya, I know I should do that, but ..."

We get so much advice. TV, radio, newspapers, web pages, strangers, even family is constantly offering up good and tasty tidbits about how we should live our lives.

There is financial advice, which you really should follow. Career advice which hugely affects your life. Health advice, the most critical of all. Advice about how to care for the car, house or cottage. Advice to be happier or more productive. Advice about relaxing. Advice about learning and advice about hobbies.

We are, it seems downing in an unlimited sea of advice. That might be OK, if underneath it all it was well-meaning, but much of it is not. Lots of the advice we receive is tainted by the fortunes of someone, somewhere. Buying advice helps corporations, lifestyle choices help consultants, charitable advice keeps not-for-profit leaders in BMWs. Living advice is sometimes pegged to political parties or other social agenda.

It seems that most advice has that stench of market manipulation silently floating in the background. Disguising propaganda as wholesome advice has become an established marketing technique.

Even if we weeded out all of the ill-will, the sea of advice doesn't help us. You see, in the midst of their hurried revolution in the sixties to 'shake up' the world, the boomers did managed to get rid of the status quo, but they entirely forgot about the other half of the issue. It does no good to tear down the establishment, if your not planning to replace it with something better. And they didn't.

We have all sorts of wonderful equalities and freedoms, but no underlying conceptual idea about how we are supposed to behave. All of the freedom in the world, but no idea how to use it. As such, we walking wounded, perpetually confused about what we should, and should not be doing. Without some idyllic reference to a perfect person, a perfect couple or a perfect family, we don't know our own roles or where we fit in. When they trashed the status quo, they trashed the models that go with it. Now what are we supposed to do?

It's funny to be complaining about too much freedom, which I wouldn't, but the circumstances have just breed opportunities for shady people to pummel us with self-serving advice.

Here we are, endless victimized constantly by unscrupulous people telling us how to find the way out of this dazed and confused state of being. We're just accidents waiting to happen.

If we don't have any role models, or people to aspire to -- even if they are fictitious -- how do we know which advice to follow and which to disregard? Is work more important than family? Education more important than health? And in the end, what the hell is normal anymore? We can't even jump on the latest trend for comfort because the Internet has fragmented our culture into a million little dirty puddles. Other than watching the sad degeneration of our imploding celebrities, we have very little in common anymore with each other; there is so little binding us together as a collective culture.

It is best, I assume, in moments of this sort of crisis, to start tuning out everyone else. If you can't trust most of the advice, then don't follow any of it. Why drive yourself nuts in an overcomplicated society trying to be responsible, polite and keep up with the Jones, when your not even sure who the f*** the Jones are? In our decaying culture we are left with the only reasonable response: trust only your own advice. Opps, now I am giving advice. I guess it has become addictive.

Oddly, that particular type of meta-advice above, suggests we look back towards the influences that we receive as kids, which at the time was the status quo of the day. Is it possible that the only way we can move forward, to get a sane model that we can follow, is to go backwards?

Saturday, February 2, 2008

The Age of Clarity

"What do we really know? Hmmm." I pondered as we walked.

I was out with the dog the other night. The quiet tranquil nature of empty suburban streets is a great place for deep thinking. The cold chill of winter keeps one from wandering too far off topic while wandering aimlessly in the streets. Dogs make wonderful intellectual companions for these types of journeys because they don't interrupt with too many questions. They are very good listeners.

I was pondering information quality, and I foolishly started to wonder about how much inaccurate information was choking up my memory. Certainly, there are lots of spin, lies, half-truths, deceptions and other stuff built up over the years from less than quality sources like politics, news and TV. Somethings in my memory are just easy simplifications. Somethings are out right fabrications. There is also the changing nature of science, and our non-stop quest for learning. Some of my knowledge is just 'relative', it wouldn't stand up to a universal judge. It is considered true here and now, but won't be in the future. In an overall sense, how much of this is really accurate?

If you factor in all of the different reasons for low quality, and take a big sweeping guess, the amount of truth in our brains could be lower than 30%. Just a wild guess, but I could easily believe that 1 in every 3 three facts in my brain are true, while the other 2 are questionable for all sorts of reasons. I am just speculating of course, but in this misinformation age, we are full of a tremendous amount of low quality knowledge. And it feels like it is growing at an ever increasing rate, although that might just be our ability to confirm that it is suspect.


In the past, mankind was mostly ignorant of the accuracy of their information. They could take pride in their depth of knowledge without ever knowing how dubious it really was. Now all we have to do is check wikipedia and we can instantly find out that truth, well, err at least a pointer towards the truth.

How often have I pulled forth some ancient fact from the depths of my brain, only to discover that it was fundamentally untrue? Worse still is how those facts actually make it into my head in the first place. Some were obviously from disreputable sources, but others had come from well-known authorities, and were still incorrect. My problem is not loss or corruption of memory, it is the opposite, these 'facts' stay for far too long. If I just dumped them faster, I might find they were more accurate overall.

It is oddly telling. It allows us to guess that this huge degree of inaccuracy in our current knowledge is actually some type of pointer towards the future. The Renaissance was an awaking about the world that we live in. A moment when we first opened our eyes and saw it for what it actually was. This in turn drove the foundations for the industrial age, where we learned to create and use an unlimited number of machines. One of those machines, the computer, has driven us into an information age, where we collect huge piles of information, about virtually everything in this world. There is a trend here. The next age will follow along in this sequence.

Even though we have built up a tremendous collection of fantastic machines, they do not serve us well. We can build things, but we have trouble maintaining them. Our massive and complex cities crumble around us. We are forever fighting a losing battle against entropy; like a runner that has leaned too far forward we are continually off balance. We must continue to build to move forward, we don't know how to preserve what we have and we don't know how to live within our environmental means. We grow at a severe cost to the world around us.

With all of our equipment and learning, collecting data is still a hit or miss proposition. We just guess at what we want to collect and how it is structured. It is not orderly and we don't have any underlying theories that drive our understanding. Computer Science is still so young that is frequently wrong. Often it is just random guessing. We are currently only utilizing a small fraction of the capabilities of our computers because we keep bumping into complexity thresholds each time we try to build truly sophisticated systems. We are trapped with crude software.

Even thought we can collect the data, we continuously fail to be able to mine or interpret it. We gather the stuff, format it and then save it to backup tapes. But we get little actual use from all of our work in collecting it. Some decisions are made from the data, but given the real amount of underlying information contained in our efforts we could actually use what we have to make really sophisticated decisions. To actually know, for fact that we are changing things for the better. If we understood what we have.


Given those trends, it is not hard at all to predicate the future. The next step in the sequence. The path we must take is the only one available: machines to labour for us physically and mentally lead to vast infrastructures and vast piles of information. We built up these things, but we don't understanding them, and we have trouble keeping them going.

It is not like we will wake up one day and the light will get turned on, but I imagine that over time like a dull and steady wind blowing away the haze, much of what we know will become clear and finally fit into place. It will be a modern day Renaissance reoccurring not with our perspective of the world around us, but with our perspective of the information and knowledge that we have collected. It will take time. Many years, decades or even a century or two, but one day there will be an 'age of clarity', where mankind can finally see the information around them for what it actually is. That is, if we survive the turmoil of our current societies; we have so many dangers that await us, because of what we know, but don't yet understand.

And what could we expect in such an age? I imagine that we will have a real understanding of information, probably based on a currently unknown science. Maybe several. We will know how to quickly, conveniently monitor and collect information for any questions. Inherently, we will understand the truthfulness of what we collect, and we'll be able to immediately use this information to ascertain whether things are improving or getting worse. The term 'immediately' being one of the very key points.

Unlike now, this won't be a big effort, but rather something simple that people do as a matter of due diligence. Government effectiveness for example, will be based on simple true numbers that show that things are improving or getting worse. Unlike the statistics of our day, these numbers and their interpretation, based on science will be irrefutable. We will be able to show cause and effect relationships between policies and real life. We will be able to measure the effectiveness, not guess at it. If we say things are getting better, it won't just be 'spin'.

Underneath, if we capture enough data, we will get a vibrant picture of all of the relationships, how they fit with each other and what they really mean. When we choose to make changes, they will not be partially-informed guesses, they will be tangible deterministic improvements to our societies that will work as expected. In the same way that the industrial age leapt from wildly building things to the reproducible industrialization of products with a tremendous amount of consistent quality, we will shift our understanding of the data around us. Like the difference between B&W photography and color we will learn how to start really capturing the information that is of real value, and we will learn how to really interpret it.


Does it sound too overly deterministic or crazy? Whatever comes in the future, it must be something that isn't here now. So, if it isn't pushing the envelop of convention, then its not really much of a prediction, is it? Jules Verne wrote about ships that travelled underwater, a famously crazy concept if ever there was one, except that it now has become common knowledge. He wrote about air ships, defying gravity and hanging in the sky with birds, clearly another bit of wackiness. Yet, this too is common, and rather boring now.

Does it sound very similar to what we have now? For all we know, we know so very little. We have many approaches and methods to really prove things, to get the the real underlying truth, but because we can't do that easily on a grand scale we are awash in misinformation. All of this low quality knowledge chokes our pathways and keeps us from progressing. It becomes food for subjective arguments, and endless discussions. And while some of us may suspect falsehoods, proving it is costly and often distracting. We can't fight all of the battles all of the time, so the low quality stuff washes over us like a tsunami.

Sometimes when I am out walking the dog, my mind drifts around to us being so sophisticated that there is not much left in this world that we don't know. That 'proposition' is comforting in many ways, but patently false. Like the pre-Renaissance societies, we think we have reached some level of sophistication, but we barely even realize how to keep our own existence from spiraling out of control. And what we don't know, is the question: "what do we really know?" We feel pride in having built up knowledge bases like the World Wide Web, but realistically the things are a mess. What good is a massive unorganized pile of data, if we can't use it to answer the serious questions in our lives? We live in an age where subjective arguments are possible for most of what we commonly deal with in our lives. Everything is up for grabs; everything is based on opinion. We can barely distinguish the quality of our facts, let alone position them into some coherent and universally correct structure of the world around us. For all that we know, we are still incredibly ignorant.

The next big thing then is obvious. If we are too survive, then we have to pass through the Clarity Age. We have no choice. If this understanding hasn't already popped into someone else's brain, it was bound to sooner or later. You can't get very far down the path, if you don't know where the path lies. It is murky now, and for us to progress it must be clear.

Woof, woof, woof! My thinking and wanderings were interrupted because the dog spotted a raccoon. I was riped from the depths by the pulling, jumping and barking. In the here and now, I am reminded that it is best if I move on quickly to keep the dog from making too much racket. I don't want to wake up my whole neighborhood with the commotion. However much I long to spend time in the future, I must live with the world around me as it is now. These dark ages are apt to last a while, possibly my entire life. I ought not to waste it, pining for enlightenment.

AUTHOR'S NOTE: I publishing this piece in both my Irrational Focus, and The Programmer's Paradox blogs because it is written for both audiences. One is a general place for ranting, while the other is aimed specifically at software developers. This piece transcends both genre.