Saturday, February 2, 2008

The Age of Clarity

"What do we really know? Hmmm." I pondered as we walked.

I was out with the dog the other night. The quiet tranquil nature of empty suburban streets is a great place for deep thinking. The cold chill of winter keeps one from wandering too far off topic while wandering aimlessly in the streets. Dogs make wonderful intellectual companions for these types of journeys because they don't interrupt with too many questions. They are very good listeners.

I was pondering information quality, and I foolishly started to wonder about how much inaccurate information was choking up my memory. Certainly, there are lots of spin, lies, half-truths, deceptions and other stuff built up over the years from less than quality sources like politics, news and TV. Somethings in my memory are just easy simplifications. Somethings are out right fabrications. There is also the changing nature of science, and our non-stop quest for learning. Some of my knowledge is just 'relative', it wouldn't stand up to a universal judge. It is considered true here and now, but won't be in the future. In an overall sense, how much of this is really accurate?

If you factor in all of the different reasons for low quality, and take a big sweeping guess, the amount of truth in our brains could be lower than 30%. Just a wild guess, but I could easily believe that 1 in every 3 three facts in my brain are true, while the other 2 are questionable for all sorts of reasons. I am just speculating of course, but in this misinformation age, we are full of a tremendous amount of low quality knowledge. And it feels like it is growing at an ever increasing rate, although that might just be our ability to confirm that it is suspect.

THOSE WHO FORGET HISTORY ARE DOOMED TO REPEAT IT

In the past, mankind was mostly ignorant of the accuracy of their information. They could take pride in their depth of knowledge without ever knowing how dubious it really was. Now all we have to do is check wikipedia and we can instantly find out that truth, well, err at least a pointer towards the truth.

How often have I pulled forth some ancient fact from the depths of my brain, only to discover that it was fundamentally untrue? Worse still is how those facts actually make it into my head in the first place. Some were obviously from disreputable sources, but others had come from well-known authorities, and were still incorrect. My problem is not loss or corruption of memory, it is the opposite, these 'facts' stay for far too long. If I just dumped them faster, I might find they were more accurate overall.

It is oddly telling. It allows us to guess that this huge degree of inaccuracy in our current knowledge is actually some type of pointer towards the future. The Renaissance was an awaking about the world that we live in. A moment when we first opened our eyes and saw it for what it actually was. This in turn drove the foundations for the industrial age, where we learned to create and use an unlimited number of machines. One of those machines, the computer, has driven us into an information age, where we collect huge piles of information, about virtually everything in this world. There is a trend here. The next age will follow along in this sequence.

Even though we have built up a tremendous collection of fantastic machines, they do not serve us well. We can build things, but we have trouble maintaining them. Our massive and complex cities crumble around us. We are forever fighting a losing battle against entropy; like a runner that has leaned too far forward we are continually off balance. We must continue to build to move forward, we don't know how to preserve what we have and we don't know how to live within our environmental means. We grow at a severe cost to the world around us.

With all of our equipment and learning, collecting data is still a hit or miss proposition. We just guess at what we want to collect and how it is structured. It is not orderly and we don't have any underlying theories that drive our understanding. Computer Science is still so young that is frequently wrong. Often it is just random guessing. We are currently only utilizing a small fraction of the capabilities of our computers because we keep bumping into complexity thresholds each time we try to build truly sophisticated systems. We are trapped with crude software.

Even thought we can collect the data, we continuously fail to be able to mine or interpret it. We gather the stuff, format it and then save it to backup tapes. But we get little actual use from all of our work in collecting it. Some decisions are made from the data, but given the real amount of underlying information contained in our efforts we could actually use what we have to make really sophisticated decisions. To actually know, for fact that we are changing things for the better. If we understood what we have.

THE YELLOW BRICK ROAD

Given those trends, it is not hard at all to predicate the future. The next step in the sequence. The path we must take is the only one available: machines to labour for us physically and mentally lead to vast infrastructures and vast piles of information. We built up these things, but we don't understanding them, and we have trouble keeping them going.

It is not like we will wake up one day and the light will get turned on, but I imagine that over time like a dull and steady wind blowing away the haze, much of what we know will become clear and finally fit into place. It will be a modern day Renaissance reoccurring not with our perspective of the world around us, but with our perspective of the information and knowledge that we have collected. It will take time. Many years, decades or even a century or two, but one day there will be an 'age of clarity', where mankind can finally see the information around them for what it actually is. That is, if we survive the turmoil of our current societies; we have so many dangers that await us, because of what we know, but don't yet understand.

And what could we expect in such an age? I imagine that we will have a real understanding of information, probably based on a currently unknown science. Maybe several. We will know how to quickly, conveniently monitor and collect information for any questions. Inherently, we will understand the truthfulness of what we collect, and we'll be able to immediately use this information to ascertain whether things are improving or getting worse. The term 'immediately' being one of the very key points.

Unlike now, this won't be a big effort, but rather something simple that people do as a matter of due diligence. Government effectiveness for example, will be based on simple true numbers that show that things are improving or getting worse. Unlike the statistics of our day, these numbers and their interpretation, based on science will be irrefutable. We will be able to show cause and effect relationships between policies and real life. We will be able to measure the effectiveness, not guess at it. If we say things are getting better, it won't just be 'spin'.

Underneath, if we capture enough data, we will get a vibrant picture of all of the relationships, how they fit with each other and what they really mean. When we choose to make changes, they will not be partially-informed guesses, they will be tangible deterministic improvements to our societies that will work as expected. In the same way that the industrial age leapt from wildly building things to the reproducible industrialization of products with a tremendous amount of consistent quality, we will shift our understanding of the data around us. Like the difference between B&W photography and color we will learn how to start really capturing the information that is of real value, and we will learn how to really interpret it.

AT THE EDGE OF REASON

Does it sound too overly deterministic or crazy? Whatever comes in the future, it must be something that isn't here now. So, if it isn't pushing the envelop of convention, then its not really much of a prediction, is it? Jules Verne wrote about ships that travelled underwater, a famously crazy concept if ever there was one, except that it now has become common knowledge. He wrote about air ships, defying gravity and hanging in the sky with birds, clearly another bit of wackiness. Yet, this too is common, and rather boring now.

Does it sound very similar to what we have now? For all we know, we know so very little. We have many approaches and methods to really prove things, to get the the real underlying truth, but because we can't do that easily on a grand scale we are awash in misinformation. All of this low quality knowledge chokes our pathways and keeps us from progressing. It becomes food for subjective arguments, and endless discussions. And while some of us may suspect falsehoods, proving it is costly and often distracting. We can't fight all of the battles all of the time, so the low quality stuff washes over us like a tsunami.

Sometimes when I am out walking the dog, my mind drifts around to us being so sophisticated that there is not much left in this world that we don't know. That 'proposition' is comforting in many ways, but patently false. Like the pre-Renaissance societies, we think we have reached some level of sophistication, but we barely even realize how to keep our own existence from spiraling out of control. And what we don't know, is the question: "what do we really know?" We feel pride in having built up knowledge bases like the World Wide Web, but realistically the things are a mess. What good is a massive unorganized pile of data, if we can't use it to answer the serious questions in our lives? We live in an age where subjective arguments are possible for most of what we commonly deal with in our lives. Everything is up for grabs; everything is based on opinion. We can barely distinguish the quality of our facts, let alone position them into some coherent and universally correct structure of the world around us. For all that we know, we are still incredibly ignorant.

The next big thing then is obvious. If we are too survive, then we have to pass through the Clarity Age. We have no choice. If this understanding hasn't already popped into someone else's brain, it was bound to sooner or later. You can't get very far down the path, if you don't know where the path lies. It is murky now, and for us to progress it must be clear.

Woof, woof, woof! My thinking and wanderings were interrupted because the dog spotted a raccoon. I was riped from the depths by the pulling, jumping and barking. In the here and now, I am reminded that it is best if I move on quickly to keep the dog from making too much racket. I don't want to wake up my whole neighborhood with the commotion. However much I long to spend time in the future, I must live with the world around me as it is now. These dark ages are apt to last a while, possibly my entire life. I ought not to waste it, pining for enlightenment.


AUTHOR'S NOTE: I publishing this piece in both my Irrational Focus, and The Programmer's Paradox blogs because it is written for both audiences. One is a general place for ranting, while the other is aimed specifically at software developers. This piece transcends both genre.

Saturday, January 26, 2008

Explosive Tendencies

I probably shouldn't writing about this. Going against what seems to be the current popular viewpoint, particularly when you are not an expert, is never a good idea. Under most circumstances I might not chime in, but for this one in particular, I've seen it going on for far too long and I though at least I might put my perspective out there. Carefully.

So I'll add a disclaimer, that says something like bla, bla, bla, I'm not an expert, bla, bla, bla, this is all observation, bla, bla, bla, use your own brain. And of course, positive comments are hugely appreciated, negative ones you should probably keep to yourself. I'm not looking to start a controversy, I just want encourage people to think a little more deeply about this. There is clearly a problem, and we are doing our best to ignore it.

The winds changed after I left high school, with the shift often dominating the news. It started with things like the zero tolerance towards violence in high schools. At the same time we could also see changes in the perspective of parenting too. Parents weren't supposed to spank their kids any more. In fact overall society was pushing for a violence-free life in all sectors.

This is one of those movements that are socially extensive, kinda of like non-smoking. Once it gets going, more and more people jump onto the bandwagon and declare themselves to be on side. More and more programs get funded, and more and more changes gradually show up in our societies. Once the ball starts to move people stop thinking about it, accepting it as fact.

At its heart, getting rid of violence is a great thing. Nobody liked being bullied when they were kids, and it is horrible for parents to beat their kids. From that perspective it is definitely a move toward being more civilized.

However, for all of its intended good, one cannot turn a blind eye towards the apparent consequences.

Humans are lost somewhere in the middle. Straddling a fence. We are not the fully civilized intellectual beings that we like to think of ourselves. Our roots are in nature, we have evolved into our present state, and we are still firmly tied with our former primitive selves. Because of this, we have two distinct sides, our intellect and our emotions. Our intellect is the direction we're headed. We are evolving into truly intellectual beings that preserve themselves because they 'think' to do so. But, we are also still rooted in our past. Our emotions are are much older fight or flee based mechanism for handling the world around us. This duality permeates our existence.

While we can control our emotions to some degree, they are definitely a part of our existence, we cannot deny that. Some cultures allow for looser expression of emotions, some expect that it is under more control. Either way, all cultures have ways of handling emotional outbursts and venting. We have to, we have no choice.

Getting back to our violence-free attempts in society: the underpinnings of violence are often emotional outbursts. Violence is how some people vent their emotions. Anger builds and then erupts into a violent action. The two are related.

Once you make that connection, you have to start thinking that suppressing violence, especially at a young age, might have a significant effect on emotion, in particular the ability to control one's own emotions. Since we all have tempers, and they need to be controlled at times, were we did we learn these skills?

Two little boys in a fight is an age old tradition that teaches both about (a) pecking order, (b) emotional control, and (c) relationships. The need to fight is the need to vent and establish a ranking, very old things, but still buried within us. It is not just the boys, the girls also go at each other although a little less physically.

A fight between two very young boys is far less damaging then a fight between two teenagers. In fact, for kids, the line between playing and fighting is often blurred, it is a little of both. The younger you are, the safer it is to experiment around with your behavior.

When you fight, you get a huge release. As well, there are important lessons that we all need to learn through experience, such as standing up to bullies. You need to conquer your fear and face your oppressors, something that will happen again and again in your life as you age, so you is is best to learn it early. We also learn social skills, such as the consequences of mouthing off to larger people. It is important to know when you are picking a fight with someone, it should never come as asurprise.

If you eliminate violence in schools and don't let the children physically interact with each other, how will they learn to control their emotions? How will they learn to interact with each other in a socially correct manner? There are so many different life-lessons that come from rough-housing with your peers.

But it's not just fighting between children that is effected. At home, without spanking, we've taken away from the parents their key tool in disciplining their children. The lack of discipline also seems to be a serious problem. If you have no 'ultimate weapon', sooner or later the kids are going to figure that out and call you on it. Beyond that, most kids act out as emotional outbursts, so not only are they not getting disciplined, but they are also not learning how to handle their emotions properly. Quiet time isn't sufficient.

You see more and more kids acting out in public. Certainly there are ages like the 'terrible twos' where you expect trouble, but it appears as if its also affecting them when they are older. In fact it seems to affect them right through their entire lives.

We appear to be seeing the incidences of violence get far worse than ever before. Over all there may be less visible violence, but each incident is far more serious. Far more explosive.

This stands to reason, if you suppress our violent nature, and don't teach people ways of venting themselves, then the emotional flare ups will be so much larger than they were in the past. Emotions, like electricity are neither created nor destroyed. You can't wipe out violence without throwing in millions of years of evolution.

We've also seen more kids packing guns, because they don't know how to protect themselves. They are afraid because they've never been in a real fight, so they think that packing a weapon will help protect them. If they knew how to slug people in the head, then they might realize that they don't need the extra help, or that escalating a conflict by pulling a weapon is a seriously dangerous thing to do. Knowledge that they should have learned in kindergarten, or junior high school.

There are just some things you have to learn in life, and it is best that they get taught early.

It is also important to realize that most successful people have very good control of their emotions. There are some notable public emotional break-downs, but in general the outbursts of emotion are delayed and then directed. Our most successful and controlling people have their emotions in check. They should, we shouldn't make choices based on anger, or depression. How would you like it if the president of the United States just picked a fight because he was feeling cranky*. Not good.

*oops, possibly not the best example :-)

If parents want their children to be successful, they have to give them the tools to do so. No self-discipline and a lack of emotional control are not winning attributes. Punishing your kids within reason is, as my parents so often like to point out, is for their own good. Letting them interact together at school helps them to learn to deal with people.

Our noble goal of getting rid of violence appears to have a serious side effect. By removing violence, I think we are suppressing our emotions, not venting them. But because they are there, as an integral part of us, suppressing them just builds them up to the point where they erupt. So, it comes as no surprise that since this anti-violence movement has gained momentum, we've seen more violence, particularly in the form of school shootings or gang violence. These are essentially kids exploding emotionally all over the place because they were never taught how to control themselves.

Clearly, one of the most critical things we need to do to fix our existing school system is to add in emotional teaching along with intellectual teaching. Kids need to be taught how to properly vent. They need to fight, and they need to establish a pecking order. We don't have to like it, but bullies are important lessons that need to be taught to young children. Banning it as a behavior is clearly causing problems, instead we need to admit to it, and find a way to 'teach it', or at very least 'structure it'.

We also need to allow parents to discipline their children in a reasonable manner. It is probably a case-by-case basis, some kids are clearly more difficult and obstinate than others. We don't want to return to the "kids should be seen and not heard" days, but parents should have a tool such as spanking available to them, and they should know how to use it correctly. Parents should never be afraid of their kids.

In the end we all need to admit, that at least for the time being, we are still emotionally driven creatures. Violence is an expression of emotion, and it will stay with us for as long as we have emotional ties, probably a few million years or so. Denial is a bad thing.

Sunday, January 6, 2008

A Telltale Honk

Beep, beep. Misunderstood and abused, the horn in your car is actually a very clever communications device. We say more with a simple tap on the horn than I think most people could ever realize. For many, their communication choices might change if they really understood what they were shouting.

For instance, with some honks we may be making a statement about another driver. A simple short honk directed at a car that has just cut you off is a quick statement of saying "hey, watch it, you almost caused an accident with your reckless driving!" Letting someone know this hopefully helps them to try and behave in a less foolish manner the next time. Short and sweet.

A longer more dramatic honk aimed at someone performing an illegal maneuver says something more serious like "if it were up to me, I'd remove your licence, you idiot!". Longer it is, the more dramatic the statement.

A series of short staccato honks, over and over again, says more emotional things like "what", "the", "hell", "do", "you", "think", "yoooour", "doing". Possibly reminding someone that there are in fact rules of the road, and that they should possibly attempt to follow them a little more literately. Good in circumstances where the other driver "shouldn't be".

With some honking we may actually be making a statement about the various injustices of our society or even the entire planet. A long series of mixed length honks often replaces the expletives in the full sentence that we are desperately trying to communicate, as in "what the beep do you beep think beeeep you are doing. You beep-head. Beep."

Or as we are often stuck in traffic one might use their horn to utter to the world "for beeps sake, beep, beep the government and beep the city planners that are beeping responsible for this beep beep beep beep pathetic beeping traffic". Sometimes venting is a healthy form of communication, although sometimes it is not.

While horn honking generally reflects our statements of the world around us it can often show us up for who we truly are. For instance while sitting outside of someone else's house in a quiet suburban neighborhood in the evening, beeping to inform them you are waiting says defiantly "I am so totally lazy that I can't be bothered to get up and ring the doorbell. I am just not an impressive person, you probably wouldn't like me." Beeping in the morning is similar, but less definitive.

Beeping at a car that actually can't get out of your way or in fact has the legitimate right way often means "I so beeping impatient that most people think I'm a huge beeep. Even my dog beeps me. I beep." Sometimes I think that people aren't actually that rude, they must just be checking to see that their horn is still working; just in case.

Certainly the most despicable honking comes from letting it rip behind anyone doing a left turn and always means "I am such a freakin beep that even though I can't see the road ahead, I will still beep you with my beeped opinion! I should rot in beep." Somethings are beyond rude, they are just stupid and dangerous.

As if you were standing in the middle of street talking to yourself, it is fair for people to judge you by the way you communicate with your horn. In our cars we like to believe we are invisible, but that pretty much isn't the case. Standing in the supermarket isle screaming to yourself is embarrassing, so is honking your horn for a stupid reason in the middle of an intersection. If you're 'barking' at less than nothing, we have the right to assume you are having sever internal difficulties, since the external ones appear to be non-existent.

It is important for people to understand that they are communicating with the world around them, and while they believe that they may be commenting on the current events, they may actually just be commenting on themselves.