Friday, November 28, 2008

On Learning

I've been reading Pragmatic Thinking and Learning by Andy Hunt (who co-wrote the The Pragmatic Programmer) lately. The book brings up idea of second-order incompetence, which means that a novice at a particular skill is so unskilled that they don't even realize how unskilled they really are. This leads beginners to greatly overestimate their own ability. In many cases, it can lead a beginner to be more confident in their own skill than an expert.

The second-order incompetence theory applies to any skill in general, but I've noticed that it applies extremely well to programmers and their particular set of skills.

I've heard some programmers say that it takes anywhere from three days to three months to learn a new programming language. I personally feel that this is an extremely low estimate (I've been learning C++ for several years). Three days is barely enough time to learn the syntax of a new language. Three months is likely enough time to learn the syntax and a few common libraries of a language, but it isn't really enough time to get you beyond the "Advanced Beginner" skill level. It's probably just enough time to give you the confidence you need to be extremely dangerous.

The book goes on to explain that it really takes more on the order of ten years to truly master any non-trivial skill, whether it be playing chess, playing music, or flying an airplane. Peter Norvig quotes some of the same research when he advocates the ten year time scale for programmers in his article Teach Yourself Programming in Ten Years.

These ideas are hardly new. Socrates was onto a similar idea almost 2500 years ago when he said

"True knowledge exists in knowing that you know nothing."

In programming, as in almost any worthwile pursuit, this means that you have to give yourself time to reach a skill level high enough that you know how much you don't know. If you catch yourself feeling confident in a given language after only three days (or even three months), try to realize that you've probably only seen a tiny fraction of the what there is to see. Remember that 90% of an iceberg is below the surface.


Anonymous said...

I completely agree.

I was lucky enough to have my parents tell me the Socrates quote as a proverb ever since I was a little kid, trying to act smart.

Bill the Lizard said...


I'll try that on my smart-mouthed 14 year old. Something tells me that it's going to be a little bit too subtle for him. :)

Ken G said...

Amazing what you can find starting on StackOverflow...

I really like that graph and I'll have to follow up on the other references. The whole thing matches my "perceptions" about my own journey.

BTW, wisdom cannot be passed on to the next generation when said generation is between the ages of 13 and 19. At least not mine!

Bill the Lizard said...

Ken G,

"Wisdom cannot be passed on to the next generation when said generation is between the ages of 13 and 19."

Awesome quote! I guess every generation has to learn this for themselves. :)

Anonymous said...

If you know the syntax of the language (half a day to 3 days depending on the language), in most cases that is enough to fool most people. There are a few languages like Haskel that are completely different from anything else, but most languages you will deal with are substantially the same (based on C) so you can look things up. Of course if you were an expert you would do a lot of things differently because you would take advantage of some features that don't fit into what you know.

I have personally had the experience of after 2 months of a new language explaining to the creator of the language a better way to solve a problem in it. There are still things I don't use in the most ideal way, but in only took a few days until I could fool other programmers into thinking I was better than I am.

troelskn said...

The problem with the ten years argument, is that it doesn't take previous experience into account. If I started now, and hardly did anything else, I could perhaps become a good guitar player in some 5..10 years time. But if someone who already is an expert bass player, picked up a guitar, he could probably achieve the same skill level in - say - a couple of months.

The same goes for programming languages. A lot of languages are very similar in their idioms and in the context they are use in. If you are already a highly skilled Python programmer, you *can* probably pick up Ruby in three months. The first three days may be spend learning the new syntax, and the rest of the time on learning the libraries and the few idioms that differ. In any case, it won't take ten years.

On the other hand, if you've never done any programming before, it may well take you ten years to learn that skill.

Bill the Lizard said...

You make a valid point about prior experience, but I don't know how far it applies to programming languages. Python and Ruby are fairly similar languages, so it feels to me like you've loaded the deck. What about C++ and Lisp? Heck, what about C and C++? You could be an expert in one and still take quite a bit longer than a few months to master the other. Still, I take your point: It wouldn't take 10 more years.

Bill the Lizard said...

There's no real danger in other people mistakenly thinking you're an expert in a language (except maybe if it's your boss). Be careful you don't fall into the trap of believing you're an expert yourself after only a few days with a new language, though. That's exactly the kind of second-order incompetence I mentioned in the body of the post. It's very difficult for you to gauge your own ability when you've just recently been exposed to something new.

Evgeni said...

There is so-called "Dunning-Kruger effect" in which "...people reach erroneous conclusions and make unfortunate choices but their incompetence robs them of the metacognitive ability to realize it"

Not sure if it's the same as the "second-order incompetence".

Bill the Lizard said...

Yes, the term "second-order incompetence" used in Pragmatic Thinking and Learning is the same as the Dunning-Kruger effect. They do cite the original paper Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments by Dunning and Kruger, but I'm not sure why they chose to use a term that doesn't appear in that paper to describe the same effect.