DUNNING AND KRUGER EFFECT

 

WE DO NOT KNOW WHAT WE DO NOT KNOW: DUNNING AND KRUGER EFFECT

 

“The less people know, the more stubbornly they know it”- Osho  

          About three and a half decades back two young pilots, with limited experience were on takeoff roll at an international airport with the younger of the two (holding D/White) as captain. The Copilot (holding C/White) decided, for inexplicable reasons, to simulate Engine Failure after takeoff (EFATO). Short of unstick he reduced power on one engine and subsequently retracted the undercarriage prematurely causing the aircraft to sink and skid on the runway on its belly. The runway remained shut for a significant time. Why would someone do such a thing?

          The copilot’s exposure to such simulations would have been limited to few in training exercises with instructors. He would have handled them well and concluded that the entire exercise was rather mundane and benign. “The instructor retards the throttle to idle and pupil continues with the take off. After all no one faced any problem and definitely it was as simple as that. Also, the other engine power can be opened up in case of anything going wrong.” Such mindset, as a result of inexperience, would have prompted him to do what he did.  He, due to his limited exposure, was incapable of knowing what all factors come into play in such an exercise.  Simply put, he did not know the things that he did not know. This ignorance gave him a false sense of security and confidence about the exercise. The self-appointed instructor could be diagnosed by what is called Dunning and Kruger effect, a cognitive bias in which people who know very little about a particular subject tend to overestimate their knowledge and skill.

          On 19 April 1995 a man named McArthur Wheeler carried out daring broad daylight robbery on two banks in Pittsburg. He wore no mask and smiled deliberately looking at security cameras. The police beamed the footage on local TV channels. A tip came within minutes and he was apprehended within hours. Wheeler was shocked to learn that he has been caught on the cameras and mumbled, “But I wore the juice”.  The police thought that he was either delusional or under drugs. But on probing it emerged that he had rubbed lemon juice on his face to make it invisible to security cameras. He knew that lemon juice is used as an invisible ink and logically, he thought, that would make his face invisible to cameras. He tested it out before the robbery by putting lemon juice on his face and snapping a selfie with a polaroid camera and found that the photo had no face (Police never probed his claim but most likely Wheeler was no more competent as a photographer than he was as a bank robber. Perhaps he could have been pointing camera in wrong direction).  The story of Wheeler caught the attention of David Dunning, a Cornell psychology professor who found him not only too stupid to be a bank robber but also too stupid to know that he was too stupid to be a bank robber. He, though, saw in him traces of a universal phenomenon- ‘Those most lacking in knowledge and skills are least able to appreciate that lack.’  Prof. Dunning along with his student Justin Kruger set out to test the hypothesis, ‘Do people really remain innocent of their own deficits even when those deficits are severe.’ They tested multiple groups of students in various subjects to assess their actual and perceived abilities. The students were asked to complete a test paper to evaluate their knowledge. They were then asked to rate their own ability in the subject, predict their test performance and also evaluate themselves in comparison with others.

          The study found that the people who had scored the lowest in the tests had the most inflated opinion of their skills. The incompetent (poor performers) in-fact face double burden. “Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it,” wrote Kruger in his research report. The study also found that incompetence doesn’t leave individuals with empty-thoughts, disillusioned or even cautious. Instead, the incompetent is filled with over confidence. The concoction of poor skill, blindness to their mistakes and overconfidence makes such people bullish. Basking in their delusional knowledge they become self-assured and ceases to learn and grow. Dunning also observed that the most dangerous situation is when people have some knowledge about a subject rather than when they know nothing about it. The research was completed in 1999 and validated the hypothesis that “incompetent people cannot recognize just how incompetent they are.” The reason for this paradox is that the qualities that are required to be good at a task are often the same that are needed to recognize that one is not good at that task and if one lacks such knowledge and intelligence, one remains ignorant about his ignorance. Psychologists call the ability to evaluate knowledge as metacognition. In general people who are knowledgeable in a given domain have better metacognitive ability than people who are not knowledgeable in that domain. An interesting observation that emerged from the research was about the high scorers. They were found to be more accurate about their performance but tended to underestimate their performance relative to other participants. Dunning concluded that people who were knowledgeable often have less confidence in their abilities as they know enough to know a lot they do not know and they also tend to mistakenly assume that everyone else is knowledgeable too. This confirms to a rather paradoxical reality bluntly stated by English Philosopher Bertrand Russel almost ninety years back, “The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubt”. Dunning also warns us not to confuse confidence with competence.

          Returning to the incident, let’s understand the exercise of simulated EFATO from the perspective of an instructor. The exercise needs authorization and a suitable environment. It definitely cannot be carried out in the midst of an operational commitment at an international airfield. Statistics are testimony to the fact that far too many accidents have taken place during emergency simulations. On 25th March 1991, HS-748 Avro crashed taking the life of 25 young trainees of AFTC and the three aircrew. The aircraft was simulating EFATO when speed washed off below VMCA. Simulating engine failure close to V1 is potentially hazardous and many organizations and individuals consider it best to wait for a safe height and speed V2. The simulation in the case under discussion started on the ground itself. The instructor caters for the psychological effects like startle, panic, freezing etc. which a trainee may experience during such exercises and anticipates mishandling of the controls like applying wrong rudder and not giving required aileron. He is also live to the fact that there could be additional actual emergency and caters for it by giving more space and time before simulating. Understanding of aerodynamics plays important role in handling engine out situations.

          When the gear is selected up there is significant increase in profile and interference drag due to opening of bay doors and change in the angle between landing gear assembly and fuselage. That’s why there is a minimum height, around 5 meters, at which the gears are selected up.  The copilot retracted the gear prematurely thinking, erroneously, that it would assist the take off by reducing the drag. Instead, the increased drag at critical time when aircraft was on one engine, close to ground and with low speed resulted in degraded performance resulting in aircraft settling down on the runway with undercarriage retracted. In such cases even if the aircraft had continued takeoff there were chances that it would have scraped the runway or worse the speed could have dropped below VMCA leading to loss of control.  A twin turbo prop loses as much as 75 percent of its performance with one engine out. The statistics on engine failures after takeoff shows that the most likely place an engine can quit is when throttles are retarded after takeoff. It is, therefore, simply dumb to reduce power before recommended height which is normally 400 feet and after cleaning the aircraft.

           Dunning-Kruger effect is seen more amongst new aircrews. The American aviation author Paul A Craig says that during the period when a newly minted pilot is building experience, they seem to be caught between two worlds. “On one hand they are fully licensed and legal pilots, but on the other hand they have not had the opportunity yet to learn from experience.”  As per Alan Gorthy, FAAs runway safety inspector, a pilot goes through four different stages of competency. The first stage is when he has just started and he is so inexperienced that he doesn’t know what he doesn’t know. They don’t even know what they do not know. At second stage they had had just enough scares to know what they don’t know and attempts to fill those gaps. At stage three despite having filled the gaps the pilot continues to seek out knowledge and challenges. Finally at level 4 the pilot becomes a natural and having reached the apex of flying skill can handle normal, abnormal and emergency procedures. Reaching the final stage require life time commitment to learning which is not the case with all. Hence this effect could be found despite huge amount of experience and age.

                    Dunning-Kruger effect was responsible for the crash in US of Pinnacle Airlines Flight 3701 Bombardier CRJ-200 on 14 Oct 2004. The airplane was on a ferry flight with no passengers and cabin crew on board. The two pilots who were the only occupants decided to test the aircraft by climbing to the max operating altitude which was 41000 feet. The FDR and CVR showed them flying dangerously during climb. Their attempts at zoom climbing recorded g-meter between 1.8 to 2 g at multiple times with prohibitive nose up attitude and high angle of attack. The speeds towards the last few thousand feet were drastically lower than mandated in the Flight manual. At 41000 feet the aircraft was not able to maintain height and speed leading to stall warning activation. The stick pusher activated 4 times but the crew overrode it all 4 times pulling the nose prematurely fixated on avoiding loss of altitude and thus, culminating all recoveries in secondary stall. These maneuvers over stressed the engines and also led to full blown aerodynamic stall resulting in upset where at one time the aircraft reached 82 degrees of bank. The upset exposed both engines to inlet airflow disruption conditions leading to engine stall and complete loss of both engine power. All attempts at windmilling restart failed as the airspeed was much lower than required to adequately spin up the engines. The low speeds gradually led to both engine cores to stop. With engine speed remaining zero the engine could not have been started through windmilling process. Finally at 13000 feet they attempted a APU assisted start which too failed. Eventually the aircraft crashed killing both the pilots.

          The pilots made multiple mistakes due to lack of exposure and knowledge about the environment in which they chose to fly. Little did they realize that the maximum operating altitude depicted in the flight manual represent the maximum capability of the airplane. The actual performance is dependent on weight, airspeed and ambient temperature which can change day to day. They never consulted any performance charts and flew at much lower speeds than recommended. They failed to appreciate that they were flying in the region of reverse command and close to coffin corner. The steep angle of attack increased the induced drag to an extent that the aircraft was unable to maintain speed and height. Finally, when the aircraft stalled, they resorted to incorrect techniques which are employed when stalling close to ground which require minimum loss of altitude. At 41000 feet the aircraft could have been un-stalled only by sacrificing altitude as the power output from engines was low. It was clear that they had no exposure to high altitude stall recovery. The restart attempts were all unsuccessful as they never reached the speeds required for windmilling start. At these altitudes the aircraft was required to be put in a massive dive to achieve those speeds. Obviously, the crew had no exposure to restart at high altitudes. The low airspeeds finally led to both engine cores to stop. With the core engine speed dropped to zero, windmilling start was out of question. The APU assisted attempt to restart also failed as by that time both engines experienced core lock which occurs when an engine is shut down at high altitude and the cold temp causes various parts of the engine to contract at different rates preventing compressor and turbine blades from spinning. The accident was the result of pilot actions which exposed engines to extreme conditions and thereafter failed to maintain target airspeed which caused the engine cores to stop rotating.

          “A little knowledge is a dangerous thing”, is well known quote of Alexander Pope. On similar lines Charles Darwin said, “Ignorance more frequently begets confidence than does knowledge”, about human nature, almost 160 years back, after he was trolled cruelly by millions of people for his theory of humans evolving from apes which contradicted with the Biblical version. Both the CRJ-200 pilots as well as the pilot who was acting as instructor were beaming confidence in their little knowledge. Their actions can also be summarized by a quote of Josh Billings, “It isn’t what you don’t know that gets you into trouble. It’s what you know for sure that just is not so”.  Dunning’s research, though, takes these ideas one step further suggesting that the less competent we are in a given area, the more likely we are to unknowingly exaggerate our competence. The keyword here is ‘unknowingly’, indicating that those effected are not aware that they are overestimating their own ability. So, it’s not that the pilots everywhere are blatantly ignoring the rules. It’s just that most do not know what they don’t know. Most of us are blind to our blindness.

          Curiosity, openness and a lifelong commitment to learning can help to minimize Dunning-Kruger effect. The more we learn, the more we know how much we do not know.  Learning more can help us to recognize more accurately our ability and performance. “The real knowledge is to know the extent of one’s ignorance” said Confucius around 500 BC. Recognizing your ignorance is the first step towards learning.  All aircrew should feel morally responsible to commit to lifelong learning. Every aviation incident, accident and close call have lessons on how to avoid them. They are all available freely and begging to be read. Accident statistics are also testimony to the fact that even the best and most skilled pilots have made poor decisions leading to catastrophes. No one is immune to errors and no one knows all that is there in aviation. We all have pockets of ignorance and blind spots.  The Dunning-Kruger effect is a good reminder that the better you think you are, the greater the risk that you are overestimating yourself-and the greater the odds you will stop improving. Being humble and accepting your ignorance and attempting to know what you do not know is the first step toward developing expertise.

“It is a fact that this phenomenon (Dunning-Kruger effect) visits all of us sooner or later. But not knowing the scope of your ignorance is part of the human condition. The problem with it is we see it in other people but we don’t see it in ourselves. The first rule of the Dunning- Kruger club is you don t know you are member of the Dunning-Kruger club.” --- Prof. David Dunning