Automation bias is closely related to automation complacency. It creeps in when people give undue weight to the information coming through their monitors. Even when the information is wrong or misleading, they believe it. Their trust in the software becomes so strong that they ignore or discount other sources of information, including their own senses. If you’ve ever found yourself lost or going around in circles after slavishly following flawed or outdated directions from a GPS device or other digital mapping tool, you’ve felt the effects of automation bias. Even people who drive for a living can display a startling lack of common sense when relying on satellite navigation. Ignoring road signs and other environmental cues, they’ll proceed down hazardous routes and sometimes end up crashing into low overpasses or getting stuck in the narrow streets of small towns. In Seattle in 2008, the driver of a twelve-foot-high bus carrying a high-school sports team ran into a concrete bridge with a nine-foot clearance. The top of the bus was sheared off, and twenty-one injured students had to be taken to the hospital. The driver told police that he had been following GPS instructions and “did not see” signs and flashing lights warning of the low bridge ahead.
Interesting excerpt that introduces the concept of automation bias. It is when we began to rely too much on technology that we no longer function consciously, as in the case of following GPS instructions.
There might be no way to avoid this as more parts of our lives get simplified by technology. But, it is between the intersections of technology and human input that we have to be aware of this potential bias and be mindful in keeping it in check.
While we can trust what computer systems output to us most (if not almost all) the time, it is still healthy to keep ourselves thinking actively and react when the technology is wrong.
