How to lie with statistics is the name of a famous book by Darrell Huff in 1954 explaining how easy it is to lie with statistics. The book is almost 70 years old, but the topic is still relevant. Why is that? “Lies, damned lies, and statistics” describes the persuasive power of statistics. Statistics can be extremely useful when it is used in the right way and deceptive when it is used in the wrong way. In some cases statistics is deliberately used to prove something that you know is not true, and sometimes it happens unconsciously. Regardless, it is a problem when the numbers can’t be trusted.
Statistics are commonly used for predictability, identifying trends and artificial intelligence. Companies are utilizing these techniques to gain insight into their businesses and stay one step ahead of their competitors. With the growing digitalization, more companies want to implement and integrate artificial intelligence and statistical models into their businesses. When an increasing amount of companies want to implement these solutions, the pool of good machine learning and mathematicians consultants is drained. It leads to poorer implementations where the quality and functionality are reduced significantly since no auditing framework is available or required. Without a solid understanding of statistics and artificial intelligence, probability of deceptive performance increases.
As it stands now, no auditing of artificial intelligence or statistical models is required, which makes the evaluation process discontinuous and difficult to interpret. These unrestricted and black-box models create problems for organizations/institutions where transparency is crucial such as public authorities and government entities. However, the large upside of implementing and utilizing artificial intelligence and machine learning models is constantly getting larger and sooner or later, the public authorities and government entities will be forced to use the cutting-edge technology.
As a result, some institutions have started forming guidelines on how an audit framework could be constructed. The Supreme Audit Institutions (SAIs) of Finland, Germany, the Netherlands, Norway and the UK have formed vague guidelines for machine learning auditing, which a handful of government agencies have applied. But for the government and public authorities to use artificial intelligence and machine learning tools, a full-scale audit framework must be in place to ensure transparent, robust and functional implementations.
