As a retired mathematician, how could I resist this book title? When I read that economic inequality was furthered by use of mathematics, I was even more intrigued. I’m also a retired political activist. In fact, I’d like to retire and wake up on November 9th.
My first thought was that like any tool, mathematics could be used for harm or good. One can use a hammer to work for Habitat for Humanity or do someone serious harm. I was being a prig in taking the clever title too literally.
Okay, so how is math being used to keep the poor, well poor? According to the author, Cathy O’Neil, algorithms and big data, intentionally or not, target the poor, reinforce racism, and amplify inequality. In a court of law or in a general education course on critical thinking, guilt by association is decried as irrational. Yet, that’s exactly what happens when one’s zip code is used to determine loan eligibility or a loan’s interest rate along with home and car insurance rates.
Employers use credit scores to measure responsibility, but this equates dependability with higher income. If your credit record is due to unemployment, your unemployment keeps you unemployed. In order to prevent unfair bias in sentencing, recidivism models were adopted by some states. However, the model includes criminal records of friends and family.
Personality tests have been devised for employment purposes. The math presumably only figures in the final score of acceptable answers. It’s not clear math has been used to verify a correlation between passing and performance on the job. The tests have been accused of measuring averageness, thereby denying outliers whose creativity could be an asset. This simply adds up (pun intended) to unfairness across the board rather than a bias against the poor.
McNeil calls these tests WMDs and labels them opaque and unfair. There is no explanation of what went wrong when these measures are used to deny life benefits.
Not mathematics, but the use of statistics is responsible for this judgment by association. More precisely, if you have a zip code, credit score, or other identifier that a software program places you in a demographic less likely to succeed, it becomes a self-fulfilling prophecy.
I read a very interesting review of the book that made me return to my thought about a tool’s capacity for good or harm. The reviewer had been poor and later lived in a neighborhood with a favorable zip code. While the reviewer suffered from the algorithms mentioned in the book, he also talked about some of the positive aspects of the tools. He appreciated more police deployment in his high crime zip code and a mechanical measure of tardiness, which prevented bosses from forgiving their pals.
If nothing else, Ms. O’Neil has heightened awareness of built-in biases in programs where we expect objectivity.