Health Tech Reads: Weapons of Math Destruction by Cathy O'Neil

This week Health Tech Reads would like to welcome Sean Erreger and thank him for contributing to our knowledge about healthcare. Sean is a great ally for improving mental health and digital tools and supporting the community. He teaches about social media and being a  LCSW. You can follow his blog and and be sure to get updates on twitter Sean is a Social Media Ambassador for #HIMSS18

From Sean:

In the last 8 months or so I have been making a shift from technology enthusiast to a bit of  a tech skeptic.  Technology is certainly becoming more ubiquitous and this comes with benefits but it also comes with costs.  In healthcare and mental health, I have been pondering what pieces of technology will bring about change. Not only that but what kinds of design will people use and is it ethical?

I recently captured a twitter thread about the unintended consequences of technology. Also spending the last month or so reading "Weapons of Math Destruction" by Cathy O'Neil has challenged a lot of my assumptions about data collection.

As somebody who still has my research core from my undergraduate education, I assumed that all this acceleration of data can only yield positive results (especially understanding those in under-served area's) but the above book has proven me wrong.   There is a lot of data collection that has potential to further divide us and often exclude other. The book goes into vast detail but here are a few examples...

  1.  Judges often use algorithms to determine sentencing of criminals. This determined by "risk of further offense". This often leads them to people in poverty and minorities creating a vicious cycle of incarcerating communities and those who associate with them.
  2. Police Departments are using algorithms to predict where crime "may happen" also create a similar feedback loop where they focus on certain neighborhoods. Leading to increased "stop and frisk practices" and increase in arrests in certain pockets.
  3. Insurance Companies (including Life, Health, and Car insurance) have always relied on math of some kind but are increasingly leaving this work up to computers. Life and car insurance companies are frequently relying on credit scores. Not only can these credit scores be inaccurate and prejudicial but some companies use their own opaque "e-score" to determine insurance rates. Leading to people being denied or being priced out without knowing why.

She calls these algorithms that create these feedback loops Weapons of Math Destruction or "WMD's".  Going into detail into many other examples she describes  3 main elements of WMD's.  First is the Opacity or lack of transparency about what is being measured and how. Second is the scale of how many individuals it may impact and lastly what is the damage or potential consequences.

On January 23,2018 the #HTreads (Health Tech reads) twitter chat will gather at 9:30pm EST to discuss the following questions...

  1. What are some of the problems Health Technology is tackling with algorithms (predictive analytics, AI etc)
  2. What are some early successes with using big data in healthcare?
  3. Given the criteria for a WMD's have you observed any in healthcare technology?
  4. What are some of the steps healthcare technologist/data scientist can use to prevent WMD's?

Moving forward we have to be critical of the ways we are using big data in healthcare. We have to ensure that we are being more intentional about asking these ethical questions.