Innovative Applications in Analytics Finalist: Detecting Preclinical Cognitive Change

This morning I attended a great talk as part of the “Innovative Applications in Analytics Finalist” track: Detecting preclinical cognitive change by Dr. Randall Davis and Dr. Cynthia Rudin of MIT. With the increased prevalence of dementia and Alzheimer’s among the elderly, the associated health care expenses and the heart-wrenching situation of relatives who, when the disease is in an advanced stage, are no longer recognized by a dear parent, it is critical to diagnose cognitive decline as soon as one can so that early action can be taken and people can enjoy as much time as they can with a dementia-afflicted relative while this person is still himself or herself. Over 5m people have been diagnosed with Alzheimer’s in the U.S. and the healthcare costs could soon be in the billions of dollars. The aging of the population also means that early diagnosis of dementia has emerged as one of the most pressing healthcare issues of our time. (The approach is applicable to other conditions such as sleep apnea.)

The talk showed how the classical Clock-Drawing test can be leveraged using new tools and technology to gain more information on a patient’s cognitive state. There are in fact two clocks: the command clock (the patient is ordered to draw a clock showing a time of ten past eleven) and the copy clock (the patient is shown an image of a clock showing a time of ten past eleven and has to reproduce it). The key is to analyze the process of drawing the clock and not just the final result. The team of Dr. Davis, Dr. Rudin and their coauthors has been able to do that using a specially designed pen (equipped with a camera) and a special paper (which lets the pen know where it is on the piece of paper.) They call their test the digital clock drawing test. This allows them to measure key metrics such as the time it takes for the patient to draw the first hand of the clock after he or she has drawn the clock face and placed the numbers. It turns out that the pre-first hand latency – the time it takes for the patient to figure out where to draw that first hand of the clock – can help distinguish Alzheimer’s from depression. Total Thinking Time is also an important metric, as was the “disappearing hooklet” on the first 1 of “11” in the numbers. (Basically when you are done drawing the first 1, you already think about drawing the second 1 starting from top to bottom so there should be a small hook at the bottom of the first 1, pointing toward the top of the second 1. A disappearing hooklet is one of the first signs of cognitive decline.)

In addition, the final result to characterize the patient-drawn clocks has traditionally been scored by physicians in widely different ways based on the distortion of the clock face, incorrect placement of the hands of the clock, and so on. The talk’s authors showed convincingly how cutting-edge machine learning algorithms such as Supersparse Linear Integer Models (SLIMs) and Bayesian Rule Lists (BRLs) could be implemented to create decision rules that resembled the operational guidelines of physician-created scoring rules. This is important because it increases transparency and makes it more likely that physicians will implement those new methods because they are close to models they know. Physician-generated scoring systems achieved AUC (area under receiver operating characteristics curve) in the range of 0.66 to 0.79 where 0.5 is random and 1.0 is perfectly predicted. Machine-learning with all features achieved an AUC of 0.93 but is not as intuitive as the traditional physician-generated scoring systems. Machine-learning models based on SLIMs or BRLs achieve a tradeoff between those extremes with AUC of the order of 0.8, improving traditional physician-driven models but retaining high interpretability. As such, they are “Centaurs”, or human-machine combinations that are better than either, applied to solving one of the greatest healthcare challenges of our time.

Digital Cognition Technologies, Inc. is marketing the technology, now pending FDA approval.

Read more about this research here (news release), here (research papers of the CSAIL Multimodal Understanding Group) and here (Dr. Rudin’s papers). Specifically, you can read the paper that accompanies the Innovations in Analytics Award entry here: “Learning Classification Models of Cognitive Conditions from Subtle Behaviors in the Digital Clock Drawing Test.” Fascinating stuff!