Epilepsy Research Benefits from the Crowd

Posted on January 20th, 2015 by Dr. Francis Collins

For millions of people with epilepsy, life comes with too many restrictions. If they just had a reliable way to predict when their next seizure will come, they could have a chance at leading more independent and productive lives.

That’s why it is so encouraging to hear that researchers have developed a new algorithm that can predict the onset of a seizure correctly 82 percent of the time. Until recently, the best algorithm was hardly better than flipping a coin, leading some to speculate that seizures are random neurological events that can’t be predicted at all. But the latest leap forward shows that seizures certainly can be predicted, and our research efforts are headed in the right direction to make them even more predictable. The other big news is how this new algorithm was developed: it’s the product of a crowdsourcing competition.

Crowdsourcing builds on the recognition among software developers in the mid-1980s that a crowd of users, not just the guy writing code in a cubicle, often knows best how to design existing products or work out the bugs in existing ones. As the credibility of the crowd has grown in recent years, an initial wave of biological crowdsourcing competitions has appeared online. The competitions often pit mathematicians, computer scientists, and other capable big-data crunchers against each other or organized into teams.  Their challenge is to solve a problem to which many often arrive at their computer screens short on expertise but long on innovative ideas to cut through the complexity. For organizers, the key is to model the right problems that lend themselves to crowdsourcing, attract the right teams, and offer the right incentives for them to drill down to an answer.

That was the idea behind the American Epilepsy Society Seizure Detection Challenge. The challenge was launched last August on the web site Kaggle.com, a well-known online platform for data prediction competitions, and co-sponsored by the American Epilepsy Society, NIH’s National Institute of Neurological Disorders and Stroke (NINDS), and the Epilepsy Foundation. The challenge involved two distinct contests: detection and prediction of seizures. A total of 504 teams from all over the world participated in both contests, which remained open until early November.

The teams analyzed a huge data set detailing the electrical activity in the brains of people while under evaluation for surgery to treat their epilepsy. They also had an even larger data set from studies with dogs, whose epilepsy closely resembles that seen in people.

All were from previous work involving collaborators from the University of Pennsylvania, the Mayo Clinic, and the University of Minnesota. It was their idea to hand off the data for the crowdsourcing competition.

The goal was to identify premonitory signatures of electrical activity in the brain during the hour prior to a seizure. In the detection contest, the team that identified the earliest changes in brain activity that led to a seizure with the fewest false alarms took home the prize. In the prediction contest, the team that generated a predictive signature of changes in brain activity that led to a seizure with the fewest false alarms won.

After a few months, the winning prize in the detection contest went to a computer engineer from Australia named Michael Hills. He competes in online contests to test his skills on the side in machine learning and digital signal processing. Hills used a special algorithm to classify various aspects of localized electrical field potential in the brain.

The seizure prediction contest was even more challenging, and it came down to a tight, seven-team race. But two of the teams—one from Australia, and the other from the United States—merged their talents down the stretch to take the prize by predicting 82 percent of seizures. Interestingly, neither team ever has met face to face.

Just for the record books, the US team members are Drew Abbot, a software engineer, and Phillip Adkins, a mathematician. Both work in California for a small company that develops motion detection for Wii and PlayStation. The Australian team members are all based at the University of Queensland’s Center for Advanced Imaging. Simone Bosshard and Min Chen are neuroscientists who work with animal models of epilepsy, while Quang Tieng is an applied mathematician.

The competition shows that sharing data to collaborate on complex problems can yield clever and unexpected solutions. Also noteworthy is that none of the winners were clinicians. This suggests that if we can find a platform to engage players from other seemingly distant disciplines, the chances are greater to find innovative, out-of-the-box solutions to current challenges.

That’s why NINDS has launched iEEG.org to catalyze collaboration and sharing of datasets, algorithms, and research tools for BigData studies of epilepsy. On this website are almost 2,000 datasets freely available for analysis, and there is already a user base of more than 670 active collaborators. It will be exciting to see how long it takes to go even beyond that 82 percent predictability.

Links:

National Institute of Neurological Disorders and Stroke: Epilepsy

IEEG.ORG, the NINDS-supported research initiative to catalyze collaboration and and data sharing for BigData studies of epilepsy

Litt Lab, University of Pennsylvania, Philadelphia

American Epilepsy Society Seizure Prediction Challenge

NIH Support: National Institute of Neurological Disorders and Stroke