October 2016

I’m sending this note as a one-off because I recently received some news that a major evaluation of the scientific literature around brain training is in progress. In fact, there is a brief, just-opened “open comment period” whose window closes on October 31st, and if you have an interest in the science or application of brain training programs to human health, I would suggest that you read the review and offer your comments.

The News

Last week, I visited the National Institute of Aging and met with several senior staff members, including Director Richard Hodes.  In passing, he let us know that the NIA had sponsored a review of the evidence on interventions associated with preventing, slowing, or delaying the onset of clinical Alzheimer’s-type dementia and MCI, and delaying or slowing age-related cognitive decline. This review will be used as input into clinical guidelines for health care professionals to advise their patients regarding interventions to maintain cognitive function in aging.

The draft version of this review was published October 4th. You can download it here. It covers cognitive training, physical exercise, and a host of other interventions.

Interestingly, it rates “cognitive training” as being supported by “moderate” evidence—which is higher than they rate any other intervention.

What do I personally think about this review, as it relates to brain training?

On the whole, this is another sad instance in which good intentions can go awry.

I could make quite a number of critical comments about the overall document—which offers little valid help for any medical professional, legislator or citizen for defining what they should be doing to best sustain their brain health and personal vitality in the latter decades of life—but I’m going to limit my remarks to the “cognitive training” part of this report. My comments below are part of a draft of a formal response that I have sent to the authors & administrators, in which I attempt to point out some of the inadequacies of their approach in generating this report, and in the conclusions that they draw within this document.

Abstracted from my comments shipped to them in the very brief, almost unannounced,  just-opened “comment period”:

I’m a recognized international authority on the science of adult brain plasticity, and have spent much of the last three decades of my research life trying to bring the neuroscience of plasticity ‘out the door’ to help older citizens (among other neurologically struggling populations) have a better and safer older life. I have 1) read the majority of the studies in this scientific domain referred to in this report—as well as many thousands of others related to this topic; 2) trained about a hundred scientists in this field, 3) worked with several thousand other scientists, engineers, technologists, and medical professionals in research and development studies in this area; 4) published several hundred peer-reviewed journal articles related to this science, 5) been awarded 58 US patents, and 6) received many prizes, awards and distinctions for these achievements (including the Russ Prize, from the National Academy of Engineering; the Kavli Prize, from the Norwegian Academy of Sciences; the Zülch Prize, from the Max Planck Institutes; the IPSEN Prize; among others).

In fact, as a member of the NAS and the NAM/IOM, one might have guessed that I might have been called upon to provide at least a little scientific input to this important national advisory effort. Such was not the case.

I would like to briefly focus on eight things (a partial list) that I think are not entirely right about the genesis of this report, or its consideration of the value of “cognitive training” for our older citizens.

First, rather astonishingly, it almost completely lacks any consideration of the brain as an organ. Since the real subject is organic brain health, this would seem to me to be a rather significant oversight. There is a massive body of literature relating to specific forms of intensive engagement (‘brain training’) to the physical status of the brain as an organ. Here, in the report, the focus is on psychological assessment. That’s akin to trying to sort how to sustain healthy kidneys by chemically analyzing urine. It’s useful, but falls short of telling anything close to the full story about the application of these strategies to drive positive changes in brain health status.

Second, the report once again fails to distinguish brain health exercises from “cognitive training” or “brain games”—indeed, appears blind to the fact that the former category actually exists. Many studies have drawn a brain training-to-brain health link. In fact, my colleagues and I have been at it for 30 years. Most “cognitive training programs” and “brain games” are NOT the same as brain exercises targeting physical/functional brain health. Throwing them into the same basket with the implication that they are the same thing is (always has been) just plain wrong.

Third, the reviewers adopted what in my view was a peculiar strategy for assuring objectivity, by organizing a review process that specifically excluded expert-scientist involvement. This is akin to organizing a Mars mission run by people who aren’t particularly good at physics, engineering, or math. I could have directed the organizers or reviewers to several hundred scientists that understand different aspects of the relationship between perceptual or cognitive ability and the physical and functional health of the organ that we call the brain. No one asked me. And no one asked them.

Fourth, the report comes to a major, damning conclusion about fundamental and American and international translational research related to brain science. Reading the review, one would think that almost all research supported by the NIA, NINDS and the other half-dozen government agencies who support studies related to adult brain health is useless. Of no value. Money wasted. Proving nothing.

Perhaps we should close the doors of these wasteful agencies—or perhaps withdraw grant support for the 99 out of 100 scientists in our field who fail to meet the standards espoused here as requisite for taking their science seriously.

Fifth, criteria for inclusion of studies are in several important respects arbitrary and unsupportable. To cite one example, if a study is shorter than 6 months, the reviewers argue that the study cannot be regarded as proving anything. That’s rather like saying that if I go to the gym for a month, and obviously improve my agility, strength and cardiovascular health, it didn’t happen if I didn’t measure it 6 months or longer after I quit going. Or to put it another way, demonstrating that an individual has benefited from brain training on a computer, measured a day or two or three after an intensive training epoch is surely a demonstration that an individual has benefited from training on a computer. SUSTAINING that benefit by controlling the ‘dosing’ of training on the path to managing their brain health is a second scientific issue. To toss such a study, claiming that it proves nothing, is just silly—especially if the reviewer has no notion of what those training-driven gains MUST mean, for the plastic physical and functional brain.

It turns out that these same exclusion criteria worked particularly well for limiting the number of instances in which reports convincingly showed that computerized training transferred to non-trained skills and domains. It’s a convenient way to dismiss a very large body of evidence that shows convincingly that training in the right forms has substantial far-transfer benefits.

Sixth, why did this review specifically exclude input from individuals who are dedicated, from the commercial side, to delivering help to our older citizens? The best of them, in this space, know as much or more about this subject as does anyone else on the planet. Does their self-interest trump their potentially useful contribution? Most NIH policy-setting activities involve academic researchers, advocacy organizations, policy groups, patients themselves, and industry representatives. Everyone should have input into the process, so that the outcome is respected by everyone.

Seventh, this report provides almost no help for people in need. When are the authorities who engage in sterile review processes like this one going to begin to understand that DOING NOTHING TO HELP WHEN WE CAN = DOING HARM. Every person who could benefit from this good and valid science who is discouraged from engaging because of a simple-minded treatment like this one is DAMAGED by it. There IS a responsibility for authoritative objectivity. There IS ALSO a responsibility for NOT applying unreasonable criteria of “proof” that result, for the public, in little useful—and a lot of useless—advice.

Eighth, the literature review of computerized brain training in this report is severely lacking. The reviewers inexplicably missed quite a few papers, and they conclude that “diffusion [transfer] to other domains was rare”—despite that fact that there are more than a hundred published papers showing such transfer in healthy aging and MCI alone.  Those studies are listed at a public reference library for computerized cognitive training publications that we host at  https://www.zotero.org/groups/cognitive_training_data/ .

What to do?

As I mentioned, the draft review has invited comments, due by October 31st. My colleague Dr. Henry Mahncke (and Posit Science’s CEO) has written a set of comprehensive comments and recommendations to improve the review. It shall probably be helpful for you to read them in draft form, here.

I’d like to ask you to consider reading the review (at least in the section that relates to brain training), and submitting your own comments. You may share some of my opinions on the issues, but shall likely also have your own significant concerns and comments about this report. Because the results of this review are likely to influence clinicians’ perspectives on the science of brain plasticity and brain training programs for quite some time, I hope you do take a few minutes to voice your opinion on how this review can and should be improved. There is still the possibility that the guiding authorities might ask the reviewers to: 1) consider all (or at least more) relevant literature; 2) bring brain science more directly into play; 3) consult with experts on the brain and its physical and functional plasticity as they related to brain health;  and 4) be of more genuine service to the concerns and needs of older people.  For gosh sake.

You can download the draft review and submit your comments here.

A final note

As always, you are getting this email because you’re a scientific comrade—or you have collaborated with Posit Science. I thought you might be interested in an update on ‘what’s happening’ from our Posit Science research group’s perspective every so often, and we want to stay in touch with scientists pursuing these issues in a serious way.  If you do not wish to receive this email every other month or so, just email me and I’ll take you off the list. We can still be friends.

A shared publication database

The scientific literature in our field is scattered across many journals. We try to keep up with it, and am committed to making our efforts public. We ask you to help us keep up with this fast-moving feast!  You can see our shared database of published randomized controlled trials in brain training online. It’s worth taking a look, as you might find something you weren’t aware of before that can be helpful for you.  As always, please email us any papers that you know about that you find to be missing from this compendium.