Mind myth 5: Brain profiling

This continues the series on mind myths especially applicable to education and training. The fifth mind myth relates to so-called brain profiles that have very little bearing on the brain, except in the very general sense that we all have brains and that all our behaviours are mediated by our brains. Brain profiles are favourite instruments for a variety of practitioners, many of whom use them as bait to involve gullible clients into pseudoscientific self-improvement schemes and quack therapies. They are widely used in schools and in the corporate sector.

My two previous posts on brain profiles described the pseudoscience concepts in more detail and can be found at Brain profiling - science or pseudoscience? and "Genetic" brain profiling in rugby. I described a brain profile as follows:

"A brain profile assessment is not as one may imagine a neurological or neuropsychological examination of brain function. As envisaged here, it is a questionnaire of thinking and behavioural preferences, putatively representing underlying differences in brain organisation. The questionnaire is sometimes combined with simple tests of motor and sensory dominance (hand, foot, eye, ear) (Brain Gym calls this a dominance profile)."

Brain profiles of this kind all involve some version of the left brain right brain mind myth, sometimes combined with the downshifts and blockages mind myth. Brain profilists essentially pretend to test the brain as pop-psychologists 40 years ago thought Roger Sperry thought it was organised, but did'nt. Ned Herrmann admitted that this view could only be a metaphor, but his later followers were careful to downplay that - such a view not being good for business. People who undergo brain profile tests after all, expect to learn something about their brains, not about the profilists' inaccurate metaphors for the brain.

A typical pseudoscience brain profile test comprises a number of simple questions about thinking or behavioural preferences and lasts about 30 minutes. Brain profiling instruments mostly seem to be derived from either Ned Herrmann's HBDI or a shorter "test" by Paul Torrance, the "Your Style of Learning and Thinking" (SOLAT).

While pseudoscience brain profiles have little value in my opinion, many find them amusing, even useful. Should amusement be your motivation to do a brain profile, there's a number of free brain profiles that can be done online. Do one for amusement value. Just don't get carried away and start referring to yourself as left brained or right brained, revealing yourself in the process to be no-brained. Here are some links for free online brain profiles, Brainworks; Hemispheric Dominance Test. Be reminded, however, that free tests are often bait to involve you in other activities you have to pay for.

Some people who have done brain profiles, quite seriously declare them accurate and useful. I believe that a number of cognitive effects, biases and logical fallacies play a role here.

Firstly a combination of gullibility and the power of suggestion. Gullible clients buy into the pretence of neuroscience and the suggestions offered to them by the profilists. It comes as no surprise that some brain profilists are good motivational speakers.

Secondly, the Forer effect. According to the Skeptic's Dictionary, the Forer effect "... refers to the tendency of people to rate sets of statements as highly accurate for them personally even though the statements could apply to many people." This could also be more specific, i.e. an engineer would see himself as ordered and logical, answer the brain profile questions (which tend to be quite transparent) accordingly, be classified in a left brain quadrant, the description of which he will find accurate. It's circular reasoning and also a Forer effect. Confirmation bias could also play a role, with people just ignoring results that do not fit their preconceptions.

Thirdly, cold reading may play a role. Cold reading is a style of telling someone something about themself that seems more accurate than it actually is by using ambiguous generalities. Dr. Terry Sandbek, in discussing brain typing (similar to brain profiles), indicated how cold reading could be at work in these type of instruments by " ... (giving) enough positive, but ambiguous, information for (them) to be influenced by the Forer Effect."

Blogger Sean Carmody at Stubborn Mule connected the dots and pointed out the similarities between brain profiling and astrology. Hat tip to him for this amusing image of a brain profiles encircled by astrology symbols.

While brain profiles may be of amusement value to adults, I believe that they are potentially harmful applied to children. It should be remembered that brain profiles are often just the bait for pseudoscientific therapy programmes. Parents of children who require evidence supported therapies, may be duped into enrolling their children into quack therapies, wasting valuable time and resources in the process. These children could also fall prey to the pseudoscientific beliefs or belief systems underlying these therapies - consider this statement by Sense about Science about children involved in Brain Gym: "These exercises are being taught with pseudoscientific explanations that undermine science teaching and mislead children about how their bodies work."

Another danger for children concerns Gattaca-esque notions that sometimes form part of brain profiling. Consider Naas Botha, an ex Springbok rugby player turned sports commentator, gushing here about "genetic" brain profiles for children in sport:

"It’s very, very scary. It’s also very exciting. I can see a future where we will establish at the age of five or six exactly who has the brain profile to make it to the top and in what position. The potential is absolutely amazing. ...I had my whole family brain-profiled and was amazed... "

Scary indeed. Gattaca-esque selections are scary enough when based on science, based on pseudoscience they're downright frightening. Consider the plight of a child of an overambitious parent, selected at an early age and programmed for "greatness". Consider also the child not selected or excluded for a team or a sport due to not having the "right" brain profile. As a parent I have no problem if my child is not selected for a sports team for valid reasons. I would have a major problem, however, should he or she not be selected because of not having the right brain profile. In today's high stakes sports environment, I would not be surprised if litigation followed the exclusion of competitors based on this kind of pseudoscience practice.

Keep in mind that not all brain profiles are pseudoscientific. Neuroscientists sometimes refer to large datasets of information about patients' brains as brain profiles. Here is one example (contrast these scientific neuropsychological assessments of more than eight hours duration, typically done in at least two sittings, with pseudoscientific brain profiles described above):

"The Brain Profiling Group (BPG) specializes in the testing and evaluation of brain function, for the purpose of detecting and quantifying abnormalities arising from psychological disorders or physical trauma. BPG provides this specialist service for professionals, medical and legal practitioners and patients seeking clarification of psychological and organic trauma, and others who seek to obtain peak brain performance."

In closing, spare a thought for brain profiling consultants who "innocently" became involved with this pseudoscience practice. Many of these are teachers or ex-teachers. Yes, they should have known better, but neuroscience, critical thinking and evidence supported practice have never featured strongly in teacher training. I can't be as charitable to the originators of these practices.

Other mind myths in this series were:

The 10% myth

The left brain right brain myth

Energy and the brain myth

The downshifts and blockages myth

Dore has arrived in South Africa

Well, it's arrived, as I knew it would (well, it's actually been here since 2006, just did not know about it). It's the Dore therapy programme for developmental disabilities. I've no personal experience of it, but I can read and I've also followed the exposés about it in several blogs, especially Ben Goldacre's Bad Science. Its rationale is unproven and there is no plausible evidence for its effectiveness.

Educational fads from America and Britain inevitably land up in South Africa (and it seems Australia). With every successive unproven fad the education system becomes more dysfunctional and more learners need therapy. We seem to have a symbiotic relationship between imported educational fads on the one hand and imported bullshit therapies on the other. Maybe the one causes the other. And yes, I know about correlation and causation.

Well, having got that off my chest, let's get back to Dore.

The Dore programme purports to treat dyslexia and other developmental disorders. It was founded by paint tycoon Wynford Dore and is based on theories on the involvement of the cerebellum in the learning process. Note that the independence of the authors of this article linked to, Nicolson and Fawcett, have been questioned - see the second link at the end of this post.

In Britain Dore was characterised by aggressive, celebrity based marketing and the bad habit of threatening critics with legal action. It was very expensive and typically required upfront payment from parents. I'm using the past tense because Dore has gone into legal administration in Britain, Australia and the USA.

Strangely enough, the South African Dore website is silent about the problems its international partners are experiencing. Where does that leave desperate parents in South Africa who may wish to consider Dore? I would suggest doing the following:

First, check the evidence, or rather lack thereof.

Should you still wish to continue, ask about their future in South Africa.

Do not agree to make upfront payments (it costs R23 000,00 for a treatment course). Let the risk be theirs, not yours.

Hold them responsible for progress, don't fall the age old scam: "Its not working because you do not have faith or are not working hard enough at it".

Here are links from various blogs and newspaper articles on Dore:

South Africa: Learning Disorder 'Miracle Cure' in Spotlight
Scientists quit in dyslexia "cure" row
Dore - The miracle cure for dyslexia
Dore - The media's miracle cure for dyslexia
Blogs vs mainstream media
Dore - The vultures start to circle
Dore shut

Mind myth 4: Downshifts and blockages

I'm continuing my series on mind myths especially applicable to education and training. This one is a lucrative favourite of many brain based pseudosciences. They typically claim that people's brains get blocked when under stress and that only their brand of snake oil can get it unblocked again. This goes far beyond the generally accepted effects of stress triggering the fight or flight response. Consider this claim about rugby players from a pseudoscientific technique called genetic brain profiling:

"There are 32 different combinations of dominance between the left and right brain and the ear, eye, hand and foot, and each has implications for the way a stressed player will perform. Lotter assesses players and is able to predict the “blockages” they would suffer as a result.

For example, in an age-group provincial team she recently assessed, she found a tighthead prop with four blockages (“basically, he was unable to function under pressure”), while the fly-half and inside-centre were among those with three blockages (“key decision makers would lose the ability to communicate and perform under stress”)." (See article here).

Where do these ideas come from and is the any evidence to back them?

It starts with another popular myth called downshifting. This mixed metaphor is based on a car's gear system on the one hand and the triune brain model on the other.

Image from The Architect of Life


The hypothesized downshifting response is seen as a negative response to even mild stress in which the brain shifts down to "lower" brain formations where primitive survival modes predominate – and then remains stuck there. Supporters of the downshifting hypothesis then each have their own special techniques to get the brain unstuck. These may be simple motor movements to activate certain brain areas, or the drinking of water in order to "instantly oxygenate" the brain and allow it to upshift again.

The triune brain is a model of brain function developed by Paul MacLean. He suggested three basic brain formations based on evolutionary history, namely a reptilian formation (brainstem structures), early mammalian formation (midbrain/limbic) and neomammalian formation (neocortex). MacLean's triune brain model is influential, but quite controversial, see some of the issues (not really applicable to this post) in this discussion by Jaak Panksepp. I have always found the triune brain model quite useful and was saddened by the way it was diminished by being appropriated by pseudoscience hucksters. The chapter by Panksepp does provide another perspective from within neuroscience itself.

Many brain based pseudoscientific techniques, such as in the rugby example referred to earlier, extend the downshift hypothesis to include the left brain right brain myth and then propose that based on the putative pattern of brain dominance, parts of the brain switch off, or become blocked. Thirty two potential brain blockages should certainly be much more lucrative to "cure" than only a few.

Little evidence is to be found in MacLean's own work to support such simplistic interpretations of his model. His triune brain is an integrated system in which the three brain formations exchange information and the whole functions better that the sum of its parts. The reaction of the brain to stress is a well studied area (see here and here, for instance), but there is no evidence to support the idea of semi-permanent downshifting and of brain blockages that would require outside intervention to upshift or clear.

Concurring with this, in a critique of Brain Gym, neurobiologist Dr Stan Lazic writing for the British organization Sense about Science, stated that:

"... in reality the only time a neurological signal would become "jammed”,“blocked” or “switched off" is during a pathological event such as a seizure, stroke, head trauma, or perhaps due to a neurodegenerative disorder."

A further critique of the downshifting concept was provided by Prof Robert Sylwester and can be read here.

Bio-Strath for ADD/ADHD?

There has lately been a spate of very expensive, full-page colour advertisements in national South African newspapers touting a product called Bio-Strath® as a solution for ADD/ADHD and learning problems in children. No specific mechanism through which Bio-Strath should work for ADHD was suggested, except its nutritional value. As a result of the advertisements, I received many enquiries from parents of children who are currently on medication such as Ritalin and Concerta. All wanted to know whether this "natural" product could be a suitable replacement. I referred them to their medical practitioners, but at the same time decided to look into the research evidence for Bio-Strath as a treatment for attention deficit and attention deficit hyperactivity disorder (ADD/ADHD).

The full text of the advertisement can be seen here.

The advertisements referred to a 2006 study in an European journal, Pädiatrie. A homeopath, Irma Schutte, made the following claims and comments about this study:

"A ground-breaking clinical trial recently revealed that Bio-Strath Elixir led to an 76% improvement in the ADD/ADHD symptoms of the children participating in the study... The clinical proof that Bio-Strath taken three times daily can solve almost 80% of these children's problems is staggering news! It would be a grave oversight if children were given medication which could carry risk and has known side effects when Bio-Strath could easily have been the answer."

Strong words indeed, but does the research referred to justify her confidence? I was unable to access the journal article in an Internet search. I then e-mailed SA Natural Products, the South African distributors of Bio-Strath, for more information. They had no problem supplying me with English copies of said article, as well as various other reprints of articles about Bio-Strath. The detail of the particular article is:
König, S. & Joller, P. 2006. Influence of a food supplement on the behaviour of children with attention deficit disorders (ADD/ADHD): Application study with the herbal yeast preparation Bio-Strath® in children. In Pädiatrie 1/2006. (no more info on page numbers, volume, etc.).

Does the article support the strong claims in Bio-Strath's advertisements? Unfortunately not. Here is why:

Author not independent

At least one of the article's authors seemed to have links with Bio-Strath. Dr. Peter Joller authored numerous other articles on Bio-Strath. Another blogger, Frank Swain of SciencePunk, questioned Joller's independence and never received an answer. There was, however, no acknowledgement of a possible conflict of interest in the article.

Was it peer-reviewed?

The lack of scientific rigour evident from the article (see below), suggests that it may not not have been peer-reviewed. A peer-review process would have given one more confidence in the findings.

Inadequate research design

Let us now consider the research itself. The research design was a simple pre-experimental one-group pretest-posttest design. This is considered a very weak design, prone to numerous threats to internal and external validity. Suffice to say, this kind of research without blinded, random assignment to experimental, control and placebo groups, is really not capable of proving anything. Bio-Strath has been on the market for more than 50 years and the first claims that it could improve concentration in children date back at least 40 years. Surely decent, so called gold standard research is long overdue?

The participants in the research were selected based on scores on an ADD/ADHD rating scale of unknown reliability and validity. The rating scale was translated from English into German, but there's no indication of adequate translation controls (i.e. a back-translation process). The participants were subjected to pretests and posttests six weeks apart on the Integrated Visual and Auditory Continuous Performance Test (IVA CPT). The IVA CPT seems to be a valid, well standardised test.

The small number of participants (18) and large number of tested variables (19), increased the probability of chance effects. The uncontrolled test-retest design is prone to many confounding effects, including history, maturation, test-retest practice, and so forth. The fact that the experiment was not blinded, made it prone to participation effects such as placebo and Hawthorne effects (and yes I know that there is some doubt about the Hawthorne experiments).

Misleading reporting

The reporting protocol used in the research was certainly very strange and misleading. It seems very much as if they cherry-picked which results to report on and how.

The results were tabled in a 18 x 19 matrix, with the participants ranked in terms of the number of variables they improved upon. König and Joller reported that 12 of the 18 subjects improved on at least one of the variables. According to their criteria, that meant that 66% of the participants improved! The range of reported improvement was from improvement on one variable, to improvement on 15 of the variables. Though not explicitly reported, the average improvement for all candidates, was on only four of the variables, or 21%. Some of the variables were composite scores, which add to questions of the appropriateness of the reporting. The authors did not distinguish between ADD, ADHD, or HD in their analysis.

They further reported 116 statistically significant changes, 93 positive, 23 neutral or negative. There was no indication how the statistical significance was determined. A neutral "change" makes no sense and I fail to see how it can be statistically significant, maybe I'm missing something. According to their criteria, 93 out of 116 represent an 80% improvement. What they seem to be ignoring, however, is the 226 scores which showed no significant change {(18 x 19) - 116}! Taking that into consideration, the percentage change would be 27% (93/342). Yes, I know this kind of reporting makes no sense, but I followed their reporting procedure.

The "statistical" analysis was followed by an anecdotal discussion of the subjective improvements (as reported by the parents) shown by the two top ranked participants, who respectively improved on 15 and 13 scores. This discussion was clearly of little value, as the parents' report was subject to among other things, confirmation bias. The limitations of anecdotal evidence are well known.

These were just some of the limitations of the research. There are also other, older studies available. These studies seem to suffer from the same limitations. The study that is hyped in the newspaper advertisements, is the one discussed here.

Why take the trouble to do such inadequate research about a product that has presented as a solution for ADHD for more than 40 years? Steven Novella on his blog, Neurologica, explains this phenomenon better than I can (I've just replaced the word "accupuncture" with "Bio-Strath"):

"Poor studies that are virtually guaranteed to generate a positive result (like this one) are also useful for marketing propaganda. They create great headlines - and most of the public are not going to read much beyond the headlines and so will be left with the sense that there is more and building evidence that Bio-Strath works. As propaganda this study is very effective."

Conclusion

The advertisement was misleading to say the least. The heading in huge, bold letters claims: "76% Improvement in ADD/ADHD symptoms". I've shown that to be incorrect - the study is of little value and does not support that claim. Even taking the study at face value, at most an improvement of 27% can be claimed and even that is highly debatable. The ADD/ADHD symptoms referred to, are from the test used and may not directly translate to real world problems.

I believe that Bio-Strath South Africa, knowingly or unknowingly, placed an advertisement that deceived many parents. The ethical thing for them to do would be to withdraw this advertisement immediately and publically admit that there is very little evidence that Bio-Strath is effective for ADD/ADHD.

See a later post on Bio-Strath for cognitive enhancement at Bio-Strath is at it again.