Skip to Content

A Behind the Scenes Look at Achieving Expert Consensus in Sports Concussion


Often, when people think about "research," it brings to mind sterilized labs where closely controlled tests are conducted by scientists in white coats on various subjects. Although lab research is fundamental to the progress of science and medicine, it isn't the only way scientific research studies are performed. Recently, I was involved in a type of research called the Modified Delphi Technique, aimed at achieving expert consensus on the assessment of on-field and sideline sports concussion in collision sports.

The Modified Delphi Technique is a formalized process for achieving consensus among participants. This scientific study method systematically and quantitatively blends clinical evidence and clinical expert opinions, in this case from medical providers and other neurologic experts on a panel, by asking them to rate the items in question, discuss those items and then re-rate them according to their importance in assessing the subject at hand.

You see, expert recommendations for anything – from health care to auto repair must first be determined by evaluating the evidence about the topic and then asking the experts involved for their opinions. For this study, the matter at hand was the evaluation of symptoms involved in sports concussion in collision sports. This critical and weighty topic has significant health implications for athletes.

Using the Modified Delphi Technique, I and a panel of more than a dozen other neurologic experts participated first in two rounds of open-ended question answering. The results from the first two rounds of questions were used to develop a questionnaire for round three utilizing the Likert Scale. Likert-style questions use a five or 7-point scale that ranges from one extreme to another and asks the person answering the question to "rate" their response. Though this might seem highly technical, you have likely answered Likert-type questions anytime you have completed a satisfaction survey for a brand you use. These are questions like: "I would recommend BRAND to people I know," or "I am deeply satisfied with BRAND's solution to X problem." The "scale" of answer prompts ranges from Strongly Agree to Strongly Disagree, with three to five other options in between.

At the end of round three, if less than 80 percent of the panel agreed on an item, if the panel members were not in consensus, or if greater than 30 percent of the panel neither agreed nor disagreed, then the process moved into round four until the desired level of consensus was achieved.

As a result, the panel achieved a consensus for 85 percent of the clinical signs indicating concussion. With this in mind, the panel concluded that on-field and sideline concussion should include observing the mechanism that resulted in suspected concussion, a clinical exam, and a cervical spine (neck) assessment.

Our team also reached a consensus that 74 percent of the concussion signs or red flags should result in removal from play. These include:

  • Loss of consciousness
  • Motor incoordination/ataxia
  • Balance disturbance
  • Confusion/disorientation
  • Memory disturbance/amnesia
  • Blurred vision/light sensitivity
  • Irritability
  • Slurred speech
  • Slow reaction time
  • Lying motionless
  • Dizziness
  • Headaches/pressure in the head
  • Falling to the ground with no protective action
  • Slow to get up after a hit
  • Dazed look
  • Posturing/seizures

The team also agreed that a normal on-field or sideline clinical examination and Health Impact Assessment with no signs of concussion are indicators of the safest return to play for athletes. In addition, a video assessment should be a mandatory part of the clinical decision-making process for professional athletes but should not replace clinical decision-making.

Though arriving at a clinical consensus may be boring for some, it illustrates neurologic and other health experts' commitment to protecting and caring for our patients. Sports concussion is a serious topic with myriad health implications for those who sustain them. Education is critical and a significant part of ensuring suspected concussions are evaluated by an expert as soon as possible. Our patients deserve to receive care from experts who are at the top of their game, and I am proud of the work this distinguished panel has done to keep us there.