As a concerned parent, you read about issues that affect your child. If your child has a learning difficulty, you’ve probably found numerous research studies in the media and on the Internet. To evaluate such information, it’s important to understand what well-founded research is — and what it is not. Valid research follows a standard scientific process and contains specific “ingredients.”
What you need to know — and why
In your role as parent and consumer, you may review research studies to help you make decisions about your child’s medical care and educational needs. You may also evaluate products and services based on research. Knowing the difference between valid and faulty research can help avoid disaster.
Just for fun, let’s apply the scientific principles of research and testing to a favorite American food: peanut butter. Suppose a new brand of peanut butter called Zippy is being tested against the top-selling national brand you’ve purchased for years. The research study is published in a national magazine. Before you purchase a jar of Zippy peanut butter, which research findings should you consider?
You probably already read the “Nutrition Facts” labels on packaged foods. Let’s apply those same consumer skills to evaluating the peanut butter research study.
Ingredients of good research
Here’s the standard recipe for scientific research, listed by key ingredients:
Independent research (also called third party research) is conducted by experts who have nothing to gain (or lose) by the results of the study. Their research is strictly objective. In the case of the peanut butter testing, any research conducted by the manufacturers of Zippy, people who have invested their time, money, and advertising dollars in the product, couldn’t be objective. An independent group such as the U.S. Food & Drug Administration (FDA), whose primary concern is assuring product safety for consumers, would better handle the testing.
Quantitative research includes qualities that can be measured, or quantified. These measures represent facts, not opinion. If the FDA were to test Zippy peanut butter on a quantitative scale, their concerns might include:
- Does the Zippy factory meet or exceed the national hygiene and safety standards established for food processing?
- Does Zippy peanut butter contain food additives in amounts higher than what is considered safe for consumers?
Qualitative research collects opinions and descriptions from subjects exposed to a particular product or treatment. Results may be reported as testimonials.
Kids who compare the taste and texture of Zippy peanut butter to the top national brand might offer qualitative opinions (testimonials) like these:
- “Zippy tastes more peanutty than the other brand.”
- “The Zippy brand stuck to the roof of my mouth like glue.”
Sample size refers to the number of people or items involved.
In quantitative research, many samples of Zippy peanut butter should be tested. Ideally, random sampling would be used. That means samples are taken at random from many different batches of peanut butter.
In qualitative testing, a large number of subjects should be exposed to the product or treatment. For example, Zippy peanut butter should be taste-tested by hundreds of kids across the country — not just a few of them living in America’s “Peanut Belt.”
In testing, certain variables or factors must stay the same for results to be valid. When comparing two brands of peanut butter, you would test Zippy’s creamy peanut butter against the top-selling creamy brand. If you compared creamy against chunky, the test results wouldn’t be meaningful.
What happens after the research study?
Credible researchers publish the details of their research in industry journals. They encourage other trustworthy researchers to replicate (or repeat) their study using the same method. For example, the FDA might encourage a national food magazine to replicate their testing of Zippy peanut butter, using the same measures.
Before a research study can be published in professional journal, a panel of other experts (peers) who conduct similar research must review it. Before the FDA can publish its research about Zippy peanut butter in the Journal of Peanut Butter & Jelly, their study must be reviewed and approved by the journal’s Peer Review Board.
Beware of False Claims
Scientific terms are often used inappropriately to make products and services appear credible. This can create a sticky situation when it affects your decision to buy peanut butter — or, more importantly, products and services to help your child with his LD. Once you know how to evaluate research published by others, you’ll be better able to detect claims and testimonials that don’t ring true.
Points to remember
Whenever you evaluate research, try to consider all the elements described in this article. If this seems overwhelming, remember you will develop better judgement the more you practice. You might start each evaluation by asking some basic questions:
- Who conducted the research? Did the researchers stand to gain or lose money or reputation depending on the outcome? Did qualified professionals publish the research?
- What was measured — fact or opinion?
- Where was the information gathered? Was quantitative research done in a well-controlled environment, such as laboratory? Was qualitative data collected from people in several geographic areas?
- When was the research conducted? Has the same research been replicated more recently, perhaps in the last year or two? Is this the most current data on the subject?
- Why was the research conducted? Why is this data important? Remember the original intent of the research as you consider the results reported. Why do you need to know this information, and how will you use this knowledge?