Conducting Web-based Research The CARS Checklist The CARS Checklist (Credibility, Accuracy, Reasonableness,
Support) is designed for ease of use. Few sources will meet every criterion
in the list, and even those that do may not possess the highest
level of quality possible. But if you learn to use the criteria in this list,
you will be much better able to separate the high-quality information
from the poor-quality information. Read each section below or jump right to the summary. CredibilityBecause people have always made important decisions based on
information, evidence of authenticity and reliability or credibility,
believability has always been important. If you read an article saying
that the area where you live will experience a major earthquake
in the next six months, it is important that you should know whether
or not to believe the information. Some questions to ask about general
credibility might include these: - Is there sufficient evidence presented to make the argument
persuasive?
- Are there compelling arguments and reasons given?
- Are there enough details for a reasonable conclusion about
the information?
There are several tests you can apply to a source to help you
judge how credible and useful it will be: Author’s Credentials. The author or source of the information
should show some evidence of being knowledgeable, reliable, and
truthful. Some questions you might ask would include the following: - What about this source makes it believable (or not)?
- How does this source know this information?
- Why should I believe this source over another?
As you can see, the key to credibility is the question of trust. Here
are some clues to credibility: - Author’s education, training, and/or
experience in a field relevant to the
information. Look for biographical
information, the author’s title, or
position of employment
- Author’s contact information (e-mail
or postal mail address, telephone
number)
- Organizational authorship from a
known and respected organization
(corporate,
governmental, or non-profit)
- Organizational authorship reflecting
an appropriate area of expertise
- Author’s reputation or standing
among peers.
Evidence of Quality Control Most
scholarly journal articles pass through a peer
review process, whereby several readers must
examine and approve content before it is
published. Statements issued in the name of
an organization have almost always been
seen and approved by several people. (But
note the difference between, “Allan
Thornton, employee of the National
Oceanographic and Atmospheric Agency,
says that a new ice age is near,” and “The
National Oceanographic and Atmospheric
Agency said today that a new ice age is near.”
The employee is speaking for himself, where-as
a statement in the name of NOAA repre-sents
the official position of NOAA.) Evaluation Tip Feel free to contact the
author of a source by
e-mail or telephone when
such contact information
is given. Many authors are
willing to clarify points
or explain details further.
Communicating with
the author will also
give you more evidence
about credibility. Evidence of quality control of Web material
includes these items: - Information presented on corporate,
government, or organizational Web sites
- Online journals that use refereeing (peer
review) by editors or others
- Postings of information taken from books or
journals that have a quality control process
Indicators of Lack of Credibility. You
can sometimes tell by the tone, style, or com-petence
of the writing whether or not the
information is suspect.
Here are a few clues
to lack of credibility: - Anonymity
- No indication of a third party editor or
publisher to ensure the quality and reliability
of the information
- Negative metainformation. If all the reviews
are critical, be careful. (See the discussion
about metainformation later in this
chapter.)
- Bad grammar or misspelled words. Most
educated people use grammar fairly well
and check their work for spelling errors. An occasional split
infinitive or comma in the wrong place is not unusual, but
more than two or three spelling or grammar errors is cause
for caution, at least. Whether the errors come from carelessness
or ignorance, neither puts the information or the writer
in a favorable light.
Evaluation Tip The top-level domain .gov
is used by state and federal
government agencies
and indicates that the
information has the sanction
of that government
agency. Information from
these domains is usually
highly reliable. For example,
you can obtain reliable
census information from
the Census Bureau at
www.census.gov or information
about medicine,
food, or cosmetics from
the Food and Drug
Administration at
www.fda.gov Back To Top
AccuracyThe goal of the accuracy test is to ensure that the information is actually
correct: up-to-date, factual, detailed, exact, and comprehensive.
For example, even though a very credible writer said something that
was correct twenty years ago, it may not be correct today. Similarly,
a reputable source might be giving up-to-date information, but the
information may be only partial and not give the full story. Here are
some concepts related to accuracy: Timeliness Some work is timeless, like the classic novels and stories,
or like the thought-provoking philosophical work of Aristotle
and Plato. Other work has a limited useful life because of advances
in the discipline (psychological theory, for example), and some work
is outdated very quickly (like technology news). You must therefore
be careful to note when the information you find was created, and
then decide whether it is still of value (and how much value). You
may need information within the past ten years, five years, or even
two weeks. But old is not necessarily bad: nineteenth-century
American history books or literary anthologies
can be highly educational because they
can function as comparisons with what is
being written or anthologized now. In many
cases, though, you want accurate, up-to-date
information. An important idea connected with timeliness
is the dynamic, fluid nature of informa-tion
and the fact that constant change means
constant changes in timeliness. The facts we
learn today may be timely now, but tomorrow
will not be. Especially in technology, science,
medicine, business, and other fields always in
flux, we must remember to check and re-check
our data from time to time, and realize
that we will always need to update our facts.
Comprehensiveness. Any source that
presents conclusions or that claims (explicitly
or implicitly) to give a full and rounded story,
should reflect the intentions of completeness
and accuracy. In other words, the information
should be comprehensive. Some writers
argue that researchers should be sure that
they have “complete” information before
making a decision or coming to a conclusion.
But with the advent of the information age,
such a goal is impossible, if by “complete” we
mean all possible information. No one can
read 20,000 articles on the same subject
before coming to a conclusion or making a
decision. On the other hand, an information
source that deliberately leaves out important facts, qualifications,
consequences, or alternatives may be misleading or even intentional-ly
deceptive. And since no single piece of information will offer the
truly complete story, even if accuracy and fairness are intended, we
must rely on more than one source to provide us with a fuller view
of the situation. Evaluation Tip You can use your browser
to find out when a Web
page was last modified,
even though there may
not be a visible date on
the page itself. In
Netscape, use “View,”
“Page Info” and you will
see a “Last Modified” field
with a date. In Internet
Explorer, using “File,”
”Properties” you will get
the date the information
was transferred to your
disc, not the date the
page was modified. Audience and Purpose For whom is this source intended and
for what purpose? If, for example, you find an article, “How Plants
Grow,” and children are the intended audience, then the material
may be too simplified for your college botany paper. More impor-tant
to the evaluation of information is the purpose for which the
information was created. For example, an article titled, “Should You
Buy or Lease a Car?” might have been written with the purpose of
being an objective analysis, but it may instead have been written
with the intention of persuading you that leasing a car is better than
buying. In such a case, the information will most likely be highly
biased or distorted. Such information is not useless, but the bias must
be taken into consideration when interpreting and using the informa-tion.
(In some cases, you may be able to find the truth by using only
biased sources, some biased in one direction and some biased in the
other.) Be sure, then, that the intended audience and purpose of the
article are appropriate to your requirements or at least clearly in evi-dence
so that you may take them into account. Information pretending
to objectivity but possessing a hidden agenda of persuasion or a hidden bias is
quite common in our culture. Indicators of a Lack of Accuracy In addition to an obvious
tone or style that reveals a carelessness with detail or accuracy, there
are several indicators that may mean the source is inaccurate, either
in whole or in part: - No date on the document
- Assertions that are vague or otherwise lacking detail
- Sweeping rather than qualified language (that is, the use of
always, never, every, completely rather than usually, seldom, some-times,
tends, and so forth)
- An old date on information known to change rapidly
- Very one-sided view that does not acknowledge opposing
views or respond to them
Back To Top
Reasonableness The test of reasonableness involves examining the information for
fairness, objectivity, moderateness, and consistency.
Fairness. Fairness includes offering a balanced, reasoned argument.
ment, not selected or slanted. Even ideas or claims made by the
source’s opponents should be presented in an accurate manner.
Pretending that the opponent has wild, irra-tional
ideas or arguments no one could accept
is to commit the straw man fallacy. A good
information source will also possess a calm,
reasoned tone, arguing or presenting material
thoughtfully and without attempting to get
you emotionally worked up. Pay attention to
the tone and be cautious of highly emotional
writing. Angry, hateful, critical, spiteful tones
often betray an irrational and unfair attack
under way rather than a reasoned argument.
And writing that attempts to inflame your
feelings to prevent you from thinking clearly
is also unfair and manipulative. Objectivity There is no such thing as
pure objectivity, but a good writer should be
able to control his or her biases. Be aware
that some organizations are naturally not neutral.
For example, a professional antibusiness
group will find, say, that some company or
industry is overcharging for widgets. The
industry trade association, on the other hand,
can be expected to find that no such over-charging
is taking place. Be on the lookout
for slanted, biased, politically distorted work.
One of the biggest hindrances to objectivity
is conflict of interest. Sometimes an infor-mation
source will benefit in some way (usually
financially, but sometimes politically or
even emotionally or psychologically) if that
source can get you to accept certain informa-tion
rather than the pure and objective truth.
For example, many sites that sell “natural”
products (cosmetics, vitamins, clothes, food)
often criticize their competitors for selling
bad, unhealthy, or dangerous products. The
criticism may be just, but because the messenger
will gain financially if you believe the
message, you should be very careful—and
check somewhere else before spending
money or believing the tale. Moderateness Moderateness is a test of
the information against how the world really is. Use your knowledge
and experience to ask if the information is really likely, possible, or
probable. Most truths are ordinary. If a claim being made is surprising
or hard to believe, use caution and demand more evidence than
you might require for a lesser claim. Claims that seem to run against
established natural laws also require more evidence. In other words,
do a reality check. Is the information believable? Does it make
sense? Or do the claims lack face validity? That is, do they seem to
conflict with what you already know in your experience, or do they
seem too exaggerated to be true? For example, does the statement,
“Half of all Americans have had their cars stolen,” pass the face
validity test? Have half of your friends had their cars stolen? Is the
subject on the news regularly (as we might assume it would be if
such a level of theft were the case)? It is important, of course, to remember that some truths are spectacular
and immoderate. If you have read about, say, the Tulipmania
in Holland in 1636–1637, where some single tulip bulbs sold for the
equivalent of tens of thousands of dollars each, the idea of immoderate
truth will not be so strange to you. Do not, therefore, automatically
reject a claim or source simply because it is astonishing. Just be
extra careful about checking it out. Evaluation Tip Many Web sites are sponsored
by news organizations
that have printed or
broadcast counterparts. As
a general rule, if the print-ed
or broadcast source is
reliable, so too is the
online counterpart. The
online sites for Time, CNN,
The New York Times, ABC
News, and so forth, have
the same reputation to
uphold and the same
resources to devote to fact
checking and accuracy as
their traditional media ver-sions,
and should therefore
be just as professional.
There have been a few
notable exceptions, but
those have indeed been
exceptions. (Similarly, the
online versions of the
incredible sources, such as
some of the tabloids,
should be viewed with the
same suspicion used for
the printed version.) Consistency The consistency test simply requires that the argument
or information does not contradict itself. Sometimes when people
spin falsehoods or distort the truth, inconsistencies or even contradictions
show up. These are evidence of unreasonableness.
Perhaps an obvious example comes from some of the chain letters
that circulate on the Internet and by fax machine: they affirm that
they have come to you “to bring you cheer and good luck,” only to
threaten you (not very indirectly) with unemployment or death if
you do not continue to circulate the letter. World View A writer’s view of the world (political, economic,
religious—including antireligious—and philosophical) often influences
his or her writing profoundly, from the subjects chosen to the slant,
the issues raised, issues ignored, fairness to opponents, kinds of
examples, and so forth. Some writers’ value systems permit them to
fabricate evidence, lie, or falsify the positions of others for the sake
of what they think is a noble cause—or perhaps political expediency.
For these writers, political agendas take precedence over truth.
Knowing about such distorting world views can therefore provide
another evaluative test for reasonableness. Indicators of a Lack of Reasonableness Writers who put
themselves in the way of the argument, either emotionally or
because of self-interest, often reveal their lack of reasonableness. If,
for example, you find a writer reviewing a book he opposes by
asserting that “the entire book is completely worthless claptrap,” you
might suspect there is more than a reasoned disagreement at work. Here are some clues to a lack of reasonableness: - Intemperate tone or language (“stupid jerks,” “shrill cries of
my extremist opponents”)
- Overclaims (“Thousands of children are murdered every
day in the United States.”)
- Sweeping statements of excessive significance (“This is the
most important idea ever conceived!”)
- Conflict of interest (“Welcome to the Old Stogie Tobacco
Company Home Page. To read our report, ‘Cigarettes Make
You Live Longer,’ click here.” or “When you buy a stereo,
beware of other brands that lack our patented circuitry.”)
Back To Top
Support The area of support is concerned with the source and corroboration
of the information. Much information, especially statistics and claims
of fact, comes from other sources. Citing sources strengthens the
credibility of the information. (Remember this when you write a
research paper.) Source Documentation or Bibliography. When facts or statis-tics
are quoted, look to see whether their source is revealed, so that
you could check their accuracy. Some source considerations include
these: - Where did this information come from?
- What sources did the information creator use?
- Are the sources listed?
- Is there a bibliography or other documentation?
- Does the author provide contact information in case you
wish to discuss an issue or request further clarification?
- What kind of support for the information is given?
- How does the writer know this?
It is especially important for statistics to be documented.
Otherwise, someone may be just making up numbers. Note that some
information from corporate sites consists of descriptions of products,
techniques, technologies, or processes with which the corporation is
involved. If you are careful to distinguish between facts (“We mix X
and Y together to get Z”) and advertising (“This protocol is the best
in the industry”), then such descriptions should be reliable. Corroboration See if other sources support this source. Corroboration or confirmability is an important test of truth. And
even in areas of judgment or opinion, if an argument is sound, there
will probably be a number of people who adhere to it or who are in
some general agreement with parts of it. Whether you’re looking for
a fact (like the lyrics to a song or the date of an event), an opinion
(such as whether paper or plastic is the more environmentally friend-ly
choice), or some advice (such as how to grow bromeliads), it is a
good idea to triangulate your findings: that is, find at least three
sources that agree. If the sources do not agree, do further research to
find out the range of opinion or disagreement before you draw your
conclusions. What you are doing with corroboration,
then, is using information to test information.
Use one source, fact, point of view, or inter-pretation
to test another. Find other informa-tion
to support and reconfirm (or to challenge
or rebut) information you have found. It is
critical, of course, to consider the quality of
the corroborative information. There is at
least one situation on the Web now where
more than a dozen “natural care” sites have
all repeated the same false story that the surfactant
sodium laureth sulfate (used in many
cosmetics) causes cancer. (The owners of
these sites sell products that do not contain
this ingredient.) Finding the same claim on
several of these sites is certainly not corroboration.
For a corroborative test, you should
look to disinterested third parties, in this case
to the federal government, toxicology reports,
and the like. Corroboration is especially important
when you find dramatic or surprising information
(information failing the moderateness
test, above). For example, a claim like the one
just mentioned, that some commonly used
substance is harmful, should be viewed with
skepticism until it can be confirmed (or
rebutted) by further research and by truly
reliable sources. The claim may be true, but it
seems unlikely that both government and
consumer organizations would let a harmful
chemical or other product go unchallenged. Evaluation Tip A quirk of human nature
causes most of us to value
what is scarce. Some information
producers who
want to manipulate us
therefore pretend to have
“secret” or “exclusive”
information—which, of
course, cannot therefore
be corroborated. Be very
careful of such informa-tion.
Remember that,
when it comes to informa-tion,
“secret” is nearly a
synonym for “gossip,” and
“exclusive” means roughly
“unsubstantiated” or
“unconfirmed.” External Consistency While the test of
corroboration involves finding out whether
other sources contain the same new informa-tion
as the source being evaluated, the test of
external consistency compares what is familiar
in the new source with what is familiar in
other sources. That is, information is usually a
mixture of old and new, of some things you
already know and some things you do not.
The test of external consistency asks, Where
this source discusses facts or ideas I already
know something about, does the source agree
or harmonize or does it conflict, exaggerate,
or distort? The reasoning is that if a source is
faulty where it discusses something you
already know, it is likely to be faulty in areas
where you do not yet know, and you should
therefore be cautious and skeptical about
trusting it. Indicators of a Lack of Support As you
can readily guess, the lack of supporting evi-dence
provides the best indication that there
is indeed no available support. Be careful then, when a source shows problems like these: Numbers or statistics presented with-out
an identified source for themAbsence of source documentation
when the discussion clearly needs
such documentationLack of any other sources that pre-sent
the same information or
acknowledge that the same informa-tion
exists (lack of corroboration)Evaluation TipAs you continue to work in
the world of information,
you should begin to devel-op
an effective “baloney
detector,” which will
enable you to sense when
a claim is too good, too
bad, too excessive, or too
weird to be true. Suppose,
for example, you receive a
chain letter e-mail claiming
that you will be paid five
thousand dollars if you for-ward
it to all your friends.
How likely is that to be
true? And if it were true,
wouldn’t you expect to
hear about such an amaz-ing
deal on the nightly
news or in the paper? You
can keep your baloney
detector tuned up by read-ing
widely and getting a
good sense of what generally
happens, and what
truths and myths are
spreading around the Web
or another part of the
information environment. Back To Top
Summary of the CARS Checklist for Research Source Evaluation Credibility Trustworthy source, the quality of evidence and
argument, author’s credentials, evidence of quality
control, known or respected authority, organizational
support. Goal: an authoritative source; a source
that supplies some good evidence that allows you to
trust it. Accuracy Up-to-date, factual, detailed, exact, comprehensive,
audience and purpose reflect intentions of completeness
and accuracy. Goal: a source that is correct
today (not yesterday); a source that gives the whole
truth. Reasonableness Fair, balanced, objective, reasoned, no conflict of
interest, absence of fallacies or slanted tone. Goal: a
source that engages the subject thoughtfully and reasonably;
a source concerned with the truth. Support Listed sources, contact information, available corroboration,
claims supported, documentation supplied.
Goal: a source that provides convincing evidence
for the claims made; a source you can triangu-late
(find at least two other sources that support it). Back To Top |