In an effort to influence the Texas State Board of Education to ban the teaching of the weaknesses apparent in evolutionary theory, the Texas Freedom Network (TFN) Education Fund sponsored and released a survey in 2008 titled “Evolution, Creationism & Public Schools: Surveying What Texas Scientists Think about Educating Our Kids in the 21st Century.”¹ ICR News previously published a review of some of the serious logical fallacies of this survey.²

Led by sociologist and evolution proponent Raymond A. Eve of the University of Texas at Arlington, the 59-question survey was sent to 1,019 biology and biological anthropology faculty members of public universities and large private institutions in Texas. There are four flaws in the methodology of the survey, any one of which would suffice to invalidate it: 1) Improper generalization, 2) lack of anonymity, 3) inadequate expertise, and 4) vague, meaningless questions.

Improper Generalization

TFN’s report repeatedly states that its data accurately represent “Texas scientists.” However, the opinions of university biology professors cannot be generalized to all Texas scientists. TFN errs by claiming that its survey represents the opinions of scientists in government, private industry, or non-biological fields.

Also, studies have shown that most American university faculty members are far more atheistic than their non-university counterparts and the public.³ Good survey construction incorporates features to control for known selection biases such as these, but the TFN made no such effort.

Lack of Anonymity

Standard survey procedures must ensure the complete anonymity of respondents, otherwise minority opinions can be underrepresented. The TFN effectively invalidated its survey by failing to use third party contractors, send out anonymous ID numbers, or employ other respondent-protective practices. In fact, on page 6 of the survey report, the authors describe how respondents could be “profiled” based on their education, teaching experience, etc. Fear of this profile being linked to their institution is exactly what discourages minority opinions and why good surveys are designed to prevent it.

The TFN is candid in its hostility towards views that dissent from Darwinism, yet this fact was not controlled for in the survey design. Without confidence that minority opinions—possibly a significant number—were not underreported, the survey findings and conclusions are not valid.

Inadequate Expertise

Over 78 percent of those surveyed selected the response that there was “no difference between ‘creationism’ and ‘intelligent design.’” Whereas the survey did contain responses to identify training or experience teaching evolution, the survey contained no verification that respondents had any training in intelligent design or creation science. There was no test showing that respondents could even accurately identify views unique to intelligent design, creation science, or views held by both groups. Such questioning elicits invalid caricatures, not legitimate conclusions.

Vague, Meaningless Questions

All five agree-disagree statements reported in the survey failed the standard of specificity. The statements’ lack of specificity force a non-scientific “mind-reader” approach as to what respondents mean by “yes” answers or in gauging their thoughts on social impact.

For example, one agree-disagree statement on the survey was rendered, “Teaching ‘Weaknesses’ of Evolution Impairs Ability to Compete for 21st Century Jobs.” But what jobs? And how many jobs? Does this include jobs as computer scientists, physicians, engineers, physicists, lawyers, or only evolutionary biologists? The statement’s lack of specificity makes it impossible to interpret what a “yes” response means. This question cannot justify TFN’s conclusion “that Emphasizing ‘Weaknesses’ of Evolution Would Substantially Harm Students’ College Readiness and Prospects for 21st Century Jobs.” This “finding” only bolsters the false claim that belief in evolution is vital for properly performing “jobs.”

Another statement read, “It is possible for someone who accepts evolutionary biology to have religious faith.” Faith in what? Faith in evolution? The Flying Spaghetti Monster? The Force? Both a nominal religious faith and a deeply held religious faith generate a “yes” response. In regard to significance, what is the difference between the 91 percent of evolutionary biologists who answered “yes” to this statement and intelligent design advocates, who also agree that evolution is compatible with religious faith, or creationists, who say evolution is a religious faith? Thus, the question yields no meaningful data.

The TFN presented its survey as a serious scientific instrument with legitimate data. But it would not survive any standard peer-review process for publication because its findings and conclusions are invalidated by at least four flawed methodologies. This survey would benefit from critical analysis in the same way that the theory of evolution would benefit from having its weaknesses discussed.

References

1. Eve, R.A, and C. A. Belhadi. 2008. Evolution, Creationism & Public Education: Surveying What Texas Scientists Think about Educating Our Kids in the 21st Century. A Report from the Texas Freedom Network Education Fund, accessed online January 19, 2009.
2. Thomas, B.Texas 'Freedom' Network, UTA Professor Oppose Academic Freedom in Public Schools. ICR News. Posted on icr.org November 26, 2008, accessed January 19, 2009.
3. Tobin, G. A. and A. K. Weinberg. 2007. Profiles of the American University, Volume II: Religious Beliefs and Behavior of College Faculty. San Francisco, CA: Institute for Jewish & Community Research.

* Dr. Guliuzza is ICR’s National Representative.

Article posted on January 22, 2009.