|
INTRODUCTION
TO THE AMERICAN INDIAN AND ALASKA NATIVE EDUCATION EVALUATION
By
Thomas R. Hopkins, Ed.D.
The four evaluation studies
(Peterson, 1948; Anderson, et. al. 1953; Coombs et al, 1958 and
Bass 1971) are classics in the field of evaluating American Indian
and Alaska Native education. * They represent
a continuum that started in 1944 and ended in 1971. The glue that
created the continuum
was L. Madison Coombs who took over the BIA testing program in
the late 1940’s and was able to maintain control until Bass
completed his report to BIA in 1971. A careful review of the four
evaluations will cover almost all issues related to evaluating
the culturally different American Indians and Alaska Natives. Basically,
the Bass evaluation contains all that was learned in the previous
evaluations by being longitudinal, tracking students and relating
test scores to background characteristics. All four were concerned
primarily with assessing school quality (public and BIA) and ethnicity
(Indian-Native and White). Anyone attempting to evaluate American
Indians and Alaska Natives in 2006 would do well to review these
evaluation reports carefully and thoroughly.
Historically, the
evaluation of American Indian and Alaska Native education has its
roots in the last quarter of the nineteenth century. It was during
this time that scientific methodology was beginning to be applied
to American Education (Cremin, 1961). Later on, from 1900 to 1945
the Progressive Education movement played a basic role in the evaluation
of American Indian and Alaska Native education. In a sense, the
Anderson and Eells research (1935) was perhaps the first effort
to apply scientific education instruments to Alaska Native education.
Anderson and Eells learned almost immediately that scientific education
instruments developed at Stanford University for one culture cannot
be successfully applied to another culture. This is a lesson which,
in general, American education keeps ignoring. Even today with
the No Child Left Behind Act (NCLBA), one culture (that of the
test norming group) fits all is the conventional approach.
The
Progressive Education influence, generally, parallels the creation
of the Progressive Education Association, a professional organization
based on the Progressive Education movement. Carson Ryan, who wrote
the Education section for the famed Meriam Report (1928) was a
professor at Swarthmore College before being selected to do the
evaluation of the education of American Indians for Meriam (Szasz
1977). Ryan became the Director of BIA Education in 1930 and lasted
to 1935. He was succeeded by Willard Beatty who was Director of
BIA Education from 1936 – 1952. Beatty was president of the
Progressive Education Association just before taking the BIA Directorship.
Following Beatty, Ryan became President of the Progressive Education
Association. The influence of Progressive Education on the evaluation of American
Indian and Alaska Native education goes something like this. The
Progressives thought their program, which emphasized basing the
curriculum on the experiences of the students and then learning
by investigating this foundation was as good as or better than
the traditional approach which was based on memorization and recall
of book knowledge. Experience did not play a big role in traditional
education. Ralph W. Tyler was selected to assess the difference
between Progressive and Traditional education by comparing, pairing,
Progressive and Traditional high school students as they progressed
in college. This became known as the “Eight Year Study.” Tyler
found the Progressive program equal to and in some cases, superior
to Traditional education.
The story now moves to the BIA Education
Program which was directed by Willard W. Beatty. Keep in mind that
the BIA took over the Alaska Native education responsibility in
1931. Beatty had been, in his mind, implementing Progressive Education
in BIA schools since 1936. He was interested in finding out if
the Federal Government’s BIA schools were as good as public
schools. For a number of years the BIA national office was located
in Chicago, not Washington, D.C., where it is now located. In 1944,
Ralph Tyler was Chairman, Department of Education, University of
Chicago. There is no record of Beatty going over to the University
and discussing with Tyler an evaluation of Indian schools. However,
Even without the record, it is highly probable that this meeting
occurred because the BIA contracted with the University of Chicago
to conduct the first of the four evaluation projects which was
carried out by Shailer Peterson (1948). Tyler has a section in
the book titled, “The Significance of this Investigation
to School Administrators, to Teachers and to Students of Education
(pp. 113 – 117)”
Prior to the Peterson “Investigation” and
except for the efforts of Anderson and Eells, the evaluation of
Alaska Native Education was virtually nonexistent. Sheldon Jackson,
the first General Agent for Education for Alaska, was basically
a Christian missionary, not an educator. His almost passionate
focus was on saving the souls of Natives and helping them step
by step scale the ladder of civilization (Hopkins, 1959). Jackson
was succeeded by Harlan Updegraff. Updegraff’s last two program
items pertained to morality and teaching religion (Hopkins, p 14)
We should keep in mind the fact that Carson Ryan came to the Meriam
study directly from Swarthmore College, a celebrated Quaker institution
of higher education. Based on the history of Alaska Native education,
it is probable that Willard Beatty implemented for the first time
a non sectarian approach to the education of Alaska Natives.
It
is important to know that L. Madison Coombs was hired by the BIA
about 1947 or 1948. His first job was as a “Counselor” at
Haskell Institute in Lawrence, Kansas. In that position he was
assigned as the technical contact for subsequent evaluation studies.
More importantly, Coombs had responsibility for the BIA standardized
testing program until it was abandoned about 1955. Madison, in
time, directed the basic BIA education evaluation and research
of American Indians and Alaska Natives for the next twenty-five
years. I worked with Madison for the first time in 1959 when he
was Assistant Area Director for Education at the BIA Juneau Area
Office. I introduced him to the Academic Department teachers at the
Mt. Edgecumbe School probably in January of 1959. His book, The
Indian Child Goes to School, had just arrived
at the school and we are excited to have in our presence such a
dignitary. Subsequently, I worked with and for Madison Coombs for
the next 15 years. Madison had an eloquent style of writing and
a clear, logical and reasoned method of thinking. It is also important
to know that Madison was the BIA technical contact for the, “National
Study of American Indian Education,” (Fuchs and Havighurst,
1972). He did all this without academic credentials in education
evaluation and research.
Historically, Ralph Tyler and Willard
Beatty designed the first evaluation of American Indian Education.
Then, Madison Coombs, with modest changes, maintained the same
basic design for the following three evaluation projects. Coombs
maintained the general assessment of the qualitative differences
between BIA schools and public schools. It is ironic that in these
days of the No Child Left Behind Act with its emphasis on the school
level of education evaluation no one has referenced or made use
of these four classic evaluations. Mistake after mistake which
more often than not reflect unfairness to Indian and Natives is
being made by states with Indian and Native populations in trying
to meet the mandates of NCLBA.
The four evaluation projects of
American Indian and Alaska Native education reflect the times from
1944 – 1971. From an evaluation perspective they all have
some things in common.
Evaluation Strategies.
Statistical Samples. All of the projects used statistical sampling which means that
the theory of probability plays a vital and basic role in the evaluations.
Over the years, one of my greatest challenges as been to satisfactorily
explain probability and sampling to Indian lay people. Most Indian
and Native cultures have been taught that reality is exacting one
to one. When probability is introduced and its sampling procedure
with a percentage of confidence and reliability, Indian and Native
peoples start frowning and looking perplexed and confused. It is
difficult for them to accept that a random sample of 30 can be
used to make reliable findings for a population of 500. It is just
simply unbelievable. None of the evaluations used data that were
based on every child enrolled in the BIA system. None of the studies
had a large enough sample to provide sub-sample findings for cultures.
By this I mean that the sample was for the entire BIA system and
insufficient in numbers to provide Eskimo, Aleut, Athbascan, Navajo,
Sioux, etc. outcomes. The samples were not randomized and depended
on administrative decisions about what tribes would be included.
- The
Peterson project included Alaska Natives, 2,085 total sample,
- The
Anderson, et. al project included 183 from a boarding school,
1,822 total sample,
- The Coombs, et. al. project did not
include any Alaska Natives, 23,608 total sample,
- The
Bass project included Alaska Natives, 2,584 total sample.
It should be noted that the large Coombs sample was possible because
he had implemented and administered a BIA total testing program.
With local BIA administrators involved data gathering was not always
the best. Reminds one of the early problems of the National Assessment
of Educational Progress (NAEP) and states concerns that it would
rank order them according achievement test scores.
Statistical
Procedures. There was a progression in the statistical procedures
used in each evaluation. Peterson used the median, only, and provides
an excellent justification based on the fact that the mean does
not handle well extreme scores at the top and the bottom. Indian-Native
scores were and are extreme in that 50 percent of the scores will
be below the 35th percentile. However, the subsequent three evaluation
with their increased sophisticated statistical procedures used
the mean as the foundation of their statistical procedures.
Purposes
of the Evaluations. Even with Tyler thinking the evaluation useful
for students, basically, the purpose of all the evaluations was
to compare the education quality for Indians between different
types of schools: BIA, public and mission. Secondly, the evaluation
compared achievement between Indians and Whites. Furthermore, only
the Coombs report mentions local use of test data as important.
On the whole, the evaluations, especially the statistical procedures
used, were for administrative use. Coombs was the only one that
had a section devoted to, “The Predictive Testing Program.” This
testing program pertained to a test battery administered to Indian
students wanting to attend college but more specifically the Haskell
Commercial Program. The data was not sufficient to produce findings.
This brings to mind the great need for predictive research on standardized
tests at all levels regarding American Indians and Alaska Natives.
Now that test data abounds perhaps some researcher should conduct
a longitudinal investigation of the “Predictive Strength
of standardized tests for Alaska Natives and American Indians regarding
the next level of education or the next phase of the students life.”
Background
Characteristics. All the evaluations gathered data on the backgrounds
of the students tested. Relationships between the background characteristics
and test scores were made.
Tracking Students. Only the Bass study
tracked students.
Culture Fairness of Standardized Tests. All the
professional evaluators understood that standardized test norms
were not valid for Indian students. All used the “Raw Score” for
most statistical purposes. The Peterson evaluation set the stage
for the use of the raw score when it said:
The “National
Norm Equivalents” have not been provided with the idea
of suggesting any levels of attainment that are expected. National
norms are commonly based upon either urban school children or,
in some cases, a combination of urban and near-urban education.
In view of this, it is obvious that national norms would not be
the
proper yardstick to use as a means of measuring or comparing
achievement in this Indian education study (Peterson, p. 59).
Later on, Ralph
Tyler (1974) revisited issues in testing. In the Introduction to
the book, Tyler describes how early tests used in World War I were
to identify individual differences, not learning. He describes
the need for tests that assist learning. Contained in this same
book is a chapter by Robert L. Thorndike (1974) titled, “Concepts
of Culture-Fairness.” In the abstract to the article Thorndike
said:
Fairness of a test relates to fair use. One definition of
fair use states that a common qualifying score may be used with
two groups if the regression line based on one group does not
systematically over- or under-predict criterion performance in
the other. However,
it is shown that when the two groups differ appreciably in mean
test score, the above procedure, which is “fair” to
individual members of the group scoring lower on the test, is “unfair” to
the lower group as a whole in the sense that the proportion qualified
on the test will be smaller, relative to the higher-scoring group,
than the proportion that will reach any specified level of criterion
performance. An alternate definition would specify that the qualifying
scores on a test should be set at levels that will qualify applicants
in the two groups in proportion to the fraction of the two groups
reaching a specified level of criterion performance. (p. 35)
Thorndike’s “Fair
Use” rule seems a good one to be applied to any use of standardized
tests with American Indians and Alaska Natives.
Thorndike did not
consider the culture that produces standardized tests, namely the
Eurocentric culture of the U.S. Regarding American Indians and
Alaska Natives there is culture based time which is very different
from linear time which is a basic variable on any standardized
test. Edward T. Hall (1984) covers the differences between culture
time by using several different cultures most of which were American
Indian and Alaska Native. I have often thought that linear time
is the one standardized test variable that presents the most trouble
to American Indians and Alaska Natives. It would be interesting
to use standardized tests in an un-timed experiment to determine
the effects of linear time on test scores. Here, again, test norms
would not be useful and the IndianNative “convention” of
using raw scores would be the appropriate test score to use.
It
is encouraging the such a concept as “Cultural Validity” is
emerging (SolanoFlores 2002). Solano-Flores has clearly described
how test scores can be influenced by the experiences (culture)
of the student. Regarding American Indians and Alaska Natives this
is not especially new knowledge. However, it perhaps may lead to
a break through in fairness in testing which has never before been
a national concern or even a concern when testing Indians and Natives.
The No Child Left Behind Act and its assessment and accountability
requirements which states are supposed to conduct has given extraordinary
importance to standardized tests. Before NCLBA professional educators
could control the uses of standardized tests. But, before and
after NCLBA made its appearance, control was shifted to state legislatures
and state departments of education. Legislatures tend to apply
that which is simple and in the case of standardized tests it
has
culminated in their gross miss use.
Based on my own experiences
in testing Indians and Natives at the school level and considering
the absence of clearly stated cultural fairness procedures,
I recommend the use of teacher grades. Specifically, from the 1977-78
school
year through the year 2,000 I have gathered estimated grade
point
averages on over 10,000 Indian and Native students (Hopkins
2005). Regardless of the ethnicity of the teacher, the results
have
consistently approached a normal distribution. It always appeared
to me that
teacher grades also had the singular advantage of communicating
the assessment to parents and elders. None of the four evaluations
or the contemporary state assessments are of much use to teachers
or parents. However, it should be mentioned that the NCLBA
does mandate parent education. Those providing parent education
regarding
the NCLBA immediately encounter statistics and probability
which leaves the parents and school board members confused whereas
the use of teacher grades would do much to improve the relationship
between assessment and parent understanding.
References Cited
Anderson, Dewey H. & Eells, Walter C. (1935)
Alaska Natives, a Survey of Their Sociological and Educational
Status. Stanford University Press, 1935.
Anderson, Kenneth E.,
Collister, E. Gordon & Ladd, Carl E. (1953) The Educational
Achievement of Indian Children. Lawrence, KS: Haskell Institute
Print Shop, March, 1953.
Bass, Willard P. (1971) "An Analysis
of Academic Achievement of Indian High School Students In Federal
and Public Schools," Albuquerque: Southwestern Cooperative
Educational Laboratory May, 1971, 142 pp.
Coombs, L. Madison, Ralph
E. Kron, E. Gordon Collister and Kenneth E. Anderson. (1958) The
Indian Child Goes to School: a Study of Interracial Differences. Lawrence, Kansas: Haskell Institute Print Shop. U.S. Bureau of
Indian Affairs, 1958, 249 pp.
Cremin, Lawrence A. (1964). The Transformation
of the School, Progressiveism in American Education, 1876 – 1957.
New York: Random House, Part I, “The Progressive Impulse
in Education, 1976 – 1917, pp. 3 – 178.
Fuchs, Estelle
and Havighurst, Robert J. (1972) To Live On This Earth, American
Indian Education. Garden City, New York: Doubleday & Co., 1972.
Hall, Edward T. (1984) The Dance of Life, The Other Dimension
of Time. New York: Anchor Press/Doubleday, 250 pp.
Hopkins, Thomas
R. (1959 reedited in 2006). “ A History of the Education
of Alaska Natives, 1867 – 1959.” Unpublished Masters
Thesis in Educational Administration, The University of Texas,
Austin.
_________________ (2005). “The RMS Educational Assessment
Program,” Albuquerque: Research and Management Specialists,
Inc. 2005. 15 pp.
Meriam, Lewis. (1928) The Problem of Indian Administration.
Baltimore: The Johns Hopkins Press, 1928, 872 pp. See Chapter IX, “Education,” pp.
346-429.
Peterson, Shailer. (1948) How Well Are Indian Children
Educated? Lawrence, KS: Haskell Institute Print Shop, Sept. 1948.
Solano-Flores, Guillermo, (2002) “Cultural Validity: The
Need for a Socio-Cultural Perspective in Educational Measurement,” WestEd/National
Center for Improving Science Education. A paper presented at the
Annual Meeting of the American Educational Research Association,
New Orleans, LA, April 1-5, 2002.
Szasz, Margaret. (1977) Education and the American Indian:
The Road to Self-Determination, 1928-1973. Albuquerque: University
of New Mexico Press, 1977, 252 pp. See Chapters 3 and 4, respectively,
for biographies of Ryan and Beatty.
Thorndike, Robert L. (1974) “Concepts
of Culture-Fairness,” Found in Crucial Issues in Testing,
Edited by Ralph W. Tyler and Richard M. Wolf, Berkeley, Calif.:
McCutchan Publishing Corp. pp. 35-46.
Tyler, Ralph W.(1974), Crucial
Issues in Testing, Berkeley, California: McCutchan Publishing
Corp., 170 pp. Edited by Ralph W. Tyler and Richard M. Wolf.
CHAPTER TWO
Organization and Plausible Uses of the Evaluation Files
All the
files of the evaluation books are in Adobe PDF format. There are
four directories: (1) Peterson, (2) Anderson, (3) Coombs and (4)
Bass and the Introduction of which this paragraph is the last one.
The first three evaluations were printed in book form by the Haskell
Print Shop, Haskell Institute, Lawrence, Kansas. The Bass report
was never published in book form.
One may print all or part of
the pages for each evaluations. The total pages per evaluation
are as follows:
Peterson
Anderson
Coombs
Bass
[included is Thomas Hopkins' Evaluating
American Indian Education] All books are in the public domain. The copy
of the Bass report received by the BIA, which funded the evaluation,
is the one used.
[PDF Instructions] Those users interested in copying parts of
the books for professional use may do so by clicking on the Adobe
Reader “Edit” button
and then the option, “Copy File to Clipboard.” Once
the file is on the Clipboard about any word processor can access
the file and allow the user to copy the desired passages. The files
in each director are based on the chapters of the books, except
for the Bass report the text of which is one file plus individual
files for Appendices.
* There is a section titled, “Organization and Plausible
Uses of the Evaluation Files,” at the end of the Introduction,
after the “References Cited.”
| 
