Saturday, October 13, 2018

Be careful of using students’ achievement data in educational research

The article about using data reminds me of a conversation on types of finals among my roommates. Two of my roommates are from the School of Engineer, and another one is from Carey business school. All of them have written examinations at the end of the semester, while I have three projects to present. At first, I was complacent because I don’t have to spend days and nights on rote learning. But after I read the article and reconsidered my process of each project, I regretted having such a superficial understanding of education research. I feel even harder to plan an educational program compared to written tests.

Hess (2008) explained three aspects of misusing data, which are using data in half-baked ways, translating research simplistically, and giving short shrift to management data. He also gave readers four suggestions to avoid the problems mentioned above (Hess, 2008, p.12). Previously, I thought students’ achievement data is vital for the assessment of a policy, or a teachers quality because, in my high school, teachers’ rewards are directly related to students’ academic performance. For example, my math teacher had more bonus as my class had higher average scores compared to other classes. Therefore, when I was thinking how to measure a teacher’s quality, a comparison between students’ previous scores and present scores is the first thought in my mind. However, my classmate was a little hesitated when I presented the idea. I was confused at that time because I believe the testing result is the most effective way for assessment. And now, Hess (2008) explained the reason that over-focusing on student achievement undermines confidence in management (Hess, 2008, p.13). Too many educators, including me, exaggerate the effect of student achievement data, while it just illustrates how students are faring. No matter that at least three methods are required when collecting data or making an assessment.

Besides, I was thinking about my own experience when I read the article. I was educated in China, whose educational system is exam-oriented. For example, when applying for college, American students have to take SAT, write personal statements, or maybe have interviews. But for Chinese students, we have only one standard, which is the college entrance examination. I am not saying that this kind of system is the worse in the world because there is no better way to realize education equity when dealing with a large population. But it does become a barrier for Chinese educators to come up with creative ideas to change the current situation.

I think the purpose of education is to develop students’ growth, such as cognitive development, academic achievement, and social-emotional status. For educators, academic achievement is the easiest ways to evaluate because we can see the tendency by quantity data. However, we cannot over-stress the importance of test scores. Is it necessary to collect student achievement data? Is it efficient to measure performance by using student achievement data? We have to ask ourselves before we conduct research.



Reference
Frederick M. Hess. (2008). The New Stupid. Retrieved from:
http://www.ascd.org/publications/educational-leadership/dec08/vol66/num04/The-New-Stupid.aspx


2 comments:

Anonymous said...

Thanks for sharing your experience Sandy! I was surprised to read that there is one standardized test that dictates college acceptances. Could you elaborate more on the Chinese college entrance exam process and what it entails? And thus, do you think the American "way" of college applications is more creative than that of the Chinese "way?"

Unknown said...

Thanks for your insight! It’s always interesting to get a glimpse into how others are measured, and it sounds as if different academic programs have very different assessment approaches. In the corporate world, there is a saying, “What gets measured, gets done.” In education, it’s more like, “What gets tested that teachers are judged on, gets taught.” Assessment can be a double-edged sword. It is essential for evaluating learning, checking for mastery of critical skills, and showing teachers whether they have been successful. But if teachers know on what results their jobs depend, they will naturally teach to those tests, perhaps to the exclusion of other subjects or skills. To some extent, then, our education is only as good as the standards we test. Many times, I fear, we are not testing the right set of skills and knowledge. High quality tests that measure content mastery, critical thinking, creativity, etc. are possible, but I don’t think they are the norm. And regardless of the quality of the test, I very much agree with you that we should not overemphasize the importance of test scores and ignore other demonstrations of mastery and capabilities.