Human vs Computer Logic vs Human Teenager Logic
Recently, a classmate of mine shared a story about an AI graded constructed response item on a district assessment. In short, the data showed that students struggled with the standard aligned to this particular constructed response item. So, what do we do? We buy more resources! We require Staurday school! We blame the teachers, no the students, no the textbook, wait we haven’t had textbooks in 12 years… we PANIC! Luckily, my classmates team was more practical in their approach. They went back to look at the student responses to look for trends in misconceptions that the computer did not evaluate. What did they find? Well, anyone who has spent any amount of time with teenagers might be able to guess. The students knew/thought their teacher would not see what they wrote and figured out the could enter anything, gibberish, song lyrics, whatever, and the system would accept it and allow them to move on in the assessment without actually completing the work.
A silly example of students being silly, but, a perfect cautionary tale of what happens when we assume the logic of artificial intelligence has all its bases covered. Now the idea of not solely relying on numerical data isn’t a novel concept. We’ve done this in data meetings for years by requiring teachers to bring in student test booklets so that we can compare the scores to the work and the work to what we know about the student; in other words, triangulate the data. However, anecdotally, their is this overwhelming rhetoric about the magic of AI. It will solve this and we won’t have to do that. This rhetoric is clouding what we know to be good data analysis in the K-12 learning space.
We have to continue to understand that artificial intelligence will augment how we do things and while we cannot know how that will ultimately materialize for learning, we do know that depending solely on algorithms, no matter how sophisticated, can and will always be usurped by the human experience and even more so by the human teenager. Humans-in-the-loop whether externally or internally designed within the UX of a system needs to remain as important as bringing test booklets to a data meeting.
