But, Is Hybrid Scoring AI? We may never know…
Okay, let’s talk about this. I have a great story. Part of my job means attending some of the TEA (Texas Education Agency) updates at various conferences. I recently attended the RLA update, and as almost all conference sessions do, the conversation turned to Artificial Intelligence. But this was different. From what I am assuming was TEA’s response to the presumed negative reaction to using artificial intelligence to auto-grade constructed response items on our state assessments (STAAR & EOC), their slide talking about auto-grading, which they call hybrid scoring, starts with this bullet point:
“Hybrid scoring is not artificial intelligence.”
Then, the second bullet point was, “Humans, including Texas educators, will continue to be involved in all parts of the scoring process.”
When they start discussing this slide, they, of course, have my full attention because it’s not making sense. They went on to use the following phrases: train the model, write the code, train the code, the computer will mimic humans, we teach the computer to grade the way a human would… I think you get the idea. In every way that they described their auto-grading technology, they described artificial intelligence. A few teachers/RLA leaders raised their hand and challenged the notion, and they were brushed off with phrases like, “Well, I don’t know how much YOU know about AI, but…” And, just straight refusals to discuss how what they were describing was not AI.
Someone smarter than me would have grimaced and moved on to their next session. But I am not smarter than me because I am me (so existential, I know, reminds me of our Gamer AI reading about quantum computing from this week where one of my notes was, “This reminds me of Loki”). So, I walked up to the presenters and let them know of my concern. Their immediate response was to be on the absolute defense. I think they assumed I was someone ready to tell them that “computers are taking over the world and not in my Texas,” blah blah blah. But, really, I see this as an answer to RLA teachers' prayers. Trying to review student papers using the released TEA rubric and give feedback for assessment writing is so incredibly hard. Just that point was brought up in the session before AI derailed us. I see this new way of grading as a tool that could be turned around and used by teachers to help calibrate their grading and give actionable feedback to students that aligns with the state expectations. This COULD be GREAT! But, what will stop this progress in its tracks; if Texas educators feel like they are not being communicated with honestly. They will assume malintent and all that could be great is lost in fear. Well, once I had the TEA team convinced I was on their side, they asked me to speak to their director because they were now concerned about their messaging (which, if you don’t know, is fed to them after their marketing and legal teams give it the thumbs up).
So, after I go through the song and dance of “I am not against you”, “I think this could be great”, “No, this is what I am currently studying, I kind of know what I am talking about”, I finally got to say, “This is messaging will derail you and I am concerned” and I then finally had the director’s ear. He said over and over that his experts assured him that this IS NOT AI. And, I kept saying, but what you are describing IS AI. I would ask, truly wanting to understand, what is the reasoning behind why this isn’t AI, and I could never get an answer. We finally landed on, well, perhaps, it is not AI but you need to reevaluate how to describe “hybrid scoring” because, as I said then, “you’re going to get roasted and this all could be lost.”
How does this long story relate to the prompt? Well, I now have a suspicion that what he was trying to describe was conditional logic, which may not technically be AI. His experts COULD be right! (but I still don’t think they are) My point in all this is that I think Artificial Intelligence has the power to do just what Pressey said, “Lift from [the teacher's] shoulders as much as possible of this burden and make her [sic] free for those inspirational and thought-stimulating activities which are, presumably, the real function of the teacher. Pressey, S.L. (1926)” (y’all are going to see this quote a lot from me, I love am obsessed with it). But, right now we are so swept up with buzzwords and dollar signs even our experts aren’t really experts because I don’t think these types of experts have had time to be fully cultivated. I think our future is promising, we all know what AI can bring to students through Intelligent Tutoring Systems and Personalized Learning Paths but we have to see it for what is, and that changes every day. We have to bring all the experts to the table- the teachers who have been seated at a kidney table and seen how the completely illogical thinking of a child can totally make sense to those who study the art of teaching, the researchers who can look at the theory and apply the data that creates the perfect scenario, the developers who understand the technical scope behind designing the technology, the content experts who understand what needs to be learned at a granular level, the instructional designers who know how to put pen to paper and make learning make sense and we have to move forward respecting and applying each expertise.
