Search Post

Post Categories

Post Tags

Post Authors

Why AI Detectors Fail and How We Can Foster AI Fluency Instead

April 8, 2025

The question isn’t if students will use AI. It’s how.

While AI literacy has been the baseline, aiming for AI fluency, where students not only understand but can critically engage with AI, is now crucial. Currently, some schools use AI detectors to address cheating, but these tools are flawed, often mislabeling work and disproportionately affecting multilingual students. In this blog, I will share the limitations of AI detectors and advocate for a shift towards cultivating AI fluency, transforming our approach from restrictive to empowering.

AI Detectors Get It Wrong. A Lot.

On the surface, AI writing detectors sound like a good idea. They claim to spot when a student is using AI instead of writing on their own. But you’re not going to believe this… They don’t actually detect AI. They use two flawed metrics called perplexity (how “predictable” writing is) and burstiness (sentence variety) to guess whether the text might be AI-generated.

The issue?

Academic writing is predictable by design. Students are taught to write like this. And multilingual students, who are often trained to write in grammatically precise, straightforward sentences, are even more at risk.

To make things worse, AI detectors don’t confirm AI use; they assign a probability. If a tool says a paper is “98% AI-generated,” that’s not proof. It’s just a statistical guess based on writing patterns. A Stanford study found that AI detectors falsely flagged over 61% of essays from the Test of English as a Foreign Language (TOEFL)—a widely used exam for students learning English as an additional language—as AI-generated. Even worse, 97% of those essays were flagged by at least one detector, and 19% were unanimously mislabeled by all seven major detectors. My thinking is, if a detector says a piece of text is 99% (probably) AI-generated, I’m not willing to go to bat and ruin the relationship I have with this student when I could be wrong. There’s no coming back from that! (This scenario gets 10x worse if you do not have a clear policy and shared understanding with students as to what “AI use” really is.)

And yet, the edtech companies keep selling AI detection tools like they’re the solution. How do they even get away with this? So frustrating. Given these flaws, it’s clear that reliance on AI detectors is misguided. Investing in developing AI fluency among our students turns a challenge into an opportunity for growth and learning.

Our Path Forward

AI isn’t the future. It’s already here! Just the other day, I suggested for a student to put their ideas into a chatbot and see what they are missing or what might add to their list. Their jaw hit the floor when a teacher was telling them to use AI! Yet when they opened a new tab and opened up the tool, all their previous chats were right there on display. They were not new to the tech, just new to openly admitting it. Clearly, if we don’t normalize responsible AI use students are going to hide it. So here’s what I think we can do moving forward.

1. Build AI Fluency (Not Just Literacy)

A student with AI literacy can tell you that AI-generated content might be biased. A student with AI fluency knows how to check for bias, challenge AI’s response, and refine their prompts to get better results. Someone who is fluent in AI uses it to amplify their human intelligence.

So, how do we help students get there?

  • Become AI fluent yourself. Choose a chatbot and pin the tab, or set Google Chrome to open the page automatically upon startup! Keep a post-it note near your screen reminding you to use your AI assistant when planning. Set a goal to use AI once a day for anything. I don’t care if you are trying to find the best wine to go with your meal or planning an itinerary for your summer vacation. Just start making it a habit to experiment and play with generative artificial intelligence. Read about it. Engage on social media (my favorite AI/edtech influencers are all buzzing on LinkedIn these days). Listen to podcasts. Read books. Attend a workshop at KCI! (You’re on the right track by reading this blog! Look at you, fancy!)
  • Let students experiment with AI, then question it. Have students compare AI-generated writing to their own. Where does it fall short? How could it be improved?
  • Teach students to refine their AI inputs. Give them a poorly written AI-generated response and ask: What prompt would get a better result? Let them test and adjust.
  • Make fact-checking a habit. Challenge students to verify AI-generated content by cross-checking sources. If AI “hallucinates” facts, how can they find the truth?
  • Ask students to explain their choices. When using AI, have them reflect: Why did you use it? How did it help? What decisions did you make?

Equity in Action

Not all students have equal access to AI knowledge outside of school. If we only teach AI literacy (basic awareness), students with more tech access at home will gain an unfair advantage over those without it. Teaching AI fluency ensures that all students, regardless of background, can engage with AI critically, ethically, and with confidence.

2. Be Transparent About AI Use

When students feel like AI is a secretive or forbidden topic, they’ll avoid asking questions, which just leads to more confusion. Instead of a “gotcha” situation, we should be having open conversations. What are good ways to use AI? What are not? How can AI be used as a tool for thinking, not just a shortcut for getting work done? Students may be more receptive to our open and honest concerns about AI over-reliance than to AI bans. 

There aren’t many people talking about what it looks like to teach students to be decision-makers with AI, but Dr. Sabba Quidwai is. Below is what she shares with school leaders, teachers, and students for how to make decisions (with AI and in life) that are WISE. Check out her free mini-course.

Alt text: How to make decisions that are WISE: How will using AI affect my well-being and those around me? Am I using AI with integrity in a way that’s honest and fair? What skills can I learn from using AI, and what might I miss learning? Does using AI help me engage better with what I’m doing?

Used with permission from Dr. Sabba Quidwai

Equity in Action 

When AI use is hidden or stigmatized, students with more tech experience feel more comfortable using it, while others may fear punishment for experimenting. A transparent approach ensures that all students have a safe space to ask questions. Better yet, if we model transparency with our own use of AI, we show them how and create that safe space!

3. Focusing on the Writing Process, Not Just the Final Product

When we shift the focus from product to process, AI becomes less of a mystery. The more we emphasize the process, the less we need to stress about AI.

One great strategy is Mike Kentz’s “Grade the Chat” approach, where students are encouraged to interact with AI during the writing process, but they submit their AI chats along with their final product. This shifts the focus from “Did you use AI?” to “How did you use AI?” It allows students to reflect on their choices and demonstrate critical engagement with the tool. How great if, along with sharing their chat you include a reflection piece where students think about what AI did with vs. for them, how it helped/harmed their thinking on the subject, and what they might do differently next time.

Here are some other ways to emphasize process over product:

  • Require drafts – Instead of just collecting a polished final paper, have students submit drafts along the way to track their thinking and development.
  • Assign reflective writing – Have students explain how they developed their ideas and whether (or how) AI played a role. 
  •  Encourage collaborative brainstorming – Ask students to compare their ideas to AI-generated ones. Which ideas are stronger? Why? This helps them see AI as a tool, not a replacement for their creativity.

The goal isn’t to “catch” AI use. The goal should be coaching students to develop confidence in their own thinking and writing. No one wants to play detective anyway! When we shift the focus from process to product, students see that they are thinkers, and AI is just another tool in their toolbox.

Equity in Action 

Traditional assessments often favor students with strong writing skills while leaving multilingual learners and students with learning differences at a disadvantage. Prioritizing process over product creates more opportunities for all learners to succeed. Additionally, and especially if you are not explicitly teaching and assessing writing and the essay or written piece is just a way for you to assess what they know, maybe try exploring alternative or even additional ways of having students demonstrate their knowledge. Think portfolios, presentations, projects, Socratic seminars…

4. Helping Students Build Confidence in Their Own Thinking

One of the best things we can do for students is to remind them that their voices, ideas, and critical thinking are valuable. AI doesn’t write with insight or passion. It doesn’t analyze in a meaningful way. When students see AI as just a tool, not a replacement for their own abilities, they’re more likely to engage with it wisely. Here’s how we can foster this understanding:

  • Encourage Critical Engagement: Integrate activities that prompt students to question AI outputs and reflect on the differences between their insights and what AI produces. For example, after using AI in a project, they could answer questions like, “What decisions did I make that AI couldn’t?”
  • Showcase Diverse Thinking: Regularly highlight how different students use AI to solve problems in unique ways. This celebrates diverse approaches.
  • Discuss AI’s Limitations: Regularly bring up examples from current events or studies that show where AI has failed or shown bias. This helps students understand the tool’s limitations and reinforces the value of human oversight.
  • Promote a Growth Mindset: Encourage students to experiment with AI, focusing on effort and learning rather than just correct outcomes. 

Equity in Action 

Imposter syndrome and self-doubt disproportionately affect marginalized students. If they believe AI is “smarter” than they are, they may stop trusting their own ideas. By reinforcing that AI is just a tool, not a superior thinker, we help all students recognize the value of their own intelligence and creativity. Shifting the focus from AI as a mystery to a manageable and comprehensible tool will empower our students to control it, use it wisely, and view themselves as capable thinkers and innovators! (YESSSSSS!)

Moving From Caution to AI Fluency

Right now, too many schools are stuck in a cycle of fear—afraid of AI, afraid of misconduct, afraid of losing control. But we don’t have to stay there! We have a choice. We can be reactive and use unreliable AI detectors, or we can be proactive and teach students to engage with AI with purpose and confidence. 

Fluency means understanding AI’s strengths and weaknesses. It means knowing when to use it, when to rely on their own thinking, and how to navigate an AI-driven world. It also means students recognize that they are the thinkers. AI might be fast, but it will never replace human creativity, curiosity, empathy, critical thinking, and other human intelligence skills. (If you agree, check out the awesome work being done by the Human Intelligence Movement.)

The journey towards fluency happens in stages. Some schools start with a cautious approach, watching and waiting to see what happens elsewhere while limiting or restricting AI apps for students or even staff. Others focus on AI literacy, helping students understand it but not necessarily integrate it. But the real goal is AI fluency, where students don’t just know about AI, they engage with it critically and with confidence.

Why Equity Matters in AI Education

AI in education isn’t really about AI. I see it as a gateway to help us find new ways to make education better. The way we implement AI can either widen or close equity gaps.

  • AI detection disproportionately harms multilingual learners.
  • Banning AI in schools leaves students in underfunded areas unprepared for the future.
  • Traditional writing assessments favor certain students, while process-based approaches make success accessible to all.

Just recently the CSU system in California made a deal with OpenAI to give access to ChatGPT Edu to ALL CSU students and faculty. Now my questions are: How will this trickle down to K-12 institutions? Will we have a greater sense of urgency around AI? Will this mean more professional learning opportunities for teachers? Increased AI education for students? Time will tell if this was a good or bad idea, but AI fluency will never be a bad idea. This kind of fluency is more than a goal. It is a necessity in today’s digital age. By fostering an environment that empowers our students to not only navigate an AI-driven world, we can also empower them to shape the future. What steps will you take to contribute to this vital educational evolution? Join us at KCI for workshops and resources designed to transform your approach to AI in education!

Reflection Questions

1. How can I design assignments that emphasize process over product?

2. What new strategies can I try to assess student learning authentically without relying on AI detectors?

3. How can I help students develop AI fluency?

4. How can I help students recognize their human intelligence and unique ideas?

 

Explore Free Resources to Learn More

Check out our AI Capstone Program

Note on AI Use:
I used ChatGPT to help me organize and refine my thoughts for this blog. I provided the ideas, notes, and direction, and the AI tool helped me pull it all together. I reviewed and edited everything to make sure it reflects my voice, stories, emojis ?, and the message I want to share.

Share Post

Post Author