Fishing for AI use in science education using modern knowledge of fishes

Artificial Intelligence (AI) is a new reality in education, presenting many opportunities and challenges for educators, students, and researchers alike (Crompton & Burke, 2023). While it’s effective and ethical use can be context-specific, and at times a bit murky, it’s general appeal is easy to understand; AI can boil down hours of note review, literature searches, and synthesis into a few keyboard taps and this will inevitably revolutionize the educational landscape (Adiguzel et al., 2023; Strielkowski et al., 2025). But beware, there’s a snag: AI can fail at fish biology, and that tempting AI-generated content may just have a hook in it! Many students enrolled in our institution’s vertebrate evolution course are experiencing their first deep dive into ichthyology. Knowledge here is rife with nomenclature and ever-evolving revisions to our understanding of the origins and relationships of fishes and other vertebrates. Grappling with the fluid nature of science is a skill itself, but the challenge it poses can incentivize shortcuts. Like many instructors, we sometimes see students leaning heavily on AI, and taking these shortcuts can lead to missing key lessons, like how to think through biological problems in an evolutionary framework. Artificial intelligence can act as a double-edged sword for learners; while it has massive potential to serve as an idea sounding board or virtual tutor, misuse can lead to “cognitive atrophy” (Hassen, 2025; Melisa et al., 2025). That is, when students outsource too much thinking to AI, they risk missing out on developing critical skills for a career-ready biologist. Therefore, we developed a way to monitor student AI use in our course while providing tangible lessons on its limitations. To do this, we angle with artificial baits that AI happily dangles for the unwary. Currently, most open-source, large-language AI models suffer pitfalls that make for a not-so-great scientist (but a decent lure maker). Artificial intelligence models do not weigh evidence like scientists do. As language scraping tools, they often overweigh popular opinion regardless of whether it aligns with the state of the science. They also answer with unmerited confidence, often hesitating to admit uncertainty or mistakes without substantial user pushback. Further troubling, instead of saying “I don’t know,” AI models will provide answers even when they have little to go on. This can result in “AI hallucination” where information is invented to produce a seemingly plausible answer (Hassen, 2025; Ji et al., 2023). Hallucination results can be particularly hard to identify without foundational skills (Hassen, 2025). Cumulatively, these attributes can make for tempting bait for unwary users looking for quick answers. With some trial, we found it simple to design “AI bait” questions for assignments that common AI models answer consistently wrong, following these “AI bait” question guidelines: Find your spot: Focus on course content involving revision of past knowledge or a commonly held misconception. Artificial intelligence models show inherent bias based on the data used (supplied by the programmer or sourced by the model itself) to train the model (Igbinovia & Mensah Danquah, 2025; Navigli et al., 2023), and we noticed that, in biology, this often manifests as a tendency toward the larger historical or popular record over emerging or nuanced theory. It is important to give your students a fighting chance, so clearly present newer information and caution students on validity of older material. Pick your bait: Frame a question with a straightforward response. One- or two-word answers are better than longer responses because they are associated with more consistent AI responses. Fill-in-the blank questions are particularly effective “AI bait.” Use a sharp hook: Choose questions that AI answers incorrectly and consistently with a specific term (i.e., something a student is unlikely to guess). Ichthyology has plenty of material (e.g., theory names, technical jargon, or revised taxonomy). It is most diagnostic if the AI response term has not been used in lecture. Fish your bait realistically: Place a few questions with plausible, but clearly incorrect, AI responses into assessments. Finding “AI bait” without being nit-picky or overt is part of the art but often made entertaining by the outlandish answers AI can offer. Early in the semester, we warned students that AI struggles with high-level biology, and, without building the fundamentals, it is difficult to filter the output to use it effectively. This was accompanied by presenting some nightmarish AI-generated hallucinations of vertebrate anatomy, included here for your amusement (Figure 1). We then employed “AI bait” questions on take-home work to gain insights into the scope and impact of AI use. Our prototype was the following: “The cartilaginous skeleton of chondrichthyans and lack of vertebrae in myxiniforms are both examples of BLANK.” Classroom discussions highlighted these skeletal conditions as “derived losses” based on current phylogenetic understanding (Marlétaz et al., 2024; M. Zhu, 2014). Common AI models answered “pleisomorphies,” a term not introduced in lecture and reflecting older theory that these trait conditions were ancestral. Of roughly 80 students, 13 of them took the bait. Sharing the AI response screenshot in lecture was met with shock and amusement and provided a useful lesson on evolving theory and the limits of AI. Four students became second-time victims of AI misinformation, and only one for a third. Vertebrate biology as seen through the lens of AI. Notice the variety of examples of “Biceps” seen throughout the vertebrate lineage. Prompt: “Generate an image of vertebrate biology” (Model: Google Gemini Version 2.5, generated: September 2025). This is an unmanipulated AI generated image and we do not claim that any component is scientifically accurate. Fish phylogeny questions are easy pickings for “AI bait” given recent revisions (Figure 2). Due to molecular phylogenetics (Betancur-R et al., 2017; Near & Thacker, 2024), gone are the days of guessing “Perciformes” on fish taxonomy exams for a 50% chance of receiving points. The reluctance of AI to admit mistakes made for an interesting AI interaction on that topic for the recently reclassified Nile Perch Lates niloticus (Figure 3; Girard et al., 2020). We also encountered examples of AI opting to cobble plausible terms rather than say “I don’t know,” such as when it rebranded the “Craniate Hypothesis” into its own “Vertebrate Hypothesis,” picking up the question’s context of vertebrate origins, but missing the focus on evolutionary relationships of jawless fishes and the distinctions between the Cyclostome and Craniate hypotheses (Janvier, 2008; Marlétaz et al., 2024). Fish phylogeny imagined by AI. Note the basal seahorse, frequent rise of sharks throughout fishes evolutionary history, and a compelling pattern of diversifying clades collapsing back into single taxa. A compass and scale help orient the reader. (Prompt: “generate an artistic cladogram of fish evolution,” Model: Google Gemini Version 2.5, generated September 2025). This is an unmanipulated AI generated image and we do not claim that any component is scientifically accurate. An abbreviated conversation with AI testing an “AI bait” question, demonstrating the model’s tendency to default to older material and hesitate to acknowledge mistakes. Conversation generated in ChatGPT Version 5.5 (April 2026). Background image generated in Google Gemini Version 3 Flash (June 2026). Prompt: “Generate an image of a smug but quietly uncertain AI thinking in cyberspace.” Of course, our system is not fool proof. Like AI, students can come across out-of-date information, and AI responses are likely to continue to improve overall. Although it is feasible that students occasionally will organically respond in parallel to AI, the odds of this being widespread or encountering repeated false positives is unlikely if “AI bait” questions are well curated (i.e., “Use a sharp hook”). Students may bypass “AI bait” if they push back with the AI model, such as asking for clarification or raising follow-up questions, but this is progress towards more responsible AI use, and engaging with AI in this way can even sharpen students’ critical thinking skills (Hassen, 2025; Melisa et al., 2025). In our experience, raising discussion around AI centered on these examples was often met with interest in understanding its use and limitations. There is a much bigger discussion to be had on how students and instructors can best use AI toward positive ends, and several systematic efforts to do so exist (Yin et al., 2025; H. Zhu et al., 2025). Regardless, solid scientific foundation is undoubtedly critical for vetting AI-generated content and putting it to the best use. Using AI ourselves to trial questions also helped us understand the contemporary landscape of education our students operate in, and, as more thinking is outsourced to AI, helped us to guide them to think like a scientist: use your resources, be critical and skeptical, and know when to say “I don’t know.” No new data were generated or analysed in support of this research. National Science Foundation Award #2222339 and University of Maine Agriculture and Forest Experiment Station (U.S. Department of Agriculture Hatch Multistate NC1189) partly supported this work. In-kind support was provided by the Maine Cooperative Fish and Wildlife Research Unit. We would like to thank Shawn Snyder and Lee Hecker for their contributions to this work. Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. government.