Artificial Intelligence (AI) Detection Tools

Artificial Intelligence (AI) Detection Tools

Generative AI and AI Detection

The use of generative artificial intelligence (AI) tools has proliferated quickly. ChatGPT, for example, is an application that produces written text at a desired length, format, and style, depending on the user’s prompts. Copilot is an AI generator provided by Minnesota State to Office 365 users. These applications and others like them perpetually “learn” how to respond to a prompt and expand their knowledgebases through millions of pieces of information being fed to them.

Students at all levels have increasingly been using tools like ChatGPT to varying degrees. With the power and easy access these tools offer, misuse may occur. There is growing concern among instructors and others over how students are using these tools. In response to this concern, AI detection tools have also emerged. One such tool is Turnitin, which has an extension used to flag submitted student work for AI. There are several other popular AI detection tools that perform a similar function. The information and guidance in this document address all AI detection tools and applications in a general sense.

Background and Saint Paul College Usage of AI Detection Tools

Within the past year, the Minnesota State System Office activated a pilot of the AI detection feature in Turnitin. Some instructors adopted this tool as part of their grading process for written assignments. At the end of the Fall 2023 semester, the System Office ended the Turnitin AI detection pilot.

Although many studies, as well as current data results from AI detection companies themselves, show the limited usefulness of AI detection tools such as Turnitin, Saint Paul College has reinstated the Turnitin AI detector tool at SPC for the remainder of the Turnitin contract term, which ends in June 2024. The College Academic Technology Team (CATT), which will be formed this semester and will include faculty representation, and leadership at the College will continue to determine the usefulness and equity of this tool for the long term.

Caution

Use caution when employing AI detection tools. Concerns have emerged from various academic institutions about the reliability of Turnitin and other AI detectors, as well as ethical and equity implications. Reports from these institutions indicate that the tool yields a sizeable percentage of “false positives” in flagging AI or plagiarized work—it often reads original, valid student writing as plagiarized. In particular, students whose first language is not English are more likely to see their work erroneously flagged as plagiarized. See Vanderbilt University’s guidance on the Turnitin AI detector for more information about this problem. Turnitin has not been forthcoming with details around how their AI detector works. This lack of transparency, as well as the substantial number of false positives, creates problems when relying on this tool for grading. For these reasons, many institutions across the country have opted to discontinue use of the Turnitin AI detector and other similar tools. For these reasons, this tool should never be used as a singular point of reference to determine the validity of student work.

How to Use it

Remember that an AI detection tool is just one data point that can be used in assessing student work. It is important not to rely on this tool alone to raise a plagiarism accusation against a student. If you use an AI detector, continue to use standard evaluation methods, comparison of current writing against the student’s previous writing, and your years of experience as an instructor when assessing student work. Trust is the foundation of good instruction.

If you suspect a student’s work has been generated with AI, here are some steps you could take:
  1. Talk with the student, in a non-accusatory manner, and ask them about their processes and sources they've used for the assignment.
  2. If you feel certain the work is not the student’s, and after you have spoken with them, ask them to resubmit the assignment for a lower grade if the situation allows for some flexibility.
  3. Submit an academic integrity violation here: Academic Integrity Violation Form
Saint Paul College is committed to becoming an anti-racist, trauma-informed institution and believes that ensuring equity, inclusion, and a sense of belonging for all students is vital to the work we do. Although we want to ensure students are creating original work at all times, communication and clear expectations help provide a sense of belonging for our student population.

Guidance

Become proactive rather than reactive.
  1. Familiarize yourself with available AI tools (i.e., Microsoft Copilot) and with conversations and implications around the still-evolving AI academic tools.
  2. Write a clear syllabus policy about what is and isn’t okay when completing work in your courses. (Some ideas to get you started from Minnesota State)
  3. Long term, rethink current assessment techniques and teaching strategies to educate students about good AI use and when it is unacceptable. Don’t change everything in your course all at once; give yourself some time to consider ways you can work within this paradigm shift in your classes and how you can mitigate the ever-changing AI environment in which our students live. We encourage you to reach out to James Smrikarov, Instructional Designer, or Eric Kline, Academic Technologist, in AEI for support as you navigate this cultural and academic shift.

Further Help

You aren’t in this alone. Across the country and the world, instructors are negotiating the widespread use of AI. For more information and ideas, you can start here:
  1. Conversations and articles on sites like The Chronicle of Higher Education (free access from the SPC Library) and Inside Higher Ed (free account required)
  2. AEI’s Teaching in the Age of Generative AI page
  3. Book a meeting with the AEI instructional designer or academic technologist to discuss how to use AI or create new, more AI-proof assessments and assignments
  4. Join the AI Community of Practice at SPC
  5. Vanderbilt University’s guidance on the Turnitin AI detector. Although this site details why Vanderbilt University has chosen not to use AI detectors, we find the links and information valuable for instructors.
Updated February 1, 2024

    • Related Articles

    • How do I use TurnItIn for AI detection in student writing assignments?

      What is TurnItIn? TurnItIn is an application used to gauge the authenticity of written text, and to flag plagiarism in student writing. It compares the text in question against a vast database of existing writing and delivers a % score (the higher ...
    • Microsoft Word Drawing Tools

      Drawing Tools allows you to add drawings to Word, Excel documents & PowerPoint presentations. By default, the Draw tab is not enabled in Word. There are two ways to get into the Ribbon settings. 1. (First option). Right click anywhere on the Ribbon ...
    • Generative AI Use Case: Creating and Fixing Students' Computer Coding

      Applicable Course/Department/Division Computer Science/STEM Specific AI Technology ChatGPT/Microsoft Copilot Background In 2023, Harvard professor David Malan and colleagues introduced a ChatGPT "bot" in Harvard's flagship Intro to Computer Science ...
    • The Prompt is Key: How to Ensure the Most Useful Generative AI Output

      Background A professor at the University of Iowa is teaching students how to get better results from an AI generator through an assignment centered around the prompts entered in to the generator. Instead of trying to prohibit her students from using ...
    • How do Tool Vouchers work?

      If you would like to use Federal Financial aid in the purchase of tools for one of the following courses: Machine Tool Automotive Technician Visit the Tuition Office to obtain a tool voucher.  The tool voucher allows the tool vendor to bill Saint ...