Use of AI in exams with limited credits

Team name: TokenAI

Use of AI tools : NotebookLM for the Mockup version of the platform

External feedback & contributions :

Hugues Benoit-Cattin (our mentor, numerical director at INSA Lyon) 

Dr. Alain Goudey - keynotes
The illustration with the activation curve was really enlightening. The fact that AI can be used as a cognitive amplifier only if we use it strategically and wisely. Good habits to keep when using AI, especially in a constrained situation like ours :

• write your intent before chatting with the AI

• keep the most important part human like formulating a research question

• struggle against the decline in intellectual independence

• protection your data as it is something very precious

Alexandra Pregent, PhD Researcher at CEA - keynotes
The ability to discern the tensions between different requirements and learn to make the right trade-offs, such as ensuring that students are not discriminated against when it comes to grading and privacy.

Initial contribution: Use of AI in exams with limited credits (initial version)(Opens in new tab)

Final contribution:

AI is already used by 79% of the young population (15-25 years old) in their daily life. Furthermore, a survey revealed that 92% of the students are using AI regularly.

Therefore, we suggest allowing artificial intelligence during exams under specific conditions, strictly contained within a restricted environment. The tool will function similarly to an open-book document or a calculator in exam mode. To ensure equity and fairness, we propose the following approach.

First, the AI will not be freely chosen by the student. Instead, the institution will deploy its own secure, centralized AI platform. While this system will rely on existing language models, it will act as an aggregator, enabling professors to select the most appropriate model for their specific subject beforehand. This setup not only allows for targeted assistance but also guarantees data privacy. By hosting and managing the portal internally, the school avoids dependency on external private tech companies and maintains complete digital sovereignty over its examination tools. Furthermore, this platform will strictly operate as a text-based chatbot, with no option to upload files or images. This restriction forces students to formulate their own prompts, requiring active thinking before interacting with the LLM. To prevent any cheating, the chatbot will only be accessible through a secure browser—a solution already widely used for online assessments—which blocks access to unauthorized materials or unrestricted internet searches.

Secondly, interactions with the AI will be limited through a token system, much like restricting open-book exams to a single cheat sheet. Professors will determine a specific token budget in advance, which can vary depending on the test. For the sake of transparency, candidates must be informed of this allowance beforehand. Secondly, interactions with the AI will be limited through a token system, much like restricting open-book exams to a single cheat sheet. Much like the countdown timer used during online Moodle assessments, the interface will feature a live tracker showing students exactly how many tokens they have left. This limitation ensures that students cannot simply delegate all their reasoning to the algorithm; they must strategically decide when assistance is truly necessary. Consequently, those lacking a fundamental understanding of the course will still struggle to achieve good grades. This system also gives teachers the flexibility to adjust their evaluation methods. One professor might grant a large budget to test prompt engineering and critical analysis (the Socratic approach), while another might provide fewer tokens for basic troubleshooting. Practically, students could spend tokens to get a hint when completely stuck (similar to the existing practice in some exams where students can ask an invigilator for a hint in exchange for a grade penalty). They could also use the AI to retrieve forgotten knowledge for a straightforward basic knowledge question, keeping in mind that spending their budget here could penalize them for subsequent, more complex problems. Finally, they could use the tool to verify their reasoning: the AI can provide an alternative point of view, which the student must then actively challenge using their critical thinking skills.

To complement this, we are incorporating a reward system for minimizing AI usage. While powerful, artificial intelligence has a significant environmental footprint, and our goal is to educate students about the real-world impact of their digital requests. The interface will feature a real-time counter displaying the carbon cost of their prompts (computed with existing tools like Code Carbon or Carbon Tracker). Any unused tokens will then be converted into a grade bonus, with a sliding scale that offers the highest reward to those who complete the exam without relying on the chatbot at all. Beyond environmental awareness, this incentivizes users to carefully consider the relevance of a prompt before submitting it. Along with their final grades, candidates will receive an environmental footprint report for their session, reinforcing the idea that AI is a finite resource requiring mindful usage.

Finally, after the assessment, all student prompts will be made accessible to the teacher for two main reasons. The first is to take into account the probabilistic side of AI. We know that two students who input the exact same prompt might receive different results, and their token consumption could also vary. To guarantee fairness between candidates, the teacher can compare the prompts and the AI's responses to apply a small correction factor to the final grade. This is similar to adjusting a score when a student points out a grading error. The second reason is to analyse how students actually formulate their requests. Gathering this data enables educators to design future lessons on effective prompt engineering and global AI literacy. It also provides a concrete way for teachers to adjust future exams, ensuring they evaluate genuine student capability rather than merely grading an LLM's output.

Prompt engineering insights

Never start a prompt without clarifying those main points :

• Context

• Aim of the request

• Constraints

• Format

• Examples

Since some of the main risks of using intensively AI are a diminution of memory capacity (in a recent MIT study, 83% of users found it difficult to recall their own essay when using AI)  and the dependency. We would like to encourage students to follow those recommendations when using AI:

• Take control of the AI and do not let it take control of you

• Know when to say "stop" if the AI takes you in a different direction to the one you intended

• The AI must remain a methodological aid, not a means of producing solutions

• Maintain an epistemic mindset (keep your critical ability)

• Learn to refine your reasoning step by step (fine-tuning)

Reflection on the process (important)

Explain how your contribution evolved during Phase 2:

• What feedback (comments, discussions, expert inputs, keynotes) influenced your work?
  Saleh bichara Abakar - concerning the inequalities between students
  Haziel Álvarez - suggest a bonus scale and not only rewarding only those who are not using AI
  Enoch Kolade - specify the exam conditions
  Dia Khurana - concerning the fact LLMs are probabilistic and therefore the outputs will be different for a given question. Moreover, LLMs can hallucinate so it might mislead the student.

• What changes did you make as a result?
  We have created a mock-up version of our project to illustrate our idea. We took some time to reflect on the questions by reading some materials and listening to the keynotes.

• How did this process strengthen your final proposal?

We were able to clarify and provide concrete details on our proposal by combining reflections using questions and the creation of the mockup. Besides, the process helped us to put things into perspective. Thanks to the diverse questions, we have deepen our thinking by reading materials such as Mr. Valluy’s(External link) article on precautions to be taken regarding the use of AI in universities. We got interesting insights regarding prompt engineering and skills that must be reinforced in the face of AI, e.g. keep a critical mind, be able to formulate research questions.  

Proof of concept

In order to better visualise our project, we encourage you to have a look at our mockup version : https://mathis-dchr.github.io/TokenAI/

Video link