How do we balance evidence of process and product when attempting to accurately assess a student’s evidence of understanding?

Frontier (2023) argues that new AI tools such as generative AI present transformational opportunities—and challenges—for teaching and learning. Therefore we need to take a transformational approach to utilizing generative AI and ask transformative questions (Frontier, 2023). Frontier (2023) says these must be big, open-ended questions, such as “What do we accept as evidence of an individual’s understanding when accomplished independently? When augmented by technology? How do we balance evidence of process and product when attempting to accurately assess a student’s evidence of understanding?” (Frontier, 2023).

Based on my experience and readings from the literature, I believe that in the era of generative AI our assessments need to put more emphasis on understanding and shift to more comprehension-based assessments. So in the era of generative AI demonstration of enduring understanding is especially important because it is challenging for teachers to know how much of the product is evidence of the student’s understanding and how much was produced by AI.

Therefore, when attempting to accurately assess a student’s evidence of understanding in the present time of generative AI we must weigh the evidence of process more heavily than products since evidence we obtain from the process should be more valid and reliable than assessing a finished product.

Generative AI cannot explain how it arrives at an answer, that is, AI cannot be asked to “show your work” whereas students are routinely asked to “show your work”. It is understood that providing an answer – a product – is only part of learning and being able to explain how one arrives at the answer – the process – reflects a deeper understanding. In the era of generative AI it becomes more valuable to refocus assessment to evidence of process rather than products. Asking students to be self-reflexive and show the steps they took to arrive at the answer shifts the balance from product to evidence of process. What this means is that the product is of lesser value within assessment rubrics than the evidence of process of how the student arrived at the product. Such assessment strategies place more importance on understanding and also more emphasis on students’ learning and their ability to explain why and how they arrived at their answer or product.

Transactional use of technology in instruction involves the use of technology for the exchange of ideas and information between teachers and students where the focus is knowledge transmission and acquisition, which has traditionally been the primary goal of learning (Kilbane & Milman, 2023). For example, I currently record my lectures in video format and include captioning to support English language learners, hearing impaired students, or those who were absent, in addition to those who may benefit from re-watching or slowing down my presentation. This is an example of transactional use of technology in teaching and learning. On the other hand, in transformational use of technology in teaching and learning, students use technology to achieve instructional outcomes that are only possible with certain uses of technology. Toward this end, both the tool and how it is used is important. A teacher may use the same technology tool in multiple ways, but one way might result in transformational learning and the other may not (Kilbane & Milman, 2023). For example, two groups of students may work on the same project to promote composting in the same community with the objective of using a photo-and-video sharing app to educate the community. One group may use the same technology to create a simple poster with information about how and where to compost. The other group, on the other hand, may use the same technology to create a video that includes polls to learn about community members composting habits and embedded links with relevant information such as composting locations in the community according to their composting habits plus information about how and where to compost. The result is that the video group also engages with the community and as a result are very much more successful.

As teachers we need to develop my knowledge of these new AI technologies to create new pedagogical possibilities for our students such as personalized learning experiences, provide opportunities for more immersive and interactive learning, enhance students’ creativity and ability to build products that have never been possible before, and help students develop skills like corroborating and thinking critically about information.

At the same time we need the ability to know when students are not using the technology responsibly and ethically such as for cheating on their assignments. As teachers we need to be proficient in AI so that we can be the driver of AI in our classrooms instead of being driven by it.

One way we can begin to refocus assessment to evidence of process rather than products, and focus on demonstration of enduring understanding, is by designing lessons and units around essential questions (Wiggins & McTighe, 2005). Problem-based learning, project-based learning and inquiry activities support broader student engagement and involve students more actively in the learning process which helps to shift the balance from product to evidence of process. In our planning and assessment we should be focusing more on the social and analytical aspects of learning – on fostering moments of collective meaning making and inquiry to help students expose gaps in their thinking and build deeper understandings. Our classrooms must be places where students gain the critical thinking skills they need to use technology constructively and strategically, rather than as passive consumers. In the era of generative AI and students with access to a powerful smartphone in their hands, no longer should we expect students to acquire a fixed body of knowledge presented by teachers and texts. Instead, teachers need to prepare students for a dynamic, unpredictable future and students need to develop an essential life-long skill – the capacity to independently direct their own learning, in school and beyond.

Schools and teachers need to shift their focus to:

prioritize self-directed learning as a fundamental goal of schooling;

explicitly teach and reinforce these skills;

provide students with regular opportunities to self-direct their own learning.

References

Frieder, Pinchetti, L., Griffiths, R.-R., Salvatori, T., Lukasiewicz, T., Petersen, P. C., Chevalier, A., & Berner, J. (2023). Mathematical Capabilities of ChatGPT. https://doi.org/10.48550/arxiv.2301.13867

Frontier, Tony (2023). Educational Leadership. Taking a Transformative Approach to AI. Available at https://ascd.org/el/articles/taking-a-transformative-approach-to-ai

Jacobs, H.H. & Fisher, M. (2023). Educational Leadership. Prompt Literacy: A Key for AI-Based Learning. Available at https://www.ascd.org/el/articles/prompt-literacy-a-key-for-ai-based-learning

Kilbane, C. & Milman, N. (2023). Educational Leadership. Differentiated Learning and Technology: A Powerful Combination. Available at https://www.ascd.org/el/articles/differentiated-learning-and-technology-a-powerful-combination

Impact Research. (March, 2023). Teachers and Students Embrace ChatGPT for Education. Walton Family Foundation. Retrieved from https://www.waltonfamilyfoundation.org/learning/teachers-and-students-embrace-chatgpt-for-education

McTighe, Jay and Tucker, Catlin (2023). Educational Leadership. Developing Self-Directed Learners by Design. Available at https://www.ascd.org/el/articles/developing-self-directed-learners-by-design

Nglia, F. (July, 2023). The UK’s top universities reached an agreement on how to deal with generative AI. Quartz. Retrieved from https://qz.com/russel-uk-universities-generative-ai-students-1850603771?mibextid=S66gvF

Ontario. Ministry of Education. (2010). Growing success: Assessment, evaluation, and reporting in Ontario schools – First edition, covering Grades 1 to 12. Toronto: Author. Available at https://www.edu.gov.on.ca/eng/policyfunding/growSuccess.pdf

Science Focus. BBC. (n.d.). ChatGPT: Everything you need to know about OpenAI’s GPT-4 tool: Author. Available at https://www.sciencefocus.com/future-technology/gpt-3/

Wiggins, G & McTighe, J. (2005). Understanding by design (2nd ed.). Alexandria, VA: Association for Supervision & Curriculum Development

Wiggers, Kyle and Stringer, Alyssa (2023). TechCrunch. ChatGPT: Everything you need to know about the AI-powered chatbot. Available at https://techcrunch.com/2023/07/10/chatgpt-everything-you-need-to-know-about-the-open-ai-powered-chatbot/

ZDNET Innovation. ZDNET. (n.d.). How to use ChatGPT Everything you need to know: Author. Available at https://www.zdnet.com/article/how-to-use-chatgpt/