International Educational Review

Student Attitudes towards Chat GPT: A Technology Acceptance Model Survey

Halit Yilmaz 1,
Samat Maxutov 1,
Azatzhan Baitekov 1 * ,
Nuri Balta 1
1 Suleyman Demirel University* Corresponding Author
Download PDF
International Educational Review, Volume 1, Issue 1, April 2023, pp. 57-83
OPEN ACCESS VIEWS: 11681 DOWNLOADS: 12999 Publication date: 15 Apr 2023
ABSTRACT
This study aimed to develop and validate an instrument to explore university students' perception of Chat GPT, while also investigating potential variations across gender, grade level, major, and prior experience with using Chat GPT. Employing a quantitative research approach, the study involved 239 students enrolled in the Science and Mathematics Education Program at a private university in Almaty, Kazakhstan. The results indicated an overall positive perception of Chat GPT among the participants. Notably, the only significant disparity in perception between male and female students was observed in the dimension of "Perceived ease of use." Moreover, no significant differences were found across any survey dimensions when comparing students from different grade levels (first to fourth grade). However, statistically significant differences emerged in the dimension of "Perceived social influence" between Mathematics majors and Chemistry-Biology majors, as well as between Chemistry-Biology majors and Physics-Informatics majors. Additionally, except for the dimension of "Perceived social influence," statistically significant differences were observed among groups based on their prior experience using artificial intelligence (AI) or chatbots. These findings provide valuable insights into university students' perceptions of Chat GPT and highlight the influence of factors such as gender, major, and prior experience on their perceptions. The implications of these findings can inform the design and implementation of educational technologies involving AI-based chat systems in higher education settings.
KEYWORDS
Student attitudes, Chat GPT, Technology acceptance model, Educational technology, Artificial intelligence, Natural language processing, Student perceptions, User acceptance, User experience, Learning outcomes, Student engagement, Technology integration, Higher education, Online learning, Pedagogy
CITATION (APA)
Yilmaz, H., Maxutov, S., Baitekov, A., & Balta, N. (2023). Student Attitudes towards Chat GPT: A Technology Acceptance Model Survey. International Educational Review, 1(1), 57-83. https://doi.org/10.58693/ier.114
REFERENCES
  1. Adams, D., Nelson, R. R., & Todd, P. M. (1992). Perceived Usefulness, Ease of Use, and Usage of Information Technology: A Replication. Management Information Systems Quarterly, 16(2), 227. https://doi.org/10.2307/249577
  2. Agarwal, R., Sambamurthy, V., & Stair, R. (2000). Research Report: The Evolving Relationship Between General and Specific Computer Self-Efficacy—An Empirical Assessment. Information Systems Research, 11(4), 418–430. https://doi.org/10.1287/isre.11.4.418.11876
  3. Araujo, T., Helberger, N., Kruikemeier, S., & De Vreese, C. H. (2020). In AI we trust? Perceptions about automated decision-making by artificial intelligence. AI & Society, 35(3), 611–623. https://doi.org/10.1007/s00146-019-00931-w
  4. Atwell, E. (1999). The language machine: the impact of speech and language technologies on English language teaching. British Council.
  5. Awang, Z. 2015. Validating the measurement model: CFA. A Handbook on SEM. 2nd edition ed: Kuala Lumpur: Universiti Sultan Zainal Abidin: 54-73.
  6. Boateng G., O, Neilands T., B, Frongillo E., A, Melgar-Quiñonez H., R & Young S., L. (2018). Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research. In Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. Routledge.
  7. Cohen, L., Manion, L., & Morrison, K. (2017). Research methods in education. Routledge.
  8. Creswell, J. W. (2002). Educational research: Planning, conducting, and evaluating quantitative (pp. 146-166). Upper Saddle River, NJ: Prentice-Hall.
  9. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science, 35(8), 982–1003. https://doi.org/10.1287/mnsc.35.8.982
  10. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. https://doi.org/10.2307/249008
  11. Demir, K., & Guraksin, G. E. (2022). Determining middle school students’ perceptions of the concept of artificial intelligence: A metaphor analysis. Participatory Educational Research, 9(2), 297–312. https://doi.org/10.17275/per.22.41.9.2
  12. Dong, Y., Xu, C., Chai, C. S., & Zhai, X. (2020). Exploring the structural relationship among teachers' technostress, technological pedagogical content knowledge (TPACK), computer self-efficacy, and school support. The Asia-Pacific Education Researcher, 29(2), 147-157.
  13. Fast, E., & Horvitz, E. (2016). Long-Term Trends in the Public Perception of Artificial Intelligence. Proceedings of the . . . AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10635
  14. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2011). How to design and evaluate research in education. New York: McGraw-Hill Humanities/Social Sciences/Languages.
  15. Firat, M. (2023). What ChatGPT means for universities: Perceptions of scholars and students. Journal of Applied Learning and Teaching, 6(1), 57-63. DOI: https://doi.org/10.37074/jalt.2023.6.1.22
  16. Gallacher, A., Thompson, A., Howarth, M., Taalas, P., Jalkanen, J., Bradley, L., & Thouësny, S. (2018). “My robot is an idiot!”–Students’ perceptions of AI in the L2 classroom. Future-proof CALL: language learning as exploration and encounters–short papers from EUROCALL, 70-76.
  17. Gefen, D., Straub, D., & Boudreau, M. C. (2000). Structural equation modeling and regression: Guidelines for research practice. Communications of the association for information systems, 4(1), 7. https://doi.org/10.17705/1cais.00407
  18. Granić, A., & Marangunić, N. (2019). Technology acceptance model in educational context: A systematic literature review. British Journal of Educational Technology, 50(5), 2572–2593. https://doi.org/10.1111/bjet.12864
  19. Hair Jr, J. F., Howard, M. C., & Nitzl, C. (2020). Assessing measurement model quality in PLS-SEM using confirmatory composite analysis. Journal of Business Research, 109, 101-110.
  20. Harrington, D. 2009. Confirmatory factor analysis. Oxford university press.
  21. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1), 1-55. DOI: https://doi.org/10.1080/10705519909540118
  22. Iqbal, N., Ahmed, H., & Azhar, K. A. (2022). Examining the acceptance of chatbots in education: A study based on the technology acceptance model. Education and Information Technologies, 27(5), 4855-4874.
  23. Iqbal, N., Ahmed, H., & Azhar, K. A. (2022). Exploring teachers’ attitudes towards using chatgpt. Global Journal for Management and Administrative Sciences, 3(4), 97–111. https://doi.org/10.46568/gjmas.v3i4.163
  24. Jeffrey, T. (2020). Understanding college student perceptions of artificial intelligence. Systemics, Cybernetics and Informatics, 18(2), 8-13.
  25. Lederer, A. L., Maupin, D. J., Sena, M. P., & Zhuang, Y. (2000). The technology acceptance model and the World Wide Web. Decision Support Systems, 29(3), 269–282. https://doi.org/10.1016/s0167-9236(00)00076-2
  26. Lee, M., H., Johanson, R. E., & Tsai, C., C. (2008). Exploring Taiwanese high school students' conceptions of and approaches to learning science through a structural equation modeling analysis, Science Education 92(2), 191–220. doi: https://doi.org/10.1002/sce.20245
  27. Liu, C., Liao, M., Chang, C., & Lin, H. M. (2022). An analysis of children’ interaction with an AI chatbot and its impact on their interest in reading. Computers & Education, 189, 104576. https://doi.org/10.1016/j.compedu.2022.104576
  28. Lozano, I. A., Molina, J. M., & Gijón, C. (2021). Perception of Artificial Intelligence in Spain. Telematics and Informatics, 63, 101672. https://doi.org/10.1016/j.tele.2021.101672
  29. Mathieson, K. (1991). Predicting User Intentions: Comparing the Technology Acceptance Model with the Theory of Planned Behavior. Information Systems Research, 2(3), 173–191. https://doi.org/10.1287/isre.2.3.173
  30. Parker, L. (2007). Gender differences in computer attitudes, ability, and use in the preschool environment. Journal of Research in Childhood Education, 22(1), 39-51.
  31. Parker, L. (2007). Technology in support of young English learners in and out of school. In L. Parker (Ed.), Technology-mediated learning environments for young English learners (pp. 213-250). Routledge.
  32. Rattray, J., & Jones, M. C. (2007). Essential elements of questionnaire design and development. Journal of clinical nursing 16(2): 234-243. DOI: https://doi.org/10.1111/j.1365-2702.2006.01573.x
  33. Swisher, L. L., Beckstead, J. W., & Bebeau, M. J. (2004). Factor analysis as a tool for survey analysis using a professional role orientation inventory as an example. In Joreskog KG, Sorbom D. LISREL Version 8.54: User’s Reference Guide [electronic manual]. Chicago, Ill: Scientific Software International Inc; 2003.
  34. Taherdoost, H. (2021). Data Collection Methods and Tools for Research; A Step-by-Step Guide to Choose Data Collection Technique for Academic and Business Research Projects. International Journal of Academic Research in Management (IJARM), 10(1), 10-38.
  35. The jamovi project (2022). jamovi. (Version 2.3) [Computer Software]. Retrieved from https://www.jamovi.org.
  36. Taylor, S., & Todd, P. M. (1995). Assessing IT Usage: The Role of Prior Experience. MIS Quarterly, 19(4), 561. https://doi.org/10.2307/249633
  37. Venkatesh, V., & Davis, F. D. (1996). A Model of the Antecedents of Perceived Ease of Use: Development and Test. Decision Sciences, 27(3), 451–481. https://doi.org/10.1111/j.1540-5915.1996.tb00860.x
  38. Venkatesh, V., & Davis, F. D. (2000). A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science, 46(2), 186–204. https://doi.org/10.1287/mnsc.46.2.186.11926
  39. Xu, X., & Lewis, J. E. (2011). Refinement of a chemistry attitude measure for college students, Journal of Chemical Education, 88(5): 561-568. DOI: https://doi.org/10.1021/ed900071q
  40. Weiss, B.A. (2011). Reliability and validity calculator for latent variables [Computer software]. Available from https://blogs.gwu.edu/weissba/teaching/calculators/reliability-validity-for-latent-variables-calculator/.
  41. Yeh, S. C., Wu, A., Yu, H., Wu, H., Kuo, Y., & Chen, P. (2021). Public Perception of Artificial Intelligence and Its Connections to the Sustainable Development Goals. Sustainability, 13(16), 9165. https://doi.org/10.3390/su13169165
  42. Zheng, C., Fu, L., & He, P. (2014). Development of an instrument for assessing the effectiveness of chemistry classroom teaching.” Journal of Science Education and Technology 23(2): 267-279. DOI: https://doi.org/10.1007/s10956-013-9459-3
LICENSE
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.