From Code Author to Code Reviewer: A Systematic Review of Practitioner Experience with Generative AI in Software Engineering

Authors

DOI:

https://doi.org/10.63158/journalisi.v8i5.1826

Keywords:

Generative AI, Software Engineering, Developer Productivity, Code Quality, Human–AI Collaboration

Abstract

Practitioner use of generative AI in software engineering has progressed faster than the evidence assessing its effects. This review examined 58 empirical studies published from 2024 to early 2026 on GitHub Copilot, ChatGPT, Claude, and Gemini. Using PRISMA-guided selection from five digital libraries, the studies addressed productivity, role shifts, code quality, adoption behaviour, and evidence gaps. Findings were synthesised into the AI Integration Consequence Model, a conceptual four-layer framework linking adoption conditions, role shifts, quality exposure, and organisational consequences. Evidence indicates that developers increasingly move from code production to code review, as more than half of AI-generated code is discarded after evaluation. Generative AI improves efficiency for boilerplate code, documentation, and unit-test scaffolding, but provides limited benefits for architectural decisions and enterprise debugging. A meta-analysis of 47 studies reported a 45.6-point gap between self-reported and objectively measured productivity. AI-generated code also showed exploitable vulnerabilities at estimated rates of 40–50%, varying by task, tool version, programming language, and measurement method. Social influence strongly predicted adoption intent, while concerns about peer judgement encouraged undisclosed use. Experienced engineers generally received an AI Boost, whereas junior engineers faced an AI Drag. Structured mentorship reduced this gap in one 14-month cohort study. Key evidence gaps include cross-cultural adoption, longitudinal skill development, and unreplicated gender disparities.

Downloads

Download data is not yet available.

References

[1] Felder et al., "Adoption of generative artificial intelligence in the German software engineering industry: An empirical study," arXiv:2601.16700, 2026.

[2] D. Shao and F. Ishengoma, "Empirical analysis of generative AI tool adoption in software development," Inf. Softw. Technol., 2026, Art. no. 108036.

[3] C. Johnson et al., "The AI productivity paradox in software development: A meta-analysis of implementation outcomes and ROI patterns (2024–2025)," SSRN 5902163, 2025.

[4] S. Sarkar, "The effect of AI tools on modern software development for frontend engineering: An empirical analysis," SSRN 5442494, 2025.

[5] M. E. Souza and E. C. S. Murillo, "How generative artificial intelligence tools can improve the quality of software produced by software development teams," 2025.

[6] S. Ferino, R. Hoda, J. Grundy, and C. Treude, "Walking the tightrope of LLMs for software development: A practitioners' perspective," arXiv:2511.06428, 2025.

[7] W. Mendes, S. Souza, and C. de Souza, "'You're on a bicycle with a little motor': Benefits and challenges of using AI code assistants," in Proc. 2024 IEEE/ACM 17th Int. Conf. Cooperative and Human Aspects of Software Engineering, 2024.

[8] M. Coutinho et al., "The role of generative AI in software development productivity: A pilot case study," in Proc. 1st ACM Int. Conf. AI-Powered Software, 2024.

[9] I. Udoidiok, H. Reza, and J. Zhang, "Exploring AI integration in software development: Case studies and insights," in Proc. NAECON 2024 IEEE National Aerospace and Electronics Conf., 2024.

[10] R. Tomaz et al., "Impacts of generative AI on agile teams' productivity: A multi-case longitudinal study," arXiv:2602.13766, 2026.

[11] S. Määttä, "How do programmers evaluate AI-generated code?" in Proc. 2025 ACM/IEEE Int. Symp. Empirical Software Engineering and Measurement, 2025, pp. 522–527, doi: 10.1109/ESEM64174.2025.00057.

[12] A. Clark, D. Igbokwe, S. Ross, and M. F. Zibran, "A quantitative analysis of quality and consistency in AI-generated code," in Proc. ICoSSE 2024, Paris, France, 2024, pp. 37–41, doi: 10.1109/ICoSSE62619.2024.00014.

[13] B. Tabarsi et al., "LLMs' reshaping of people, processes, products, and society in software development," arXiv:2503.05012, 2025.

[14] A. Yalçıner, E. Gökalp, and A. Dikici, "Transforming software engineering processes through generative AI: A framework for integration and implementation," Computer, vol. 58, no. 7, pp. 53–65, Jul. 2025, doi: 10.1109/MC.2025.3539347.

[15] Z. S. Li et al., "AI tool use and adoption in software development by individuals and organisations: A grounded theory study," arXiv:2406.17325, 2024.

[16] P. Diebold, "From backlogs to bots: Generative AI's impact on agile role evolution," J. Softw. Evol. Process, vol. 37, no. 1, Art. no. e2740, 2025.

[17] M. O. Ahmad, H. Ghanbari, T. Gustavsson, and B. R. Upreti, "It all starts with structure: Investigating learning dynamics in large-scale agile software development," J. Syst. Softw., vol. 230, 2025.

[18] L. M. Restrepo-Tamayo, G. P. Gasca-Hurtado, and J. Valencia-Calvo, "Characterising social and human factors in software development team productivity," IEEE Access, vol. 12, pp. 59739–59755, 2024.

[19] B. Isong and T. Vhahangwele, "AI-assisted development tools and team dynamics in South African software engineering teams," J. Inf. Syst. Inform., vol. 8, no. 2, pp. 1–29, Jun. 2026, doi: 10.51519/journalisi.v8i2.1615.

[20] S. A. Besser, E. A. Jensen, and D. S. Katz, "How generative AI is shaping research software development and maintenance at a research-intensive university," Open Res. Eur., vol. 6, p. 56, 2026, doi: 10.12688/openreseurope.22009.1.

[21] Zhou et al., "Exploring the problems, their causes and solutions of AI pair programming," J. Syst. Softw., vol. 219, Art. no. 112204, 2025.

[22] R. Mo, D. Wang, W. Zhan et al., "Assessing and analyzing the correctness of GitHub Copilot's code suggestions," ACM Trans. Softw. Eng. Methodol., vol. 34, no. 7, Art. no. 194, 2025, doi: 10.1145/3715108.

[23] O. Dunay, D. Cheng, A. Tait et al., "Multi-line AI-assisted code authoring," in Proc. FSE 2024 Companion, 2024, doi: 10.1145/3663529.3663836.

[24] J. Liu et al., "Large language model-based agents for software engineering: A survey," arXiv:2409.02977, 2024.

[25] V. Tawosi, K. Ramani, S. Alamir, and X. Liu, "ALMAS: An autonomous LLM-based multi-agent software engineering framework," in Proc. ASEW 2025, 2025, doi: 10.1109/ASEW67777.2025.00059.

[26] K. Cinkusz and J. A. Chudziak, "Towards LLM-augmented multi-agent systems for agile software engineering," in Proc. ASE '24, 2024, doi: 10.1145/3691620.3695336.

[27] S. Zhang, Z. Xing, R. Guo et al., "Empowering agile-based generative software development through human-AI teamwork," ACM Trans. Softw. Eng. Methodol., vol. 34, no. 6, Art. no. 156, 2025, doi: 10.1145/3702987.

[28] N. V. Malladi and S. R. K, "Generative AI in agile software development: A comprehensive survey," in Proc. ICIDCA 2025, 2025, pp. 1199–1208.

[29] A. Welter et al., "An empirical study of knowledge transfer in AI pair programming," in Proc. ASE 2025, IEEE, 2025.

[30] S. Panyam and P. Gujar, "How AI agents are transforming software engineering and the future of product development," Computer, vol. 58, no. 5, pp. 71–77, 2025.

[31] T. Chen, "The impact of AI-pair programmers on code quality and developer satisfaction: Evidence from Timi Studio," in Proc. 2024 Int. Conf. Generative AI and Information Security, 2024.

[32] A. Kumar et al., "Intuition to evidence: Measuring AI's true impact on developer productivity," arXiv:2509.19708, 2025.

[33] E. Paradis et al., "How much does AI impact development speed? An enterprise-based randomised controlled trial," in Proc. ICSE-SEIP 2025, IEEE, 2025.

[34] Muñoz et al., "Comparative study of AI code generation tools: Quality assessment and performance analysis," LatIA, vol. 2, p. 104, 2024.

[35] F. Muratovic, D. Kearns-Manolatos, and A. Alibage, "Generative AI in software development: Challenges, opportunities, and new paradigms for quality assurance," Computer, vol. 58, no. 7, pp. 31–39, Jul. 2025.

[36] M. Hamza, D. Siemon, M. A. Akbar, and T. Rahman, "Human-AI collaboration in software engineering: Lessons learned from a hands-on workshop," in Proc. ACM/IEEE Workshop on Software-Intensive Business, 2024.

[37] T. G. Erdoğan, H. Altunel, and A. Tarhan, "A process model for AI-enabled software development: A synthesis from validation studies in white literature," J. Softw. Evol. Process, vol. 37, no. 1, 2025, doi: 10.1002/smr.2743.

[38] S. Inman et al., "Seamful AI for creative software engineering," IEEE Softw., vol. 42, no. 3, pp. 59–66, May–Jun. 2025.

[39] B. Kitchenham et al., "Systematic literature reviews in software engineering: A tertiary study," Inf. Softw. Technol., vol. 52, no. 8, pp. 792–805, 2010.

[40] M. J. Page et al., "The PRISMA 2020 statement: An updated guideline for reporting systematic reviews," BMJ, vol. 372, 2021.

[41] D. Tosi, "Studying the quality of source code generated by different AI generative engines," Future Internet, vol. 16, no. 6, p. 188, 2024.

[42] K. Jin et al., "Can ChatGPT support developers? An empirical evaluation of large language models for code generation," in Proc. MSR 2024, 2024.

[43] M. A. Kuhail, S. S. Mathew, A. Khalil, J. Berengueres, and S. J. H. Shah, "'Will I be replaced?' Assessing ChatGPT's effect on software development and programmer perceptions of AI tools," Sci. Comput. Program., vol. 235, p. 103111, 2024, doi: 10.1016/j.scico.2024.103111.

[44] S. Gudimetla, S. Gangula, R. Adinarayana, C. Challa, S. Vunnam, and S. Buhyavarapu, "From AI drag to AI boost: A preceptorship framework for developing software engineering expertise in the agentic era," in Proc. 2026 Int. Conf. Artificial Intelligence, Systems, and Emerging Technologies, Cairo, Egypt, 2026, pp. 1–12, doi: 10.1109/ICAISET66439.2026.11542057.

[45] J. Seo, E. Deldari, and H. M. Mentis, "Whose code is it? How AI autonomy reshapes ownership, responsibility, and disclosure in AI-assisted programming," in Proc. 31st Int. Conf. Intelligent User Interfaces, 2026.

[46] J. Zacharias, A. Popova, M. von Zahn et al., "Developers' dilemma: Opportunities and pitfalls of generative AI for software development," Bus. Inf. Syst. Eng., 2026, doi: 10.1007/s12599-026-00998-y.

[47] Hasan et al., "Human-AI collaboration in software design: A framework for efficient co-creation," Adv. Int. J. Multidiscip. Res., 2025.

[48] K. Li et al., "Utilising deep learning to optimise software development processes," arXiv:2404.13630, 2024.

[49] C. R. B. de Souza et al., "The fine balance between helping with your job and taking it," IEEE Softw., vol. 41, no. 6, pp. 111–118, 2024.

[50] V. Stray, A. Barbala, and V. T. Wivestad, "Human-AI collaboration in software development: A mixed-methods study of developers' use of GitHub Copilot and ChatGPT," in Proc. FSE Companion '25, 2025, doi: 10.1145/3696630.3730566.

[51] K. Z. Cui et al., "The effects of generative AI on high-skilled work: Evidence from three field experiments with software developers," Manage. Sci., 2026.

[52] L. Gambacorta, H. Qiu, S. Shan, and D. M. Rees, "Generative AI and labour productivity: A quasi-experiment on coding," J. Financ. Stab., vol. 84, p. 101543, 2026, doi: 10.1016/j.jfs.2026.101543.

[53] S. Afroz et al., "The fast and spurious: Developer productivity with GenAI," in Proc. ACM Int. Conf. Foundations of Software Engineering, 2026.

[54] S. C. Winckler et al., "AI-assisted collaboration: Exploring developer experience with GitHub Copilot and Windsurf," IEEE Softw., 2026.

[55] S. E. Farrag, "The productivity-reliability paradox: Specification-driven governance for AI-augmented software development," arXiv:2605.01160, 2026.

[56] D. Miranda et al., "A multimodal experimental dataset on agile software development team interactions," Data Brief, vol. 61, p. 111828, 2025.

[57] S. Pal, A. Nair, and Z. Zuo, "Collaborative dynamics in open source software development," J. Oper. Manag., vol. 70, no. 7, pp. 1076–1099, 2024.

[58] V. V. Jensen et al., "Managing expectations towards AI tools for software development: A multiple-case study," Inf. Syst. e-Bus. Manag., vol. 23, pp. 869–901, 2025.

[59] T. Xiao, Y. Fan, F. Calefato et al., "Self-admitted GenAI usage in open-source software," IEEE Trans. Softw. Eng., vol. 52, no. 6, pp. 1891–1910, Jun. 2026, doi: 10.1109/TSE.2026.3681886.

[60] A. V. Voutama, "Generative artificial intelligence in agile product management," Indones. J. Comput. Sci., vol. 14, no. 3, 2025.

[61] K. Cinkusz, J. A. Chudziak, and E. Niewiadomska-Szynkiewicz, "Cognitive agents powered by large language models for agile software project management," Electronics, vol. 14, no. 1, p. 87, 2025.

[62] N. Gupta, "The rise of AI copilots: Redefining human-machine collaboration in knowledge work," Int. J. Humanit. Inf. Technol., vol. 7, no. 3, 2025.

[63] P. Hoffmann, D. Zercher, and T. C. Schimmer, "How AI communication capabilities affect developer productivity in AI pair programming," 2025.

[64] K. Koyanagi et al., "Exploring the effect of multiple natural languages on code suggestion using GitHub Copilot," in Proc. MSR 2024, 2024.

[65] J. Melegati et al., "Exploring potential implications of intelligent tools for human aspects of software engineering," in Proc. CHASE 2024, 2024.

[66] A. Al-Ahmad, H. Kahtan, L. Tahat, and T. Tahat, "Enhancing software engineering with AI: Key insights from ChatGPT," in Proc. DASA 2024, 2024.

[67] M. L. Gillenson et al., "Benchmarking software test cases produced by generative AI against traditional methods: A design science approach," JDM, vol. 37, no. 1, pp. 1–32, 2026.

[68] A. Gheventer et al., "Generative AI solutions for software quality: Assessing industrial readiness," Softw. Qual. J., vol. 34, no. 2, p. 24, 2026.

[69] C. Zhong, H. Huang, J. Lyv, and S. Li, "Quality attributes, challenges, and solutions for designing GenAI-enabled systems," Inf. Softw. Technol., vol. 198, p. 108207, 2026, doi: 10.1016/j.infsof.2026.108207.

Downloads

Published

2026-10-24

Issue

Vol. 8 No. 5 (2026): October

Section

Articles

License

Copyright (c) 2026 Journal of Information Systems and Informatics

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Author's Declaration

  1. The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
  2. The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
  3. The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
  4. The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
  5. The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
  6. The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
  7. If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
  8. The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights

How to Cite

[1]
B. Isong, T. Vahangwele, and N. Dladlu, “From Code Author to Code Reviewer: A Systematic Review of Practitioner Experience with Generative AI in Software Engineering”, journalisi, vol. 8, no. 5, pp. 5717–5776, Oct. 2026, doi: 10.63158/journalisi.v8i5.1826.