David Gewirtz/ZDNET has been exploring the capabilities of AI chatbots in writing code since the emergence of ChatGPT and generative artificial intelligence (AI) in 2022. Initially viewed as a novelty, the technology has proven to be a valuable productivity tool and programming partner. Gewirtz has conducted in-depth testing on ten large language models (LLMs) to evaluate their performance in real-world scenarios. He has compiled a set of four tests to assess the AI’s coding abilities and plans to continue testing and updating the results over time. The tests include tasks like writing a WordPress plugin and rewriting code segments to enhance functionality. By using his day-to-day programming work as a basis for the tests, Gewirtz aims to determine how well AI chatbots can assist developers in practical coding tasks. The results of the tests are documented in a living document for others to replicate and explore.
