Why ChatGPT Won’t Replace Coders Just Yet — written by → Clive Thompson

Clive makes some good points.

Me, I’ve been using Copilot for over a year now, and this is more or less how I use it: To help me quickly blast through boilerplate code so I can more quickly get to the tricky bits.

There’s a more subtle problem with ChatGPT’s code generation, which is that it suffers from ChatGPT’s general “bullshit” problem.

After giving some examples of trickier code that ChatGPT was asked to write…

With each case, ChatGPT produced something that kind of worked — but not quite. The key thing is that it didn’t give up; it didn’t say “no, I can’t really do this.” It just confidently spouted code and said, sure, I got this. But it didn’t.

…Now, the thing about bullshitting with code is that computers, unlike humans, quickly reject obvious bullshit. None of this bullshitted code actually runs!

Ah, but subtler bullshit? That’ll run! As I’ve found in using Copilot, it’s quite possible for the AI to make subtler errors; the code will run at first, but quickly collapse when it hits an edge case. So one needs to be pretty vigilant even in using AI as an assistant.