[leourbina]

I've been using co-pilot as well as Claude (Sonnet 3-3.5) and ChatGPT (4-4o) since the beginning of the year with somewhat mixed success. My general take is that, at least so far, the improvement is very much marginal. These tools save me a few seconds by quickly surfacing information that otherwise would require me having to google and search through docs/stackoverflow/etc, or by writing code that's generally tedious (like tests).

Co-pilot is mostly useful to stay in the zone, allowing me to focus on a larger task and letting it get some of the details right (which it does ~90% of the time).

On the other hand, I use ChatGPT/Claude for more open ended tasks (e.g. "I got this <insert obscure> error", "how do I configure this framework so that xxx") which previously I would have googled hoping to find a stackoverflow answer or a doc page somewhere. For this use case I'd say it's ~50% successful, but I often have to deal with hallucinations - some times just following up with "Are you sure?" does help, but it's hit/miss.

As I said a the beginning, mostly marginal improvement. It's definitely saved me time, but thus far nothing that I couldn't do myself by spending a little bit more time. Largely it is a nice to have, not a need to have.