The real danger of AI
When people think about the danger of AI, it might be about "it's going to replace us", or "it's going to pwn all the software in the world". All of these claims are not relevant. The worst thing that could happen is if we lose our craft and are unable to think for ourselves. If this day comes, we are thoroughly f*cked. Because if this day come, it means we have surended to Anthropic, OpenAI and the other tech giants and they now can control the software.
People often tell we are "only going up the abstraction ladder" like we did with compilers. We're not. Everyone claiming that today is a moron. Compilers are deterministic and most often open-source and they don't require a $10K hardware rig to operate. If you give up your skills to write software, you are only a serf to the all powerful Silicon Valley and they own you. They can raise their prices, lower their quotas, do whatever they want with your code.
If you want to protect your independence, work with humans. You can still use an AI to speed up your tasks, but it still must be a 1-1 mapping and humans must know the code, know how to debug it and how it works.