If AI could genuinely become an existential threat, why are we treating its development as a race? Kyle Taylor faces an uncomfortable truth: we don’t know where this race ends or why we are running it
The parallels are there with nuclear - energy and weapons. The potential for good and the potential for utter disaster, literally destroying the planet.
A key difference is that AI is in the hands of a very few people, who have been driven primarily by profit, egged on by greedy finance. Both groups using their wealth to grab political power and block the regulation that is really needed.
Nuclear has by and large stayed in the hands of state actors. Whilst they are by no means all well intentioned, there is not that same lust for personal reward and power. Where nuclear power is concerned, the idea that there might be a 10% chance of disaster would be seen as utterly unacceptable
When you look into what the companies were actually doing when they allowed their products to "go rogue", it was so unutterably stupid, you wonder why anyone believes anything they say, or leaves them in charge of the tea fund, never mind a whole massive company.
We all know that LLMs "hallucinate" (ie make things up) all the time, and anyone with any sense examines their proposed course of action carefully before following it. LLMs don't understand what they're proposing, they just provide examples of what the data they were fed might suggest might work - and there's no reason to think that what went into those datasets actually did work, or that the circumstances were in any real way similar.
The claims of "autonomous hacking" are just a sales pitch, to try to keep the investments rolling in. Because the moment enough people see what's really going on, the bubble will burst.
Don't get me wrong, specialised AI that doesn't just depend on LLMs is powerful & could be dangerous, although it's where the real value is, too. But that's not what these idiots were using, and it's not where the big investments are being made.
Fear of AI is mainly being led by those in the business. Ok they are the ones who might be expected to know, but when Open AI cite these fears as a reason to hold back from public trading, might we also be suspicious that it's a convenient cover for concerns that the sector's profits are not so far leading to dazzling performance on the stock exchanges.
The parallels are there with nuclear - energy and weapons. The potential for good and the potential for utter disaster, literally destroying the planet.
A key difference is that AI is in the hands of a very few people, who have been driven primarily by profit, egged on by greedy finance. Both groups using their wealth to grab political power and block the regulation that is really needed.
Nuclear has by and large stayed in the hands of state actors. Whilst they are by no means all well intentioned, there is not that same lust for personal reward and power. Where nuclear power is concerned, the idea that there might be a 10% chance of disaster would be seen as utterly unacceptable
When you look into what the companies were actually doing when they allowed their products to "go rogue", it was so unutterably stupid, you wonder why anyone believes anything they say, or leaves them in charge of the tea fund, never mind a whole massive company.
According to a very well-informed podcast (https://podcasts.apple.com/us/podcast/no-ai-is-not-autonomously-hacking-with-cal-newport/id1730587238?i=1000785935670) that I found because Cory Doctorow linked to it, what they actually did was used an LLM to generate ideas of how to hack a file, and kept an "agent" following the LLM's instructions to the letter for days, no matter how deeply idiotic they were.
We all know that LLMs "hallucinate" (ie make things up) all the time, and anyone with any sense examines their proposed course of action carefully before following it. LLMs don't understand what they're proposing, they just provide examples of what the data they were fed might suggest might work - and there's no reason to think that what went into those datasets actually did work, or that the circumstances were in any real way similar.
The claims of "autonomous hacking" are just a sales pitch, to try to keep the investments rolling in. Because the moment enough people see what's really going on, the bubble will burst.
Don't get me wrong, specialised AI that doesn't just depend on LLMs is powerful & could be dangerous, although it's where the real value is, too. But that's not what these idiots were using, and it's not where the big investments are being made.
Fear of AI is mainly being led by those in the business. Ok they are the ones who might be expected to know, but when Open AI cite these fears as a reason to hold back from public trading, might we also be suspicious that it's a convenient cover for concerns that the sector's profits are not so far leading to dazzling performance on the stock exchanges.