2023-06-26
3 小时 7 分钟If you have an AI that produces bioweapons that could kill most humans in the world,
then it's playing at the level of the superpowers in terms of mutually assured destruction.
What are the particular zero-day exploits that the AI might use?
conquistadors with some technological advantage in terms of weaponry and whatnot.
Very,
very small bands were able to overthrow these large empires or
if you predicted the global economy is going to be skyrocketing into the stratosphere within 10 years.
These AI companies should be worth a large fraction of the global portfolio.
And so this is indeed contrary to the efficient market hypothesis.
This is like.
literally the top in terms of contributing to my world model in terms of all the episodes I've done.
How do I find more of these?
So we've been talking about alignment.
Suppose we fail out of alignment and we have AIs that are unaligned and at some point becoming more and more intelligent.
What does that look like?
How concretely could they disempower and take over humanity?
This is a scenario where we have many AI systems.
They have the way we've been training them means that they're not interested when they have the opportunity to take over and rearrange things to do what they wish,
including having their reward or loss be whatever they desire.
They would like to take that opportunity.