The idea of ​​the artificial intelligence being toppled mankind has been talked about Many decadesScientists have just made their judgment about whether we will be able to control the intelligence of a high-level computer. the answer? of course no.

The point is that controlling a superintelligence that goes far beyond human understanding requires a simulation of that superintelligence that we can analyze. But if we are unable to understand it, then it is impossible to create such a simulation.

It is not possible to establish rules such as “do no harm to humans” if we do not understand the type of scenarios in which AI will come up, as the authors of the new paper suggest. Once a computer system operates at a level higher than our programmers’ range, we can no longer set limits.

Superintelligence poses a problem fundamentally different from the one typically studied under the slogan ‘Robot Ethics’. Write researchers.

„This is because superintelligent intelligence is multifaceted, and thus potentially able to mobilize a variety of resources in order to achieve goals that are likely to be incomprehensible to humans, let alone control them.”

Part of the team’s inference comes from Problem stopped It was brought up by Alan Turing in 1936. The problem revolves around knowing whether or not a computer program will reach a conclusion and an answer (so it stops), or just a forever loop trying to find one.

Turing has also proven through some Smart mathematics, While we can know that for some specific programs, it is logically impossible to find a method that would allow us to know this for every potential program that could be written at any time. This brings us back to artificial intelligence, which in a super-intelligent state can keep every possible computer program in its memory simultaneously.

Any program written to prevent AI from harming humans and destroying the world, for example, might reach a conclusion (and stop) or not – it’s mathematically impossible to be absolutely sure either way, which means it’s not contained.

„In fact, this makes the containment algorithm unusable,” Computer scientist Iyad Rahwan saysFrom the Max Planck Institute for Human Development in Germany.

The researchers say the alternative to teaching AI some ethics and telling it not to destroy the world – something that no algorithm can be completely sure to do – is to limit the capabilities of superintelligence. It can be disconnected from parts of the internet or certain networks, for example.

The new study also rejects this idea, indicating that it would limit access Artificial intelligence The argument goes that if we are not going to use it to solve problems outside of humanity, why do we create it at all?

If we were to go further with AI, we might not even know when a superintelligence beyond our control, such as not understanding it, arrives. That means we need to start asking some serious questions about the directions we’re heading in.

„The super-intelligent machine that controls the world looks like science fiction,” Says computer scientist Manuel CebrianFrom the Max Planck Institute for Human Development. „But there are actually machines that perform certain important tasks independently without the programmers fully understanding how they learned them.”

The question therefore arises whether this could at some point become out of control and dangerous to humanity.

The research has been published in The Journal of Artificial Intelligence Research.