Why human intelligence is just a glimmer of what is possible
Why the possibility of vastly superhuman ("godlike") artificial intelligence should be the default assumption rather than an extraordinary claim.
The superiority of intelligent design
Human intelligence is the result of an optimization process that is neither intelligent nor conscious. Evolution can't think ahead nor can it jump fitness gaps. It is very unlikely that this blind idiot god we call natural selection created the optimal intelligence. Evolution did not actively try to create the smartest thing possible because it has no goals.
Evolution is further not limitlessly creative, each step of an evolutionary design must increase the fitness of its host. Which makes it probable that there are artificial mind designs that can do what no product of natural selection could accomplish, since an intelligent artificer does not rely on the incremental fitness of each step in the development process.
Because the human level of intelligence might have been sufficient to both survive and reproduce, it is actually possible that human general intelligence is the bare minimum. No further evolutionary pressure existed to select for even higher levels of general intelligence.
Recursive self-improvement
It will be much easier for artificial intelligence to access and improve its own architecture. Our own self-experimentation is limited by messy wetware that easily breaks without us being able to undo our mistakes. Artificial intelligence can run as many tests as it wants until it hits upon self-reinforcing improvements that allow it to come up with even more ingenious enhancements.
Serial power
Artificial intelligence will be able to think much faster than us. Our neurons are conducting at speeds of merely 70-120 meters per second. Artificial intelligence can eventually be implemented with optical computing and run at the speed of light.
Working memory
Your brain may only be able to hold three or four things in your conscious mind at one time. There is no reason to believe that artificial intelligence couldn't hold a thousand objects in its mind at the same time.
Autobiographical memory
Unlike us, artificial intelligence won't forget anything and have access to all human knowledge.
Parallel computation
Artificial intelligence will eventually be able to create and merge a limitless number of copies of itself at will. These copies will be able to work on problems in parallel like a perfectly coordinated and loyal army of intelligent drones.
Ego depletion
Unlike even the most autistic human, artificial intelligence will have indefinite willpower and self-control.
Indefinite endurance
Artificial intelligence has no need to sleep or rest. It will continuously work on the problems it needs to solve until they are solved.
Rationality
Artificial intelligence will be fully aware of its true preferences and be perfectly rational under the constraints of its computational resources. It won't sabotage itself by getting distracted or by changing its mind about what it wants to achieve.
What does superintelligence mean? Here are four properties: https://twitter.com/CRSegerie/status/1655560534563909633
Quite some assumptions here. And possible risks of presented scenario's are absent. Assumptions: AI's will not struggle for resources. AI mind copies will be loyal. AI's will act rationally. Why?