The Economist welcomes your views. Please stay on topic and be respectful of other readers. Review our comments policy.
You must be logged in to post a comment. Log in to your account.Don't have an account? Register
Moore's Law has "stumbled" because those in a position to move our computing power forward seem to have hit the pause button. There are any number of new technologies, including graphene, that have the potential to deliver 100 times the current computing power in their first generation. Why have we not seen these in the marketplace? I don't know, but maybe we shouldn't trust what everyone would do with that sort of computing power.
If you are wondering why someone might worry about Artificial General Intelligences getting out of control, I would recommend reading or watching Nate Soares' "Ensuring smarter-than-human intelligence has a positive outcome" talk.
Basically, we (humans) don't know how to program computers to do what we want, not what we tell them to do.
"with genetically modified humans bound together by brain implants into a solar-system spanning hivemind, or perhaps uploading their minds into a silicon utopia. "
I know I may come across as a reactionary Neanderthal but, this is supposed to be the optimistic view??
Singularity or not, our cleverness so far outstrips our sense that it is almost certain that we will find a way to destroy ourselves when some technology slips out of our control. Nuclear weapons are already waiting to do the job.
The big-brained ape is an evolutionary dead end.
For 70 years the megacephalic simian has found ways to avoid using those nuclear weapons, a credit to his species and a relief to all the others. Until the recent American presidential election that condition very probably would have continued, Vladimir Putin's injured-ego saber-rattling notwithstanding. With an unpredictable Tasmanian Devil in the White House bragging about the size of his button, all bets are off. Invest in home fallout shelter stocks before the rush.
Tom M makes a valid point that, in nature, exponentials always morph into sigmoids. And the failure to date of "thinking machine" research reinforces his point. My view, from life observation, is that 'intelligence" is composed of memory, deductive reasoning and inductive reasoning. Machines already exceed humans in the first two (and will continue to improve), but I've seen no research able to implement (or even rudimentarily explain) inductive reasoning. So I fully agree with Tom's last paragraph.
"Tom M makes a valid point that, in nature, exponentials always morph into sigmoids. And the failure to date of 'thinking machine' research reinforces his point."
"Thinking machines" are at the same point in their development as the airplane was in 1906. Your spiritual forebears had their say about the future of the new technology back then:
"As it is not at all likely that any means of suspending the effect of air-resistance can ever be devised, a flying-machine must always be slow and cumbersome… . But as a means of amusement, the idea of aerial travel has great promise. — T. Baron Russell, author of "A Hundred Years Hence: The Expectations of An Optimist,'' 1905
"The popular mind often pictures gigantic flying machines speeding across the Atlantic, carrying innumerable passengers. It seems safe to say that such ideas must be wholly visionary." — William Pickering, Harvard astronomer, 1910.
"The demonstration that no possible combination of known substances, known forms of machinery and known forms of force, can be united in a practical machine by which man shall fly long distances through the air, seems to the writer as complete as it is possible for the demonstration of any physical fact to be." — Simon Newcomb, professor of mathematics and astronomy at Johns Hopkins University, 1906
"All attempts at artificial aviation are not only dangerous to human life, but foredoomed to failure from the engineering standpoint." — Engineering [former name for Technology] Editor, New York Times, 1906.
When a quantity in nature is released from a constraint, it tends to grow as a positive exponential until it finds a new constraint, which it will approach as a negative exponential. This behavior describes a sigmoid (S) shape when graphed against time. Up to the inflection point, a sigmoid is generally indistinguishable (in a statistically significant sense) from a simple exponential. So until we pass the inflection point (second derivative changes sign), which is the point of highest velocity, to the best of our predictive ability we can only say that it appears that this trend will continue until the quantity in question reaches infinity.
Except nothing keeps going to infinity. There is always another constraint. The global population crossed its inflection point in the late 1980s, but we didn't have the demographic data to discover this until well into the 1990s. We have likely passed the inflection point in Moore's law, but don't have the data to really prove it yet. But passing the inflection point doesn't mean that we'll get to infinity, only slower. It means that we are now on a negative exponential glide-path to a new upper limit for computing speed and efficiency (computations/$). To climb back onto an exponential rise will require as profound a new technology as the combination of the transistor/semi-conductor logic gate/microchip.
As far as AI is concerned, the neural network is a non-linear interpolation tool developed 40-50 years ago. Applications are coming to light now that we have sufficient computing power and the huge stores of data to train more of them in useful ways. Back in the 1980s-1990s, AI researchers generally split off neural networks as a separate field of research from actual Artificial Intelligence, as there was nothing terribly intelligent about them; neural networks were brute force statistical tools. Researchers were working on the problem of General Artificial Intelligence, i.e. actual thinking machines.
And they failed, repeatedly. And they continue to fail. There is no research likely to yield anything approaching a general Artificial Intelligence within 10 years, which is a nice way of saying that researchers still aren't really sure how to even grapple with the problem. They're still trying to ask the right questions. Generalized thinking machines remain purely a subject for speculative fiction. There would be a great deal more clarity on the subject if people refrained from putting the AI label on neural networks. Neural networks, despite the name, are not a field of work that anybody thinks is going to lead to a general intelligence, an "artificial brain", i.e. a machine that can learn and think about arbitrary new topics without a well-defined and narrow domain.
So no, there will be no singularity, and we are not going to see the world taken over by thinking computers.
I challenge anyone to find me a serious AI researcher who says differently.
Challenge accepted, from Russell and Norvig, one of my college AI textbooks:
One threat in particular is worthy of further consideration: that ultraintelligent machines might lead to a future that is very different from today--we may not like it, and at that point we may not have a choice. Such considerations lead inevitably to the conclusion that we must weigh carefully, and soon, the possible consequences of AI research for the future of the human race
From Artificial Intelligence, a Modern Approach, 2nd Ed. by Stuart Russell and Peter Norvig pg 964
Sure, I'd put that on pg. 964 of a textbook too, just like I try to end the last lecture of the week with something that sounds vaguely profound, or a good joke. But if Russell and Norvig were standing in front of either of us and were asked about generally intelligent machines, machines that could learn in an unstructured way as humans learn, they'd beg off making any predictions of when or how that was going to happen. Yes, there will be many more specialized machines which are better and faster at doing some feat of analysis than a person, just as there are now, but the machine's genius will be narrow and focused, and utterly incapable of leaving its domain of interest. And those machines will change our world, but arguably less than did electricity, the automobile, or spaceflight, which made life not just better and faster, but very different from what came before.
First of all, I do agree with you that we are not on a never ending exponential curve. At some point the growth curve has to slow down and pass an inflection point and eventually stop growing.
Neural networks are in a sense statistical processing, but it is impressive what can be achieved with it. For example AlphaZero combines neutral networks with a monte carlo tree search, and with just being given the rules of chess, shogi, and go, with a day of playing games against itself, was able to achieve superhuman ability to play chess, shogi or go. So in short neural nets can achieve things that we would have considered intelligent were they to be done by a human. (See https://arxiv.org/abs/1712.01815 )
You may be right that Russell and Norvig would not make any predictions about when or how general intelligence would happen. However, that actually would worry me more. If they could say this is why general intelligence cannot be done until X happens, and X will not happen until we have computers with specification Y, that would increase my confidence. However, lack of ability to make predictions is a bad sign for safety. It means we don't understand how to stay in the safe zone.
I leave you with a question (and this is not original to me, Elizer Yudkowsky has asked a variant of it: https://intelligence.org/2018/02/28/sam-harris-and-eliezer-yudkowsky/ ): What, short of the world being taken over by thinking computers, would actually change your mind? What lesser ability would you actually consider a sign that thinking computers were close?
Good Question. It is tempting to give the old supreme court answer about pornography, "I'll know it when I see it", but that was a bad answer then and would be a bad answer now.
I'm not sure I have an answer, but the quality I'm looking for has to address this: Humans are capable of learning new concepts and performing new tasks without thousands of trial and error attempts because they can draw analogies from widely disparate experiences and found knowledge (things that they've read or heard about) and construct a strategy to execute a task which they've never done before, or very few times. Computers can be taught to become very good at analysis, but we struggle to make them good at synthesis, and in particular drawing analogies. This is in part because the type of decisions required to complete these tasks draw on the emotional subconscious part of the brain, in addition to the conscious, "thinking" part of the brain which the computer is more capable of mimicking. I want a series of examples where a computer makes sensible decisions to questions that are vaguely posed, and where the computer lacks enough data to know that it has the best, or even a good answer.
I agree with you that general learning, or synthesis is lacking in computer algorithms currently. I also agree that humans use their emotional parts of their brain for part of their thinking and decision making ability.
I think that an algorithm can do learning and decision making perfectly fine without anything we would call emotion. As in it runs its utility function on the possible states and calculates the value from the utility function, and gets a number and tries to maximize expected utility.
I think computers could take over the world without ever being able to answer a vaguely posed question in a way that a human would find satisfactory.
I think that a general ability to do learning is very close to having the ability to take over the world. I don't think waiting until that happens would give you much time to solve any problems.
Actually, scientific output is growing exponentially. And it can only increase as computational power increases. I think that the AI will be serve human since it will lack a creative thinking for, probably, a very long time. I understand the singularity as a moment when the AI will start to think creatively, which can even take centuries. But there is not reason to believe that humans and AIs cannot cooperate.
I am glad that TE is starting to take AI threats at least semi-seriously. Reading the comments on this article, it seems that some people still snicker at the idea of AI taking over. For me it is all about weighted probabilities (probability of something happening multiples by the magnitude of the effects it would have). Can I say that a future Singularity scenario is likely? No. But it does seem possible. And because the potential effects are disastrous, it is worth taking seriously.
My attention is drawn to the following singular idea asserted by Mr. Son:
"Robots will have IQs of 10,000 within the next 30 years, he says."
Who constructs this IQ test? And what IQ is the constrcutor of this test?
Singularity is 20 years-old idea out of sci-fi from the early 2000s. I have soft spot for this time, because of the dotcom boom. But objectively speaking, intelligent machines failed to materialize. I am surprised that TE revived this old trend.
"Robots will have IQs of 10,000 within the next 30 years,"
Total crap! Robert cannot be smarter than people who make them.
Robert, despite his humble beginnings in a London hovel and his illiterate parents, went on to add more to human knowledge than those who made him. :-)
Exponential growth....oh well it is The Economist after all.
Regarding the optimists: Would infinite leisure really be a utopia? What is leisure? What is labor? What is our purpose in being? Are either of those things really our purpose? Or were both labor and leisure always a fantasy of purpose, a distraction from purpose?
Regarding the pessimists: Why would an AI maliciously wipe out humanity at all? I could foresee this happening but it would not be the AI who destroys us; rather we destroy ourselves having been made useless by the AI when we are forced to confront the emptiness of existence without the illusion of meaning found through work.
Regarding your first Q, @Peace, my personal answer is not only would infinite leisure not be a utopia, it would be living hell. I cannot imagine anything quite so boring!!! I think I'll kill myself if subjected to that torture.
Regarding your second Q, I'd go with your answer 100%. Who could destroy ourselves but ourselves (and every now and again, we make wars to accomplish that)? And why on earth would we want to make a bunch of silly things to replace ourselves just so we don't have to think and work any more??????!!! A wise man said (I don't want to mention his name because he is not a popular guy) love (not the kind that end up in various positions but the kind that benefit our fellow human beings without taking positions) and work (so we would know how to do love) are the 2 paramount purposes in life. Worked out for him. In all the books he wrote and work he did, no one ever read or saw any complaint.
The Singularity idea appeared before the wealth was concentrated in the top 1%. With the current situation, the hypothetical utopia would at best concern the rich 1%. The rest of the population would stay in poverty, with no chance of ever getting better. Maybe the rogue AI would also wipe out only the top 1% of humanity and believe that most people are irrelevant to them.