Computer superintelligence is a long, long way from the stuff of sci-fi movies, but several high-profile leaders and thinkers have been worrying quite publicly about what they see as the risks to come.

　　Our economics correspondent, Paul Solman, explores that. It's part of his weekly series, Making Sense.

　　I want to talk to you about the greatest scientific event in the history of man.

　　Are you building an A.I.?

　　A.I., artificial intelligence.

　　Do you think I might be switched off?

　　It's not up to me.

　　Why is it up to anyone?

　　Some version of this scenario has had prominent tech luminaries and scientists worried for years.

　　In 2014, cosmologist Stephen Hawking told the BBC:

　　I think the development of full artificial intelligence could spell the end of the human race.

　　And just this week, Tesla and SpaceX entrepreneur Elon Musk told the ?MDNMˉNational Governors Association:

　　A.I. is a fundamental existential risk for human civilization. And I don't think people fully appreciate that.

　　OK, but what's the economics angle? Well, at Oxford University's Future of Humanity Institute,

　　founding director Nick Bostrom leads a team trying to figure out how best to invest in, well, the future of humanity.

　　We are in this very peculiar situation of looking back at the history of our species, 100,000 years old,

　　and now finding ourselves just before the threshold to what looks like it will be this transition to some post-human era of superintelligence that can colonize the universe, and then maybe last for billions of years.

　　Philosopher Bostrom has been perhaps the most prominent thinker about the benefits and dangers to humanity of what he calls superintelligence for many years.

　　Once there is superintelligence, the fate of humanity may depend on what that superintelligence does.

　　There are plenty of ways to invest in humanity, he says, giving money to anti-disease charities, for example.

　　But Bostrom thinks longer-term, about investing to lessen existential risks, those that threaten to wipe out the human species entirely.

　　Global warming might be one. But plenty of other people are worrying about that, he says. So, he thinks about other risks.

　　What are the greatest of those risks?

　　The greatest existential risks arise from certain anticipated technological breakthroughs that we might make,

　　in particular, machine superintelligence, nanotechnology, and synthetic biology, fundamentally because we don't have the ability to uninvent anything that we invent.

　　We don't, as a human civilization, have the ability to put the genie back into the bottle. Once something has been published, then we are stuck with that knowledge.

　　So Bostrom wants money invested in how to manage A.I.

　　Specifically on the question, if and when in the future you could build machines that were really smart, maybe superintelligent, smarter than humans,

　　how could you then ensure that you could control what those machines do, that they were beneficial, that they were aligned with human intentions?

　　How likely is it that machines would develop basically a mind of their own, which is what you're saying, right?

　　I do think that advanced A.I., including superintelligence, is a sort of portal through which humanity will have passage, assuming we don't destroy ourselves prematurely in some other way.

　　Right now, the human brain is where it's at. It's the source of almost all of the technologies we have.

　　I'm relieved to hear that. And the complex social organization we have. Right.

　　It's why the modern condition is so different from the way that the chimpanzees live.

　　It's all through the human brain's ability to discover and communicate.

　　But there is no reason to think that human intelligence is anywhere near the greatest possible level of intelligence that could exist, that we are sort of the smartest possible species.

　　I think, rather, that we are the stupidest possible species that is capable of creating technological civilization.

　　For the PBS NewsHour, this is economics correspondent Paul Solman, reporting from Oxford, England. undefined