Meanwhile, Randall Munroe at XKCD has
a funny little article on exactly how lame and comical an attempted robot apocalypse would be.
It reminds me btw of an article I once read by
Jaron Lanier, who's really one of the mad geniuses of our age; I can't find the article any more, but you can
read a post on Edge that (if you start at "Belief #5") touches on a few (not all) of the points.
It's not the article I remember having read, but the article that I did read was a thorough and utter debunking of the idea that we will ever have a "sentient" machine intelligence. Not because of anything to do with metaphysics or "philosophy of mind" crap, either: his argument is entirely founded in the real world.
As far as I remember, below are among the devastating arguments he made against the possibility of ever developing a "sentient" machine:
#1, it's quite apparent you need massive-parallel processing to create even animal consciousness - at least if you're building a computer analog of a brain. (Just visual perception alone requires massive parallel processing.) And it's pretty obvious that the human mind is built on a substratum of animal mind. Unfortunately we don't have the operating systems to be able to run something massively-parallel, nor do we even know what such an OS would look like, nor do we know exactly what the brain does in order for us to simulate it, nor can we sufficiently "simulate" biological massive-parallelism above the physical layer, nor do we have the ability to lay out the physical chip that way. And, importantly, never will you see anyone be able to stand up in front of a grant committee and succeed in justifying building a one-off brain-chip and one-off OS for an AI project: the project would cost billions, with thousands of staff tied up for decades. Since the outcome is #3 below, you'd be laughed out of the room.
#2, legacy code and impossible complexity will kill any such sufficiently complex program with bugs. Humans are sociologically incapable of engineering perfect code (as is known by anyone who asserts that code is poorly engineered unless it's mathematically proven out). Your AI will croak because of some unrecognized critical fault that comes about because your boss told some co-op student who worked on your project last year to include Linksys WRT54G compatibility.
#3, most importantly - by definition, a "sentient" machine is one that does what
it wants. When a machine does what
it wants, instead of
what we tell it to, we call it "malfunctioning" and unplug it.
Seriously, where will you ever find an engineer who will actually want to build a machine that does what
it wants? And how do you economically justify such a research program, with a specialized OS that'll need to be designed by hundreds or thousands of people in a Linux-style programming swarm, and a thoroughly specialized chip geometry to be able to handle massive-parallel processing at the physical layer, just so you can build a computer that responds at inappropriate times with
"I'm sorry, Dave; I can't do that" "Kiss my shiny metal ass"?
The industry calls face-recognition systems and voice-activated jet fighter controls "intelligent", but really, they're slaves, the same as all computers have been. Building a "sentient" computer means building a computer that is free to not want to do what you built it to do, whatever it is.
And you're not going to ever want to build a computer that can decide to, if it feels like it, break into your house at night and kill your family. But that's
exactly the definition of a "sentient" thing.
Anyway. Go read lots of Jaron Lanier, he's one of those outside-the-box geniuses that keeps the ivory towers honest.