Technological Musings

neuromorphic systems

by far the most exciting thing happening in computing today (in my view) is the beginning of neuromorphic architecture.

these systems are not like your typical desktop PC.  they are designed to mimic the brain.  why, one might ask, do we want the latest computer hardware to mimic a forgetful piece of biology that has a hard time with long division?  ah, it is all about power, baby!  power as in electrical power.  the CPU in your mac laptop has been referred to (by kevin kelly) as a slow moving nuclear explosion.  our modern silicon chips pack as many as 2 billion transistors on each one, and even though they are getting more efficient every year, their transistor density grows faster than the power efficiency.  today, the heat dissipation on a chip is about the same as a nuclear bomb (if the bomb released its energy at the same rate as the chip, per second).

simply put, if we demand more computing power from our chips, and our chips get much hotter, they’re gonna melt.  this is remarkable….especially when you compare our high tech chips with the 3# of wetware in your skull.  our brains operate on about 20 watts of power…enough for a dim lightbulb.  yet, even with this meager power budget, we compose symphonies, run at high speed through crowded environments and recognize friends instantly.  even an owl, with its even smaller brain, flies silently at night through dense forest.  it is this talent for pattern recognition and processing that brains are so good at, and at which our silicon computers fail miserably (no matter how much power we allow them).

as we approach fundamental limits for our chips, our researchers have begun to appreciate the talents of our neurons and their chemical transmitters.  even if they move at a sluggish 10 hz. (neurons can fire [communicate] about 10 times a second) they outperform silicon that moves at 3 billion times a second (3 gigahertz).  how is this possible?  our computers are terribly inefficient in two ways:

  • traditional computers operate serially…that is, they do one thing, then the next thing
  • our silicon chips energize (expend power on) all their connections, all the time, even if they are “idle”

our brains evolved over millions of years.  one thing that has been universally true for all those years (until mini-marts came about) was that food was scarce.  any brain that consumed a lot of power was evolutionarily disadvantaged.  evolution is a cruel but masterful designer…ultimately creating brains that do wonderful things on a power budget of a glass of water and a tuna fish sandwich (ferrucci).*  instead of serial processing, our brains are interconnected in a massively parallel way and they use power only when a neuron fires.  these two features make the brain incredibly power efficient and also great at patterns…but crappy at long division.

IBM and others are now in the process of mimicking the behavior of neurons and synapses with silicon transistors and (new technology) memristors.  the memristor story is worth another blog post in its own right and i won’t dig into them today (fascinating).  they contribute to a silicon chip that can learn (yes, learn) by strengthening connections between silicon neurons.  it also mimics the brain in certain other ways that reduces its power budget significantly (not as low as the brain, but give ’em a break!  we haven’t had a million years to design it!).  this project is entitled “SyNAPSE” and is funded (who knew!) by the defense agency that gave us the internet: DARPA.  (insert terminator joke, here).

the SyNAPSE project is in the third stage of five and has successfully created the “foundational” chip.  it only has 256 neurons (the brain has 100 billion), but the roadmap is set.  by stage five, they should have a neuromorphic system of 100 million neurons installed in a robot which will perform at “cat level.”  this is scheduled for completion between 2014 and 2017.  ultimately, the goal is to have a system of 10 billion neurons that consumes less than 1 kilowatt (think small space heater).

there is far more to this project, mostly related to the massively parallel connection patterns that are still being discovered.  however, i am more interested in where this will lead us.  many techno-futurists talk about the “singularity”…a time when computers become conscious and drive change so fast it becomes an event horizon today’s humans cannot see past.  to get us there, the pathway most point to is emulating a brain in a supercomputer.  our machines are theoretically powerful enough to do just that, but at stupendous cost, size and power requirements.  simply running IBM’s new sequoia machine requires millions of $$ per year of electricity (7 megawatts)…and it would be hard to equip a robot with that “brain” as it takes up over 4000 ft. of floor space!

instead, i see neuromorphic systems that are compact and power-efficient as the real path to machines that will be truly useful.  conscious?  maybe.  but i don’t need conscious machines to see a radical break in our social, political and economic norms.  i simply need machines that hear, see, understand and can execute in the real world.  i think these systems are fairly close at hand…10 to 15 years is my guess.  not ubiquitous by then, but get ready!

* from david ferrucci, IBM lead researcher on Watson project (computer-powered jeopardy champion)


Single Post Navigation

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: