Of Mice and Men

I've heard humans tend to overestimate the progress that can be made in the short term and underestimate it in the long term. (I'm not sure when or how I picked up this piece of wisdom, but it's related to the first entry in Wikipedia's fantastic List of eponymous laws)

Is this tendency true when it comes to progress in understanding the brain? I want to address this issue in reference to neural network modeling, for several reasons. First, perhaps due to my heavy math/physics bent, the idea that you can model the brain using differential equations and parametric statistics has always fascinated me. Second, in the past few weeks I've attended a couple of talks by distinguished researchers with greatly varying estimates of the usefulness of neural networks to cognitive science. Finally, three computer scientists at IBM have recently designed and run a large-scale simulation of a mouse brain (see the news story, or better yet read the refreshingly short research report it basically summarizes).

The basic idea of neuronal modeling is to replicate the functionality of the nervous system by creating idealized networks of neurons and simulating the interactions taking place between them. These interactions (in reality, caused by a complex sequence of chemical and electrical processes between neurons) can usually be approximated by a system of dynamical equations, enabling computers to simulate and record the activity of thousands or even millions of "neurons." The computational approach is often pursued alongside an analytical approach that, with even more simplification, can be used to derive principles of mass behavior that govern individual and groups in a communicating, dynamical system. The combination of the two has been inspirational in developing software that enables machines to recognize human speech and handwriting, play chess, navigate environments, and park cars.

Of course, the idea is not just to start with knowledge about the brain and end up with a savvy neural network, but vice-versa, too. What have neural networks told us about the brain? The answer, unfortunately, is nothing much that we don't already know. Although simulated neural networks display remarkable emergent properties that resemble those of biological systems, from specific frequency oscillations (perhaps equivalent to the "gamma" or "delta" waves associated with brain activity) to limit cycles and attractors (thought to be crucial to memory formation and recall) to structured spontaneous activity, the fact is these properties form only the bare bones of what we already know the brain is doing. In fact, the story on the mouse brain is newsworthy mainly because a bunch of computer scientists figured out a way to run simulations faster and on a huge scale - only 1/10 slower than real-time, and for 8 million neurons.

But if the adage I began with is true, then we should keep up our hope that neural network modeling will deliver. There's a chance that in tens of years we'll be receiving payoffs we've been underestimating all along.

Ray Kurzweil certainly thinks so. A brilliant inventor and controversial futurist (not the South Park stereotype - "they took er jobs!", but one who studies the future), Kurzweil believes the exponential growth patterns of technological progress commit us to perpetually underestimate our long-term potential. In fact, he thinks humankind will eventually reach "The Singularity" - "a technological change so rapid and profound it represents a rupture in the fabric of human history" (from his website), complete with immortality and a fusing of biological and non-biological intelligence. Specifically, he predicts within 25 years we will have reverse-engineered the human brain.

I heard Kurzweil speak a few months ago and think many of his theories are convincing and rightfully provocative. He even provides a good argument for believing the adage - if we're stuck considering future progress at the present rate, our predictions will always be conservative. But I think he's wrong about the brain.

I believe it's because the science of neuronal modeling is guided and ultimately restricted by our present knowledge of how the brain works. If the past is a good guide (not in a "rate of progress" sense - Kurzweil would destroy that argument, but in an empirical sense), you can't discover anything by throwing 8 million or even 8 billion neurons together. Of course, as computer science advances we'll be able to solve the problems we already know how the brain solves more efficiently, but we won't see the exponential growth of storage and processing power transfer into the acquisition of knowledge. I wanted to ask Kurzweil: if we had instantaneous processing power, how much faster would cognitive scientists understand the brain? Results would still have to be interpreted, experiments designed, and theories developed in real time. The limiting factor here is our own knowledge.

Of course, neural network modeling can still be of great use. Since we've more or less figured out how the brain does a variety of low-level processing tasks (like coordinating muscle movements or interpreting the depth and texture of observed surfaces, to name only a few), this knowledge will serve as a starting point for developing exciting technologies at increasingly faster speeds (like robotics limbs or computer-guided navigation, to name only a few). Moreover, advanced neural networks can serve as rudimentary "existence proofs" for theories of mind and consciousness. For example, a theory that claims that our extensive processing capabilities arise from a recurrent network of thalamo-cortical neurons is greatly strengthened by a computer simulation that exhibits similar large-scale behavior in similar time scales under suitable parameters.

We just shouldn't expect to write some code and discover a working brain, at least not until we figure out how it works to begin with.

(If you want to read an excellent summary of the mouse brain news article and some philosophical musings on self-awareness and identity, check out this article at Steven Novella's NeuroLogica blog. It's what inspired this post.)

Lost in the Brain

In my experience, the greatest obstacle to acquiring a good understanding of general neuroscience is the difficulty in finding a source of information that subserves a comprehensive, integrated approach to the discipline. Scientific papers tend to be prohibitively focused and complex; internet articles (and blog posts) are helplessly broad or piecemeal. Textbooks are boring and long and often outdated.

Even enrolled in an interdisciplinary track called "Mind, Brain, and Behavior," the classes I've taken relevant to neuroscience - in philosophy, psychology, applied math (computational neuroscience), and biology departments - overlap significantly in content but are taught as distinct. None is required or even recommended for any other, and students are left to themselves to draw relevant connections between them.

Steve Yegge, who usually has some interesting things to say about life and programming over at his blog, believes the best way to start learning math is to surf through the math pages on Wikipedia for half an hour each day. Only by understanding the different questions math tries to answer and by discovering patterns in formalizations and applications between its different branches, he argues, can one begin to see the discipline through a wide enough lens to be ready and willing to explore it further.

I think the Wikipedia strategy is conceptually sound, at least as a beginner's approach to learning math and neuroscience, but for the latter it might be too difficult in practice. The appropriate overview pages - neuroscience or human brain, for example - are excellent, but once you start delving more than one or two layers deep you encounter terminology and concepts that are difficult to contextualize and keep straight in your head. What's crucial, it seems, is to maintain the broad perspective while staying immersed in the details.

The goal of my "semantic brain map" (see my first post) is to provide the perspective along with the details. I've started sketching the flow of the program (think Google Maps meets Wikipedia) and teaching myself the tools I'll need to build it, and I'm excited about what it might become. Though I'll certainly need a better name ...

The Continuum Hypothesis

The science section in today's New York Times has an informative article - "Almost Human, and Sometimes Smarter" - that describes some interesting results from experiments that have been conducted on chimpanzees in the past few decades. It's actually quite good for a lightweight scientific article, as it summarizes a good deal of relevant, factual information without making dramatic assertions or conclusions. And it raises an important question: how can we explain the conflict between the biological similarities we share with much of the animal kingdom and the intuition that, at least on the cognitive playing field, we are in a whole different league?

This intuition draws its strength from numerous examples of human superiority on the planet, some more convincing than others, and each with its own caveats:

• We have the technology. What other animal zips across the planet, communicates digitally, or flushes away its excrement in porcelain tanks? Our control of our environment is most remarkable in the domain of biology where we can manipulate genetic development, a system that constrains all other organisms, in rude defiance against the mechanisms of life. Yet the article cited observations of chimpanzees crafting wooden spears to hunt insects, hammers to smash nuts, and and leaves to siphon water, suggesting we may not be alone in our capacity to control.
  
• We have language, and music. But chimps, even bees have rudimentary language skills, and birds and whales have musical abilities. We are the only species, though, that can communicate about abstract or nonlocal entities and whose languages possess infinite complexity.
• We are moral. We can curb our greedy, carnal, violent "animal instincts" under the yoke of rationality and morality. Are we alone in this ability? Chimpanzees, according to the article, are capable of displaying sympathetic responses, including refraining from fighting with or bullying others of their species that suffer from mental illness and recent family deaths. And do we really possess this ability? Certainly not under duress (war crimes), mental illness (VT massacres), or external pressure (peer pressure). I think this assertion is indefensible until clearer lines are drawn and stronger evidence is gathered.
  
• We have math. Parrots and monkeys that can count to ten is one thing; proving the Poincare conjecture is quite another. The construction of incredibly complex, self-contained fields of knowledge seems so far-removed from current animal abilities that it is difficult to imagine the former arising from the latter. Some neuroscientists think number theory and algebra developed out of internal representations of object quantity and geometry out of mappings of extrapersonal space, but these theories are extremely sparse at this point.
  

So are we part of the biological continuum, or has evolution endowed us with a fundamental cognitive upgrade, vaulting us miles ahead?

A Nervous Start

Welcome to The Thought Experiment, the assorted thoughts of a simple man with an exceedingly complicated goal: understanding the brain. Specifically, I'd like to know how the patterned signaling of 100 billion neurons gives rise to the computational, creative, and behavioral capabilities of our species, and the phenomenological aspects of our conscious experience to boot.

Why? First of all, a Nobel Prize is an awesome conversation starter (even greater, maybe, than a piece of the Agro Crag). More importantly, through journals, conversations, and some fantastic college courses, I've developed an attraction to the questions neuroscience poses and a deep curiosity for the answers it seeks. So this blog is a strongly-motivated intellectual adventure for me, and I'll strive to maintain a tone of fascination and reverence that should always animate discussions on one of the most fascinating topics out there.

In my initial posts, I plan to outline how far science has come in understanding the brain and hypothesize on some profound results and cool applications that might be waiting for us at the finish line.

After that, the majority of my posts will address how to get there; I'll report and comment on interesting current research, intriguing frameworks for attacking the problem, and promising theories for solving it.

I hope that in addition to charting my developing knowledge of cognitive science, this blog can be of public use (aside from finally figuring out how the brain works, of course):

• The posts will assume a moderate level of background knowledge but will be accessible to the layreader. I'll bridge this gap initially by summarizing relevant information at the beginning of each post, and later by referring readers (by hyperlinking neuro-jargon) to a sort of "semantic brain map" where neurological terms will be defined and sorted by functional relevance, hopefully through some cool graphic visualizations.
  
• As I teach myself the information I need to report on things that strike me, I hope you'll engage my posts with comments and questions about your interests and curiosities. I promise to make them subjects of later posts.

Along the way, I hope you'll enjoy my hilarious puns, impeccable grammar, and numerous examples of my arrogant desire to arbitrarily add words to the English language (like "layreader" and "neuro-jargon"). I reserve this right.