Some more indications of the importance of memory bandwidth for the effective power of a computer. In this paper there's some statistics on neuron update speed, transport speed of neuronal information and brain size. It appears that the entire memory of the brain could theoretically be available to every single computation performed - at least as far as raw speed is concerned.

Also the latest Crays mentioned below localize the memory subsystems close to the processor. In the design for IBM's
Blue Gene machine the entire architecture of the machine will use merged DRAM embedding DRAM completely in the processor core (about 8 MB per processor) This may not sound like a lot, but according to the IBM scientists it suffices for the code they need to run.

On another note - as an aside to the mention of the latest Crays : This machine will pack a whopping 2^15 processor nodes - 31 times the Cray. Each node will yield 32 GFlops (3 times the Cray nodes I believe) for a total of one peta-flop i.e. 10^15 Flops. The power efficiency of the machine will be 2Megawatts per peta-flop. Orders of magnitude more efficient than the Crays mentioned below.