Scalability is definitely a challenge -- in fact, scaling to deal with SatoshiDice is what triggered the recent fork.

https://en.bitcoin.it/wiki/Scalability (which has some recent research on CPU optimizations) and https://bitcointalk.org/index.php?topic=34597.0 (a response from a year ago) answer some of Kaminsky's scalability criticisms.

There's also the possibility of off-the-chain transactions: http://gavintech.blogspot.co.il/2012/07/off-chain-transactio...