The Google TV remote has trainable buttons. I can operate the chromecast and but also adjust volume direct to TV. And turn TV on and off.

So, rarely have to use the TV remote. It's used only for live TV (rare) and switching input, also rare.