Modern audio systems actually do work like this, here's a good write up from PulseAudio about dynamic buffer sizes and resampling when a new sound comes in

I do not know how a2dp works and if it supports such buffers though.

http://0pointer.de/blog/projects/pulse-glitch-free.html