That's ffmpeg. Neither mplayer nor VLC were doing anything special that let them "play everything". They just used ffmpeg.

(nb they did often use their own demuxers instead of libavformat)