The act of scrolling is only noticeable by the browser + Javascript per whatever cycle it's on; it's not always watched except for at certain checkpoints.

To more accurately be affected by quick scrolls, save the last known y-axis value and compare to the current y-axis value. Act accordingly!