There's not a built-in way currently, but probably what I'd propose is not using the On Drag handler, and instead use one of the solutions in the forums that are targeting the scroll wheel.
The basic technique is here:
And there's some solutions including one for scene transitioning here:
I believe there are a few other threads on the topic.