It’d be good to see a .hype document of your setup and how it is interacting with your zoom and pan library; any suggestions otherwise are a shot in the dark.
I assume you are using Hype’s vector shapes on top of the SVG? You may want to group them and then use the zoom and pan on the group, if you are not already doing that.
If you’re using a group it could also just have the Content Overflow (in the Metrics Inspector) set to Scrollbars and the user can just scroll, though this does not solve zoom.
If it is more of a guided tour where the user cannot arbitrarily move the file, you could think about a solution like a camera element outlined in this post.