Hype document don’t have any support for being dynamically removed. This isn’t a very common thing to want to do, though React-like libraries make it a bit more of a priority now. If you need to fully tear-down and build up Hype documents the best way to do it is with an iframe.
window["HYPE"]["documents"][documentName] only contains a reference to the API object; there will still be plenty of other entities that are holding onto references (timers etc).
This has been there since v1 but my recollection is this is required for some older browser support that would not pick up the style if it were injected directly. It should be harmless - are you encountering issues with it being there?
You are correct that it should probably be ID’d to the document. What goes here is general CSS default styling, but it could change if there’s a different version of Hype or if the Use WebKit graphics acceleration box is different.