I don’t think anything changed in 10.11.5 specifically, but since you reinstalled OS X it is likely these reverted back to the default (the default may have been different on 10.7 vs. a clean install of 10.8+). The options are in the System Preferences’ General pref pane:
- Ask to keep changes when closing documents
- Close windows when quitting an app
Also note, as part of this OS X “Versions” feature Hype implements, you can always choose ‘File > Revert To > Browse All Versions…’, ‘Last Opened’, or ‘Last Saved’.