As you are not changing the height, the video size is actually not changing. So you can have the element not actually grow in width, and then the video controls will stay the same size to the video. If you still want the letterbox effect you could make another black element behind the video that is set to grow in width.
This isn’t possible per-se; you could make the height larger but you’ll wind up with letter boxing. The only workaround is to not show controls and then re-implement them yourself (calling into the video element). I think there might be some libraries around that can offer alternate controls.