SVGs are great for Vectors, where you can take advantage of solid colors, pre-defined gradients, or vector-based content. Once you start requiring bitmap-embedded image data in an SVG, the SVG will get pretty large. There are of course ways to optimize the bitmap data in SVGs, but it's easier to keep those image types separate.
So for the image types below, there should be a third top-level item that says: "Is it a vector-based image?" --> SVG.