You're correct that "Streaming Media" is a bit of an overloaded term. I tend to think that the vast majority of streaming media content is delivered via vanilla HTTP requests.
"I just noticed that Firefox now performs this magic automatically. Why did it take 20 years to add this rather obvious feature?"
I think that many browsers have had the capability to play at least simple audio formats natively for sometime (I think versions of Netscape from 1995 would handle some plain PCM WAV, AIFF, and SND files). About being able to handle MP3 natively, there were longstanding legal, licensing, and patent battles that are still in process. That adds to the friction. By now, I think most of the major browsers can handle MP3 audio natively via the audio tag.
Regarding seeking: A sufficiently intelligent client can do that via plain HTTP. If the user issues a seek request and the portion has not been downloaded yet, the client could close the HTTP connection, create a new one, and request a certain range of bytes. And that's only if the entire file hasn't been downloaded already. It's possible that Media Element already does something like this.
In your playlist example, the 10 tracks should be separate files, rather than squashed together into one big file. Once playback finished on the first track, the JavaScript can receive a signal that tells it to update the UI and request the second file for playback. If the user elects to skip to track 10 while playing track 1, then the client just requests track 10.
I hope I've helped. I know the feeling you're expressing-- not sure about the right question to ask in the first place.