If you are using Apache with mod_wsgi then you can use mod_xsendfile
You are essentially looking to run the authorisation for some resources via Django, pass a header back to Apache saying 'Hey dude, lighten up. This user is okay to access this' Apache will then handle returning the resource.
Rough steps (as in, rough enough that you will need to do a little more research using the links I provide as a starting point)
Apache needs to know which resources are public and which aren't. Create a sub directory under media for both of these types (Why not go crazxy and call them /media/public/ and /media/private/)
Set up an alias for the public directory and a WSGIScriptAlias for the protected dir, the protected alias will be pointing to your main site handler (probably django.wsgi)
Add settings to vhost:
XSendFile On
XSendFileAllowAbove On
Add an urlconf to your Django app that handles /media/protected/{whatever} and routes it through your auth Django app auth logic. An example of this is here
A useful snippet for the above is here
and another example for good measure here