While it is true that subprocess.call does fork the process, and this child process does have its own memory space that is initially identical to the parent process (your python program), modern operating systems will use copy-on-write memory. The memory overhead of the forked process is initially relatively small, only requiring a few KB of memory in the kernel for process accounting. It's not until that child process starts making changes to its memory that extra memory is required.
After forking, the child process spawned by subprocess.call will call one of the exec system calls, which loads ffmpeg into memory and starts executing it.
Furthermore, fork is usually the only mechanism to create a new process on a POSIX system (see here), so I don't think there's an alternative to the fork-then-exec sequence that subprocess.call implements.
You may try running your program through strace or Valgrind to see exactly what system call is not getting the memory it requests. This may help you determine how to lower your memory requirements.