Even in 64-bit mode, wherein kernel space is reserved at the top of the address space (i.e. 0xffffff8000xxxxxx), and even when there's no swapping, you can't access user space memory, unless you are in kernel mode acting on behalf of that user space process. The reason is because the addresses are all virtual, and you are relying on CR3 (a control register) to tell the MMU which physical pages belong to which process. So although in principle you have access to all memory in kernel mode, without CR3 you will not be able to figure out which pages belong to which process.
Thus, inside a system call, you can move data in and out of user mode memory (there's copyin/copyout for that, similar to Linux's copy_[from/to]_user) - and those also handle page faults, in case pages are actually swapped, as is the normal case. But that is only for the active user space memory - i.e. the active process. Sure, there are hacks to access other processes' CR3, but those are quite discouraged (unless you're authoring quality malware).