UPDATE: This question may be less relevant now. As of January 2014, Angular plans to move from their own E2E suite to the protractor library for function tests, which use Selenium Webdriver.
If you're starting a new Angular project, you may want to look into using Protractor, as it is going to replace the current method of E2E Testing in the near future.
(Source: http://docs.angularjs.org/guide/dev_guide.e2e-testing)
I am struggling with this question as well.
Here is my thinking today:
1) Use Angular e2e testing for integration/regression testing the UI with mocks of your external dependencies (like the app server).
2) Use Selenium for integration/regression testing the complete system in a test/qa environment.
It seems Angular e2e testing helps fill in the gaps that unit testing leaves in testing view and controller logic that is tied to UI events. All of the Angular tests should mock external dependencies.
Selenium seems like the best fit for testing the entire system as the user will experience it.