Your first attempt (number 1) is the best way to focus on getting this to work. The code sample Marc provided uses the built-in App Engine App Identity module to securely identify the App Engine app to other Google APIs: https://developers.google.com/appengine/docs/java/appidentity/#Asserting_Identity_to_Google_APIs
However, there are 2 gotchas to check:
The App Identity module will not work using the local devserver. When running locally there is no way for the local server to securely identity itself to Google production APIs - so you should either look at using a service account with a downloadable key for local testing, or, easiest option (and what I'd recommend for now), is to deploy into a production App Engine environment to test.
Make sure the App Engine service account email address is added to the access control list for the Google Cloud Storage bucket that you are trying to access. Get this email address by looking under the Admin Console (for your App Engine app) -> Application Settings:
Service Account Name:
xxxxxxx@appspot.gserviceaccount.com
To add this to a Google Cloud Storage access control list, either add it to the project team for the project that owns the Google Cloud Storage bucket, or, if you're not using the default access control settings, add it directly to an ACL:
https://developers.google.com/storage/docs/accesscontrol
Service accounts with downloadable key - use this for local testing if you can't test in a production App Engine environment:
https://developers.google.com/accounts/docs/OAuth2ServiceAccount