You are correct that getting CloudKit data via the server-to-server key or a JS API token can only get you access to the public database. User authentication is required for getting anything in a private or shared database.
This is sensible because your application wouldn't know which private database to query if it didn't have a CloudKit user as a reference. It would also be less secure if an anonymous user (even if it's your application) could query private user data.
As a workaround, you could have the iOS/macOS app push certain parts of the users' data that you want available on the web into your public database. You could make a record type in your public database of something like Web
and create fields for all the data that you need. This shared data would then be available, but everything would be available to anyone. The user would still have to identify themselves in your web app to get certain data (which could be done if you had a userId
field in your Web
record type).
Hopefully that makes sense. Let me know if you have follow-up questions.