Issue
When deploying a Ktor Kotlin application to AppEngine per the Ktor tutorial, the Firestore server authentication is not working, thus data is not being written to the specified Firestore database.
Data is written as expected to Firestore both when the app is run directly in the IntelliJ IDE as well as when it is run with ktor's implementation via the gradle appengineRun command.
There are two sets of AppEngine/Firebase projects for both a staging and production environment. Prior to deploying with the gradle appengineDeploy command the correct SDK configuration been activated and verified via the command gcloud config configurations list.
The strange part is that a few of the apps deployed with these strategies did write to Firestore, however upon deploying the app again Firestore did not show new data being written to it.
Implementation
Ktor Setup
I have the standard ktor required files. I also have an old MANIFEST.MF file from an older implementation. Could that be causing issues?
src/main/resources/application.conf
ktor {
application {
modules = [ Initialization.main ]
}
}
src/main/resources/webapp/WEB-INF/
appengine-web.xml
<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<threadsafe>true</threadsafe>
<runtime>java8</runtime>
</appengine-web-app>
web.xml
<?xml version="1.0" encoding="ISO-8859-1" ?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" version="3.0">
<servlet>
<display-name>KtorServlet</display-name>
<servlet-name>KtorServlet</servlet-name>
<servlet-class>io.ktor.server.servlet.ServletApplicationEngine</servlet-class>
<!-- path to application.conf file, required -->
<init-param>
<param-name>io.ktor.config</param-name>
<param-value>application.conf</param-value>
</init-param>
</servlet>
<servlet-mapping>
<servlet-name>KtorServlet</servlet-name>
<url-pattern>/</url-pattern>
</servlet-mapping>
logging.properties
.level = INFO
src/main/META-INF/MANIFEST>MF
Manifest-Version: 1.0
Main-Class: Initialization
Dependencies
For authentication strategies outlined below #1-3 the Firebase Admin library is used: compile 'com.google.firebase:firebase-admin:6.5.0'
For authentication strategy #4 the Google Cloud Firestore library is used: compile 'com.google.cloud:google-cloud-firestore:0.58.0-beta'
build.gradle
group 'coinverse'
version '1.0-SNAPSHOT'
buildscript {
ext.kotlin_version = '1.2.61'
ext.junitJupiterVersion = '5.0.3'
ext.ktor_version = '0.9.4'
ext.appengine_version = '1.9.60'
ext.appengine_plugin_version = '1.3.4'
repositories {
mavenCentral()
jcenter()
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
classpath 'org.junit.platform:junit-platform-gradle-plugin:1.0.3'
classpath "com.google.cloud.tools:appengine-gradle-plugin:$appengine_plugin_version"
}
}
apply plugin: 'java'
apply plugin: 'kotlin'
apply plugin: 'war'
apply plugin: 'com.google.cloud.tools.appengine'
sourceSets {
main.kotlin.srcDirs = [ 'src/main/kotlin' ]
}
sourceCompatibility = 1.8
repositories {
mavenCentral()
jcenter()
maven { url "https://kotlin.bintray.com/ktor" }
}
dependencies {
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
testCompile group: 'junit', name: 'junit', version: '4.12'
testCompile("org.junit.jupiter:junit-jupiter-api:${junitJupiterVersion}")
testRuntime("org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}")
testCompile("org.assertj:assertj-core:3.10.0")
testCompileOnly('org.apiguardian:apiguardian-api:1.0.0')
compile 'com.squareup.retrofit2:retrofit:2.3.0'
compile 'com.squareup.retrofit2:converter-gson:2.3.0'
compile 'com.squareup.retrofit2:adapter-rxjava:2.3.0'
compile 'io.reactivex.rxjava2:rxjava:2.2.0'
compile 'com.google.cloud:google-cloud-firestore:0.58.0-beta'
// Or compile 'com.google.cloud:google-cloud-firestore:0.58.0-beta'
compile 'com.google.firebase:firebase-admin:6.5.0'
compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
compile "io.ktor:ktor-server-servlet:$ktor_version"
compile "io.ktor:ktor-html-builder:$ktor_version"
providedCompile "com.google.appengine:appengine:$appengine_version"
}
kotlin.experimental.coroutines = 'enable'
compileKotlin {
kotlinOptions.jvmTarget = "1.8"
}
compileTestKotlin {
kotlinOptions.jvmTarget = "1.8"
}
task run(dependsOn: appengineRun)
appengine {
deploy {
version = 'price-staging-1021653pm'
stopPreviousVersion = false
}
}
Initializing Firebase Strategies
1. Initialize on Google Cloud Platform
This method is promising as credentials are managed automatically.
// Use the application default credentials
GoogleCredentials credentials = GoogleCredentials.getApplicationDefault();
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(credentials)
.setProjectId(projectId)
.build();
FirebaseApp.initializeApp(options);
Firestore db = FirestoreClient.getFirestore();
2. Initialize on your own server
I've confirmed in GCPs IAM & admin > Service accounts that the key id's match with the Json object being used to authenticate.
I'm using this strategy successfully in another Firestore connected app deployed to AppEngine. The working app is built as a .Jar and deployed directly to AppEngine without using ktor, but rather with the steps outlined here.
// Use a service account
InputStream serviceAccount = new FileInputStream("path/to/serviceAccount.json");
GoogleCredentials credentials = GoogleCredentials.fromStream(serviceAccount);
FirebaseOptions options = new FirebaseOptions.Builder()
.setCredentials(credentials)
.build();
FirebaseApp.initializeApp(options);
Firestore db = FirestoreClient.getFirestore();
In my working .Jar built app I'm passing in the Json object programmatically to avoid issues with the file not being found. I tried the same programmatic implementation for this ktor application. It worked with gradle appengineRun but not when deployed.
val credentials = GoogleCredentials.fromStream(Gson().toJson(FirebaseCredentials(
"service_account",
"project-name",
"asdfghjkl",
"keyStringHere",
"firebase-adminsdk-dhr30@project-name.iam.gserviceaccount.com",
"1234567890",
"https://accounts.google.com/o/oauth2/auth",
"https://oauth2.googleapis.com/token",
"https://www.googleapis.com/oauth2/v1/certs",
"https://www.googleapis.com/robot/v1/metadata/x509/firebase-adminsdk-dhr30%40project-name-staging.iam.gserviceaccount.com"
)).byteInputStream())
val options = FirebaseOptions.Builder()
.setCredentials(credentials)
.setDatabaseUrl("https://project-name-staging.firebaseio.com")
.build()
FirebaseApp.initializeApp(options)
3. Initialize on your own server (Firebase console setup)
The only difference between #2 is this setup adds .setDatabaseUrl("https://yourProjectName.firebaseio.com").
FirestoreOptions firestoreOptions =
FirestoreOptions.getDefaultInstance().toBuilder()
.setProjectId(projectId)
.build();
Firestore db = firestoreOptions.getService();
Accessing Firestore Object
For #1-3 the Firebase app is initialized right away in the application's main() method. Then, the Firestore object is accessed from an object.
FirebaseClient.Kt
object FirebaseClient {
val firestore: Firestore
init {
firestore = FirestoreClient.getFirestore()
}
}
For #4 the Firestore object is created in the Kotlin object's init{...} and stored in the object as a value.
FirebaseClient.Kt
object FirebaseClient {
val firestore: Firestore
init {
val firestoreOptions = FirestoreOptions.getDefaultInstance().toBuilder()
.setTimestampsInSnapshotsEnabled(true)
.setProjectId("project-name")
.build()
firestore = firestoreOptions.service
}
}
Writing to Firestore
FirebaseClient.firestore.collection(someCollection).document(someDocument).collection(anotherCollection).add(someObject)