11

我最近尝试在 Spring 应用程序中使用新的 AWS DocumentDB 服务作为我的数据库。

集群已在与我部署应用程序的 EKS 相同的 VPC 中创建。安全组允许 VPC 中所有节点之间的连接。

AWS 为我的数据库集群公开了一个这样的 mongo URI:

mongodb://<my-user>:<insertYourPassword>@<my-cluster-endpoint>:27017/?ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0

我的问题:

如何使我的 Spring 代码与这种连接一起工作?

我尝试将以下内容添加到我的application.properties文件中:

spring.data.mongodb.uri=mongodb://<my-user>:<insertYourPassword>@<my-cluster-endpoint>:27017/admin?ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs00
spring.data.mongodb.database=admin
server.ssl.key-store=classpath:rds-combined-ca-bundle.pem

并将 PEM 文件放入/src/main/resources

但是,代码仍然无法连接到数据库集群。

我收到此消息作为错误:No server chosen by com.mongodb.client.internal.MongoClientDelegate

紧随其后的是Exception in monitor thread while connecting to server ...

最后是超时异常:com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message

它看起来有点像安全组问题,但我从运行 Spring 应用程序 Pod 的同一 EC2 连接 mongo shell 没有问题。

有任何想法吗?

4

6 回答 6

10

如文档中所述,

根据设计,您可以从与 Amazon DocumentDB 资源相同的 Amazon VPC 中的 Amazon EC2 实例访问 Amazon DocumentDB(与 MongoDB 兼容)资源。但是,假设您的使用案例要求您或您的应用程序从集群的 Amazon VPC 外部访问您的 Amazon DocumentDB 资源。在这种情况下,您可以使用 SSH 隧道(也称为“端口转发”)来访问您的 Amazon DocumentDB 资源。

从外部 VPC 连接

您的 Amazon DocumentDB 集群应该在您的默认虚拟私有云 (VPC) 中运行。要与您的 Amazon DocumentDB 集群交互,您必须在您的默认 VPC 中启动一个 Amazon Elastic Compute Cloud (Amazon EC2) 实例,该实例位于您创建 Amazon DocumentDB 集群的同一 AWS 区域中。

按照指南连接到集群 AWS DocumentDB 集群

GitHub 参考spring-boot-aws-documentdb

更新

要通过 SSL 连接,请通过将SSL_CERTIFICATE设置为指向 aws 区域特定的中间证书来使用以下逻辑。

这可以从SSL 证书下载并将其复制到基本目录。或者,您可以提供变量SSL_CERTIFICATE的绝对路径。

     private static final String SSL_CERTIFICATE = "rds-ca-2015-us-east-1.pem";
     private static final String KEY_STORE_TYPE = "JKS";
     private static final String KEY_STORE_PROVIDER = "SUN";
     private static final String KEY_STORE_FILE_PREFIX = "sys-connect-via-ssl-test-cacerts";
     private static final String KEY_STORE_FILE_SUFFIX = ".jks";
     private static final String DEFAULT_KEY_STORE_PASSWORD = "changeit";

    public static void main(String[] args) {
        SSLContextHelper.setSslProperties();
        SpringApplication.run(Application.class, args);
    }


    protected static class SSLContextHelper{
    /**
     * This method sets the SSL properties which specify the key store file, its type and password:
     * @throws Exception
     */
    private static void setSslProperties()  {

        try {
            System.setProperty("javax.net.ssl.trustStore", createKeyStoreFile());
        } catch (Exception e) {

            e.printStackTrace();
        }
        System.setProperty("javax.net.ssl.trustStoreType", KEY_STORE_TYPE);
        System.setProperty("javax.net.ssl.trustStorePassword", DEFAULT_KEY_STORE_PASSWORD);
    }


    private static String createKeyStoreFile() throws Exception {
        return createKeyStoreFile(createCertificate()).getPath();
    }

    /**
     *  This method generates the SSL certificate
     * @return
     * @throws Exception
     */
    private static X509Certificate createCertificate() throws Exception {
        CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
        URL url = new File(SSL_CERTIFICATE).toURI().toURL();
        if (url == null) {
            throw new Exception();
        }
        try (InputStream certInputStream = url.openStream()) {
            return (X509Certificate) certFactory.generateCertificate(certInputStream);
        }
    }

    /**
     * This method creates the Key Store File
     * @param rootX509Certificate - the SSL certificate to be stored in the KeyStore
     * @return
     * @throws Exception
     */
    private static File createKeyStoreFile(X509Certificate rootX509Certificate) throws Exception {
        File keyStoreFile = File.createTempFile(KEY_STORE_FILE_PREFIX, KEY_STORE_FILE_SUFFIX);
        try (FileOutputStream fos = new FileOutputStream(keyStoreFile.getPath())) {
            KeyStore ks = KeyStore.getInstance(KEY_STORE_TYPE, KEY_STORE_PROVIDER);
            ks.load(null);
            ks.setCertificateEntry("rootCaCertificate", rootX509Certificate);
            ks.store(fos, DEFAULT_KEY_STORE_PASSWORD.toCharArray());
        }
        return keyStoreFile;
    }


    }

连接输出

019-01-17 13:33:22.316  INFO 3598 --- [onaws.com:27017] org.mongodb.driver.cluster               : Canonical address mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017 does not match server address.  Removing mongodb.cluster-cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017 from client view of cluster
2019-01-17 13:33:22.401  INFO 3598 --- [onaws.com:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:2}] to mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017
2019-01-17 13:33:22.403  INFO 3598 --- [onaws.com:27017] org.mongodb.driver.cluster               : Monitor thread successfully connected to server with description ServerDescription{address=mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 0]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=null, roundTripTimeNanos=2132149, setName='rs0', canonicalAddress=mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017, hosts=[mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017], passives=[], arbiters=[], primary='mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017', tagSet=TagSet{[]}, electionId=7fffffff0000000000000001, setVersion=null, lastWriteDate=Thu Jan 17 13:33:21 UTC 2019, lastUpdateTimeNanos=516261208876}
2019-01-17 13:33:22.406  INFO 3598 --- [onaws.com:27017] org.mongodb.driver.cluster               : Discovered replica set primary mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017
2019-01-17 13:33:22.595  INFO 3598 --- [           main] com.barath.app.CustomerService           : Saving the customer with customer details com.barath.app.Customer@6c130c45
2019-01-17 13:33:22.912  INFO 3598 --- [           main] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:3}] to mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017
2019-01-17 13:33:23.936  INFO 3598 --- [           main] pertySourcedRequestMappingHandlerMapping : Mapped URL path [/v2/api-docs] onto method [public org.springframework.http.ResponseEntity<springfox.documentation.spring.web.json.Json> springfox.documentation.swagger2.web.Swagger2Controller.getDocumentation(java.lang.String,javax.servlet.http.HttpServletRequest)]
于 2019-01-17T08:02:51.173 回答
4

我可以确认@Barath 提供的解决方案允许您保护 Java 应用程序本身内部的 AWS DocumentDB TLS 连接。与 AWS 在https://docs.aws.amazon.com/documentdb/latest/developerguide/connect_programmatically.html上描述的方法相比,这是一种更简洁的方法,后者需要您在服务器上运行更复杂且更复杂的脚本难以自动化部署等。

为了在 Spring 应用程序中进一步设置连接本身,我使用了以下 @Configuration 类,它允许您连接到本地 MongoDB 以在开发期间进行测试,而 AWS 则使用属性文件中的设置进行部署。

import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.core.convert.converter.Converter;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;
import org.springframework.data.mongodb.core.convert.MongoCustomConversions;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;
    
@Configuration
@EnableMongoRepositories(basePackages = "YOUR.PACKAGE.WITH.repository")
public class MongoDbConfig extends AbstractMongoClientConfiguration {
    
    @Value("${spring.profiles.active}")
    private String activeProfile;
    
    @Value("${mongodb.host:localhost}")
    private String dbUri;
    @Value("${mongodb.port:27017}")
    private int dbPort;
    @Value("${mongodb.database.name:YOUR_DOCUMENTDB_NAME}")
    private String dbName;
    @Value("${mongodb.username:}")
    private String dbUser;
    @Value("${mongodb.password:}")
    private String dbPassword;
    
    @Override
    public String getDatabaseName() {
        return dbName;
    }
    
    @Override
    public MongoClient mongoClient() {
        ConnectionString connectionString = new ConnectionString(getConnectionString());
        MongoClientSettings mongoClientSettings = MongoClientSettings.builder()
               .applyConnectionString(connectionString)
               .build();
        return MongoClients.create(mongoClientSettings);
    }
    
    private String getConnectionString() {
        if (activeProfile.contains("local")) {
            return String.format("mongodb://%s:%s/%s", dbUri, dbPort, dbName);
        }
        return String.format("mongodb://%s:%s@%s:%s/%s?ssl=true&replicaSet=rs0&readpreference=secondaryPreferred&retrywrites=false",
                dbUser, dbPassword, dbUri, dbPort, dbName);
    }
}
于 2021-03-08T09:04:03.650 回答
3

@Sunny Pelletier提供的答案在我们的 Java 设置中与 @Frank 的答案混搭对我有用。

所以对我来说,我想要一个适用于我们的本地 docker 设置以及我们的任何 AWS 环境的解决方案,这些环境通过 CDK 在我们的环境中设置了活动配置文件和其他环境变量。

我首先从一个简单的配置 POJO 开始,在spring.data.mongo.*范式之外设置我的属性。您不必这样做,只需让 Spring 像通常那样处理它来创建MongoClient.

我的默认本地开发application.yml和相应的配置类。

mongo:
  user: mongo
  password: mongo
  host: localhost
  port: 27017
  database: my-service

@Data
@Configuration
@ConfigurationProperties(prefix = "mongo")
public class MongoConnectConfig {

    private int port;

    private String host;

    private String user;

    private String database;

    private String password;

}

然后,我创建了两个AbstractMongoClientConfiguration子类;一种用于本地,一种用于非本地。这里的关键是我没有创建自己的MongoClient. 原因是因为我想要你从框架中获得的所有好的 Spring Boot 初始化东西。例如,所有转换器的自动注册等。

相反,我利用提供的自定义钩子AbstractMongoClientConfiguration.configureClientSettings(MongoClientSettings.Builder builder)来聚合自定义设置,如.pem作品。

另一部分是我利用配置文件来启用/禁用配置,以使其对本地开发人员“无缝”;除了用于本地开发之外,我们不使用任何配置文件default,因此无需从一开始就“了解”太多,就可以更轻松地进行设置。

import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;

@Slf4j
@Configuration
@RequiredArgsConstructor
@Profile({"!dev && !qa && !prod"})
@EnableMongoRepositories(basePackages = "co.my.data.repositories")
public class LocalDevMongoConfig extends AbstractMongoClientConfiguration {
    
    private final MongoConnectConfig config;
    
    @Override
    public String getDatabaseName() {
        return config.getDatabase();
    }
    
    @Override
    protected void configureClientSettings(MongoClientSettings.Builder builder) {
        log.info("Applying Local Dev MongoDB Configuration");
        builder.applyConnectionString(new ConnectionString(getConnectionString()));
    }

    //mongodb://${mongo.user}:${mongo.password}@${mongo.host}:${mongo.port}/${mongo.database}?authSource=admin
    private String getConnectionString() {
        return String.format("mongodb://%s:%s@%s:%s/%s?authSource=admin",
                config.getUser(),
                config.getPassword(),
                config.getHost(),
                config.getPort(),
                config.getDatabase()
        );
    }
}


import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;

import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.nio.file.Files;
import java.security.KeyStore;
import java.security.cert.CertificateFactory;
import java.util.Arrays;
import java.util.stream.Collectors;

@Slf4j
@Configuration
@RequiredArgsConstructor
@Profile({"dev || qa || prod"})
@EnableMongoRepositories(basePackages = "co.my.data.repositories")
public class DocumentDbMongoConfig extends AbstractMongoClientConfiguration {

    private final MongoConnectConfig config;

    @Override
    public String getDatabaseName() {
        return config.getDatabase();
    }

    @SneakyThrows
    @Override
    protected void configureClientSettings(MongoClientSettings.Builder builder) {
        log.info("Applying AWS DocumentDB Configuration");
        builder.applyConnectionString(new ConnectionString(getConnectionString()));
        var endOfCertificateDelimiter = "-----END CERTIFICATE-----";
        File resource = new ClassPathResource("certs/rds-combined-ca-bundle.pem").getFile();
        String pemContents = new String(Files.readAllBytes(resource.toPath()));
        var allCertificates = Arrays.stream(pemContents
                .split(endOfCertificateDelimiter))
                .filter(line -> !line.isBlank())
                .map(line -> line + endOfCertificateDelimiter)
                .collect(Collectors.toUnmodifiableList());


        var certificateFactory = CertificateFactory.getInstance("X.509");
        var keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        // This allows us to use an in-memory key-store
        keyStore.load(null);

        for (int i = 0; i < allCertificates.size(); i++) {
            var certString = allCertificates.get(i);
            var caCert = certificateFactory.generateCertificate(new ByteArrayInputStream(certString.getBytes()));
            keyStore.setCertificateEntry(String.format("AWS-certificate-%s", i), caCert);
        }

        var trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        trustManagerFactory.init(keyStore);

        var sslContext = SSLContext.getInstance("TLS");
        sslContext.init(null, trustManagerFactory.getTrustManagers(), null);

        builder.applyToSslSettings(ssl -> {
            ssl.enabled(true).context(sslContext);
        });
    }

    /**
     * Partly based on the AWS Console "Connectivity & security " section in the DocumentDB Cluster View.
     *   Since we register the pem above, we don't need to add the ssl & sslCAFile piece
     *   mongodb://${user}:${password}@${host}:${port}/?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false
     */
    private String getConnectionString() {
        return String.format("mongodb://%s:%s@%s:%s/%s?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false",
                config.getUser(),
                config.getPassword(),
                config.getHost(),
                config.getPort(),
                config.getDatabase()
        );
    }
}

最后,我们将其rds-combined-ca-bundle.pem放入src/main/resources/certs/文件夹中。

旁注:

  • 同样,我相信您应该能够摆脱使用默认spring.data*属性并且您MongoClient应该使用它们。
  • 忽略@SneakyThrows这里,我只是为了代码简洁的目的,按照你认为合适的方式处理你的检查异常。
  • 我想我们可以看到为什么 Kotlin 语法可以被认为是“更干净”的吧?:)
于 2021-07-15T15:32:48.970 回答
3

我实际上遇到了与您相同的问题,但现在 AWS 使用rds-combined-ca-bundle.pem将许多证书组合在一起。

如果您不想使用他们过时的文档创建信任库,您可以自己创建并rds-combined-ca-bundle.pem让应用程序在运行时生成密钥库。

我设法让它与这个代码示例一起工作。这已经通过spring:2.4,mongo-driver: 4.1.1和 documentDB 的mongo 4.0兼容性进行了测试。

val endOfCertificateDelimiter = "-----END CERTIFICATE-----"

// rds-combined-ca-bundle.pem contains more than one certificate. We need to add them all to the trust-store independantly. 

val allCertificates = ClassPathResource("certificates/rds-combined-ca-bundle.pem").file.readText()
    .split(endOfCertificateDelimiter)
    .filter { it.isNotBlank() }
    .map { it + endOfCertificateDelimiter }

val certificateFactory = CertificateFactory.getInstance("X.509")
val keyStore = KeyStore.getInstance(KeyStore.getDefaultType())
keyStore.load(null) // This allows us to use an in-memory key-store

allCertificates.forEachIndexed { index, certificate ->
    val caCert = certificateFactory.generateCertificate(certificate.byteInputStream()) as X509Certificate
    keyStore.setCertificateEntry("AWS-certificate-$index", caCert)
}

val trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm())
trustManagerFactory.init(keyStore)

val sslContext = SSLContext.getInstance("TLS")
sslContext.init(null, trustManagerFactory.trustManagers, null)

builder.applyToSslSettings {
    it.enabled(true)
        .context(sslContext)
}
于 2021-03-22T17:59:47.990 回答
0

这是一个对我有用的解决方案,只需在连接到 documentdb 之前调用 setSslProperties 方法。

    /**
 * This method sets the SSL properties which specify the key store file, its type and password.
 *
 * @throws Exception
 */
private static void setSslProperties() throws Exception {

    System.setProperty("javax.net.ssl.trustStore", createKeyStoreFile());
    System.setProperty("javax.net.ssl.trustStoreType", KEY_STORE_TYPE);
    System.setProperty("javax.net.ssl.trustStorePassword", DEFAULT_KEY_STORE_PASSWORD);
}

/**
 * This method returns the path of the Key Store File needed for the SSL verification during the IAM Database Authentication to
 * the db instance.
 *
 * @return
 * @throws Exception
 */
private static String createKeyStoreFile() throws Exception {
    return createKeyStoreFile(createCertificate()).getPath();
}

/**
 * This method generates the SSL certificate.
 *
 * @return
 * @throws Exception
 */
private static X509Certificate createCertificate() throws Exception {
    final CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
    final ClassLoader classLoader = MyClass.class.getClassLoader();
    final InputStream is = classLoader.getResourceAsStream(SSL_CERTIFICATE);
    return (X509Certificate) certFactory.generateCertificate(is);

}

/**
 * This method creates the Key Store File.
 *
 * @param rootX509Certificate - the SSL certificate to be stored in the KeyStore
 * @return
 * @throws Exception
 */
private static File createKeyStoreFile(final X509Certificate rootX509Certificate) throws Exception {
    final File keyStoreFile = File.createTempFile(KEY_STORE_FILE_PREFIX, KEY_STORE_FILE_SUFFIX);
    try (final FileOutputStream fos = new FileOutputStream(keyStoreFile.getPath())) {
        final KeyStore ks = KeyStore.getInstance(KEY_STORE_TYPE, KEY_STORE_PROVIDER);
        ks.load(null);
        ks.setCertificateEntry("rootCaCertificate", rootX509Certificate);
        ks.store(fos, DEFAULT_KEY_STORE_PASSWORD.toCharArray());
    }
    return keyStoreFile;
}

这是常数。

public static final String SSL_CERTIFICATE = "rds-ca-2019-root.pem";
public static final String KEY_STORE_TYPE = "JKS";
public static final String KEY_STORE_PROVIDER = "SUN";
public static final String KEY_STORE_FILE_PREFIX = "sys-connect-via-ssl-test-cacerts";
public static final String KEY_STORE_FILE_SUFFIX = ".jks";
public static final String DEFAULT_KEY_STORE_PASSWORD = "changeit";

这是 rds-ca-2019-root.pem 文件的链接,该文件位于该文件 inder 资源文件夹中。

让我知道这对你有用。

这是一个示例

       setSslProperties();
        final MongoCredential credential = MongoCredential.createCredential(userName, mongoProps.getDatabaseName(), password.toCharArray());
        final MongoClientSettings settings = MongoClientSettings.builder()
                .credential(credential)
                .readPreference(ReadPreference.secondaryPreferred())
                .retryWrites(false)
                .applyToSslSettings(builder -> builder.enabled(true))
                .applyToConnectionPoolSettings(connPoolBuilder ->
                        ConnectionPoolSettings.builder().
                                maxSize(1).build())
                .applyToClusterSettings(builder ->
                        builder.hosts(Arrays.asList(new ServerAddress(clusterEndPoint, 27017))))
                .build();
        mongoClient = MongoClients.create(settings);
于 2021-09-28T03:44:01.940 回答
-3

简单的解决方案是您可以删除 AWS 中的 TLS (SSL) 选项,然后您可以从连接字符串中删除“ssl_ca_certs=rds-combined-ca-bundle.pem”。但是如果应用程序需要 SSL DB 连接,那么您可以使用 AWS 指南

于 2020-03-19T07:10:01.787 回答