87

我将 Alembic 与 SQL Alchemy 一起使用。对于 SQL Alchemy,我倾向于遵循一种模式,即不将连接字符串与版本化代码一起存储。相反,我有secret.py包含任何机密信息的文件。我把这个文件名扔进我的.gitignore,所以它不会出现在 GitHub 上。

这种模式很好用,但现在我开始使用 Alembic 进行迁移。看来我无法隐藏连接字符串。而是在 alembic.ini 中,将连接字符串作为配置参数

# the 'revision' command, regardless of autogenerate
# revision_environment = false

sqlalchemy.url = driver://user:pass@localhost/dbname

# Logging configuration
[loggers]
keys = root,sqlalchemy,alembi

我担心我会不小心为我的数据库提交一个包含用户名/密码信息的文件。我宁愿将此连接字符串存储在一个地方,并避免意外将其提交给版本控制的风险。

我有什么选择?

4

12 回答 12

86

我昨天遇到了同样的问题,并找到了以下解决方案。我执行以下操作alembic/env.py

# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# this will overwrite the ini-file sqlalchemy.url path
# with the path given in the config of the main code
import config as ems_config
config.set_main_option('sqlalchemy.url', ems_config.config.get('sql', 'database'))

ems_config是一个保存我的配置数据的外部模块。

config.set_main_option(...)本质上会覆盖文件部分中的sqlalchemy.url键。在我的配置中,我只是将其保留为黑色。[alembic]alembic.ini

于 2014-12-02T18:45:20.080 回答
44

为了避免提交我的用户/通行证,我能想到的最简单的方法是 a) 将插值字符串添加到alembic.ini文件中,b) 将这些插值值设置为env.py

alembic.ini

sqlalchemy.url = postgresql://%(DB_USER)s:%(DB_PASS)s@35.197.196.146/nozzle-website

环境.py

import os

from logging.config import fileConfig

from sqlalchemy import engine_from_config
from sqlalchemy import pool

from alembic import context

# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# here we allow ourselves to pass interpolation vars to alembic.ini
# fron the host env
section = config.config_ini_section
config.set_section_option(section, "DB_USER", os.environ.get("DB_USER"))
config.set_section_option(section, "DB_PASS", os.environ.get("DB_PASS"))

...
于 2019-03-15T20:49:34.173 回答
31

Alembic 文档建议使用create_engine数据库 URL(而不是在代码中修改 sqlalchemy.url)。

您还应该修改 run_migrations_offline 以使用新的 URL。Allan Simon 在他的博客上有一个例子,但总而言之,将 env.py 修改为:

  1. 提供一个共享函数以某种方式获取 URL(这里它来自命令行):

    def get_url():
        url = context.get_x_argument(as_dictionary=True).get('url')
        assert url, "Database URL must be specified on command line with -x url=<DB_URL>"
        return url
    
  2. 在离线模式下使用 URL:

    def run_migrations_offline():
        ...
        url = get_url()
        context.configure(
            url=url, target_metadata=target_metadata, literal_binds=True)
        ...
    
  3. 通过使用create_engine而不是在线模式使用 URL engine_from_config

    def run_migrations_online():
        ...
        connectable = create_engine(get_url())
        with connectable.connect() as connection:
        ...
    
于 2017-01-09T11:24:31.677 回答
10

因此,似乎可行的是在 env.py 中重新实现引擎创建,这显然是进行这种自定义的地方,而不是在 ini 中使用 sqlalchemy 连接字符串:

engine = engine_from_config(
            config.get_section(config.config_ini_section),
            prefix='sqlalchemy.',
           poolclass=pool.NullPool)

您可以替换并指定自己的引擎配置:

import store
engine = store.engine

事实上,文档似乎暗示这是可以的:

sqlalchemy.url - 通过 SQLAlchemy 连接到数据库的 URL。这个键实际上只在特定于“通用”配置的 env.py 文件中被引用;开发人员可以自定义的文件。多数据库配置可能会在此处响应多个键,或者可能会引用文件的其他部分。

于 2014-03-04T17:38:00.947 回答
6

我一直在寻找如何为多数据库管理这个

这就是我所做的。我有两个数据库:日志ohlc

根据文档,我已经像这样设置了蒸馏器

alembic init --template multidb

alembic.ini

databases = logs, ohlc
[logs]
sqlalchemy.url = postgresql://botcrypto:botcrypto@localhost/logs
[ohlc]
sqlalchemy.url = postgresql://botcrypto:botcrypto@localhost/ohlc

环境.py

[...]
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)
logger = logging.getLogger('alembic.env')

# overwrite alembic.ini db urls from the config file
settings_path = os.environ.get('SETTINGS')
if settings_path:
    with open(settings_path) as fd:
        settings = conf.load(fd, context=os.environ) # loads the config.yml
    config.set_section_option("ohlc", "sqlalchemy.url", settings["databases"]["ohlc"])
    config.set_section_option("logs", "sqlalchemy.url", settings["databases"]["logs"])
else:
    logger.warning('Environment variable SETTINGS missing - use default alembic.ini configuration')
[...]

配置.yml

databases:
    logs: postgresql://botcrypto:botcrypto@127.0.0.1:5432/logs
    ohlc: postgresql://botcrypto:botcrypto@127.0.0.1:5432/ohlc

用法

SETTINGS=config.yml alembic upgrade head

希望它可以帮助!

于 2018-12-18T10:38:03.033 回答
3

对于MultiDB设置(SingleDB 相同),您可以使用config.set_section_option('section_name', 'variable_name', 'db_URL')修改 alembic.ini 文件中数据库 URL 的值。

例如:

alembic.init

[engine1]
sqlalchemy.url = 

[engine2]
sqlalchemy.url = 

然后,

环境.py

config = context.config

config.set_section_option('engine1', 'sqlalchemy.url', os.environ.get('URL_DB1'))
config.set_section_option('engine2', 'sqlalchemy.url', os.environ.get('URL_DB2'))
于 2021-06-02T16:08:41.030 回答
1

环境.py:

from alembic.config import Config

alembic_cfg = Config()
alembic_cfg.set_main_option("sqlalchemy.url", getenv('PG_URI'))

https://alembic.sqlalchemy.org/en/latest/api/config.html

于 2021-04-17T21:46:56.563 回答
0

另一种解决方案是创建模板 alembic.ini.dist 文件并使用您的版本化代码跟踪它,同时忽略 VCS 中的 alembic.ini。

不要在 alembic.ini.dist 中添加任何机密信息:

sqlalchemy.url = ...

将代码部署到平台时,将 alembic.ini.dist 复制到 alembic.ini(您的 VCS 不会跟踪此代码)并使用平台的凭据修改 alembic.ini。

于 2014-07-22T12:57:03.947 回答
0

我也遇到了这个问题,因为我们正在从本地机器运行迁移。我的解决方案是将环境部分放在alembic.ini存储数据库配置(减去凭据)中:

[local]
host = localhost
db = dbname

[test]
host = x.x.x.x
db = dbname

[prod]
host = x.x.x.x
db = dbname

然后我将以下内容放入其中,env.py以便用户可以选择他们的环境并提示输入凭据:

from alembic import context
from getpass import getpass

...

envs = ['local', 'test', 'prod']

print('Warning: Do not commit your database credentials to source control!')
print(f'Available migration environments: {", ".join(envs)}')

env = input('Environment: ')

if env not in envs:
    print(f'{env} is not a valid environment')
    exit(0)

env_config = context.config.get_section(env)
host = env_config['host']
db = env_config['db']

username = input('Username: ')
password = getpass()
connection_string = f'postgresql://{username}:{password}@{host}/{db}'

context.config.set_main_option('sqlalchemy.url', connection_string)

您应该将您的凭据存储在整个团队都可以访问的密码管理器中,或者您可以使用的任何配置/秘密存储中。但是,通过这种方法,密码会暴露给您的本地剪贴板 - 更好的方法是env.py直接连接到您的配置/秘密存储 API 并直接提取用户名/密码,但这会增加第三方依赖性。

于 2020-06-03T07:13:47.547 回答
0

正如 Doug T. 所说,您可以编辑 env.py 以从 ini 文件以外的其他位置提供 URL。url您可以向函数传递一个附加参数,而不是创建新引擎engine_from_config(kwargs 稍后会合并到从 ini 文件中获取的选项)。在这种情况下,您可以例如将加密的密码存储在 ini 文件中,并在运行时通过存储在 ENV 变量中的密码对其进行解密。

connectable = engine_from_config(                 
    config.get_section(config.config_ini_section),
    prefix='sqlalchemy.',                         
    poolclass=pool.NullPool,                      
    url=some_decrypted_endpoint)                                   
于 2016-04-21T11:53:46.117 回答
-1

只需env.py添加

config.set_main_option('sqlalchemy.url', os.environ['DB_URL'])

config = context.config

喜欢

config = context.config
config.set_main_option('sqlalchemy.url', os.environ['DB_URL'])

然后像这样执行:

DB_URL="mysql://atuamae:de4@127.0.0.1/db" \
  alembic upgrade head
于 2020-10-20T11:21:13.390 回答
-1

我已经在这里尝试了所有答案,但没有奏效。然后我尝试自己处理,如下:

.ini 文件:

# A generic, single database configuration.

[alembic]
# path to migration scripts
script_location = alembic

# template used to generate migration files
file_template = %%(rev)s_%%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d_%%(minute).2d_%%(second).2d

# timezone to use when rendering the date
# within the migration file as well as the filename.
# string value is passed to dateutil.tz.gettz()
# leave blank for localtime
# timezone =

# max length of characters to apply to the
# "slug" field
#truncate_slug_length = 40

# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false

# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false

# version location specification; this defaults
# to alembic/versions.  When using multiple version
# directories, initial revisions must be specified with --version-path
# version_locations = %(here)s/bar %(here)s/bat alembic/versions

# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8

databases = auth_engine


[auth_engine]
sqlalchemy.url = mysql+mysqldb://{}:{}@{}:{}/auth_db

# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARN
handlers = console
qualname =

[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine

[logger_alembic]
level = INFO
handlers =
qualname = alembic

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

.env 文件(它位于我项目的根文件夹中):

DB_USER='root'
DB_PASS='12345678'
DB_HOST='127.0.0.1'
DB_PORT='3306'

env.py 文件:

from __future__ import with_statement

import os
import re
import sys
from logging.config import fileConfig

from sqlalchemy import engine_from_config
from sqlalchemy import pool

from alembic import context

DB_USER = os.getenv("DB_USER")
DB_PASS = os.getenv("DB_PASS")
DB_HOST = os.getenv("DB_HOST")
DB_PORT = os.getenv("DB_PORT")

# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)

# gather section names referring to different
# databases.  These are named "engine1", "engine2"
# in the sample .ini file.
db_names = config.get_main_option('databases')

# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata

sys.path.append(os.path.join(os.path.dirname(__file__), "../../../"))
from db_models.auth_db import auth_db_base

target_metadata = {
    'auth_engine': auth_db_base.auth_metadata
}


# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.


def run_migrations_offline():
    """Run migrations in 'offline' mode.

    This configures the context with just a URL
    and not an Engine, though an Engine is acceptable
    here as well.  By skipping the Engine creation
    we don't even need a DBAPI to be available.

    Calls to context.execute() here emit the given string to the
    script output.

    """
    engines = {}
    for name in re.split(r',\s*', db_names):
        engines[name] = rec = {}
        section = context.config.get_section(name)
        url = section['sqlalchemy.url'].format(DB_USER, DB_PASS, DB_HOST, DB_PORT)
        section['sqlalchemy.url'] = url
        rec['url'] = url
        # rec['url'] = context.config.get_section_option(name, "sqlalchemy.url")

    for name, rec in engines.items():
        print("Migrating database %s" % name)
        file_ = "%s.sql" % name
        print("Writing output to %s" % file_)
        with open(file_, 'w') as buffer:
            context.configure(url=rec['url'], output_buffer=buffer,
                              target_metadata=target_metadata.get(name),
                              compare_type=True,
                              compare_server_default=True
                              )

            with context.begin_transaction():
                context.run_migrations(engine_name=name)


def run_migrations_online():
    """Run migrations in 'online' mode.

    In this scenario we need to create an Engine
    and associate a connection with the context.

    """
    engines = {}
    for name in re.split(r',\s*', db_names):
        engines[name] = rec = {}
        section = context.config.get_section(name)
        url = section['sqlalchemy.url'].format(DB_USER, DB_PASS, DB_HOST, DB_PORT)
        section['sqlalchemy.url'] = url
        rec['engine'] = engine_from_config(
            section,
            prefix='sqlalchemy.',
            poolclass=pool.NullPool)

    for name, rec in engines.items():
        engine = rec['engine']
        rec['connection'] = conn = engine.connect()
        rec['transaction'] = conn.begin()

    try:
        for name, rec in engines.items():
            print("Migrating database %s" % name)
            context.configure(
                connection=rec['connection'],
                upgrade_token="%s_upgrades" % name,
                downgrade_token="%s_downgrades" % name,
                target_metadata=target_metadata.get(name),
                compare_type=True,
                compare_server_default=True
            )
            context.run_migrations(engine_name=name)

        for rec in engines.values():
            rec['transaction'].commit()
    except:
        for rec in engines.values():
            rec['transaction'].rollback()
        raise
    finally:
        for rec in engines.values():
            rec['connection'].close()


if context.is_offline_mode():
    run_migrations_offline()
else:
    run_migrations_online()

Wish可以帮助别人。

于 2019-11-29T07:52:29.853 回答