1

1个问题

我正在通过 tff.learning.build_federated_averaging_process() 生成一个迭代过程。并收到错误：


    Traceback (most recent call last):
      File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\IPython\core\interactiveshell.py", line 3331, in run_code
        exec(code_obj, self.user_global_ns, self.user_ns)
      File "<ipython-input-2-47998fd56829>", line 1, in <module>
        runfile('B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/train_v04.py', args=['--experiment_name=temp', '--client_batch_size=20', '--client_optimizer=sgd', '--client_learning_rate=0.2', '--server_optimizer=sgd', '--server_learning_rate=1.0', '--total_rounds=200', '--rounds_per_eval=1', '--rounds_per_checkpoint=50', '--rounds_per_profile=0', '--root_output_dir=B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/logs/fed_out/'], wdir='B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection')
      File "B:\tools and software\PyCharm 2020.1\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
        pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
      File "B:\tools and software\PyCharm 2020.1\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
        exec(compile(contents+"\n", file, 'exec'), glob, loc)
      File "B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/train_v04.py", line 306, in <module>
        app.run(main)
      File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\absl\app.py", line 299, in run
        _run_main(main, args)
      File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\absl\app.py", line 250, in _run_main
        sys.exit(main(argv))
      File "B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/train_v04.py", line 299, in main
        train_main()
      File "B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/train_v04.py", line 262, in train_main
        server_optimizer_fn=server_optimizer_fn,
      File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\learning\federated_averaging.py", line 211, in build_federated_averaging_process
        stateful_delta_aggregate_fn, stateful_model_broadcast_fn)
      File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\learning\framework\optimizer_utils.py", line 498, in build_model_delta_optimizer_process
        py_typecheck.check_callable(model_fn)
      File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\common_libs\py_typecheck.py", line 106, in check_callable
        type_string(type(target))))
    TypeError: Expected a callable, found non-callable tensorflow_federated.python.learning.model_utils.EnhancedModel.

突出显示：

in build_federated_averaging_process
    stateful_delta_aggregate_fn, stateful_model_broadcast_fn)

和

TypeError: Expected a callable, found non-callable tensorflow_federated.python.learning.model_utils.EnhancedModel.

2人试过

  1. 在这里查看了另一个类似的问题， 试图使 model_fn 成为 collection.abc 可调用的，model_fn=Callable[[], model_fn] 只会创建一个新错误。

3 一些代码：

  • 迭代过程：

    

 model_fn = model_builder(input_dim=sysarg,
                             input_spec=input_spec)
    iterative_process = tff.learning.build_federated_averaging_process(
        model_fn=model_fn,
        client_optimizer_fn=client_optimizer_fn,
        server_optimizer_fn=server_optimizer_fn,
    )
    iterative_process = compression_process_adapter.CompressionProcessAdapter(iterative_process)```

  • 模型生成器：


    def model_builder(input_dim, input_spec):
           model = create_model(input_dim)
            return tff.learning.from_keras_model(keras_model=model,
                                                 loss=tf.keras.losses.MeanSquaredError(),
                                                 input_spec=input_spec,
                                                 metrics=[tf.keras.metrics.Accuracy()],
                                                 )
  • 创建模型（良好的衡量标准）

    def create_model(input_dim):
           autoencoder = Sequential([
                tf.keras.layers.Dense(int(0.75 * input_dim), activation="tanh", input_shape=(input_dim,)),
                tf.keras.layers.Dense(int(0.5 * input_dim), activation="tanh"),
                tf.keras.layers.Dense(int(0.33 * input_dim), activation="tanh"),
                tf.keras.layers.Dense(int(0.25 * input_dim), activation="tanh"),
                tf.keras.layers.Dense(int(0.33 * input_dim), activation="tanh"),
                tf.keras.layers.Dense(int(0.5 * input_dim), activation="tanh"),
                tf.keras.layers.Dense(int(0.75 * input_dim), activation="tanh"),
                tf.keras.layers.Dense(input_dim)
                ])
4

1 回答 1

2

的model_fn参数tff.learning.build_federated_averaging_process需要是一个可调用的（什么是可调用的？），它不接受任何参数并返回一个tff.learning.Model。

从代码中（为了便于阅读，在此处复制）：

def model_builder(input_dim, input_spec):
  model = create_model(input_dim)
  return tff.learning.from_keras_model(
    keras_model=model,
    loss=tf.keras.losses.MeanSquaredError(),
    input_spec=input_spec,
    metrics=[tf.keras.metrics.Accuracy()])

model_fn = model_builder(input_dim=sysarg, input_spec=input_spec)

iterative_process = tff.learning.build_federated_averaging_process(
  model_fn=model_fn,
  client_optimizer_fn=client_optimizer_fn,
  server_optimizer_fn=server_optimizer_fn)

model_fn实际上是 的实例，tff.learning.Model而不是构造和返回模型的可调用对象。

model_builder应该传递给tff.learning.build_federated_averaging_process，但由于它需要参数，因此无法按原样工作。

一种替代方法是使用functools.partial：

iterative_process = tff.learning.build_federated_averaging_process(
  model_fn=functools.partial(model_builder, input_dim=sysarg, input_spec=input_spec),
  client_optimizer_fn=client_optimizer_fn,
  server_optimizer_fn=server_optimizer_fn)

甚至使用无参数 lambda：

iterative_process = tff.learning.build_federated_averaging_process(
  model_fn=lambda: model_builder(input_dim=sysarg, input_spec=input_spec),
  client_optimizer_fn=client_optimizer_fn,
  server_optimizer_fn=server_optimizer_fn)

以下问题对 lambda 和 partials 之间的区别进行了很好的讨论：Python：为什么 functools.partial 是必要的？

于 2020-06-16T01:53:58.497 回答