Having issues with kedro. The 'register_pipelines' function doesn't seem to be running or creating the default Pipeline that I'm returning from it.
The error is
(kedro-environment) C:\Users\cc667216\OneDrive\DCS_Pipeline\dcs_files>kedro run
2021-03-22 13:30:28,201 - kedro.framework.session.store - INFO - `read()` not implemented for `BaseSessionStore`. Assuming empty store.
fatal: not a git repository (or any of the parent directories): .git
2021-03-22 13:30:28,447 - kedro.framework.session.session - WARNING - Unable to git describe C:\Users\cc667216\OneDrive\DCS_Pipeline\dcs_files
2021-03-22 13:30:28,476 - root - INFO - ** Kedro project dcs_files
2021-03-22 13:30:28,486 - kedro.framework.session.store - INFO - `save()` not implemented for `BaseSessionStore`. Skipping the step.
Traceback (most recent call last):
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\kedro\framework\context\context.py", line 304, in _get_pipeline
return pipelines[name]
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\dynaconf\utils\functional.py", line 17, in inner
return func(self._wrapped, *args)
KeyError: '__default__'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Anaconda3\envs\kedro-environment\Scripts\kedro-script.py", line 9, in <module>
sys.exit(main())
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\kedro\framework\cli\cli.py", line 228, in main
cli_collection(**cli_context)
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "C:\Users\cc667216\OneDrive\DCS_Pipeline\dcs_files\src\dcs_package\cli.py", line 240, in run
pipeline_name=pipeline,
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\kedro\framework\session\session.py", line 344, in run
pipeline = context._get_pipeline(name=pipeline_name)
File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\kedro\framework\context\context.py", line 310, in _get_pipeline
) from exc
kedro.framework.context.context.KedroContextError: Failed to find the pipeline named '__default__'. It needs to be generated and returned by the 'register_pipelines' function.
My src\dcs_package\pipeline_registry.py looks like this:
# Copyright 2021 QuantumBlack Visual Analytics Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND
# NONINFRINGEMENT. IN NO EVENT WILL THE LICENSOR OR OTHER CONTRIBUTORS
# BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF, OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#
# The QuantumBlack Visual Analytics Limited ("QuantumBlack") name and logo
# (either separately or in combination, "QuantumBlack Trademarks") are
# trademarks of QuantumBlack. The License does not grant you any right or
# license to the QuantumBlack Trademarks. You may not use the QuantumBlack
# Trademarks or any confusingly similar mark as a trademark for your product,
# or use the QuantumBlack Trademarks in any other manner that might cause
# confusion in the marketplace, including but not limited to in advertising,
# on websites, or on software.
#
# See the License for the specific language governing permissions and
# limitations under the License.
"""Project pipelines."""
from typing import Dict
from kedro.pipeline import Pipeline, node
from .pipelines.data_processing.pipeline import create_pipeline
import logging
def register_pipelines() -> Dict[str, Pipeline]:
"""Register the project's pipelines.
Returns:
A mapping from a pipeline name to a ``Pipeline`` object.
"""
log = logging.getLogger(__name__)
log.info("Start register_pipelines")
data_processing_pipeline = create_pipeline()
log.info("create pipeline done")
return {
"__default__": data_processing_pipeline,
"dp": data_processing_pipeline
}
Then I have a "src\dcs_package\pipelines\data_processing\pipeline.py" file with a real simple function that outputs "test string" and nothing else.
I was able to read a few items from my catalog (a csv and a xlsx) so I think all the dependencies are working fine.