spark_pipeline_framework.utilities.run_if_else¶
Module Contents¶
Functions¶
|
If enable flag is true then runs stages else runs else_stages |
- spark_pipeline_framework.utilities.run_if_else.run_if_else(enable: Union[bool, Callable[[pyspark.sql.dataframe.DataFrame], bool]], stages: Union[List[pyspark.ml.Transformer], Callable[[], List[pyspark.ml.Transformer]]], else_stages: Optional[Union[List[pyspark.ml.Transformer], Callable[[], List[pyspark.ml.Transformer]]]] = None) spark_pipeline_framework.transformers.framework_if_else_transformer.v1.framework_if_else_transformer.FrameworkIfElseTransformer¶
If enable flag is true then runs stages else runs else_stages :param enable: a boolean or a function that takes a DataFrame and returns a boolean :param stages: list of transformers or a function that returns a list of transformers :param else_stages: list of transformers or a function that returns a list of transformers