spark_auto_mapper.data_types.complex.complex_base

Module Contents

Classes

AutoMapperDataTypeComplexBase

Base class for all Automapper data types

class spark_auto_mapper.data_types.complex.complex_base.AutoMapperDataTypeComplexBase(**kwargs)

Bases: spark_auto_mapper.data_types.data_type_base.AutoMapperDataTypeBase

Base class for all Automapper data types

base class for complex types :param kwargs:

Parameters

kwargs (Any) –

include_null_properties(self, include_null_properties)

Parameters

include_null_properties (bool) –

Return type

None

get_column_spec(self, source_df, current_column)

Gets the column spec for this automapper data type

Parameters

• source_df (Optional[pyspark.sql.DataFrame]) – source data frame in case the automapper type needs that data to decide what to do

• current_column (Optional[pyspark.sql.Column]) – (Optional) this is set when we are inside an array

Return type

pyspark.sql.Column

get_child_mappers(self)

Return type

Dict[str, spark_auto_mapper.data_types.data_type_base.AutoMapperDataTypeBase]

get_schema(self, include_extension)

Parameters

include_extension (bool) –

Return type

Optional[Union[pyspark.sql.types.StructType, pyspark.sql.types.DataType]]