`spark_auto_mapper.data_types.hash`¶

Module Contents¶

Calculates the hash code of given columns, and returns the result as an int column.

class spark_auto_mapper.data_types.hash.AutoMapperHashDataType(*args)¶

Calculates the hash code of given columns, and returns the result as an int column.

Parameters: args (Union[spark_auto_mapper.type_definitions.native_types.AutoMapperNativeTextType, spark_auto_mapper.type_definitions.wrapper_types.AutoMapperWrapperType, spark_auto_mapper.data_types.text_like_base.AutoMapperTextLikeBase]) –

get_column_spec(self, source_df, current_column)¶

Gets the column spec for this automapper data type

Parameters

source_df (Optional[pyspark.sql.DataFrame]) – source data frame in case the automapper type needs that data to decide what to do
current_column (Optional[pyspark.sql.Column]) – (Optional) this is set when we are inside an array

Return type

pyspark.sql.Column