`spark_auto_mapper.data_types.unix_timestamp`¶

Module Contents¶

Base class for all Automapper data types

class spark_auto_mapper.data_types.unix_timestamp.AutoMapperUnixTimestampType(value)¶

Base class for all Automapper data types

Converts the value to a timestamp type in Spark

Parameters

value (spark_auto_mapper.type_definitions.defined_types.AutoMapperNumberInputType) – value
formats – (Optional) formats to use for trying to parse the value otherwise uses Spark defaults

get_column_spec(self, source_df, current_column)¶

Gets the column spec for this automapper data type

Parameters

source_df (Optional[pyspark.sql.DataFrame]) – source data frame in case the automapper type needs that data to decide what to do
current_column (Optional[pyspark.sql.Column]) – (Optional) this is set when we are inside an array

Return type

pyspark.sql.Column