create empty partition
Create empty partitions for partition values not yet existing
Creates the read schema based on a given write schema.
Delete given partitions.
Returns the factory that can parse this type (that is, type
the factory (object) for this class.
Handle class cast exception when getting objects from instance registry
A unique identifier for this instance.
Initialize callback before writing data out to disk/sinks.
list hive table partitions
Additional metadata for the DataObject
Definition of partition columns
Runs operations after reading from DataObject
Runs operations after writing to DataObject
Runs operations before reading from DataObject
Runs operations before writing to DataObject
Prepare & test DataObject's prerequisits
An optional, minimal schema that a DataObject schema must have to pass schema validation.
Validate the schema of a given Spark Data Frame
The data frame to validate.
SchemaViolationException is the
schemaMin does not validate.
Writes DataFrame to HDFS/Parquet and creates Hive table.
Write Spark structured streaming DataFrame The default implementation uses foreachBatch and this traits writeDataFrame method to write the DataFrame.
The Streaming DataFrame to write
Trigger frequency for stream
location for checkpoints of streaming query