Class

io.smartdatalake.workflow.action.customlogic

CustomDfTransformerConfig

Related Doc: package customlogic

Permalink

case class CustomDfTransformerConfig(className: Option[String] = None, scalaFile: Option[String] = None, scalaCode: Option[String] = None, sqlCode: Option[String] = None, options: Map[String, String] = Map()) extends Product with Serializable

Configuration of a custom Spark-DataFrame transformation between one input and one output (1:1)

className

Optional class name to load transformer code from

scalaFile

Optional file where scala code for transformation is loaded from

scalaCode

Optional scala code for transformation

sqlCode

Optional map of DataObjectId and corresponding SQL Code

options

Options to pass to the transformation

Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CustomDfTransformerConfig
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CustomDfTransformerConfig(className: Option[String] = None, scalaFile: Option[String] = None, scalaCode: Option[String] = None, sqlCode: Option[String] = None, options: Map[String, String] = Map())

    Permalink

    className

    Optional class name to load transformer code from

    scalaFile

    Optional file where scala code for transformation is loaded from

    scalaCode

    Optional scala code for transformation

    sqlCode

    Optional map of DataObjectId and corresponding SQL Code

    options

    Options to pass to the transformation

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. val className: Option[String]

    Permalink

    Optional class name to load transformer code from

  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  10. val impl: Option[CustomDfTransformer]

    Permalink
  11. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  14. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  15. val options: Map[String, String]

    Permalink

    Options to pass to the transformation

  16. val scalaCode: Option[String]

    Permalink

    Optional scala code for transformation

  17. val scalaFile: Option[String]

    Permalink

    Optional file where scala code for transformation is loaded from

  18. val sqlCode: Option[String]

    Permalink

    Optional map of DataObjectId and corresponding SQL Code

  19. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  20. def toString(): String

    Permalink
    Definition Classes
    CustomDfTransformerConfig → AnyRef → Any
  21. def transform(df: DataFrame, dataObjectId: DataObjectId)(implicit session: SparkSession): DataFrame

    Permalink
  22. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped