DataFrameNaFunctions — Working With Missing Data

DataFrameNaFunctions is used to work with missing data in a structured query (a DataFrame).

Table 1. DataFrameNaFunctions API
Method Description

drop

 
drop(): DataFrame
drop(cols: Array[String]): DataFrame
drop(minNonNulls: Int): DataFrame
drop(minNonNulls: Int, cols: Array[String]): DataFrame
drop(minNonNulls: Int, cols: Seq[String]): DataFrame
drop(cols: Seq[String]): DataFrame
drop(how: String): DataFrame
drop(how: String, cols: Array[String]): DataFrame
drop(how: String, cols: Seq[String]): DataFrame

fill

 
fill(value: Boolean): DataFrame
fill(value: Boolean, cols: Array[String]): DataFrame
fill(value: Boolean, cols: Seq[String]): DataFrame
fill(value: Double): DataFrame
fill(value: Double, cols: Array[String]): DataFrame
fill(value: Double, cols: Seq[String]): DataFrame
fill(value: Long): DataFrame
fill(value: Long, cols: Array[String]): DataFrame
fill(value: Long, cols: Seq[String]): DataFrame
fill(valueMap: Map[String, Any]): DataFrame
fill(value: String): DataFrame
fill(value: String, cols: Array[String]): DataFrame
fill(value: String, cols: Seq[String]): DataFrame

replace

 
replace[T](cols: Seq[String], replacement: Map[T, T]): DataFrame
replace[T](col: String, replacement: Map[T, T]): DataFrame

DataFrameNaFunctions is available using na untyped transformation.

val q: DataFrame = ...
q.na

convertToDouble Internal Method

convertToDouble(v: Any): Double

convertToDouble…​FIXME

Note
 convertToDouble is used when…​FIXME

drop Method

drop(): DataFrame
drop(cols: Array[String]): DataFrame
drop(minNonNulls: Int): DataFrame
drop(minNonNulls: Int, cols: Array[String]): DataFrame
drop(minNonNulls: Int, cols: Seq[String]): DataFrame
drop(cols: Seq[String]): DataFrame
drop(how: String): DataFrame
drop(how: String, cols: Array[String]): DataFrame
drop(how: String, cols: Seq[String]): DataFrame

drop…​FIXME

fill Method

fill(value: Boolean): DataFrame
fill(value: Boolean, cols: Array[String]): DataFrame
fill(value: Boolean, cols: Seq[String]): DataFrame
fill(value: Double): DataFrame
fill(value: Double, cols: Array[String]): DataFrame
fill(value: Double, cols: Seq[String]): DataFrame
fill(value: Long): DataFrame
fill(value: Long, cols: Array[String]): DataFrame
fill(value: Long, cols: Seq[String]): DataFrame
fill(valueMap: Map[String, Any]): DataFrame
fill(value: String): DataFrame
fill(value: String, cols: Array[String]): DataFrame
fill(value: String, cols: Seq[String]): DataFrame

fill…​FIXME

fillCol Internal Method

fillCol[T](col: StructField, replacement: T): Column

fillCol…​FIXME

Note
 fillCol is used when…​FIXME

fillMap Internal Method

fillMap(values: Seq[(String, Any)]): DataFrame

fillMap…​FIXME

Note
 fillMap is used when…​FIXME

fillValue Internal Method

fillValue[T](value: T, cols: Seq[String]): DataFrame

fillValue…​FIXME

Note
 fillValue is used when…​FIXME

replace0 Internal Method

replace0[T](cols: Seq[String], replacement: Map[T, T]): DataFrame

replace0…​FIXME

Note
 replace0 is used when…​FIXME

replace Method

replace[T](cols: Seq[String], replacement: Map[T, T]): DataFrame
replace[T](col: String, replacement: Map[T, T]): DataFrame

replace…​FIXME

replaceCol Internal Method

replaceCol(col: StructField, replacementMap: Map[_, _]): Column

replaceCol…​FIXME

Note
 replaceCol is used when…​FIXME

results matching ""

    No results matching ""