DataFrameStatFunctions — Working With Statistic Functions

DataFrameStatFunctions is used to work with statistic functions in a structured query (a DataFrame).

Table 1. DataFrameStatFunctions API
Method Description

approxQuantile

 
approxQuantile(
  cols: Array[String],
  probabilities: Array[Double],
  relativeError: Double): Array[Array[Double]]
approxQuantile(
  col: String,
  probabilities: Array[Double],
  relativeError: Double): Array[Double]

bloomFilter

 
bloomFilter(col: Column, expectedNumItems: Long, fpp: Double): BloomFilter
bloomFilter(col: Column, expectedNumItems: Long, numBits: Long): BloomFilter
bloomFilter(colName: String, expectedNumItems: Long, fpp: Double): BloomFilter
bloomFilter(colName: String, expectedNumItems: Long, numBits: Long): BloomFilter

corr

 
corr(col1: String, col2: String): Double
corr(col1: String, col2: String, method: String): Double

countMinSketch

 
countMinSketch(col: Column, eps: Double, confidence: Double, seed: Int): CountMinSketch
countMinSketch(col: Column, depth: Int, width: Int, seed: Int): CountMinSketch
countMinSketch(colName: String, eps: Double, confidence: Double, seed: Int): CountMinSketch
countMinSketch(colName: String, depth: Int, width: Int, seed: Int): CountMinSketch

cov

 
cov(col1: String, col2: String): Double

crosstab

 
crosstab(col1: String, col2: String): DataFrame

freqItems

 
freqItems(cols: Array[String]): DataFrame
freqItems(cols: Array[String], support: Double): DataFrame
freqItems(cols: Seq[String]): DataFrame
freqItems(cols: Seq[String], support: Double): DataFrame

sampleBy

 
sampleBy[T](col: String, fractions: Map[T, Double], seed: Long): DataFrame

DataFrameStatFunctions is available using stat untyped transformation.

val q: DataFrame = ...
q.stat

approxQuantile Method

approxQuantile(
  cols: Array[String],
  probabilities: Array[Double],
  relativeError: Double): Array[Array[Double]]
approxQuantile(
  col: String,
  probabilities: Array[Double],
  relativeError: Double): Array[Double]

approxQuantile…​FIXME

bloomFilter Method

bloomFilter(col: Column, expectedNumItems: Long, fpp: Double): BloomFilter
bloomFilter(col: Column, expectedNumItems: Long, numBits: Long): BloomFilter
bloomFilter(colName: String, expectedNumItems: Long, fpp: Double): BloomFilter
bloomFilter(colName: String, expectedNumItems: Long, numBits: Long): BloomFilter

bloomFilter…​FIXME

buildBloomFilter Internal Method

buildBloomFilter(col: Column, zero: BloomFilter): BloomFilter

buildBloomFilter…​FIXME

Note
 convertToDouble is used when…​FIXME

corr Method

corr(col1: String, col2: String): Double
corr(col1: String, col2: String, method: String): Double

corr…​FIXME

countMinSketch Method

countMinSketch(col: Column, eps: Double, confidence: Double, seed: Int): CountMinSketch
countMinSketch(col: Column, depth: Int, width: Int, seed: Int): CountMinSketch
countMinSketch(colName: String, eps: Double, confidence: Double, seed: Int): CountMinSketch
countMinSketch(colName: String, depth: Int, width: Int, seed: Int): CountMinSketch
// PRIVATE API
countMinSketch(col: Column, zero: CountMinSketch): CountMinSketch

countMinSketch…​FIXME

cov Method

cov(col1: String, col2: String): Double

cov…​FIXME

crosstab Method

crosstab(col1: String, col2: String): DataFrame

crosstab…​FIXME

freqItems Method

freqItems(cols: Array[String]): DataFrame
freqItems(cols: Array[String], support: Double): DataFrame
freqItems(cols: Seq[String]): DataFrame
freqItems(cols: Seq[String], support: Double): DataFrame

freqItems…​FIXME

sampleBy Method

sampleBy[T](col: String, fractions: Map[T, Double], seed: Long): DataFrame

sampleBy…​FIXME

results matching ""

    No results matching ""