object KolmogorovSmirnovTest
Conduct the two-sided Kolmogorov Smirnov (KS) test for data sampled from a continuous distribution. By comparing the largest difference between the empirical cumulative distribution of the sample data and the theoretical distribution we can provide a test for the the null hypothesis that the sample data comes from that theoretical distribution. For more information on KS Test:
- Annotations
- @Since( "2.4.0" )
- Source
- KolmogorovSmirnovTest.scala
- See also
- Alphabetic
- By Inheritance
- KolmogorovSmirnovTest
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
def
test(dataset: Dataset[_], sampleCol: String, distName: String, params: Double*): DataFrame
Convenience function to conduct a one-sample, two-sided Kolmogorov-Smirnov test for probability distribution equality.
Convenience function to conduct a one-sample, two-sided Kolmogorov-Smirnov test for probability distribution equality. Currently supports the normal distribution, taking as parameters the mean and standard deviation.
- dataset
A
Dataset
or aDataFrame
containing the sample of data to test- sampleCol
Name of sample column in dataset, of any numerical type
- distName
a
String
name for a theoretical distribution, currently only support "norm".- params
Double*
specifying the parameters to be used for the theoretical distribution. For "norm" distribution, the parameters includes mean and variance.- returns
DataFrame containing the test result for the input sampled data. This DataFrame will contain a single Row with the following fields:
pValue: Double
statistic: Double
- Annotations
- @Since( "2.4.0" ) @varargs()
-
def
test(dataset: Dataset[_], sampleCol: String, cdf: Function[Double, Double]): DataFrame
Java-friendly version of
test(dataset: DataFrame, sampleCol: String, cdf: Double => Double)
Java-friendly version of
test(dataset: DataFrame, sampleCol: String, cdf: Double => Double)
- Annotations
- @Since( "2.4.0" )
-
def
test(dataset: Dataset[_], sampleCol: String, cdf: (Double) ⇒ Double): DataFrame
Conduct the two-sided Kolmogorov-Smirnov (KS) test for data sampled from a continuous distribution.
Conduct the two-sided Kolmogorov-Smirnov (KS) test for data sampled from a continuous distribution. By comparing the largest difference between the empirical cumulative distribution of the sample data and the theoretical distribution we can provide a test for the the null hypothesis that the sample data comes from that theoretical distribution.
- dataset
A
Dataset
or aDataFrame
containing the sample of data to test- sampleCol
Name of sample column in dataset, of any numerical type
- cdf
a
Double => Double
function to calculate the theoretical CDF at a given value- returns
DataFrame containing the test result for the input sampled data. This DataFrame will contain a single Row with the following fields:
pValue: Double
statistic: Double
- Annotations
- @Since( "2.4.0" )