 # functions

#### object functions

:: Experimental :: Functions available for DataFrame.

Annotations
()
Linear Supertypes
AnyRef, Any
Ordering
1. Grouped
2. Alphabetic
3. By inheritance
Inherited
1. functions
2. AnyRef
3. Any
1. Hide All
2. Show all
Visibility
1. Public
2. All

### Value Members

1. #### final def !=(arg0: AnyRef): Boolean

Definition Classes
AnyRef
2. #### final def !=(arg0: Any): Boolean

Definition Classes
Any
3. #### final def ##(): Int

Definition Classes
AnyRef → Any
4. #### final def ==(arg0: AnyRef): Boolean

Definition Classes
AnyRef
5. #### final def ==(arg0: Any): Boolean

Definition Classes
Any
6. #### def abs(e: Column): Column

Computes the absolutle value.

7. #### def approxCountDistinct(columnName: String, rsd: Double): Column

Aggregate function: returns the approximate number of distinct items in a group.

8. #### def approxCountDistinct(e: Column, rsd: Double): Column

Aggregate function: returns the approximate number of distinct items in a group.

9. #### def approxCountDistinct(columnName: String): Column

Aggregate function: returns the approximate number of distinct items in a group.

10. #### def approxCountDistinct(e: Column): Column

Aggregate function: returns the approximate number of distinct items in a group.

11. #### final def asInstanceOf[T0]: T0

Definition Classes
Any
12. #### def asc(columnName: String): Column

Returns a sort expression based on ascending order of the column.

Returns a sort expression based on ascending order of the column. {{ // Sort by dept in ascending order, and then age in descending order. df.sort(asc("dept"), desc("age")) }}

13. #### def avg(columnName: String): Column

Aggregate function: returns the average of the values in a group.

14. #### def avg(e: Column): Column

Aggregate function: returns the average of the values in a group.

15. #### def callUDF(f: Function10[_, _, _, _, _, _, _, _, _, _, _], returnType: DataType, arg1: Column, arg2: Column, arg3: Column, arg4: Column, arg5: Column, arg6: Column, arg7: Column, arg8: Column, arg9: Column, arg10: Column): Column

Call a Scala function of 10 arguments as user-defined function (UDF).

Call a Scala function of 10 arguments as user-defined function (UDF). This requires you to specify the return data type.

16. #### def callUDF(f: Function9[_, _, _, _, _, _, _, _, _, _], returnType: DataType, arg1: Column, arg2: Column, arg3: Column, arg4: Column, arg5: Column, arg6: Column, arg7: Column, arg8: Column, arg9: Column): Column

Call a Scala function of 9 arguments as user-defined function (UDF).

Call a Scala function of 9 arguments as user-defined function (UDF). This requires you to specify the return data type.

17. #### def callUDF(f: Function8[_, _, _, _, _, _, _, _, _], returnType: DataType, arg1: Column, arg2: Column, arg3: Column, arg4: Column, arg5: Column, arg6: Column, arg7: Column, arg8: Column): Column

Call a Scala function of 8 arguments as user-defined function (UDF).

Call a Scala function of 8 arguments as user-defined function (UDF). This requires you to specify the return data type.

18. #### def callUDF(f: Function7[_, _, _, _, _, _, _, _], returnType: DataType, arg1: Column, arg2: Column, arg3: Column, arg4: Column, arg5: Column, arg6: Column, arg7: Column): Column

Call a Scala function of 7 arguments as user-defined function (UDF).

Call a Scala function of 7 arguments as user-defined function (UDF). This requires you to specify the return data type.

19. #### def callUDF(f: Function6[_, _, _, _, _, _, _], returnType: DataType, arg1: Column, arg2: Column, arg3: Column, arg4: Column, arg5: Column, arg6: Column): Column

Call a Scala function of 6 arguments as user-defined function (UDF).

Call a Scala function of 6 arguments as user-defined function (UDF). This requires you to specify the return data type.

20. #### def callUDF(f: Function5[_, _, _, _, _, _], returnType: DataType, arg1: Column, arg2: Column, arg3: Column, arg4: Column, arg5: Column): Column

Call a Scala function of 5 arguments as user-defined function (UDF).

Call a Scala function of 5 arguments as user-defined function (UDF). This requires you to specify the return data type.

21. #### def callUDF(f: Function4[_, _, _, _, _], returnType: DataType, arg1: Column, arg2: Column, arg3: Column, arg4: Column): Column

Call a Scala function of 4 arguments as user-defined function (UDF).

Call a Scala function of 4 arguments as user-defined function (UDF). This requires you to specify the return data type.

22. #### def callUDF(f: Function3[_, _, _, _], returnType: DataType, arg1: Column, arg2: Column, arg3: Column): Column

Call a Scala function of 3 arguments as user-defined function (UDF).

Call a Scala function of 3 arguments as user-defined function (UDF). This requires you to specify the return data type.

23. #### def callUDF(f: Function2[_, _, _], returnType: DataType, arg1: Column, arg2: Column): Column

Call a Scala function of 2 arguments as user-defined function (UDF).

Call a Scala function of 2 arguments as user-defined function (UDF). This requires you to specify the return data type.

24. #### def callUDF(f: Function1[_, _], returnType: DataType, arg1: Column): Column

Call a Scala function of 1 arguments as user-defined function (UDF).

Call a Scala function of 1 arguments as user-defined function (UDF). This requires you to specify the return data type.

25. #### def callUDF(f: Function0[_], returnType: DataType): Column

Call a Scala function of 0 arguments as user-defined function (UDF).

Call a Scala function of 0 arguments as user-defined function (UDF). This requires you to specify the return data type.

26. #### def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
27. #### def coalesce(e: Column*): Column

Returns the first column that is not null.

Returns the first column that is not null.

`df.select(coalesce(df("a"), df("b")))`
Annotations
@varargs()
28. #### def col(colName: String): Column

Returns a Column based on the given column name.

29. #### def column(colName: String): Column

Returns a Column based on the given column name.

Returns a Column based on the given column name. Alias of col.

30. #### def count(columnName: String): Column

Aggregate function: returns the number of items in a group.

31. #### def count(e: Column): Column

Aggregate function: returns the number of items in a group.

32. #### def countDistinct(columnName: String, columnNames: String*): Column

Aggregate function: returns the number of distinct items in a group.

Aggregate function: returns the number of distinct items in a group.

Annotations
@varargs()
33. #### def countDistinct(expr: Column, exprs: Column*): Column

Aggregate function: returns the number of distinct items in a group.

Aggregate function: returns the number of distinct items in a group.

Annotations
@varargs()
34. #### def desc(columnName: String): Column

Returns a sort expression based on the descending order of the column.

Returns a sort expression based on the descending order of the column. {{ // Sort by dept in ascending order, and then age in descending order. df.sort(asc("dept"), desc("age")) }}

35. #### final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
36. #### def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
37. #### def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
38. #### def first(columnName: String): Column

Aggregate function: returns the first value of a column in a group.

39. #### def first(e: Column): Column

Aggregate function: returns the first value in a group.

40. #### final def getClass(): Class[_]

Definition Classes
AnyRef → Any
41. #### def hashCode(): Int

Definition Classes
AnyRef → Any
42. #### final def isInstanceOf[T0]: Boolean

Definition Classes
Any
43. #### def last(columnName: String): Column

Aggregate function: returns the last value of the column in a group.

44. #### def last(e: Column): Column

Aggregate function: returns the last value in a group.

45. #### def lit(literal: Any): Column

Creates a Column of literal value.

Creates a Column of literal value.

The passed in object is returned directly if it is already a Column. If the object is a Scala Symbol, it is converted into a Column also. Otherwise, a new Column is created to represent the literal value.

46. #### def lower(e: Column): Column

Converts a string exprsesion to lower case.

47. #### def max(columnName: String): Column

Aggregate function: returns the maximum value of the column in a group.

48. #### def max(e: Column): Column

Aggregate function: returns the maximum value of the expression in a group.

49. #### def min(columnName: String): Column

Aggregate function: returns the minimum value of the column in a group.

50. #### def min(e: Column): Column

Aggregate function: returns the minimum value of the expression in a group.

51. #### final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
52. #### def negate(e: Column): Column

Unary minus, i.

Unary minus, i.e. negate the expression.

```// Select the amount column and negates all values.
// Scala:
df.select( -df("amount") )

// Java:
df.select( negate(df.col("amount")) );```
53. #### def not(e: Column): Column

Inversion of boolean expression, i.

Inversion of boolean expression, i.e. NOT. {{ // Scala: select rows that are not active (isActive === false) df.filter( !df("isActive") )

// Java: df.filter( not(df.col("isActive")) ); }}

54. #### final def notify(): Unit

Definition Classes
AnyRef
55. #### final def notifyAll(): Unit

Definition Classes
AnyRef
56. #### def sqrt(e: Column): Column

Computes the square root of the specified float value.

57. #### def sum(columnName: String): Column

Aggregate function: returns the sum of all values in the given column.

58. #### def sum(e: Column): Column

Aggregate function: returns the sum of all values in the expression.

59. #### def sumDistinct(columnName: String): Column

Aggregate function: returns the sum of distinct values in the expression.

60. #### def sumDistinct(e: Column): Column

Aggregate function: returns the sum of distinct values in the expression.

61. #### final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
62. #### def toString(): String

Definition Classes
AnyRef → Any
63. #### def udf[RT, A1, A2, A3, A4, A5, A6, A7, A8, A9, A10](f: (A1, A2, A3, A4, A5, A6, A7, A8, A9, A10) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2], arg3: scala.reflect.api.JavaUniverse.TypeTag[A3], arg4: scala.reflect.api.JavaUniverse.TypeTag[A4], arg5: scala.reflect.api.JavaUniverse.TypeTag[A5], arg6: scala.reflect.api.JavaUniverse.TypeTag[A6], arg7: scala.reflect.api.JavaUniverse.TypeTag[A7], arg8: scala.reflect.api.JavaUniverse.TypeTag[A8], arg9: scala.reflect.api.JavaUniverse.TypeTag[A9], arg10: scala.reflect.api.JavaUniverse.TypeTag[A10]): UserDefinedFunction

Defines a user-defined function of 10 arguments as user-defined function (UDF).

Defines a user-defined function of 10 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

64. #### def udf[RT, A1, A2, A3, A4, A5, A6, A7, A8, A9](f: (A1, A2, A3, A4, A5, A6, A7, A8, A9) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2], arg3: scala.reflect.api.JavaUniverse.TypeTag[A3], arg4: scala.reflect.api.JavaUniverse.TypeTag[A4], arg5: scala.reflect.api.JavaUniverse.TypeTag[A5], arg6: scala.reflect.api.JavaUniverse.TypeTag[A6], arg7: scala.reflect.api.JavaUniverse.TypeTag[A7], arg8: scala.reflect.api.JavaUniverse.TypeTag[A8], arg9: scala.reflect.api.JavaUniverse.TypeTag[A9]): UserDefinedFunction

Defines a user-defined function of 9 arguments as user-defined function (UDF).

Defines a user-defined function of 9 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

65. #### def udf[RT, A1, A2, A3, A4, A5, A6, A7, A8](f: (A1, A2, A3, A4, A5, A6, A7, A8) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2], arg3: scala.reflect.api.JavaUniverse.TypeTag[A3], arg4: scala.reflect.api.JavaUniverse.TypeTag[A4], arg5: scala.reflect.api.JavaUniverse.TypeTag[A5], arg6: scala.reflect.api.JavaUniverse.TypeTag[A6], arg7: scala.reflect.api.JavaUniverse.TypeTag[A7], arg8: scala.reflect.api.JavaUniverse.TypeTag[A8]): UserDefinedFunction

Defines a user-defined function of 8 arguments as user-defined function (UDF).

Defines a user-defined function of 8 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

66. #### def udf[RT, A1, A2, A3, A4, A5, A6, A7](f: (A1, A2, A3, A4, A5, A6, A7) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2], arg3: scala.reflect.api.JavaUniverse.TypeTag[A3], arg4: scala.reflect.api.JavaUniverse.TypeTag[A4], arg5: scala.reflect.api.JavaUniverse.TypeTag[A5], arg6: scala.reflect.api.JavaUniverse.TypeTag[A6], arg7: scala.reflect.api.JavaUniverse.TypeTag[A7]): UserDefinedFunction

Defines a user-defined function of 7 arguments as user-defined function (UDF).

Defines a user-defined function of 7 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

67. #### def udf[RT, A1, A2, A3, A4, A5, A6](f: (A1, A2, A3, A4, A5, A6) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2], arg3: scala.reflect.api.JavaUniverse.TypeTag[A3], arg4: scala.reflect.api.JavaUniverse.TypeTag[A4], arg5: scala.reflect.api.JavaUniverse.TypeTag[A5], arg6: scala.reflect.api.JavaUniverse.TypeTag[A6]): UserDefinedFunction

Defines a user-defined function of 6 arguments as user-defined function (UDF).

Defines a user-defined function of 6 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

68. #### def udf[RT, A1, A2, A3, A4, A5](f: (A1, A2, A3, A4, A5) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2], arg3: scala.reflect.api.JavaUniverse.TypeTag[A3], arg4: scala.reflect.api.JavaUniverse.TypeTag[A4], arg5: scala.reflect.api.JavaUniverse.TypeTag[A5]): UserDefinedFunction

Defines a user-defined function of 5 arguments as user-defined function (UDF).

Defines a user-defined function of 5 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

69. #### def udf[RT, A1, A2, A3, A4](f: (A1, A2, A3, A4) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2], arg3: scala.reflect.api.JavaUniverse.TypeTag[A3], arg4: scala.reflect.api.JavaUniverse.TypeTag[A4]): UserDefinedFunction

Defines a user-defined function of 4 arguments as user-defined function (UDF).

Defines a user-defined function of 4 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

70. #### def udf[RT, A1, A2, A3](f: (A1, A2, A3) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2], arg3: scala.reflect.api.JavaUniverse.TypeTag[A3]): UserDefinedFunction

Defines a user-defined function of 3 arguments as user-defined function (UDF).

Defines a user-defined function of 3 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

71. #### def udf[RT, A1, A2](f: (A1, A2) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1], arg2: scala.reflect.api.JavaUniverse.TypeTag[A2]): UserDefinedFunction

Defines a user-defined function of 2 arguments as user-defined function (UDF).

Defines a user-defined function of 2 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

72. #### def udf[RT, A1](f: (A1) ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT], arg1: scala.reflect.api.JavaUniverse.TypeTag[A1]): UserDefinedFunction

Defines a user-defined function of 1 arguments as user-defined function (UDF).

Defines a user-defined function of 1 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

73. #### def udf[RT](f: () ⇒ RT)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[RT]): UserDefinedFunction

Defines a user-defined function of 0 arguments as user-defined function (UDF).

Defines a user-defined function of 0 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.

74. #### def upper(e: Column): Column

Converts a string expression to upper case.

75. #### final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
76. #### final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
77. #### final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )