org.apache.spark.paths

Class SparkPath

Object

org.apache.spark.paths.SparkPath

All Implemented Interfaces:

java.io.Serializable, scala.Equals, scala.Product

public class SparkPath
extends Object
implements scala.Product, scala.Serializable

A canonical representation of a file path. This class is intended to provide type-safety to the way that Spark handles Paths. Paths can be represented as Strings in multiple ways, which are not always compatible. Spark regularly uses two ways: 1. hadoop Path.toString and java URI.toString.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SparkPath()

Method Summary

Modifier and Type	Method and Description
static SparkPath	fromFileStatus(org.apache.hadoop.fs.FileStatus fs)
static SparkPath	fromPath(org.apache.hadoop.fs.Path path)
static SparkPath	fromPathString(String str) Creates a SparkPath from a hadoop Path string.
static SparkPath	fromUri(java.net.URI uri)
static SparkPath	fromUrlString(String str) Creates a SparkPath from a url-encoded string.
org.apache.hadoop.fs.Path	toPath()
String	toString()
java.net.URI	toUri()
String	urlEncoded()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface scala.Product

$init$, productArity, productElement, productIterator, productPrefix

Methods inherited from interface scala.Equals

canEqual, equals

Constructor Detail

SparkPath

public SparkPath()

Method Detail

fromPathString

public static SparkPath fromPathString(String str)

Creates a SparkPath from a hadoop Path string. Please be very sure that the provided string is encoded (or not encoded) in the right way.

Please see the hadoop Path documentation here: https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/Path.html#Path-java.lang.String-

Parameters:

str - (undocumented)

Returns:

(undocumented)

fromPath

public static SparkPath fromPath(org.apache.hadoop.fs.Path path)

fromFileStatus

public static SparkPath fromFileStatus(org.apache.hadoop.fs.FileStatus fs)

fromUrlString

public static SparkPath fromUrlString(String str)

Creates a SparkPath from a url-encoded string. Note: It is the responsibility of the caller to ensure that str is a valid url-encoded string.

Parameters:

str - (undocumented)

Returns:

(undocumented)

fromUri

public static SparkPath fromUri(java.net.URI uri)

urlEncoded

public String urlEncoded()

toUri

public java.net.URI toUri()

toPath

public org.apache.hadoop.fs.Path toPath()

toString

public String toString()

Overrides:

toString in class Object