Index
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form
A
- abort() - Method in interface org.apache.spark.sql.connector.write.DataWriter
-
Aborts this writer if it is failed.
- abort(long, WriterCommitMessage[]) - Method in interface org.apache.spark.sql.connector.write.streaming.StreamingWrite
-
Aborts this writing job because some data writers are failed and keep failing when retried, or the Spark job fails with some unknown reasons, or
StreamingWrite.commit(long, WriterCommitMessage[])
fails. - abort(Throwable) - Method in interface org.apache.spark.shuffle.api.ShuffleMapOutputWriter
-
Abort all of the writes done by any writers returned by
ShuffleMapOutputWriter.getPartitionWriter(int)
. - abort(WriterCommitMessage[]) - Method in interface org.apache.spark.sql.connector.write.BatchWrite
-
Aborts this writing job because some data writers are failed and keep failing when retry, or the Spark job fails with some unknown reasons, or
BatchWrite.onDataWriterCommit(WriterCommitMessage)
fails, orBatchWrite.commit(WriterCommitMessage[])
fails. - abortStagedChanges() - Method in interface org.apache.spark.sql.connector.catalog.StagedTable
-
Abort the changes that were staged, both in metadata and from temporary outputs of this table's writers.
- abs() - Method in class org.apache.spark.sql.types.Decimal
- abs(double) - Static method in class org.apache.spark.sql.types.DoubleExactNumeric
- abs(double) - Method in interface org.apache.spark.sql.types.DoubleType.DoubleIsConflicted
- abs(float) - Static method in class org.apache.spark.sql.types.FloatExactNumeric
- abs(float) - Method in interface org.apache.spark.sql.types.FloatType.FloatIsConflicted
- abs(Column) - Static method in class org.apache.spark.sql.functions
-
Computes the absolute value of a numeric value.
- abs(T) - Static method in class org.apache.spark.sql.types.ByteExactNumeric
- abs(T) - Static method in class org.apache.spark.sql.types.DecimalExactNumeric
- abs(T) - Static method in class org.apache.spark.sql.types.IntegerExactNumeric
- abs(T) - Static method in class org.apache.spark.sql.types.LongExactNumeric
- abs(T) - Static method in class org.apache.spark.sql.types.ShortExactNumeric
- absent() - Static method in class org.apache.spark.api.java.Optional
- AbsoluteError - Class in org.apache.spark.mllib.tree.loss
-
Class for absolute error loss calculation (for regression).
- AbsoluteError() - Constructor for class org.apache.spark.mllib.tree.loss.AbsoluteError
- AbstractLauncher<T extends AbstractLauncher<T>> - Class in org.apache.spark.launcher
-
Base class for launcher implementations.
- accept(ES, Function1<ES, List<Object>>) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- accept(String, PartialFunction<Object, U>) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- accept(Path) - Method in class org.apache.spark.ml.image.SamplePathFilter
- accept(Parsers) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- ACCEPT_ANY_SCHEMA - Enum constant in enum class org.apache.spark.sql.connector.catalog.TableCapability
-
Signals that the table accepts input of any schema in a write operation.
- acceptIf(Function1<Object, Object>, Function1<Object, String>) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- acceptMatch(String, PartialFunction<Object, U>) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- acceptSeq(ES, Function1<ES, Iterable<Object>>) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- AcceptsLatestSeenOffset - Interface in org.apache.spark.sql.connector.read.streaming
-
Indicates that the source accepts the latest seen offset, which requires streaming execution to provide the latest seen offset when restarting the streaming query from checkpoint.
- acceptsType(DataType) - Method in class org.apache.spark.sql.types.ObjectType
- accessNonExistentAccumulatorError(long) - Static method in class org.apache.spark.errors.SparkCoreErrors
- accId() - Method in class org.apache.spark.CleanAccum
- accumCleaned(long) - Method in interface org.apache.spark.CleanerListener
- AccumulableInfo - Class in org.apache.spark.scheduler
-
:: DeveloperApi :: Information about an
AccumulatorV2
modified during a task or stage. - AccumulableInfo - Class in org.apache.spark.status.api.v1
- accumulableInfoFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- AccumulableInfoSerializer - Class in org.apache.spark.status.protobuf
- AccumulableInfoSerializer() - Constructor for class org.apache.spark.status.protobuf.AccumulableInfoSerializer
- accumulableInfoToJson(AccumulableInfo, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- accumulables() - Method in class org.apache.spark.scheduler.StageInfo
-
Terminal values of accumulables updated during this stage, including all the user-defined accumulators.
- accumulables() - Method in class org.apache.spark.scheduler.TaskInfo
-
Intermediate updates to accumulables during this task.
- accumulablesToJson(Iterable<AccumulableInfo>, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- ACCUMULATOR_ID_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.SQLPlanMetric
- ACCUMULATOR_UPDATES_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.StageData
- ACCUMULATOR_UPDATES_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.TaskData
- ACCUMULATOR_UPDATES_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper
- AccumulatorContext - Class in org.apache.spark.util
-
An internal class used to track accumulators by Spark itself.
- AccumulatorContext() - Constructor for class org.apache.spark.util.AccumulatorContext
- ACCUMULATORS() - Static method in class org.apache.spark.status.TaskIndexNames
- accumulatorUpdates() - Method in class org.apache.spark.status.api.v1.StageData
- accumulatorUpdates() - Method in class org.apache.spark.status.api.v1.TaskData
- AccumulatorV2<IN,
OUT> - Class in org.apache.spark.util -
The base class for accumulators, that can accumulate inputs of type
IN
, and produce output of typeOUT
. - AccumulatorV2() - Constructor for class org.apache.spark.util.AccumulatorV2
- accumUpdates() - Method in class org.apache.spark.ExceptionFailure
- accumUpdates() - Method in class org.apache.spark.scheduler.SparkListenerExecutorMetricsUpdate
- accumUpdates() - Method in class org.apache.spark.TaskKilled
- accuracy() - Method in interface org.apache.spark.ml.classification.ClassificationSummary
-
Returns accuracy.
- accuracy() - Method in class org.apache.spark.mllib.evaluation.MulticlassMetrics
- accuracy() - Method in class org.apache.spark.mllib.evaluation.MultilabelMetrics
-
Returns accuracy
- acos(String) - Static method in class org.apache.spark.sql.functions
- acos(Column) - Static method in class org.apache.spark.sql.functions
- acosh(String) - Static method in class org.apache.spark.sql.functions
- acosh(Column) - Static method in class org.apache.spark.sql.functions
- acquire(Seq<String>) - Method in interface org.apache.spark.resource.ResourceAllocator
-
Acquire a sequence of resource addresses (to a launched task), these addresses must be available.
- actionNotAllowedOnTableSincePartitionMetadataNotStoredError(String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- actionNotAllowedOnTableWithFilesourcePartitionManagementDisabledError(String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- ActivationFunction - Interface in org.apache.spark.ml.ann
-
Trait for functions and their derivatives for functional layers
- active() - Static method in class org.apache.spark.sql.SparkSession
-
Returns the currently active SparkSession, otherwise the default one.
- active() - Method in class org.apache.spark.sql.streaming.StreamingQueryManager
-
Returns a list of active queries associated with this SQLContext
- active() - Method in class org.apache.spark.streaming.scheduler.ReceiverInfo
- ACTIVE - Enum constant in enum class org.apache.spark.status.api.v1.StageStatus
- ACTIVE - Enum constant in enum class org.apache.spark.streaming.StreamingContextState
-
The context has been started, and been not stopped.
- ACTIVE() - Static method in class org.apache.spark.streaming.scheduler.ReceiverState
- ACTIVE_TASKS_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary
- activeIterator() - Method in interface org.apache.spark.ml.linalg.Vector
-
Returns an iterator over all the active elements of this vector.
- activeIterator() - Method in interface org.apache.spark.mllib.linalg.Vector
-
Returns an iterator over all the active elements of this vector.
- activeStages() - Method in class org.apache.spark.status.LiveJob
- activeTasks() - Method in class org.apache.spark.status.api.v1.ExecutorSummary
- activeTasks() - Method in class org.apache.spark.status.LiveJob
- activeTasks() - Method in class org.apache.spark.status.LiveStage
- activeTasksPerExecutor() - Method in class org.apache.spark.status.LiveStage
- add(double) - Method in class org.apache.spark.sql.util.NumericHistogram
-
Adds a new data point to the histogram approximation.
- add(double) - Method in class org.apache.spark.util.DoubleAccumulator
-
Adds v to the accumulator, i.e. increment sum by v and count by 1.
- add(double[], MultivariateGaussian[], ExpectationSum, Vector<Object>) - Static method in class org.apache.spark.mllib.clustering.ExpectationSum
- add(long) - Method in class org.apache.spark.util.LongAccumulator
-
Adds v to the accumulator, i.e. increment sum by v and count by 1.
- add(long, long) - Static method in class org.apache.spark.streaming.util.RawTextHelper
- add(Datum) - Method in interface org.apache.spark.ml.optim.aggregator.DifferentiableLossAggregator
-
Add a single data point to this aggregator.
- add(IN) - Method in class org.apache.spark.util.AccumulatorV2
-
Takes the inputs and accumulates.
- add(Double) - Method in class org.apache.spark.util.DoubleAccumulator
-
Adds v to the accumulator, i.e. increment sum by v and count by 1.
- add(Long) - Method in class org.apache.spark.sql.util.MapperRowCounter
- add(Long) - Method in class org.apache.spark.util.LongAccumulator
-
Adds v to the accumulator, i.e. increment sum by v and count by 1.
- add(Object) - Method in class org.apache.spark.util.sketch.CountMinSketch
-
Increments
item
's count by one. - add(Object, long) - Method in class org.apache.spark.util.sketch.CountMinSketch
-
Increments
item
's count bycount
. - add(String, String) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new nullable field with no metadata where the dataType is specified as a String. - add(String, String, boolean) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new field with no metadata where the dataType is specified as a String. - add(String, String, boolean, String) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new field and specifying metadata where the dataType is specified as a String. - add(String, String, boolean, Metadata) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new field and specifying metadata where the dataType is specified as a String. - add(String, DataType) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new nullable field with no metadata. - add(String, DataType, boolean) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new field with no metadata. - add(String, DataType, boolean, String) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new field and specifying metadata. - add(String, DataType, boolean, Metadata) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new field and specifying metadata. - add(org.apache.spark.ml.feature.InstanceBlock) - Method in class org.apache.spark.ml.clustering.KMeansAggregator
- add(Term) - Static method in class org.apache.spark.ml.feature.Dot
- add(Term) - Static method in class org.apache.spark.ml.feature.EmptyTerm
- add(Term) - Method in interface org.apache.spark.ml.feature.Term
-
Creates a summation term by concatenation of terms.
- add(BlockMatrix) - Method in class org.apache.spark.mllib.linalg.distributed.BlockMatrix
-
Adds the given block matrix
other
tothis
block matrix:this + other
. - add(Vector) - Method in class org.apache.spark.mllib.feature.IDF.DocumentFrequencyAggregator
-
Adds a new document.
- add(Vector) - Method in class org.apache.spark.mllib.stat.MultivariateOnlineSummarizer
-
Add a new sample to this summarizer, and update the statistical summary.
- add(StructField) - Method in class org.apache.spark.sql.types.StructType
-
Creates a new
StructType
by adding a new field. - add(Tuple2<Vector, Object>) - Method in class org.apache.spark.ml.clustering.ExpectationAggregator
-
Add a new training instance to this ExpectationAggregator, update the weights, means and covariances for each distributions, and update the log likelihood.
- add(T) - Method in class org.apache.spark.sql.util.SQLOpenHashSet
- add(T) - Method in class org.apache.spark.util.CollectionAccumulator
- add_months(Column, int) - Static method in class org.apache.spark.sql.functions
-
Returns the date that is
numMonths
afterstartDate
. - add_months(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns the date that is
numMonths
afterstartDate
. - ADD_TIME_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary
- ADD_TIME_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ProcessSummary
- addAccumulatorUpdates(int, StoreTypes.AccumulableInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 44;
- addAccumulatorUpdates(int, StoreTypes.AccumulableInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdates(int, StoreTypes.AccumulableInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdates(int, StoreTypes.AccumulableInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 44;
- addAccumulatorUpdates(int, StoreTypes.AccumulableInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdates(int, StoreTypes.AccumulableInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdates(StoreTypes.AccumulableInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 44;
- addAccumulatorUpdates(StoreTypes.AccumulableInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdates(StoreTypes.AccumulableInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdates(StoreTypes.AccumulableInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 44;
- addAccumulatorUpdates(StoreTypes.AccumulableInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdates(StoreTypes.AccumulableInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdatesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 44;
- addAccumulatorUpdatesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdatesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdatesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 44;
- addAccumulatorUpdatesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAccumulatorUpdatesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAddresses(String) - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceInformation.Builder
-
repeated string addresses = 2;
- addAddressesBytes(ByteString) - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceInformation.Builder
-
repeated string addresses = 2;
- addAllAccumulatorUpdates(Iterable<? extends StoreTypes.AccumulableInfo>) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 44;
- addAllAccumulatorUpdates(Iterable<? extends StoreTypes.AccumulableInfo>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAllAccumulatorUpdates(Iterable<? extends StoreTypes.AccumulableInfo>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.AccumulableInfo accumulator_updates = 13;
- addAllAddresses(Iterable<String>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceInformation.Builder
-
repeated string addresses = 2;
- addAllAttempts(Iterable<? extends StoreTypes.ApplicationAttemptInfo>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ApplicationAttemptInfo attempts = 7;
- addAllBlacklistedInStages(Iterable<? extends Long>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary.Builder
-
repeated int64 blacklisted_in_stages = 25;
- addAllBytesRead(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetricDistributions.Builder
-
repeated double bytes_read = 1;
- addAllBytesWritten(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetricDistributions.Builder
-
repeated double bytes_written = 1;
- addAllChildClusters(Iterable<? extends StoreTypes.RDDOperationClusterWrapper>) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationClusterWrapper child_clusters = 4;
- addAllChildNodes(Iterable<? extends StoreTypes.RDDOperationNode>) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationNode child_nodes = 3;
- addAllClasspathEntries(Iterable<? extends StoreTypes.PairStrings>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings classpath_entries = 6;
- addAllCorruptMergedBlockChunks(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double corrupt_merged_block_chunks = 1;
- addAllDataDistribution(Iterable<? extends StoreTypes.RDDDataDistribution>) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDDataDistribution data_distribution = 8;
- addAllDiskBytesSpilled(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double disk_bytes_spilled = 15;
- addAllDiskBytesSpilled(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double disk_bytes_spilled = 14;
- addAllDuration(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double duration = 2;
- addAllEdges(Iterable<? extends StoreTypes.RDDOperationEdge>) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge edges = 2;
- addAllEdges(Iterable<? extends StoreTypes.SparkPlanGraphEdge>) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphEdge edges = 3;
- addAllExcludedInStages(Iterable<? extends Long>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary.Builder
-
repeated int64 excluded_in_stages = 31;
- addAllExecutorCpuTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double executor_cpu_time = 6;
- addAllExecutorDeserializeCpuTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double executor_deserialize_cpu_time = 4;
- addAllExecutorDeserializeTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double executor_deserialize_time = 3;
- addAllExecutorMetrics(Iterable<? extends StoreTypes.ExecutorMetrics>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated .org.apache.spark.status.protobuf.ExecutorMetrics executor_metrics = 2;
- addAllExecutorRunTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double executor_run_time = 5;
- addAllExecutors(Iterable<String>) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDPartitionInfo.Builder
-
repeated string executors = 5;
- addAllFailedTasks(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double failed_tasks = 3;
- addAllFetchWaitTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double fetch_wait_time = 5;
- addAllGettingResultTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double getting_result_time = 10;
- addAllHadoopProperties(Iterable<? extends StoreTypes.PairStrings>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings hadoop_properties = 3;
- addAllIncomingEdges(Iterable<? extends StoreTypes.RDDOperationEdge>) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge incoming_edges = 4;
- addAllInputBytes(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double input_bytes = 6;
- addAllInputRecords(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double input_records = 7;
- addAllJobIds(Iterable<? extends Long>) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageDataWrapper.Builder
-
repeated int64 job_ids = 2;
- addAllJobTags(Iterable<String>) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
-
repeated string job_tags = 21;
- addAllJvmGcTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double jvm_gc_time = 8;
- addAllKilledTasks(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double killed_tasks = 5;
- addAllLocalBlocksFetched(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double local_blocks_fetched = 4;
- addAllLocalMergedBlocksFetched(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double local_merged_blocks_fetched = 4;
- addAllLocalMergedBytesRead(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double local_merged_bytes_read = 8;
- addAllLocalMergedChunksFetched(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double local_merged_chunks_fetched = 6;
- addAllMemoryBytesSpilled(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double memory_bytes_spilled = 14;
- addAllMemoryBytesSpilled(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double memory_bytes_spilled = 13;
- addAllMergedFetchFallbackCount(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double merged_fetch_fallback_count = 2;
- addAllMetrics(Iterable<? extends StoreTypes.SQLPlanMetric>) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 5;
- addAllMetrics(Iterable<? extends StoreTypes.SQLPlanMetric>) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 4;
- addAllMetrics(Iterable<? extends StoreTypes.SQLPlanMetric>) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 7;
- addAllMetricsProperties(Iterable<? extends StoreTypes.PairStrings>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings metrics_properties = 5;
- addAllNodes(Iterable<? extends StoreTypes.SparkPlanGraphNodeWrapper>) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 4;
- addAllNodes(Iterable<? extends StoreTypes.SparkPlanGraphNodeWrapper>) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 2;
- addAllOutgoingEdges(Iterable<? extends StoreTypes.RDDOperationEdge>) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge outgoing_edges = 3;
- addAllOutputBytes(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double output_bytes = 8;
- addAllOutputRecords(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double output_records = 9;
- addAllPartitions(Iterable<? extends StoreTypes.RDDPartitionInfo>) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDPartitionInfo partitions = 9;
- addAllPeakExecutionMemory(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double peak_execution_memory = 12;
- addAllQuantiles(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double quantiles = 1;
- addAllQuantiles(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated double quantiles = 1;
- addAllQuantiles(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double quantiles = 1;
- addAllRddIds(Iterable<? extends Long>) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated int64 rdd_ids = 43;
- addAllReadBytes(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double read_bytes = 1;
- addAllReadRecords(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double read_records = 2;
- addAllRecordsRead(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetricDistributions.Builder
-
repeated double records_read = 2;
- addAllRecordsWritten(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetricDistributions.Builder
-
repeated double records_written = 2;
- addAllRemoteBlocksFetched(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double remote_blocks_fetched = 3;
- addAllRemoteBytesRead(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double remote_bytes_read = 6;
- addAllRemoteBytesReadToDisk(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double remote_bytes_read_to_disk = 7;
- addAllRemoteMergedBlocksFetched(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double remote_merged_blocks_fetched = 3;
- addAllRemoteMergedBytesRead(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double remote_merged_bytes_read = 7;
- addAllRemoteMergedChunksFetched(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double remote_merged_chunks_fetched = 5;
- addAllRemoteMergedReqsDuration(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double remote_merged_reqs_duration = 9;
- addAllRemoteReqsDuration(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double remote_reqs_duration = 9;
- addAllResourceProfiles(Iterable<? extends StoreTypes.ResourceProfileInfo>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ResourceProfileInfo resource_profiles = 7;
- addAllResultSerializationTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double result_serialization_time = 9;
- addAllResultSize(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double result_size = 7;
- addAllSchedulerDelay(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double scheduler_delay = 11;
- addAllShuffleRead(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double shuffle_read = 10;
- addAllShuffleReadRecords(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double shuffle_read_records = 11;
- addAllShuffleWrite(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double shuffle_write = 12;
- addAllShuffleWriteRecords(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double shuffle_write_records = 13;
- addAllSkippedStages(Iterable<? extends Integer>) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobDataWrapper.Builder
-
repeated int32 skipped_stages = 2;
- addAllSources(Iterable<? extends StoreTypes.SourceProgress>) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.SourceProgress sources = 10;
- addAllSparkProperties(Iterable<? extends StoreTypes.PairStrings>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings spark_properties = 2;
- addAllStageIds(Iterable<? extends Long>) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
-
repeated int64 stage_ids = 6;
- addAllStageIds(Iterable<? extends Long>) - Method in class org.apache.spark.status.protobuf.StoreTypes.PoolData.Builder
-
repeated int64 stage_ids = 2;
- addAllStages(Iterable<? extends Long>) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated int64 stages = 12;
- addAllStateOperators(Iterable<? extends StoreTypes.StateOperatorProgress>) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.StateOperatorProgress state_operators = 9;
- addAllSucceededTasks(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double succeeded_tasks = 4;
- addAllSystemProperties(Iterable<? extends StoreTypes.PairStrings>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings system_properties = 4;
- addAllTaskTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double task_time = 2;
- addAllTotalBlocksFetched(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double total_blocks_fetched = 8;
- addAllWriteBytes(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
-
repeated double write_bytes = 1;
- addAllWriteRecords(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
-
repeated double write_records = 2;
- addAllWriteTime(Iterable<? extends Double>) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
-
repeated double write_time = 3;
- addAppArgs(String...) - Method in class org.apache.spark.launcher.AbstractLauncher
-
Adds command line arguments for the application.
- addAppArgs(String...) - Method in class org.apache.spark.launcher.SparkLauncher
- addArchive(String) - Method in class org.apache.spark.SparkContext
-
:: Experimental :: Add an archive to be downloaded and unpacked with this Spark job on every node.
- addAttempts(int, StoreTypes.ApplicationAttemptInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ApplicationAttemptInfo attempts = 7;
- addAttempts(int, StoreTypes.ApplicationAttemptInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ApplicationAttemptInfo attempts = 7;
- addAttempts(StoreTypes.ApplicationAttemptInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ApplicationAttemptInfo attempts = 7;
- addAttempts(StoreTypes.ApplicationAttemptInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ApplicationAttemptInfo attempts = 7;
- addAttemptsBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ApplicationAttemptInfo attempts = 7;
- addAttemptsBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ApplicationAttemptInfo attempts = 7;
- addBin(double, double, int) - Method in class org.apache.spark.sql.util.NumericHistogram
-
Set a particular histogram bin with index.
- addBinary(byte[]) - Method in class org.apache.spark.util.sketch.CountMinSketch
-
Increments
item
's count by one. - addBinary(byte[], long) - Method in class org.apache.spark.util.sketch.CountMinSketch
-
Increments
item
's count bycount
. - addBlacklistedInStages(long) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary.Builder
-
repeated int64 blacklisted_in_stages = 25;
- addBytesRead(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetricDistributions.Builder
-
repeated double bytes_read = 1;
- addBytesWritten(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetricDistributions.Builder
-
repeated double bytes_written = 1;
- addCatalogInCacheTableAsSelectNotAllowedError(String, SqlBaseParser.CacheTableContext) - Static method in class org.apache.spark.sql.errors.QueryParsingErrors
- addChildClusters(int, StoreTypes.RDDOperationClusterWrapper) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationClusterWrapper child_clusters = 4;
- addChildClusters(int, StoreTypes.RDDOperationClusterWrapper.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationClusterWrapper child_clusters = 4;
- addChildClusters(StoreTypes.RDDOperationClusterWrapper) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationClusterWrapper child_clusters = 4;
- addChildClusters(StoreTypes.RDDOperationClusterWrapper.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationClusterWrapper child_clusters = 4;
- addChildClustersBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationClusterWrapper child_clusters = 4;
- addChildClustersBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationClusterWrapper child_clusters = 4;
- addChildNodes(int, StoreTypes.RDDOperationNode) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationNode child_nodes = 3;
- addChildNodes(int, StoreTypes.RDDOperationNode.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationNode child_nodes = 3;
- addChildNodes(StoreTypes.RDDOperationNode) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationNode child_nodes = 3;
- addChildNodes(StoreTypes.RDDOperationNode.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationNode child_nodes = 3;
- addChildNodesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationNode child_nodes = 3;
- addChildNodesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationNode child_nodes = 3;
- addChunk(ShuffleBlockChunkId, RoaringBitmap) - Method in class org.apache.spark.storage.PushBasedFetchHelper
-
This is executed by the task thread when the
iterator.next()
is invoked and the iterator processes a response of typeShuffleBlockFetcherIterator.PushMergedLocalMetaFetchResult
. - addClasspathEntries(int, StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings classpath_entries = 6;
- addClasspathEntries(int, StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings classpath_entries = 6;
- addClasspathEntries(StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings classpath_entries = 6;
- addClasspathEntries(StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings classpath_entries = 6;
- addClasspathEntriesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings classpath_entries = 6;
- addClasspathEntriesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings classpath_entries = 6;
- addColumn(String[], DataType) - Static method in interface org.apache.spark.sql.connector.catalog.TableChange
-
Create a TableChange for adding an optional column.
- addColumn(String[], DataType, boolean) - Static method in interface org.apache.spark.sql.connector.catalog.TableChange
-
Create a TableChange for adding a column.
- addColumn(String[], DataType, boolean, String) - Static method in interface org.apache.spark.sql.connector.catalog.TableChange
-
Create a TableChange for adding a column.
- addColumn(String[], DataType, boolean, String, TableChange.ColumnPosition, ColumnDefaultValue) - Static method in interface org.apache.spark.sql.connector.catalog.TableChange
-
Create a TableChange for adding a column.
- addColumnWithV1TableCannotSpecifyNotNullError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- addCorruptMergedBlockChunks(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double corrupt_merged_block_chunks = 1;
- addDataDistribution(int, StoreTypes.RDDDataDistribution) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDDataDistribution data_distribution = 8;
- addDataDistribution(int, StoreTypes.RDDDataDistribution.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDDataDistribution data_distribution = 8;
- addDataDistribution(StoreTypes.RDDDataDistribution) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDDataDistribution data_distribution = 8;
- addDataDistribution(StoreTypes.RDDDataDistribution.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDDataDistribution data_distribution = 8;
- addDataDistributionBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDDataDistribution data_distribution = 8;
- addDataDistributionBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDDataDistribution data_distribution = 8;
- addDirectoryError(Path) - Static method in class org.apache.spark.errors.SparkCoreErrors
- addDiskBytesSpilled(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double disk_bytes_spilled = 15;
- addDiskBytesSpilled(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double disk_bytes_spilled = 14;
- addDuration(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double duration = 2;
- addEdges(int, StoreTypes.RDDOperationEdge) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge edges = 2;
- addEdges(int, StoreTypes.RDDOperationEdge.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge edges = 2;
- addEdges(int, StoreTypes.SparkPlanGraphEdge) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphEdge edges = 3;
- addEdges(int, StoreTypes.SparkPlanGraphEdge.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphEdge edges = 3;
- addEdges(StoreTypes.RDDOperationEdge) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge edges = 2;
- addEdges(StoreTypes.RDDOperationEdge.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge edges = 2;
- addEdges(StoreTypes.SparkPlanGraphEdge) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphEdge edges = 3;
- addEdges(StoreTypes.SparkPlanGraphEdge.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphEdge edges = 3;
- addEdgesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge edges = 2;
- addEdgesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphEdge edges = 3;
- addEdgesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge edges = 2;
- addEdgesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphEdge edges = 3;
- addExcludedInStages(long) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary.Builder
-
repeated int64 excluded_in_stages = 31;
- addExecutorCpuTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double executor_cpu_time = 6;
- addExecutorDeserializeCpuTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double executor_deserialize_cpu_time = 4;
- addExecutorDeserializeTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double executor_deserialize_time = 3;
- addExecutorMetrics(int, StoreTypes.ExecutorMetrics) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated .org.apache.spark.status.protobuf.ExecutorMetrics executor_metrics = 2;
- addExecutorMetrics(int, StoreTypes.ExecutorMetrics.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated .org.apache.spark.status.protobuf.ExecutorMetrics executor_metrics = 2;
- addExecutorMetrics(StoreTypes.ExecutorMetrics) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated .org.apache.spark.status.protobuf.ExecutorMetrics executor_metrics = 2;
- addExecutorMetrics(StoreTypes.ExecutorMetrics.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated .org.apache.spark.status.protobuf.ExecutorMetrics executor_metrics = 2;
- addExecutorMetricsBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated .org.apache.spark.status.protobuf.ExecutorMetrics executor_metrics = 2;
- addExecutorMetricsBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated .org.apache.spark.status.protobuf.ExecutorMetrics executor_metrics = 2;
- addExecutorRunTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double executor_run_time = 5;
- addExecutors(String) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDPartitionInfo.Builder
-
repeated string executors = 5;
- addExecutorsBytes(ByteString) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDPartitionInfo.Builder
-
repeated string executors = 5;
- addFailedTasks(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double failed_tasks = 3;
- addFetchWaitTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double fetch_wait_time = 5;
- addFile(String) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Add a file to be downloaded with this Spark job on every node.
- addFile(String) - Method in class org.apache.spark.launcher.AbstractLauncher
-
Adds a file to be submitted with the application.
- addFile(String) - Method in class org.apache.spark.launcher.SparkLauncher
- addFile(String) - Method in class org.apache.spark.SparkContext
-
Add a file to be downloaded with this Spark job on every node.
- addFile(String, boolean) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Add a file to be downloaded with this Spark job on every node.
- addFile(String, boolean) - Method in class org.apache.spark.SparkContext
-
Add a file to be downloaded with this Spark job on every node.
- addFilesWithAbsolutePathUnsupportedError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- addFilter(ServletContextHandler, String, Map<String, String>) - Static method in class org.apache.spark.ui.JettyUtils
- addGettingResultTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double getting_result_time = 10;
- addGrid(BooleanParam) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Adds a boolean param with true and false.
- addGrid(DoubleParam, double[]) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Adds a double param with multiple values.
- addGrid(FloatParam, float[]) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Adds a float param with multiple values.
- addGrid(IntParam, int[]) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Adds an int param with multiple values.
- addGrid(LongParam, long[]) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Adds a long param with multiple values.
- addGrid(Param<T>, Iterable<T>) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Adds a param with multiple values (overwrites if the input param exists).
- addHadoopProperties(int, StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings hadoop_properties = 3;
- addHadoopProperties(int, StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings hadoop_properties = 3;
- addHadoopProperties(StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings hadoop_properties = 3;
- addHadoopProperties(StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings hadoop_properties = 3;
- addHadoopPropertiesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings hadoop_properties = 3;
- addHadoopPropertiesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings hadoop_properties = 3;
- addIncomingEdges(int, StoreTypes.RDDOperationEdge) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge incoming_edges = 4;
- addIncomingEdges(int, StoreTypes.RDDOperationEdge.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge incoming_edges = 4;
- addIncomingEdges(StoreTypes.RDDOperationEdge) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge incoming_edges = 4;
- addIncomingEdges(StoreTypes.RDDOperationEdge.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge incoming_edges = 4;
- addIncomingEdgesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge incoming_edges = 4;
- addIncomingEdgesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge incoming_edges = 4;
- addInputBytes(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double input_bytes = 6;
- addInputRecords(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double input_records = 7;
- addJar(String) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Adds a JAR dependency for all tasks to be executed on this SparkContext in the future.
- addJar(String) - Method in class org.apache.spark.launcher.AbstractLauncher
-
Adds a jar file to be submitted with the application.
- addJar(String) - Method in class org.apache.spark.launcher.SparkLauncher
- addJar(String) - Method in class org.apache.spark.SparkContext
-
Adds a JAR dependency for all tasks to be executed on this
SparkContext
in the future. - addJarsToClassPath(String, MutableURLClassLoader) - Static method in class org.apache.spark.util.DependencyUtils
- addJarToClasspath(String, MutableURLClassLoader) - Static method in class org.apache.spark.util.DependencyUtils
- addJobIds(long) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageDataWrapper.Builder
-
repeated int64 job_ids = 2;
- addJobTag(String) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Add a tag to be assigned to all the jobs started by this thread.
- addJobTag(String) - Method in class org.apache.spark.SparkContext
-
Add a tag to be assigned to all the jobs started by this thread.
- addJobTags(String) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
-
repeated string job_tags = 21;
- addJobTagsBytes(ByteString) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
-
repeated string job_tags = 21;
- addJvmGcTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double jvm_gc_time = 8;
- addKilledTasks(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double killed_tasks = 5;
- addListener(L) - Method in interface org.apache.spark.util.ListenerBus
-
Add a listener to listen events.
- addListener(SparkAppHandle.Listener) - Method in interface org.apache.spark.launcher.SparkAppHandle
-
Adds a listener to be notified of changes to the handle's information.
- addListener(StreamingQueryListener) - Method in class org.apache.spark.sql.streaming.StreamingQueryManager
-
Register a
StreamingQueryListener
to receive up-calls for life cycle events ofStreamingQuery
. - addLocalBlocksFetched(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double local_blocks_fetched = 4;
- addLocalConfiguration(String, int, int, int, JobConf) - Static method in class org.apache.spark.rdd.HadoopRDD
-
Add Hadoop configuration specific to a single partition and attempt.
- addLocalDirectoryError(Path) - Static method in class org.apache.spark.errors.SparkCoreErrors
- addLocalMergedBlocksFetched(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double local_merged_blocks_fetched = 4;
- addLocalMergedBytesRead(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double local_merged_bytes_read = 8;
- addLocalMergedChunksFetched(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double local_merged_chunks_fetched = 6;
- addLong(long) - Method in class org.apache.spark.util.sketch.CountMinSketch
-
Increments
item
's count by one. - addLong(long, long) - Method in class org.apache.spark.util.sketch.CountMinSketch
-
Increments
item
's count bycount
. - addMapOutput(int, MapStatus) - Method in class org.apache.spark.ShuffleStatus
-
Register a map output.
- addMemoryBytesSpilled(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double memory_bytes_spilled = 14;
- addMemoryBytesSpilled(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double memory_bytes_spilled = 13;
- addMergedFetchFallbackCount(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double merged_fetch_fallback_count = 2;
- addMergeResult(int, org.apache.spark.scheduler.MergeStatus) - Method in class org.apache.spark.ShuffleStatus
-
Register a merge result.
- addMetrics(int, StoreTypes.SQLPlanMetric) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 5;
- addMetrics(int, StoreTypes.SQLPlanMetric) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 4;
- addMetrics(int, StoreTypes.SQLPlanMetric) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 7;
- addMetrics(int, StoreTypes.SQLPlanMetric.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 5;
- addMetrics(int, StoreTypes.SQLPlanMetric.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 4;
- addMetrics(int, StoreTypes.SQLPlanMetric.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 7;
- addMetrics(TaskMetrics, TaskMetrics) - Static method in class org.apache.spark.status.LiveEntityHelpers
-
Add m2 values to m1.
- addMetrics(StoreTypes.SQLPlanMetric) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 5;
- addMetrics(StoreTypes.SQLPlanMetric) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 4;
- addMetrics(StoreTypes.SQLPlanMetric) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 7;
- addMetrics(StoreTypes.SQLPlanMetric.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 5;
- addMetrics(StoreTypes.SQLPlanMetric.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 4;
- addMetrics(StoreTypes.SQLPlanMetric.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 7;
- addMetricsBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 5;
- addMetricsBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 4;
- addMetricsBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 7;
- addMetricsBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 5;
- addMetricsBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 4;
- addMetricsBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated .org.apache.spark.status.protobuf.SQLPlanMetric metrics = 7;
- addMetricsProperties(int, StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings metrics_properties = 5;
- addMetricsProperties(int, StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings metrics_properties = 5;
- addMetricsProperties(StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings metrics_properties = 5;
- addMetricsProperties(StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings metrics_properties = 5;
- addMetricsPropertiesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings metrics_properties = 5;
- addMetricsPropertiesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings metrics_properties = 5;
- addNaN() - Method in class org.apache.spark.sql.util.SQLOpenHashSet
- addNewDefaultColumnToExistingTableNotAllowed(String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- addNewFunctionMismatchedWithFunctionError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- addNodes(int, StoreTypes.SparkPlanGraphNodeWrapper) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 4;
- addNodes(int, StoreTypes.SparkPlanGraphNodeWrapper) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 2;
- addNodes(int, StoreTypes.SparkPlanGraphNodeWrapper.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 4;
- addNodes(int, StoreTypes.SparkPlanGraphNodeWrapper.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 2;
- addNodes(StoreTypes.SparkPlanGraphNodeWrapper) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 4;
- addNodes(StoreTypes.SparkPlanGraphNodeWrapper) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 2;
- addNodes(StoreTypes.SparkPlanGraphNodeWrapper.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 4;
- addNodes(StoreTypes.SparkPlanGraphNodeWrapper.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 2;
- addNodesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 4;
- addNodesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 2;
- addNodesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 4;
- addNodesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.SparkPlanGraphNodeWrapper nodes = 2;
- addNull() - Method in class org.apache.spark.sql.util.SQLOpenHashSet
- addOutgoingEdges(int, StoreTypes.RDDOperationEdge) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge outgoing_edges = 3;
- addOutgoingEdges(int, StoreTypes.RDDOperationEdge.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge outgoing_edges = 3;
- addOutgoingEdges(StoreTypes.RDDOperationEdge) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge outgoing_edges = 3;
- addOutgoingEdges(StoreTypes.RDDOperationEdge.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge outgoing_edges = 3;
- addOutgoingEdgesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge outgoing_edges = 3;
- addOutgoingEdgesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
-
repeated .org.apache.spark.status.protobuf.RDDOperationEdge outgoing_edges = 3;
- addOutputBytes(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double output_bytes = 8;
- addOutputRecords(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double output_records = 9;
- addPartition(LiveRDDPartition) - Method in class org.apache.spark.status.RDDPartitionSeq
- addPartitions(int, StoreTypes.RDDPartitionInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDPartitionInfo partitions = 9;
- addPartitions(int, StoreTypes.RDDPartitionInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDPartitionInfo partitions = 9;
- addPartitions(StoreTypes.RDDPartitionInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDPartitionInfo partitions = 9;
- addPartitions(StoreTypes.RDDPartitionInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDPartitionInfo partitions = 9;
- addPartitionsBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDPartitionInfo partitions = 9;
- addPartitionsBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
-
repeated .org.apache.spark.status.protobuf.RDDPartitionInfo partitions = 9;
- addPartToPGroup(Partition, PartitionGroup) - Method in class org.apache.spark.rdd.DefaultPartitionCoalescer
- addPeakExecutionMemory(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double peak_execution_memory = 12;
- addPyFile(String) - Method in class org.apache.spark.launcher.AbstractLauncher
-
Adds a python file / zip / egg to be submitted with the application.
- addPyFile(String) - Method in class org.apache.spark.launcher.SparkLauncher
- addQuantiles(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double quantiles = 1;
- addQuantiles(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
-
repeated double quantiles = 1;
- addQuantiles(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double quantiles = 1;
- addRddIds(long) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
-
repeated int64 rdd_ids = 43;
- addReadBytes(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double read_bytes = 1;
- addReadRecords(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double read_records = 2;
- addRecordsRead(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetricDistributions.Builder
-
repeated double records_read = 2;
- addRecordsWritten(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetricDistributions.Builder
-
repeated double records_written = 2;
- addRemoteBlocksFetched(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double remote_blocks_fetched = 3;
- addRemoteBytesRead(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double remote_bytes_read = 6;
- addRemoteBytesReadToDisk(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double remote_bytes_read_to_disk = 7;
- addRemoteMergedBlocksFetched(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double remote_merged_blocks_fetched = 3;
- addRemoteMergedBytesRead(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double remote_merged_bytes_read = 7;
- addRemoteMergedChunksFetched(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double remote_merged_chunks_fetched = 5;
- addRemoteMergedReqsDuration(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
-
repeated double remote_merged_reqs_duration = 9;
- addRemoteReqsDuration(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double remote_reqs_duration = 9;
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.AccumulableInfo.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationAttemptInfo.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfoWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfoWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.AppSummary.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.CachedQuantile.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetrics.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorResourceRequest.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorStageSummary.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorStageSummaryWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummaryWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetricDistributions.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetrics.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobDataWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.MemoryMetrics.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetricDistributions.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetrics.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.PairStrings.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.PoolData.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ProcessSummary.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ProcessSummaryWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDDataDistribution.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationEdge.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationNode.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDPartitionInfo.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfoWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceInformation.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceProfileInfo.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceProfileWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.RuntimeInfo.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetrics.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetrics.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetrics.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SinkProgress.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SourceProgress.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphEdge.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNodeWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SpeculationStageSummary.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SpeculationStageSummaryWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLPlanMetric.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.StageDataWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.StateOperatorProgress.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamBlockData.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryData.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgressWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetrics.Builder
- addRepeatedField(Descriptors.FieldDescriptor, Object) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskResourceRequest.Builder
- addRequest(TaskResourceRequest) - Method in class org.apache.spark.resource.TaskResourceRequests
-
Add a certain
TaskResourceRequest
to the request set. - addResourceProfiles(int, StoreTypes.ResourceProfileInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ResourceProfileInfo resource_profiles = 7;
- addResourceProfiles(int, StoreTypes.ResourceProfileInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ResourceProfileInfo resource_profiles = 7;
- addResourceProfiles(StoreTypes.ResourceProfileInfo) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ResourceProfileInfo resource_profiles = 7;
- addResourceProfiles(StoreTypes.ResourceProfileInfo.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ResourceProfileInfo resource_profiles = 7;
- addResourceProfilesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ResourceProfileInfo resource_profiles = 7;
- addResourceProfilesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.ResourceProfileInfo resource_profiles = 7;
- address() - Method in class org.apache.spark.BarrierTaskInfo
- address() - Method in class org.apache.spark.status.api.v1.RDDDataDistribution
- ADDRESS_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.RDDDataDistribution
- addresses() - Method in class org.apache.spark.resource.ResourceInformation
- addresses() - Method in class org.apache.spark.resource.ResourceInformationJson
- ADDRESSES_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ResourceInformation
- addResultSerializationTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double result_serialization_time = 9;
- addResultSize(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double result_size = 7;
- addSchedulable(Schedulable) - Method in interface org.apache.spark.scheduler.Schedulable
- addSchedulerDelay(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
-
repeated double scheduler_delay = 11;
- addShuffleRead(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double shuffle_read = 10;
- addShuffleReadRecords(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double shuffle_read_records = 11;
- addShuffleWrite(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double shuffle_write = 12;
- addShuffleWriteRecords(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double shuffle_write_records = 13;
- addShutdownHook(int, Function0<BoxedUnit>) - Static method in class org.apache.spark.util.ShutdownHookManager
-
Adds a shutdown hook with the given priority.
- addShutdownHook(Function0<BoxedUnit>) - Static method in class org.apache.spark.util.ShutdownHookManager
-
Adds a shutdown hook with default priority.
- addSkippedStages(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobDataWrapper.Builder
-
repeated int32 skipped_stages = 2;
- addSources(int, StoreTypes.SourceProgress) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.SourceProgress sources = 10;
- addSources(int, StoreTypes.SourceProgress.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.SourceProgress sources = 10;
- addSources(StoreTypes.SourceProgress) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.SourceProgress sources = 10;
- addSources(StoreTypes.SourceProgress.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.SourceProgress sources = 10;
- addSourcesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.SourceProgress sources = 10;
- addSourcesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.SourceProgress sources = 10;
- addSparkArg(String) - Method in class org.apache.spark.launcher.AbstractLauncher
-
Adds a no-value argument to the Spark invocation.
- addSparkArg(String) - Method in class org.apache.spark.launcher.SparkLauncher
- addSparkArg(String, String) - Method in class org.apache.spark.launcher.AbstractLauncher
-
Adds an argument with a value to the Spark invocation.
- addSparkArg(String, String) - Method in class org.apache.spark.launcher.SparkLauncher
- addSparkListener(SparkListenerInterface) - Method in class org.apache.spark.SparkContext
-
:: DeveloperApi :: Register a listener to receive up-calls from events that happen during execution.
- addSparkProperties(int, StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings spark_properties = 2;
- addSparkProperties(int, StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings spark_properties = 2;
- addSparkProperties(StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings spark_properties = 2;
- addSparkProperties(StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings spark_properties = 2;
- addSparkPropertiesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings spark_properties = 2;
- addSparkPropertiesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings spark_properties = 2;
- addStageIds(long) - Method in class org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
-
repeated int64 stage_ids = 6;
- addStageIds(long) - Method in class org.apache.spark.status.protobuf.StoreTypes.PoolData.Builder
-
repeated int64 stage_ids = 2;
- addStages(long) - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
-
repeated int64 stages = 12;
- addStateOperators(int, StoreTypes.StateOperatorProgress) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.StateOperatorProgress state_operators = 9;
- addStateOperators(int, StoreTypes.StateOperatorProgress.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.StateOperatorProgress state_operators = 9;
- addStateOperators(StoreTypes.StateOperatorProgress) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.StateOperatorProgress state_operators = 9;
- addStateOperators(StoreTypes.StateOperatorProgress.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.StateOperatorProgress state_operators = 9;
- addStateOperatorsBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.StateOperatorProgress state_operators = 9;
- addStateOperatorsBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
-
repeated .org.apache.spark.status.protobuf.StateOperatorProgress state_operators = 9;
- addStreamingListener(StreamingListener) - Method in class org.apache.spark.streaming.api.java.JavaStreamingContext
-
Deprecated.Add a
StreamingListener
object for receiving system events related to streaming. - addStreamingListener(StreamingListener) - Method in class org.apache.spark.streaming.StreamingContext
-
Deprecated.Add a
StreamingListener
object for receiving system events related to streaming. - addString(String) - Method in class org.apache.spark.util.sketch.CountMinSketch
-
Increments
item
's count by one. - addString(String, long) - Method in class org.apache.spark.util.sketch.CountMinSketch
-
Increments
item
's count bycount
. - addSucceededTasks(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double succeeded_tasks = 4;
- addSystemProperties(int, StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings system_properties = 4;
- addSystemProperties(int, StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings system_properties = 4;
- addSystemProperties(StoreTypes.PairStrings) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings system_properties = 4;
- addSystemProperties(StoreTypes.PairStrings.Builder) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings system_properties = 4;
- addSystemPropertiesBuilder() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings system_properties = 4;
- addSystemPropertiesBuilder(int) - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
-
repeated .org.apache.spark.status.protobuf.PairStrings system_properties = 4;
- addTaskCompletionListener(TaskCompletionListener) - Method in class org.apache.spark.BarrierTaskContext
- addTaskCompletionListener(TaskCompletionListener) - Method in class org.apache.spark.TaskContext
-
Adds a (Java friendly) listener to be executed on task completion.
- addTaskCompletionListener(Function1<TaskContext, U>) - Method in class org.apache.spark.TaskContext
-
Adds a listener in the form of a Scala closure to be executed on task completion.
- addTaskFailureListener(TaskFailureListener) - Method in class org.apache.spark.BarrierTaskContext
- addTaskFailureListener(TaskFailureListener) - Method in class org.apache.spark.TaskContext
-
Adds a listener to be executed on task failure (which includes completion listener failure, if the task body did not already fail).
- addTaskFailureListener(Function2<TaskContext, Throwable, BoxedUnit>) - Method in class org.apache.spark.TaskContext
-
Adds a listener to be executed on task failure (which includes completion listener failure, if the task body did not already fail).
- addTaskResourceRequests(SparkConf, TaskResourceRequests) - Static method in class org.apache.spark.resource.ResourceUtils
- addTaskSetManager(Schedulable, Properties) - Method in interface org.apache.spark.scheduler.SchedulableBuilder
- addTaskTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
-
repeated double task_time = 2;
- addTime() - Method in class org.apache.spark.status.api.v1.ExecutorSummary
- addTime() - Method in class org.apache.spark.status.api.v1.ProcessSummary
- addTotalBlocksFetched(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
-
repeated double total_blocks_fetched = 8;
- addURL(URL) - Method in class org.apache.spark.util.MutableURLClassLoader
- AddWebUIFilter(String, Map<String, String>, String) - Constructor for class org.apache.spark.scheduler.cluster.CoarseGrainedClusterMessages.AddWebUIFilter
- AddWebUIFilter$() - Constructor for class org.apache.spark.scheduler.cluster.CoarseGrainedClusterMessages.AddWebUIFilter$
- addWriteBytes(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
-
repeated double write_bytes = 1;
- addWriteRecords(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
-
repeated double write_records = 2;
- addWriteTime(double) - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
-
repeated double write_time = 3;
- advisoryPartitionSizeInBytes() - Method in interface org.apache.spark.sql.connector.write.RequiresDistributionAndOrdering
-
Returns the advisory (not guaranteed) shuffle partition size in bytes for this write.
- aes_decrypt(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns a decrypted value of
input
. - aes_decrypt(Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns a decrypted value of
input
. - aes_decrypt(Column, Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns a decrypted value of
input
. - aes_decrypt(Column, Column, Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns a decrypted value of
input
using AES inmode
withpadding
. - aes_encrypt(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns an encrypted value of
input
. - aes_encrypt(Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns an encrypted value of
input
. - aes_encrypt(Column, Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns an encrypted value of
input
. - aes_encrypt(Column, Column, Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns an encrypted value of
input
. - aes_encrypt(Column, Column, Column, Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns an encrypted value of
input
using AES in givenmode
with the specifiedpadding
. - aesCryptoError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- aesModeUnsupportedError(String, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- aesUnsupportedAad(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- aesUnsupportedIv(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- after(String) - Static method in interface org.apache.spark.sql.connector.catalog.TableChange.ColumnPosition
- AFTSurvivalRegression - Class in org.apache.spark.ml.regression
-
Fit a parametric survival regression model named accelerated failure time (AFT) model (see Accelerated failure time model (Wikipedia)) based on the Weibull distribution of the survival time.
- AFTSurvivalRegression() - Constructor for class org.apache.spark.ml.regression.AFTSurvivalRegression
- AFTSurvivalRegression(String) - Constructor for class org.apache.spark.ml.regression.AFTSurvivalRegression
- AFTSurvivalRegressionModel - Class in org.apache.spark.ml.regression
-
Model produced by
AFTSurvivalRegression
. - AFTSurvivalRegressionParams - Interface in org.apache.spark.ml.regression
-
Params for accelerated failure time (AFT) regression.
- agg(Map<String, String>) - Method in class org.apache.spark.sql.Dataset
-
(Java-specific) Aggregates on the entire Dataset without groups.
- agg(Map<String, String>) - Method in class org.apache.spark.sql.RelationalGroupedDataset
-
(Java-specific) Compute aggregates by specifying a map from column name to aggregate methods.
- agg(Column, Column...) - Method in class org.apache.spark.sql.Dataset
-
Aggregates on the entire Dataset without groups.
- agg(Column, Column...) - Method in class org.apache.spark.sql.RelationalGroupedDataset
-
Compute aggregates by specifying a series of aggregate columns.
- agg(Column, Seq<Column>) - Method in class org.apache.spark.sql.Dataset
-
Aggregates on the entire Dataset without groups.
- agg(Column, Seq<Column>) - Method in class org.apache.spark.sql.RelationalGroupedDataset
-
Compute aggregates by specifying a series of aggregate columns.
- agg(TypedColumn<V, U1>) - Method in class org.apache.spark.sql.KeyValueGroupedDataset
-
Computes the given aggregation, returning a
Dataset
of tuples for each unique key and the result of computing this aggregation over all elements in the group. - agg(TypedColumn<V, U1>, TypedColumn<V, U2>) - Method in class org.apache.spark.sql.KeyValueGroupedDataset
-
Computes the given aggregations, returning a
Dataset
of tuples for each unique key and the result of computing these aggregations over all elements in the group. - agg(TypedColumn<V, U1>, TypedColumn<V, U2>, TypedColumn<V, U3>) - Method in class org.apache.spark.sql.KeyValueGroupedDataset
-
Computes the given aggregations, returning a
Dataset
of tuples for each unique key and the result of computing these aggregations over all elements in the group. - agg(TypedColumn<V, U1>, TypedColumn<V, U2>, TypedColumn<V, U3>, TypedColumn<V, U4>) - Method in class org.apache.spark.sql.KeyValueGroupedDataset
-
Computes the given aggregations, returning a
Dataset
of tuples for each unique key and the result of computing these aggregations over all elements in the group. - agg(TypedColumn<V, U1>, TypedColumn<V, U2>, TypedColumn<V, U3>, TypedColumn<V, U4>, TypedColumn<V, U5>) - Method in class org.apache.spark.sql.KeyValueGroupedDataset
-
Computes the given aggregations, returning a
Dataset
of tuples for each unique key and the result of computing these aggregations over all elements in the group. - agg(TypedColumn<V, U1>, TypedColumn<V, U2>, TypedColumn<V, U3>, TypedColumn<V, U4>, TypedColumn<V, U5>, TypedColumn<V, U6>) - Method in class org.apache.spark.sql.KeyValueGroupedDataset
-
Computes the given aggregations, returning a
Dataset
of tuples for each unique key and the result of computing these aggregations over all elements in the group. - agg(TypedColumn<V, U1>, TypedColumn<V, U2>, TypedColumn<V, U3>, TypedColumn<V, U4>, TypedColumn<V, U5>, TypedColumn<V, U6>, TypedColumn<V, U7>) - Method in class org.apache.spark.sql.KeyValueGroupedDataset
-
Computes the given aggregations, returning a
Dataset
of tuples for each unique key and the result of computing these aggregations over all elements in the group. - agg(TypedColumn<V, U1>, TypedColumn<V, U2>, TypedColumn<V, U3>, TypedColumn<V, U4>, TypedColumn<V, U5>, TypedColumn<V, U6>, TypedColumn<V, U7>, TypedColumn<V, U8>) - Method in class org.apache.spark.sql.KeyValueGroupedDataset
-
Computes the given aggregations, returning a
Dataset
of tuples for each unique key and the result of computing these aggregations over all elements in the group. - agg(Map<String, String>) - Method in class org.apache.spark.sql.Dataset
-
(Scala-specific) Aggregates on the entire Dataset without groups.
- agg(Map<String, String>) - Method in class org.apache.spark.sql.RelationalGroupedDataset
-
(Scala-specific) Compute aggregates by specifying a map from column name to aggregate methods.
- agg(Tuple2<String, String>, Seq<Tuple2<String, String>>) - Method in class org.apache.spark.sql.Dataset
-
(Scala-specific) Aggregates on the entire Dataset without groups.
- agg(Tuple2<String, String>, Seq<Tuple2<String, String>>) - Method in class org.apache.spark.sql.RelationalGroupedDataset
-
(Scala-specific) Compute aggregates by specifying the column names and aggregate methods.
- aggregate(Column, Column, Function2<Column, Column, Column>) - Static method in class org.apache.spark.sql.functions
-
Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state.
- aggregate(Column, Column, Function2<Column, Column, Column>, Function1<Column, Column>) - Static method in class org.apache.spark.sql.functions
-
Applies a binary operator to an initial state and all elements in the array, and reduces this to a single state.
- aggregate(U, Function2<U, T, U>, Function2<U, U, U>) - Method in interface org.apache.spark.api.java.JavaRDDLike
-
Aggregate the elements of each partition, and then the results for all the partitions, using given combine functions and a neutral "zero value".
- aggregate(U, Function2<U, T, U>, Function2<U, U, U>, ClassTag<U>) - Method in class org.apache.spark.rdd.RDD
-
Aggregate the elements of each partition, and then the results for all the partitions, using given combine functions and a neutral "zero value".
- aggregateByKey(U, int, Function2<U, V, U>, Function2<U, U, U>) - Method in class org.apache.spark.api.java.JavaPairRDD
-
Aggregate the values of each key, using given combine functions and a neutral "zero value".
- aggregateByKey(U, int, Function2<U, V, U>, Function2<U, U, U>, ClassTag<U>) - Method in class org.apache.spark.rdd.PairRDDFunctions
-
Aggregate the values of each key, using given combine functions and a neutral "zero value".
- aggregateByKey(U, Function2<U, V, U>, Function2<U, U, U>) - Method in class org.apache.spark.api.java.JavaPairRDD
-
Aggregate the values of each key, using given combine functions and a neutral "zero value".
- aggregateByKey(U, Partitioner, Function2<U, V, U>, Function2<U, U, U>) - Method in class org.apache.spark.api.java.JavaPairRDD
-
Aggregate the values of each key, using given combine functions and a neutral "zero value".
- aggregateByKey(U, Partitioner, Function2<U, V, U>, Function2<U, U, U>, ClassTag<U>) - Method in class org.apache.spark.rdd.PairRDDFunctions
-
Aggregate the values of each key, using given combine functions and a neutral "zero value".
- aggregateByKey(U, Function2<U, V, U>, Function2<U, U, U>, ClassTag<U>) - Method in class org.apache.spark.rdd.PairRDDFunctions
-
Aggregate the values of each key, using given combine functions and a neutral "zero value".
- AggregatedDialect - Class in org.apache.spark.sql.jdbc
-
AggregatedDialect can unify multiple dialects into one virtual Dialect.
- AggregatedDialect(List<JdbcDialect>) - Constructor for class org.apache.spark.sql.jdbc.AggregatedDialect
- aggregateExpressionRequiredForPivotError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- aggregateExpressions() - Method in class org.apache.spark.sql.connector.expressions.aggregate.Aggregation
- AggregateFunc - Interface in org.apache.spark.sql.connector.expressions.aggregate
-
Base class of the Aggregate Functions.
- AggregateFunction<S extends Serializable,
R> - Interface in org.apache.spark.sql.connector.catalog.functions -
Interface for a function that produces a result value by aggregating over multiple input rows.
- aggregateInAggregateFilterError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- aggregateMessages(Function1<EdgeContext<VD, ED, A>, BoxedUnit>, Function2<A, A, A>, TripletFields, ClassTag<A>) - Method in class org.apache.spark.graphx.Graph
-
Aggregates values from the neighboring edges and vertices of each vertex.
- aggregateMessagesWithActiveSet(Function1<EdgeContext<VD, ED, A>, BoxedUnit>, Function2<A, A, A>, TripletFields, Option<Tuple2<VertexRDD<?>, EdgeDirection>>, ClassTag<A>) - Method in class org.apache.spark.graphx.impl.GraphImpl
- aggregateTaskMetrics(long[]) - Method in class org.apache.spark.sql.connector.metric.CustomAvgMetric
- aggregateTaskMetrics(long[]) - Method in interface org.apache.spark.sql.connector.metric.CustomMetric
-
Given an array of task metric values, returns aggregated final metric value.
- aggregateTaskMetrics(long[]) - Method in class org.apache.spark.sql.connector.metric.CustomSumMetric
- aggregateUsingIndex(RDD<Tuple2<Object, VD2>>, Function2<VD2, VD2, VD2>, ClassTag<VD2>) - Method in class org.apache.spark.graphx.impl.VertexRDDImpl
- aggregateUsingIndex(RDD<Tuple2<Object, VD2>>, Function2<VD2, VD2, VD2>, ClassTag<VD2>) - Method in class org.apache.spark.graphx.VertexRDD
-
Aggregates vertices in
messages
that have the same ids usingreduceFunc
, returning a VertexRDD co-indexed withthis
. - AggregatingEdgeContext<VD,
ED, A> - Class in org.apache.spark.graphx.impl - AggregatingEdgeContext(Function2<A, A, A>, Object, BitSet) - Constructor for class org.apache.spark.graphx.impl.AggregatingEdgeContext
- Aggregation - Class in org.apache.spark.sql.connector.expressions.aggregate
-
Aggregation in SQL statement.
- Aggregation(AggregateFunc[], Expression[]) - Constructor for class org.apache.spark.sql.connector.expressions.aggregate.Aggregation
- aggregationDepth() - Method in class org.apache.spark.ml.classification.LinearSVC
- aggregationDepth() - Method in class org.apache.spark.ml.classification.LinearSVCModel
- aggregationDepth() - Method in class org.apache.spark.ml.classification.LogisticRegression
- aggregationDepth() - Method in class org.apache.spark.ml.classification.LogisticRegressionModel
- aggregationDepth() - Method in class org.apache.spark.ml.clustering.GaussianMixture
- aggregationDepth() - Method in class org.apache.spark.ml.clustering.GaussianMixtureModel
- aggregationDepth() - Method in interface org.apache.spark.ml.param.shared.HasAggregationDepth
-
Param for suggested depth for treeAggregate (>= 2).
- aggregationDepth() - Method in class org.apache.spark.ml.regression.AFTSurvivalRegression
- aggregationDepth() - Method in class org.apache.spark.ml.regression.AFTSurvivalRegressionModel
- aggregationDepth() - Method in class org.apache.spark.ml.regression.GeneralizedLinearRegression
- aggregationDepth() - Method in class org.apache.spark.ml.regression.GeneralizedLinearRegressionModel
- aggregationDepth() - Method in class org.apache.spark.ml.regression.LinearRegression
- aggregationDepth() - Method in class org.apache.spark.ml.regression.LinearRegressionModel
- aggregationFunctionAppliedOnNonNumericColumnError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- aggregationFunctionAppliedOnNonNumericColumnError(String, int) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- aggregationNotAllowedInMergeCondition(String, Expression) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- aggregator() - Method in class org.apache.spark.ShuffleDependency
- Aggregator<K,
V, C> - Class in org.apache.spark -
:: DeveloperApi :: A set of functions used to aggregate data.
- Aggregator<IN,
BUF, OUT> - Class in org.apache.spark.sql.expressions -
A base class for user-defined aggregations, which can be used in
Dataset
operations to take all of the elements of a group and reduce them to a single value. - Aggregator() - Constructor for class org.apache.spark.sql.expressions.Aggregator
- Aggregator(Function1<V, C>, Function2<C, V, C>, Function2<C, C, C>) - Constructor for class org.apache.spark.Aggregator
- aic() - Method in class org.apache.spark.ml.regression.GeneralizedLinearRegressionSummary
- aic(RDD<Tuple3<Object, Object, Object>>, double, double, double) - Method in class org.apache.spark.ml.regression.GeneralizedLinearRegression.Binomial$
- aic(RDD<Tuple3<Object, Object, Object>>, double, double, double) - Method in class org.apache.spark.ml.regression.GeneralizedLinearRegression.Gamma$
- aic(RDD<Tuple3<Object, Object, Object>>, double, double, double) - Method in class org.apache.spark.ml.regression.GeneralizedLinearRegression.Gaussian$
- aic(RDD<Tuple3<Object, Object, Object>>, double, double, double) - Method in class org.apache.spark.ml.regression.GeneralizedLinearRegression.Poisson$
- algo() - Method in class org.apache.spark.mllib.tree.configuration.Strategy
- algo() - Method in class org.apache.spark.mllib.tree.model.DecisionTreeModel
- algo() - Method in class org.apache.spark.mllib.tree.model.GradientBoostedTreesModel
- algo() - Method in class org.apache.spark.mllib.tree.model.RandomForestModel
- Algo - Class in org.apache.spark.mllib.tree.configuration
-
Enum to select the algorithm for the decision tree
- Algo() - Constructor for class org.apache.spark.mllib.tree.configuration.Algo
- algorithm() - Method in class org.apache.spark.mllib.regression.StreamingLinearRegressionWithSGD
- alias(String) - Method in class org.apache.spark.sql.Column
-
Gives the column an alias.
- alias(String) - Method in class org.apache.spark.sql.Dataset
-
Returns a new Dataset with an alias set.
- alias(Symbol) - Method in class org.apache.spark.sql.Dataset
-
(Scala-specific) Returns a new Dataset with an alias set.
- aliasesNumberNotMatchUDTFOutputError(int, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- aliasNumberNotMatchColumnNumberError(int, int, TreeNode<?>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- All - Static variable in class org.apache.spark.graphx.TripletFields
-
Expose all the fields (source, edge, and destination).
- ALL_GATHER() - Static method in class org.apache.spark.RequestMethod
- ALL_REMOVALS_TIME_MS_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.StateOperatorProgress
- ALL_UPDATES_TIME_MS_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.StateOperatorProgress
- allAvailable() - Static method in interface org.apache.spark.sql.connector.read.streaming.ReadLimit
- allGather(String) - Method in class org.apache.spark.BarrierTaskContext
-
:: Experimental :: Blocks until all tasks in the same stage have reached this routine.
- AllJobsCancelled - Class in org.apache.spark.scheduler
- AllJobsCancelled() - Constructor for class org.apache.spark.scheduler.AllJobsCancelled
- allocate(int) - Method in class org.apache.spark.sql.util.NumericHistogram
-
Sets the number of histogram bins to use for approximating data.
- allocator() - Method in class org.apache.spark.storage.memory.SerializedValuesHolder
- AllReceiverIds - Class in org.apache.spark.streaming.scheduler
-
A message used by ReceiverTracker to ask all receiver's ids still stored in ReceiverTrackerEndpoint.
- AllReceiverIds() - Constructor for class org.apache.spark.streaming.scheduler.AllReceiverIds
- allRemovalsTimeMs() - Method in class org.apache.spark.sql.streaming.StateOperatorProgress
- allSources() - Static method in class org.apache.spark.metrics.source.StaticSources
-
The set of all static sources.
- allSupportedExecutorResources() - Static method in class org.apache.spark.resource.ResourceProfile
-
Return all supported Spark built-in executor resources, custom resources like GPUs/FPGAs are excluded.
- allUpdatesTimeMs() - Method in class org.apache.spark.sql.streaming.StateOperatorProgress
- alpha() - Method in class org.apache.spark.ml.recommendation.ALS
- alpha() - Method in interface org.apache.spark.ml.recommendation.ALSParams
-
Param for the alpha parameter in the implicit preference formulation (nonnegative).
- alpha() - Method in class org.apache.spark.mllib.random.WeibullGenerator
- ALS - Class in org.apache.spark.ml.recommendation
-
Alternating Least Squares (ALS) matrix factorization.
- ALS - Class in org.apache.spark.mllib.recommendation
-
Alternating Least Squares matrix factorization.
- ALS() - Constructor for class org.apache.spark.ml.recommendation.ALS
- ALS() - Constructor for class org.apache.spark.mllib.recommendation.ALS
-
Constructs an ALS instance with default parameters: {numBlocks: -1, rank: 10, iterations: 10, lambda: 0.01, implicitPrefs: false, alpha: 1.0}.
- ALS(String) - Constructor for class org.apache.spark.ml.recommendation.ALS
- ALS.InBlock$ - Class in org.apache.spark.ml.recommendation
- ALS.LeastSquaresNESolver - Interface in org.apache.spark.ml.recommendation
-
Trait for least squares solvers applied to the normal equation.
- ALS.Rating<ID> - Class in org.apache.spark.ml.recommendation
-
Rating class for better code readability.
- ALS.Rating$ - Class in org.apache.spark.ml.recommendation
- ALS.RatingBlock$ - Class in org.apache.spark.ml.recommendation
- ALSModel - Class in org.apache.spark.ml.recommendation
-
Model fitted by ALS.
- ALSModelParams - Interface in org.apache.spark.ml.recommendation
-
Common params for ALS and ALSModel.
- ALSParams - Interface in org.apache.spark.ml.recommendation
-
Common params for ALS.
- alterAddColNotSupportDatasourceTableError(Object, TableIdentifier) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterAddColNotSupportViewError(TableIdentifier) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterColumnCannotFindColumnInV1TableError(String, org.apache.spark.sql.connector.catalog.V1Table) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterColumnWithV1TableCannotSpecifyNotNullError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterDatabaseLocationUnsupportedError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterNamespace(String[], NamespaceChange...) - Method in class org.apache.spark.sql.connector.catalog.DelegatingCatalogExtension
- alterNamespace(String[], NamespaceChange...) - Method in interface org.apache.spark.sql.connector.catalog.SupportsNamespaces
-
Apply a set of metadata changes to a namespace in the catalog.
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.DB2Dialect
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.DerbyDialect
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.H2Dialect
- alterTable(String, Seq<TableChange>, int) - Method in class org.apache.spark.sql.jdbc.JdbcDialect
-
Alter an existing table.
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.MsSqlServerDialect
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.MySQLDialect
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.NoopDialect
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.OracleDialect
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.PostgresDialect
- alterTable(String, Seq<TableChange>, int) - Static method in class org.apache.spark.sql.jdbc.TeradataDialect
- alterTable(Identifier, TableChange...) - Method in class org.apache.spark.sql.connector.catalog.DelegatingCatalogExtension
- alterTable(Identifier, TableChange...) - Method in interface org.apache.spark.sql.connector.catalog.TableCatalog
-
Apply a set of
changes
to a table in the catalog. - alterTableChangeColumnNotSupportedForColumnTypeError(String, StructField, StructField) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterTableRecoverPartitionsNotSupportedForV2TablesError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterTableSerDePropertiesNotSupportedForV2TablesError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterTableSetSerdeForSpecificPartitionNotSupportedError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterTableSetSerdeNotSupportedError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterTableWithDropPartitionAndPurgeUnsupportedError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- alterV2TableSetLocationWithPartitionNotSupportedError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- alterView(Identifier, ViewChange...) - Method in interface org.apache.spark.sql.connector.catalog.ViewCatalog
-
Apply
changes
to a view in the catalog. - AlwaysFalse - Class in org.apache.spark.sql.connector.expressions.filter
-
A predicate that always evaluates to
false
. - AlwaysFalse - Class in org.apache.spark.sql.sources
-
A filter that always evaluates to
false
. - AlwaysFalse() - Constructor for class org.apache.spark.sql.connector.expressions.filter.AlwaysFalse
- AlwaysFalse() - Constructor for class org.apache.spark.sql.sources.AlwaysFalse
- AlwaysTrue - Class in org.apache.spark.sql.connector.expressions.filter
-
A predicate that always evaluates to
true
. - AlwaysTrue - Class in org.apache.spark.sql.sources
-
A filter that always evaluates to
true
. - AlwaysTrue() - Constructor for class org.apache.spark.sql.connector.expressions.filter.AlwaysTrue
- AlwaysTrue() - Constructor for class org.apache.spark.sql.sources.AlwaysTrue
- am() - Method in class org.apache.spark.scheduler.cluster.CoarseGrainedClusterMessages.RegisterClusterManager
- ambiguousAttributesInSelfJoinError(Seq<AttributeReference>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- ambiguousColumnOrFieldError(Seq<String>, int) - Method in interface org.apache.spark.sql.errors.CompilationErrors
- ambiguousColumnOrFieldError(Seq<String>, int, Origin) - Static method in class org.apache.spark.sql.errors.DataTypeErrors
- ambiguousColumnOrFieldError(Seq<String>, int, Origin) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- ambiguousLateralColumnAliasError(String, int) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- ambiguousLateralColumnAliasError(Seq<String>, int) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- ambiguousReferenceError(String, Seq<Attribute>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- ambiguousReferenceToFieldsError(String, int) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- ambiguousRelationAliasNameInNestedCTEError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- amount() - Method in class org.apache.spark.resource.ExecutorResourceRequest
- amount() - Method in class org.apache.spark.resource.ResourceRequest
- amount() - Method in class org.apache.spark.resource.TaskResourceRequest
- AMOUNT() - Static method in class org.apache.spark.resource.ResourceUtils
- AMOUNT_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ExecutorResourceRequest
- AMOUNT_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.TaskResourceRequest
- AnalysisException - Exception in org.apache.spark.sql
-
Thrown when a query fails to analyze, usually because the query itself is invalid.
- AnalysisException(String, Map<String, String>) - Constructor for exception org.apache.spark.sql.AnalysisException
- AnalysisException(String, Map<String, String>, QueryContext[], String) - Constructor for exception org.apache.spark.sql.AnalysisException
- AnalysisException(String, Map<String, String>, Origin) - Constructor for exception org.apache.spark.sql.AnalysisException
- AnalysisException(String, Map<String, String>, Origin, Option<Throwable>) - Constructor for exception org.apache.spark.sql.AnalysisException
- AnalysisException(String, Map<String, String>, Option<Throwable>) - Constructor for exception org.apache.spark.sql.AnalysisException
- analyzeTableNotSupportedForV2TablesError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- analyzeTableNotSupportedOnViewsError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- analyzingColumnStatisticsNotSupportedForColumnTypeError(String, DataType) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- and(Column) - Method in class org.apache.spark.sql.Column
-
Boolean AND.
- And - Class in org.apache.spark.sql.connector.expressions.filter
-
A predicate that evaluates to
true
iff bothleft
andright
evaluate totrue
. - And - Class in org.apache.spark.sql.sources
-
A filter that evaluates to
true
iff bothleft
orright
evaluate totrue
. - And(Predicate, Predicate) - Constructor for class org.apache.spark.sql.connector.expressions.filter.And
- And(Filter, Filter) - Constructor for class org.apache.spark.sql.sources.And
- ANOVATest - Class in org.apache.spark.ml.stat
-
ANOVA Test for continuous data.
- ANOVATest() - Constructor for class org.apache.spark.ml.stat.ANOVATest
- ansiDateTimeError(Exception) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- ansiDateTimeParseError(Exception) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- ansiIllegalArgumentError(IllegalArgumentException) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- ansiIllegalArgumentError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- antecedent() - Method in class org.apache.spark.mllib.fpm.AssociationRules.Rule
- any(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns true if at least one value of
e
is true. - ANY() - Static method in class org.apache.spark.scheduler.TaskLocality
- any_value(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns some value of
e
for a group of rows. - any_value(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns some value of
e
for a group of rows. - AnyDataType - Class in org.apache.spark.sql.types
-
An
AbstractDataType
that matches any concrete data types. - AnyDataType() - Constructor for class org.apache.spark.sql.types.AnyDataType
- anyNull() - Method in interface org.apache.spark.sql.Row
-
Returns true if there are any NULL values in this row.
- anyNull() - Method in class org.apache.spark.sql.vectorized.ColumnarBatchRow
- anyNull() - Method in class org.apache.spark.sql.vectorized.ColumnarRow
- AnyTimestampType - Class in org.apache.spark.sql.types
- AnyTimestampType() - Constructor for class org.apache.spark.sql.types.AnyTimestampType
- AnyTimestampTypeExpression - Class in org.apache.spark.sql.types
- AnyTimestampTypeExpression() - Constructor for class org.apache.spark.sql.types.AnyTimestampTypeExpression
- ApiHelper - Class in org.apache.spark.ui.jobs
- ApiHelper() - Constructor for class org.apache.spark.ui.jobs.ApiHelper
- ApiRequestContext - Interface in org.apache.spark.status.api.v1
- APP_SPARK_VERSION_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ApplicationAttemptInfo
- appAttemptId() - Method in class org.apache.spark.scheduler.SparkListenerApplicationStart
- append() - Method in class org.apache.spark.sql.DataFrameWriterV2
-
Append the contents of the data frame to the output table.
- Append - Enum constant in enum class org.apache.spark.sql.SaveMode
-
Append mode means that when saving a DataFrame to a data source, if data/table already exists, contents of the DataFrame are expected to be appended to existing data.
- Append() - Static method in class org.apache.spark.sql.streaming.OutputMode
-
OutputMode in which only the new rows in the streaming DataFrame/Dataset will be written to the sink.
- appendBias(Vector) - Static method in class org.apache.spark.mllib.util.MLUtils
-
Returns a new vector with
1.0
(bias) appended to the input vector. - appendColumn(StructType, String, DataType, boolean) - Static method in class org.apache.spark.ml.util.SchemaUtils
-
Appends a new column to the input schema.
- appendColumn(StructType, StructField) - Static method in class org.apache.spark.ml.util.SchemaUtils
-
Appends a new column to the input schema.
- AppHistoryServerPlugin - Interface in org.apache.spark.status
-
An interface for creating history listeners(to replay event logs) defined in other modules like SQL, and setup the UI of the plugin to rebuild the history UI.
- appId() - Method in class org.apache.spark.scheduler.SparkListenerApplicationStart
- appId() - Method in interface org.apache.spark.status.api.v1.BaseAppResource
- appId() - Method in class org.apache.spark.storage.ShuffleMergedDataBlockId
- appId() - Method in class org.apache.spark.storage.ShuffleMergedIndexBlockId
- appId() - Method in class org.apache.spark.storage.ShuffleMergedMetaBlockId
- APPLICATION_EXECUTOR_LIMIT() - Static method in class org.apache.spark.ui.ToolTips
- APPLICATION_MASTER() - Static method in class org.apache.spark.metrics.MetricsSystemInstances
- applicationAttemptId() - Method in interface org.apache.spark.scheduler.SchedulerBackend
-
Get the attempt ID for this run, if the cluster manager supports multiple attempts.
- applicationAttemptId() - Method in interface org.apache.spark.scheduler.TaskScheduler
-
Get an application's attempt ID associated with the job.
- applicationAttemptId() - Method in class org.apache.spark.SparkContext
- ApplicationAttemptInfo - Class in org.apache.spark.status.api.v1
- applicationEndFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- applicationEndToJson(SparkListenerApplicationEnd, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- ApplicationEnvironmentInfo - Class in org.apache.spark.status.api.v1
- applicationId() - Method in interface org.apache.spark.scheduler.SchedulerBackend
-
Get an application ID associated with the job.
- applicationId() - Method in interface org.apache.spark.scheduler.TaskScheduler
-
Get an application ID associated with the job.
- applicationId() - Method in class org.apache.spark.SparkContext
-
A unique identifier for the Spark application.
- ApplicationInfo - Class in org.apache.spark.status.api.v1
- APPLICATIONS() - Static method in class org.apache.spark.metrics.MetricsSystemInstances
- applicationStartFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- applicationStartToJson(SparkListenerApplicationStart, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- ApplicationStatus - Enum Class in org.apache.spark.status.api.v1
- apply() - Static method in class org.apache.spark.ml.TransformEnd
- apply() - Static method in class org.apache.spark.ml.TransformStart
- apply() - Static method in class org.apache.spark.scheduler.local.ReviveOffers
- apply() - Static method in class org.apache.spark.scheduler.local.StopExecutor
- apply() - Static method in class org.apache.spark.sql.Observation
-
Observation constructor for creating an anonymous observation.
- apply() - Static method in class org.apache.spark.sql.types.DayTimeIntervalType
- apply() - Static method in class org.apache.spark.sql.types.YearMonthIntervalType
- apply(boolean, boolean, boolean, boolean, int) - Static method in class org.apache.spark.storage.StorageLevel
-
:: DeveloperApi :: Create a new StorageLevel object.
- apply(boolean, boolean, boolean, int) - Static method in class org.apache.spark.storage.StorageLevel
-
:: DeveloperApi :: Create a new StorageLevel object without setting useOffHeap.
- apply(byte) - Static method in class org.apache.spark.sql.types.DayTimeIntervalType
- apply(byte) - Static method in class org.apache.spark.sql.types.YearMonthIntervalType
- apply(double) - Static method in class org.apache.spark.sql.types.Decimal
- apply(int) - Static method in class org.apache.spark.ErrorMessageFormat
- apply(int) - Method in class org.apache.spark.ml.attribute.AttributeGroup
-
Gets an attribute by its index.
- apply(int) - Method in class org.apache.spark.ml.linalg.DenseVector
- apply(int) - Method in class org.apache.spark.ml.linalg.SparseVector
- apply(int) - Method in interface org.apache.spark.ml.linalg.Vector
-
Gets the value of the ith element.
- apply(int) - Method in class org.apache.spark.mllib.linalg.DenseVector
- apply(int) - Method in class org.apache.spark.mllib.linalg.SparseVector
- apply(int) - Method in interface org.apache.spark.mllib.linalg.Vector
-
Gets the value of the ith element.
- apply(int) - Static method in class org.apache.spark.mllib.tree.configuration.Algo
- apply(int) - Static method in class org.apache.spark.mllib.tree.configuration.EnsembleCombiningStrategy
- apply(int) - Static method in class org.apache.spark.mllib.tree.configuration.FeatureType
- apply(int) - Static method in class org.apache.spark.mllib.tree.configuration.QuantileStrategy
- apply(int) - Static method in class org.apache.spark.rdd.CheckpointState
- apply(int) - Static method in class org.apache.spark.rdd.DeterministicLevel
- apply(int) - Static method in class org.apache.spark.RequestMethod
- apply(int) - Static method in class org.apache.spark.scheduler.SchedulingMode
- apply(int) - Static method in class org.apache.spark.scheduler.TaskLocality
- apply(int) - Method in interface org.apache.spark.sql.Row
-
Returns the value at position i.
- apply(int) - Static method in class org.apache.spark.sql.types.Decimal
- apply(int) - Method in class org.apache.spark.sql.types.StructType
- apply(int) - Method in class org.apache.spark.status.RDDPartitionSeq
- apply(int) - Static method in class org.apache.spark.streaming.scheduler.ReceiverState
- apply(int) - Static method in class org.apache.spark.TaskState
- apply(int, int) - Method in class org.apache.spark.ml.linalg.DenseMatrix
- apply(int, int) - Method in interface org.apache.spark.ml.linalg.Matrix
-
Gets the (i, j)-th element.
- apply(int, int) - Method in class org.apache.spark.ml.linalg.SparseMatrix
- apply(int, int) - Method in class org.apache.spark.mllib.linalg.DenseMatrix
- apply(int, int) - Method in interface org.apache.spark.mllib.linalg.Matrix
-
Gets the (i, j)-th element.
- apply(int, int) - Method in class org.apache.spark.mllib.linalg.SparseMatrix
- apply(int, int) - Static method in class org.apache.spark.storage.StorageLevel
-
:: DeveloperApi :: Create a new StorageLevel object from its integer representation.
- apply(int, Node) - Static method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.NodeData
- apply(int, Node) - Method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.NodeData$
- apply(int, Predict, double, boolean) - Static method in class org.apache.spark.mllib.tree.model.Node
-
Construct a node with nodeIndex, predict, impurity and isLeaf parameters.
- apply(long) - Static method in class org.apache.spark.sql.types.Decimal
- apply(long) - Static method in class org.apache.spark.streaming.Milliseconds
- apply(long) - Static method in class org.apache.spark.streaming.Minutes
- apply(long) - Static method in class org.apache.spark.streaming.Seconds
- apply(long, int, int) - Static method in class org.apache.spark.sql.types.Decimal
- apply(long, TaskMetrics) - Static method in class org.apache.spark.scheduler.RuntimePercentage
- apply(DenseMatrix<Object>, DenseMatrix<Object>, DenseMatrix<Object>, Function2<Object, Object, Object>) - Static method in class org.apache.spark.ml.ann.ApplyInPlace
- apply(DenseMatrix<Object>, DenseMatrix<Object>, Function1<Object, Object>) - Static method in class org.apache.spark.ml.ann.ApplyInPlace
- apply(ObjectInput) - Static method in class org.apache.spark.storage.BlockManagerId
- apply(ObjectInput) - Static method in class org.apache.spark.storage.StorageLevel
-
:: DeveloperApi :: Read StorageLevel object from ObjectInput stream.
- apply(Object) - Method in class org.apache.spark.sql.Column
-
Extracts a value or values from a complex type.
- apply(String) - Method in class org.apache.spark.ml.attribute.AttributeGroup
-
Gets an attribute by its name.
- apply(String) - Method in class org.apache.spark.sql.Dataset
-
Selects column based on the column name and returns it as a
Column
. - apply(String) - Static method in class org.apache.spark.sql.Observation
-
Observation constructor for creating a named observation.
- apply(String) - Static method in class org.apache.spark.sql.types.Decimal
- apply(String) - Method in class org.apache.spark.sql.types.StructType
-
Extracts the
StructField
with the given name. - apply(String) - Static method in class org.apache.spark.storage.BlockId
- apply(String, long, Enumeration.Value, ByteBuffer, int, Map<String, ResourceInformation>) - Method in class org.apache.spark.scheduler.cluster.CoarseGrainedClusterMessages.StatusUpdate$
-
Alternate factory method that takes a ByteBuffer directly for the data field
- apply(String, String, int, Option<String>) - Static method in class org.apache.spark.storage.BlockManagerId
-
Returns a
BlockManagerId
for the given configuration. - apply(String, Expression...) - Static method in class org.apache.spark.sql.connector.expressions.Expressions
-
Create a logical transform for applying a named transform.
- apply(String, Seq<Expression>) - Static method in class org.apache.spark.sql.connector.expressions.LogicalExpressions
- apply(String, Option<Object>, Map<String, String>) - Static method in class org.apache.spark.sql.streaming.SinkProgress
- apply(BigDecimal) - Static method in class org.apache.spark.sql.types.Decimal
- apply(BigDecimal, int, int) - Static method in class org.apache.spark.sql.types.Decimal
- apply(BigInteger) - Static method in class org.apache.spark.sql.types.Decimal
- apply(Graph<VD, ED>, A, int, EdgeDirection, Function3<Object, VD, A, VD>, Function1<EdgeTriplet<VD, ED>, Iterator<Tuple2<Object, A>>>, Function2<A, A, A>, ClassTag<VD>, ClassTag<ED>, ClassTag<A>) - Static method in class org.apache.spark.graphx.Pregel
-
Execute a Pregel-like iterative vertex-parallel abstraction.
- apply(VertexRDD<VD>, EdgeRDD<ED>, ClassTag<VD>, ClassTag<ED>) - Static method in class org.apache.spark.graphx.impl.GraphImpl
-
Create a graph from a VertexRDD and an EdgeRDD with arbitrary replicated vertices.
- apply(Param<T>) - Method in class org.apache.spark.ml.param.ParamMap
-
Gets the value of the input param or its default value if it does not exist.
- apply(GeneralizedLinearRegressionBase) - Method in class org.apache.spark.ml.regression.GeneralizedLinearRegression.FamilyAndLink$
-
Constructs the FamilyAndLink object from a parameter map
- apply(Split) - Method in class org.apache.spark.ml.tree.DecisionTreeModelReadWrite.SplitData$
- apply(BinaryConfusionMatrix) - Method in interface org.apache.spark.mllib.evaluation.binary.BinaryClassificationMetricComputer
- apply(BinaryConfusionMatrix) - Static method in class org.apache.spark.mllib.evaluation.binary.FalsePositiveRate
- apply(BinaryConfusionMatrix) - Static method in class org.apache.spark.mllib.evaluation.binary.Precision
- apply(BinaryConfusionMatrix) - Static method in class org.apache.spark.mllib.evaluation.binary.Recall
- apply(Predict) - Static method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.PredictData
- apply(Predict) - Method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.PredictData$
- apply(Split) - Static method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.SplitData
- apply(Split) - Method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.SplitData$
- apply(RDD<Edge<ED>>, VD, StorageLevel, StorageLevel, ClassTag<VD>, ClassTag<ED>) - Static method in class org.apache.spark.graphx.impl.GraphImpl
-
Create a graph from edges, setting referenced vertices to
defaultVertexAttr
. - apply(RDD<Tuple2<Object, VD>>, EdgeRDD<?>, VD, Function2<VD, VD, VD>, ClassTag<VD>) - Static method in class org.apache.spark.graphx.VertexRDD
-
Constructs a
VertexRDD
from an RDD of vertex-attribute pairs. - apply(RDD<Tuple2<Object, VD>>, EdgeRDD<?>, VD, ClassTag<VD>) - Static method in class org.apache.spark.graphx.VertexRDD
-
Constructs a
VertexRDD
from an RDD of vertex-attribute pairs. - apply(RDD<Tuple2<Object, VD>>, RDD<Edge<ED>>, VD, StorageLevel, StorageLevel, ClassTag<VD>, ClassTag<ED>) - Static method in class org.apache.spark.graphx.Graph
-
Construct a graph from a collection of vertices and edges with attributes.
- apply(RDD<Tuple2<Object, VD>>, RDD<Edge<ED>>, VD, StorageLevel, StorageLevel, ClassTag<VD>, ClassTag<ED>) - Static method in class org.apache.spark.graphx.impl.GraphImpl
-
Create a graph from vertices and edges, setting missing vertices to
defaultVertexAttr
. - apply(RDD<Tuple2<Object, VD>>, ClassTag<VD>) - Static method in class org.apache.spark.graphx.VertexRDD
-
Constructs a standalone
VertexRDD
(one that is not set up for efficient joins with anEdgeRDD
) from an RDD of vertex-attribute pairs. - apply(Column...) - Method in class org.apache.spark.sql.expressions.UserDefinedAggregateFunction
-
Deprecated.Creates a
Column
for this UDAF using givenColumn
s as input arguments. - apply(Column...) - Method in class org.apache.spark.sql.expressions.UserDefinedFunction
-
Returns an expression that invokes the UDF, using the given arguments.
- apply(Dataset<Row>, Seq<Expression>, RelationalGroupedDataset.GroupType) - Static method in class org.apache.spark.sql.RelationalGroupedDataset
- apply(Row) - Method in class org.apache.spark.mllib.clustering.KMeansModel.Cluster$
- apply(Row) - Static method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.NodeData
- apply(Row) - Method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.NodeData$
- apply(Row) - Static method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.PredictData
- apply(Row) - Method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.PredictData$
- apply(Row) - Static method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.SplitData
- apply(Row) - Method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.SplitData$
- apply(DataType) - Static method in class org.apache.spark.sql.types.ArrayType
-
Construct a
ArrayType
object with the given element type. - apply(DataType, DataType) - Static method in class org.apache.spark.sql.types.MapType
-
Construct a
MapType
object with the given key type and value type. - apply(Set<String>) - Method in class org.apache.spark.sql.types.StructType
-
Returns a
StructType
containingStructField
s of the given names, preserving the original order of fields. - apply(Seq<Object>) - Static method in class org.apache.spark.util.StatCounter
-
Build a StatCounter from a list of values passed as variable-length arguments.
- apply(Seq<Column>) - Method in class org.apache.spark.sql.expressions.UserDefinedAggregateFunction
-
Deprecated.Creates a
Column
for this UDAF using givenColumn
s as input arguments. - apply(Seq<Column>) - Method in class org.apache.spark.sql.expressions.UserDefinedFunction
-
Returns an expression that invokes the UDF, using the given arguments.
- apply(TraversableOnce<Object>) - Static method in class org.apache.spark.util.StatCounter
-
Build a StatCounter from a list of values.
- apply(BigDecimal) - Static method in class org.apache.spark.sql.types.Decimal
- apply(BigDecimal, int, int) - Static method in class org.apache.spark.sql.types.Decimal
- apply(BigInt) - Static method in class org.apache.spark.sql.types.Decimal
- apply(T1) - Static method in class org.apache.spark.CleanAccum
- apply(T1) - Static method in class org.apache.spark.CleanBroadcast
- apply(T1) - Static method in class org.apache.spark.CleanCheckpoint
- apply(T1) - Static method in class org.apache.spark.CleanRDD
- apply(T1) - Static method in class org.apache.spark.CleanShuffle
- apply(T1) - Static method in class org.apache.spark.CleanSparkListener
- apply(T1) - Static method in class org.apache.spark.ErrorSubInfo
- apply(T1) - Static method in class org.apache.spark.ExecutorRegistered
- apply(T1) - Static method in class org.apache.spark.ExecutorRemoved
- apply(T1) - Static method in class org.apache.spark.ml.SaveInstanceEnd
- apply(T1) - Static method in class org.apache.spark.ml.SaveInstanceStart
- apply(T1) - Static method in class org.apache.spark.mllib.feature.ChiSqSelectorModel.SaveLoadV1_0$.Data
- apply(T1) - Static method in class org.apache.spark.scheduler.SparkListenerApplicationEnd
- apply(T1) - Static method in class org.apache.spark.scheduler.SparkListenerBlockUpdated
- apply(T1) - Static method in class org.apache.spark.scheduler.SparkListenerEnvironmentUpdate
- apply(T1) - Static method in class org.apache.spark.scheduler.SparkListenerLogStart
- apply(T1) - Static method in class org.apache.spark.scheduler.SparkListenerResourceProfileAdded
- apply(T1) - Static method in class org.apache.spark.scheduler.SparkListenerStageCompleted
- apply(T1) - Static method in class org.apache.spark.scheduler.SparkListenerTaskGettingResult
- apply(T1) - Static method in class org.apache.spark.scheduler.SparkListenerUnpersistRDD
- apply(T1) - Static method in class org.apache.spark.sql.sources.IsNotNull
- apply(T1) - Static method in class org.apache.spark.sql.sources.IsNull
- apply(T1) - Static method in class org.apache.spark.sql.sources.Not
- apply(T1) - Static method in class org.apache.spark.sql.types.CharType
- apply(T1) - Static method in class org.apache.spark.sql.types.VarcharType
- apply(T1) - Static method in class org.apache.spark.status.api.v1.StackTrace
- apply(T1) - Static method in class org.apache.spark.storage.TaskResultBlockId
- apply(T1) - Static method in class org.apache.spark.streaming.Duration
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerBatchCompleted
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerBatchStarted
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerBatchSubmitted
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerOutputOperationCompleted
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerOutputOperationStarted
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerReceiverError
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerReceiverStarted
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerReceiverStopped
- apply(T1) - Static method in class org.apache.spark.streaming.scheduler.StreamingListenerStreamingStarted
- apply(T1, T2) - Static method in class org.apache.spark.ContextBarrierId
- apply(T1, T2) - Static method in class org.apache.spark.ml.clustering.ClusterData
- apply(T1, T2) - Static method in class org.apache.spark.ml.feature.LabeledPoint
- apply(T1, T2) - Static method in class org.apache.spark.mllib.linalg.distributed.IndexedRow
- apply(T1, T2) - Static method in class org.apache.spark.mllib.regression.impl.GLMRegressionModel.SaveLoadV1_0$.Data
- apply(T1, T2) - Static method in class org.apache.spark.mllib.stat.test.BinarySample
- apply(T1, T2) - Static method in class org.apache.spark.resource.ResourceInformationJson
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.ExcludedExecutor
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerBlockManagerRemoved
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorUnblacklisted
-
Deprecated.
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorUnexcluded
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerNodeUnblacklisted
-
Deprecated.
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerNodeUnexcluded
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerSpeculativeTaskSubmitted
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerStageSubmitted
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerUnschedulableTaskSetAdded
- apply(T1, T2) - Static method in class org.apache.spark.scheduler.SparkListenerUnschedulableTaskSetRemoved
- apply(T1, T2) - Static method in class org.apache.spark.sql.jdbc.JdbcType
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.And
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.EqualNullSafe
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.EqualTo
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.GreaterThan
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.GreaterThanOrEqual
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.In
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.LessThan
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.LessThanOrEqual
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.Or
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.StringContains
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.StringEndsWith
- apply(T1, T2) - Static method in class org.apache.spark.sql.sources.StringStartsWith
- apply(T1, T2) - Static method in class org.apache.spark.status.api.v1.sql.Metric
- apply(T1, T2) - Static method in class org.apache.spark.storage.BroadcastBlockId
- apply(T1, T2) - Static method in class org.apache.spark.storage.RDDBlockId
- apply(T1, T2) - Static method in class org.apache.spark.storage.StreamBlockId
- apply(T1, T2, T3) - Static method in class org.apache.spark.ErrorInfo
- apply(T1, T2, T3) - Static method in class org.apache.spark.ExecutorLostFailure
- apply(T1, T2, T3) - Static method in class org.apache.spark.mllib.classification.impl.GLMClassificationModel.SaveLoadV1_0$.Data
- apply(T1, T2, T3) - Static method in class org.apache.spark.mllib.classification.NaiveBayesModel.SaveLoadV1_0$.Data
- apply(T1, T2, T3) - Static method in class org.apache.spark.mllib.linalg.distributed.MatrixEntry
- apply(T1, T2, T3) - Static method in class org.apache.spark.mllib.recommendation.Rating
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.local.KillTask
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.local.StatusUpdate
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorAdded
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorBlacklisted
-
Deprecated.
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorExcluded
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorMetricsUpdate
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorRemoved
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerJobEnd
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerMiscellaneousProcessAdded
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerNodeBlacklisted
-
Deprecated.
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerNodeExcluded
- apply(T1, T2, T3) - Static method in class org.apache.spark.scheduler.SparkListenerTaskStart
- apply(T1, T2, T3) - Static method in class org.apache.spark.storage.CacheId
- apply(T1, T2, T3) - Static method in class org.apache.spark.storage.ShuffleBlockId
- apply(T1, T2, T3) - Static method in class org.apache.spark.storage.ShuffleChecksumBlockId
- apply(T1, T2, T3) - Static method in class org.apache.spark.storage.ShuffleDataBlockId
- apply(T1, T2, T3) - Static method in class org.apache.spark.storage.ShuffleIndexBlockId
- apply(T1, T2, T3) - Static method in class org.apache.spark.storage.ShuffleMergedBlockId
- apply(T1, T2, T3) - Static method in class org.apache.spark.TaskCommitDenied
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.mllib.classification.NaiveBayesModel.SaveLoadV2_0$.Data
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.mllib.tree.model.Split
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.scheduler.AskPermissionToCommitOutput
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.scheduler.SparkListenerJobStart
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.scheduler.SparkListenerStageExecutorMetrics
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.sql.types.StructField
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.status.api.v1.sql.Node
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.storage.ShuffleBlockBatchId
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.storage.ShuffleBlockChunkId
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.storage.ShuffleMergedDataBlockId
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.storage.ShuffleMergedIndexBlockId
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.storage.ShuffleMergedMetaBlockId
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.storage.ShufflePushBlockId
- apply(T1, T2, T3, T4) - Static method in class org.apache.spark.TaskKilled
- apply(T1, T2, T3, T4, T5) - Static method in class org.apache.spark.mllib.feature.VocabWord
- apply(T1, T2, T3, T4, T5) - Static method in class org.apache.spark.scheduler.SparkListenerBlockManagerAdded
- apply(T1, T2, T3, T4, T5) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorBlacklistedForStage
-
Deprecated.
- apply(T1, T2, T3, T4, T5) - Static method in class org.apache.spark.scheduler.SparkListenerExecutorExcludedForStage
- apply(T1, T2, T3, T4, T5) - Static method in class org.apache.spark.scheduler.SparkListenerNodeBlacklistedForStage
-
Deprecated.
- apply(T1, T2, T3, T4, T5) - Static method in class org.apache.spark.scheduler.SparkListenerNodeExcludedForStage
- apply(T1, T2, T3, T4, T5, T6) - Static method in class org.apache.spark.FetchFailed
- apply(T1, T2, T3, T4, T5, T6) - Static method in class org.apache.spark.streaming.scheduler.BatchInfo
- apply(T1, T2, T3, T4, T5, T6, T7) - Static method in class org.apache.spark.scheduler.AccumulableInfo
- apply(T1, T2, T3, T4, T5, T6, T7) - Static method in class org.apache.spark.scheduler.SparkListenerApplicationStart
- apply(T1, T2, T3, T4, T5, T6, T7) - Static method in class org.apache.spark.scheduler.SparkListenerTaskEnd
- apply(T1, T2, T3, T4, T5, T6, T7) - Static method in class org.apache.spark.status.api.v1.ApplicationInfo
- apply(T1, T2, T3, T4, T5, T6, T7) - Static method in class org.apache.spark.status.api.v1.ThreadStackTrace
- apply(T1, T2, T3, T4, T5, T6, T7) - Static method in class org.apache.spark.streaming.scheduler.OutputOperationInfo
- apply(T1, T2, T3, T4, T5, T6, T7, T8) - Static method in class org.apache.spark.ExceptionFailure
- apply(T1, T2, T3, T4, T5, T6, T7, T8) - Static method in class org.apache.spark.status.api.v1.ApplicationAttemptInfo
- apply(T1, T2, T3, T4, T5, T6, T7, T8) - Static method in class org.apache.spark.streaming.scheduler.ReceiverInfo
- ApplyInPlace - Class in org.apache.spark.ml.ann
-
Implements in-place application of functions in the arrays
- ApplyInPlace() - Constructor for class org.apache.spark.ml.ann.ApplyInPlace
- applyNamespaceChanges(Map<String, String>, Seq<NamespaceChange>) - Static method in class org.apache.spark.sql.connector.catalog.CatalogV2Util
-
Apply properties changes to a Java map and return the result.
- applyNamespaceChanges(Map<String, String>, Seq<NamespaceChange>) - Static method in class org.apache.spark.sql.connector.catalog.CatalogV2Util
-
Apply properties changes to a map and return the result.
- applyPropertiesChanges(Map<String, String>, Seq<TableChange>) - Static method in class org.apache.spark.sql.connector.catalog.CatalogV2Util
-
Apply properties changes to a Java map and return the result.
- applyPropertiesChanges(Map<String, String>, Seq<TableChange>) - Static method in class org.apache.spark.sql.connector.catalog.CatalogV2Util
-
Apply properties changes to a map and return the result.
- applySchema(JavaRDD<?>, Class<?>) - Method in class org.apache.spark.sql.SQLContext
-
Deprecated.Use createDataFrame instead. Since 1.3.0.
- applySchema(JavaRDD<Row>, StructType) - Method in class org.apache.spark.sql.SQLContext
-
Deprecated.Use createDataFrame instead. Since 1.3.0.
- applySchema(RDD<?>, Class<?>) - Method in class org.apache.spark.sql.SQLContext
-
Deprecated.Use createDataFrame instead. Since 1.3.0.
- applySchema(RDD<Row>, StructType) - Method in class org.apache.spark.sql.SQLContext
-
Deprecated.Use createDataFrame instead. Since 1.3.0.
- applySchemaChanges(StructType, Seq<TableChange>, Option<String>, String) - Static method in class org.apache.spark.sql.connector.catalog.CatalogV2Util
-
Apply schema changes to a schema and return the result.
- appName() - Method in class org.apache.spark.api.java.JavaSparkContext
- appName() - Method in class org.apache.spark.scheduler.SparkListenerApplicationStart
- appName() - Method in class org.apache.spark.SparkContext
- appName(String) - Method in class org.apache.spark.sql.SparkSession.Builder
-
Sets a name for the application, which will be shown in the Spark web UI.
- approx_count_distinct(String) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the approximate number of distinct items in a group.
- approx_count_distinct(String, double) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the approximate number of distinct items in a group.
- approx_count_distinct(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the approximate number of distinct items in a group.
- approx_count_distinct(Column, double) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the approximate number of distinct items in a group.
- approx_percentile(Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the approximate
percentile
of the numeric columncol
which is the smallest value in the orderedcol
values (sorted from least to greatest) such that no more thanpercentage
ofcol
values is less than the value or equal to that value. - approxCountDistinct(String) - Static method in class org.apache.spark.sql.functions
-
Deprecated.Use approx_count_distinct. Since 2.1.0.
- approxCountDistinct(String, double) - Static method in class org.apache.spark.sql.functions
-
Deprecated.Use approx_count_distinct. Since 2.1.0.
- approxCountDistinct(Column) - Static method in class org.apache.spark.sql.functions
-
Deprecated.Use approx_count_distinct. Since 2.1.0.
- approxCountDistinct(Column, double) - Static method in class org.apache.spark.sql.functions
-
Deprecated.Use approx_count_distinct. Since 2.1.0.
- ApproxHist() - Static method in class org.apache.spark.mllib.tree.configuration.QuantileStrategy
- ApproximateEvaluator<U,
R> - Interface in org.apache.spark.partial -
An object that computes a function incrementally by merging in results of type U from multiple tasks.
- approxNearestNeighbors(Dataset<?>, Vector, int) - Method in class org.apache.spark.ml.feature.BucketedRandomProjectionLSHModel
-
Overloaded method for approxNearestNeighbors.
- approxNearestNeighbors(Dataset<?>, Vector, int, String) - Method in class org.apache.spark.ml.feature.BucketedRandomProjectionLSHModel
-
Given a large dataset and an item, approximately find at most k items which have the closest distance to the item.
- approxQuantile(String[], double[], double) - Method in class org.apache.spark.sql.DataFrameStatFunctions
-
Calculates the approximate quantiles of numerical columns of a DataFrame.
- approxQuantile(String, double[], double) - Method in class org.apache.spark.sql.DataFrameStatFunctions
-
Calculates the approximate quantiles of a numerical column of a DataFrame.
- approxSimilarityJoin(Dataset<?>, Dataset<?>, double) - Method in class org.apache.spark.ml.feature.BucketedRandomProjectionLSHModel
-
Overloaded method for approxSimilarityJoin.
- approxSimilarityJoin(Dataset<?>, Dataset<?>, double, String) - Method in class org.apache.spark.ml.feature.BucketedRandomProjectionLSHModel
-
Join two datasets to approximately find all pairs of rows whose distance are smaller than the threshold.
- appSparkVersion() - Method in class org.apache.spark.status.api.v1.ApplicationAttemptInfo
- AppStatusUtils - Class in org.apache.spark.status
- AppStatusUtils() - Constructor for class org.apache.spark.status.AppStatusUtils
- archives() - Method in class org.apache.spark.SparkContext
- AreaUnderCurve - Class in org.apache.spark.mllib.evaluation
-
Computes the area under the curve (AUC) using the trapezoidal rule.
- AreaUnderCurve() - Constructor for class org.apache.spark.mllib.evaluation.AreaUnderCurve
- areaUnderPR() - Method in class org.apache.spark.mllib.evaluation.BinaryClassificationMetrics
-
Computes the area under the precision-recall curve.
- areaUnderROC() - Method in interface org.apache.spark.ml.classification.BinaryClassificationSummary
-
Computes the area under the receiver operating characteristic (ROC) curve.
- areaUnderROC() - Method in class org.apache.spark.ml.classification.BinaryLogisticRegressionSummaryImpl
- areaUnderROC() - Method in class org.apache.spark.ml.classification.BinaryRandomForestClassificationSummaryImpl
- areaUnderROC() - Method in class org.apache.spark.ml.classification.FMClassificationSummaryImpl
- areaUnderROC() - Method in class org.apache.spark.ml.classification.LinearSVCSummaryImpl
- areaUnderROC() - Method in class org.apache.spark.mllib.evaluation.BinaryClassificationMetrics
-
Computes the area under the receiver operating characteristic (ROC) curve.
- argmax() - Method in class org.apache.spark.ml.linalg.DenseVector
- argmax() - Method in class org.apache.spark.ml.linalg.SparseVector
- argmax() - Method in interface org.apache.spark.ml.linalg.Vector
-
Find the index of a maximal element.
- argmax() - Method in class org.apache.spark.mllib.linalg.DenseVector
- argmax() - Method in class org.apache.spark.mllib.linalg.SparseVector
- argmax() - Method in interface org.apache.spark.mllib.linalg.Vector
-
Find the index of a maximal element.
- arguments() - Method in interface org.apache.spark.sql.connector.expressions.Transform
-
Returns the arguments passed to the transform function.
- arithmeticOverflowError(ArithmeticException) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- arithmeticOverflowError(String, String, SQLQueryContext) - Method in interface org.apache.spark.sql.errors.ExecutionErrors
- arithmeticOverflowError$default$2() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- arithmeticOverflowError$default$3() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- ARPACK - Class in org.apache.spark.mllib.linalg
-
ARPACK routines for MLlib's vectors and matrices.
- ARPACK() - Constructor for class org.apache.spark.mllib.linalg.ARPACK
- array() - Method in class org.apache.spark.sql.vectorized.ColumnarArray
- array(String, String...) - Static method in class org.apache.spark.sql.functions
-
Creates a new array column.
- array(String, Seq<String>) - Static method in class org.apache.spark.sql.functions
-
Creates a new array column.
- array(Column...) - Static method in class org.apache.spark.sql.functions
-
Creates a new array column.
- array(DataType) - Method in class org.apache.spark.sql.ColumnName
-
Creates a new
StructField
of type array. - array(Seq<Column>) - Static method in class org.apache.spark.sql.functions
-
Creates a new array column.
- array_agg(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns a list of objects with duplicates.
- array_append(Column, Object) - Static method in class org.apache.spark.sql.functions
-
Returns an ARRAY containing all elements from the source ARRAY as well as the new element.
- array_compact(Column) - Static method in class org.apache.spark.sql.functions
-
Remove all null elements from the given array.
- array_contains(Column, Object) - Static method in class org.apache.spark.sql.functions
-
Returns null if the array is null, true if the array contains
value
, and false otherwise. - array_distinct(Column) - Static method in class org.apache.spark.sql.functions
-
Removes duplicate values from the array.
- array_except(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns an array of the elements in the first array but not in the second array, without duplicates.
- array_insert(Column, Column, Column) - Static method in class org.apache.spark.sql.functions
-
Adds an item into a given array at a specified position
- array_intersect(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns an array of the elements in the intersection of the given two arrays, without duplicates.
- array_join(Column, String) - Static method in class org.apache.spark.sql.functions
-
Concatenates the elements of
column
using thedelimiter
. - array_join(Column, String, String) - Static method in class org.apache.spark.sql.functions
-
Concatenates the elements of
column
using thedelimiter
. - array_max(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the maximum value in the array.
- array_min(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the minimum value in the array.
- array_position(Column, Object) - Static method in class org.apache.spark.sql.functions
-
Locates the position of the first occurrence of the value in the given array as long.
- array_prepend(Column, Object) - Static method in class org.apache.spark.sql.functions
-
Returns an array containing value as well as all elements from array.
- array_remove(Column, Object) - Static method in class org.apache.spark.sql.functions
-
Remove all elements that equal to element from the given array.
- array_repeat(Column, int) - Static method in class org.apache.spark.sql.functions
-
Creates an array containing the left argument repeated the number of times given by the right argument.
- array_repeat(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Creates an array containing the left argument repeated the number of times given by the right argument.
- array_size(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the total number of elements in the array.
- array_sort(Column) - Static method in class org.apache.spark.sql.functions
-
Sorts the input array in ascending order.
- array_sort(Column, Function2<Column, Column, Column>) - Static method in class org.apache.spark.sql.functions
-
Sorts the input array based on the given comparator function.
- array_to_vector(Column) - Static method in class org.apache.spark.ml.functions
-
Converts a column of array of numeric type into a column of dense vectors in MLlib.
- array_union(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns an array of the elements in the union of the given two arrays, without duplicates.
- arrayComponentTypeUnsupportedError(Class<?>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- arrayLengthGt(double) - Static method in class org.apache.spark.ml.param.ParamValidators
-
Check that the array length is greater than lowerBound.
- arrays_overlap(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns
true
ifa1
anda2
have at least one non-null element in common. - arrays_zip(Column...) - Static method in class org.apache.spark.sql.functions
-
Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays.
- arrays_zip(Seq<Column>) - Static method in class org.apache.spark.sql.functions
-
Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays.
- ArrayType - Class in org.apache.spark.sql.types
- ArrayType(DataType, boolean) - Constructor for class org.apache.spark.sql.types.ArrayType
- arrayValues() - Method in class org.apache.spark.storage.memory.DeserializedValuesHolder
- ArrowColumnVector - Class in org.apache.spark.sql.vectorized
-
A column vector backed by Apache Arrow.
- ArrowColumnVector(ValueVector) - Constructor for class org.apache.spark.sql.vectorized.ArrowColumnVector
- ArrowUtils - Class in org.apache.spark.sql.util
- ArrowUtils() - Constructor for class org.apache.spark.sql.util.ArrowUtils
- as(String) - Method in class org.apache.spark.sql.Column
-
Gives the column an alias.
- as(String) - Method in class org.apache.spark.sql.Dataset
-
Returns a new Dataset with an alias set.
- as(String[]) - Method in class org.apache.spark.sql.Column
-
Assigns the given aliases to the results of a table generating function.
- as(String, Metadata) - Method in class org.apache.spark.sql.Column
-
Gives the column an alias with metadata.
- as(Encoder<K>, Encoder<T>) - Method in class org.apache.spark.sql.RelationalGroupedDataset
-
Returns a
KeyValueGroupedDataset
where the data is grouped by the grouping expressions of currentRelationalGroupedDataset
. - as(Encoder<U>) - Method in class org.apache.spark.sql.Column
-
Provides a type hint about the expected return value of this column.
- as(Encoder<U>) - Method in class org.apache.spark.sql.Dataset
-
Returns a new Dataset where each record has been mapped on to the specified type.
- as(Seq<String>) - Method in class org.apache.spark.sql.Column
-
(Scala-specific) Assigns the given aliases to the results of a table generating function.
- as(Symbol) - Method in class org.apache.spark.sql.Column
-
Gives the column an alias.
- as(Symbol) - Method in class org.apache.spark.sql.Dataset
-
(Scala-specific) Returns a new Dataset with an alias set.
- asBinary() - Method in interface org.apache.spark.ml.classification.LogisticRegressionSummary
-
Convenient method for casting to binary logistic regression summary.
- asBinary() - Method in interface org.apache.spark.ml.classification.RandomForestClassificationSummary
-
Convenient method for casting to BinaryRandomForestClassificationSummary.
- asBreeze() - Method in interface org.apache.spark.ml.linalg.Matrix
-
Converts to a breeze matrix.
- asBreeze() - Method in interface org.apache.spark.ml.linalg.Vector
-
Converts the instance to a breeze vector.
- asBreeze() - Method in interface org.apache.spark.mllib.linalg.Matrix
-
Converts to a breeze matrix.
- asBreeze() - Method in interface org.apache.spark.mllib.linalg.Vector
-
Converts the instance to a breeze vector.
- asc() - Method in class org.apache.spark.sql.Column
-
Returns a sort expression based on ascending order of the column.
- asc(String) - Static method in class org.apache.spark.sql.functions
-
Returns a sort expression based on ascending order of the column.
- asc_nulls_first() - Method in class org.apache.spark.sql.Column
-
Returns a sort expression based on ascending order of the column, and null values return before non-null values.
- asc_nulls_first(String) - Static method in class org.apache.spark.sql.functions
-
Returns a sort expression based on ascending order of the column, and null values return before non-null values.
- asc_nulls_last() - Method in class org.apache.spark.sql.Column
-
Returns a sort expression based on ascending order of the column, and null values appear after non-null values.
- asc_nulls_last(String) - Static method in class org.apache.spark.sql.functions
-
Returns a sort expression based on ascending order of the column, and null values appear after non-null values.
- asCaseSensitiveMap() - Method in class org.apache.spark.sql.util.CaseInsensitiveStringMap
-
Returns the original case-sensitive map.
- ASCENDING - Enum constant in enum class org.apache.spark.sql.connector.expressions.SortDirection
- ascii(Column) - Static method in class org.apache.spark.sql.functions
-
Computes the numeric value of the first character of the string column, and returns the result as an int column.
- asFunctionCatalog() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.CatalogHelper
- asFunctionIdentifier() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.IdentifierHelper
- asFunctionIdentifier() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.MultipartIdentifierHelper
- asIdentifier() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.MultipartIdentifierHelper
- asin(String) - Static method in class org.apache.spark.sql.functions
- asin(Column) - Static method in class org.apache.spark.sql.functions
- asinh(String) - Static method in class org.apache.spark.sql.functions
- asinh(Column) - Static method in class org.apache.spark.sql.functions
- asInteraction() - Static method in class org.apache.spark.ml.feature.Dot
- asInteraction() - Method in interface org.apache.spark.ml.feature.InteractableTerm
-
Convert to ColumnInteraction to wrap all interactions.
- asIterator() - Method in class org.apache.spark.serializer.DeserializationStream
-
Read the elements of this stream through an iterator.
- asJavaPairRDD() - Method in class org.apache.spark.api.r.PairwiseRRDD
- asJavaRDD() - Method in class org.apache.spark.api.r.RRDD
- asJavaRDD() - Method in class org.apache.spark.api.r.StringRRDD
- ask(Object) - Method in interface org.apache.spark.api.plugin.PluginContext
-
Send an RPC to the plugin's driver-side component.
- asKeyValueIterator() - Method in class org.apache.spark.serializer.DeserializationStream
-
Read the elements of this stream through an iterator over key-value pairs.
- AskPermissionToCommitOutput - Class in org.apache.spark.scheduler
- AskPermissionToCommitOutput(int, int, int, int) - Constructor for class org.apache.spark.scheduler.AskPermissionToCommitOutput
- askRpcTimeout(SparkConf) - Static method in class org.apache.spark.util.RpcUtils
-
Returns the default Spark timeout to use for RPC ask operations.
- askStandaloneSchedulerToShutDownExecutorsError(Exception) - Static method in class org.apache.spark.errors.SparkCoreErrors
- askStorageEndpoints() - Method in class org.apache.spark.storage.BlockManagerMessages.GetBlockStatus
- askStorageEndpoints() - Method in class org.apache.spark.storage.BlockManagerMessages.GetMatchingBlockIds
- asML() - Method in class org.apache.spark.mllib.linalg.DenseMatrix
- asML() - Method in class org.apache.spark.mllib.linalg.DenseVector
- asML() - Method in interface org.apache.spark.mllib.linalg.Matrix
-
Convert this matrix to the new mllib-local representation.
- asML() - Method in class org.apache.spark.mllib.linalg.SparseMatrix
- asML() - Method in class org.apache.spark.mllib.linalg.SparseVector
- asML() - Method in interface org.apache.spark.mllib.linalg.Vector
-
Convert this vector to the new mllib-local representation.
- asMultipart() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.FunctionIdentifierHelper
- asMultipartIdentifier() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.IdentifierHelper
- asNamespaceCatalog() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.CatalogHelper
- asNondeterministic() - Method in class org.apache.spark.sql.expressions.UserDefinedFunction
-
Updates UserDefinedFunction to nondeterministic.
- asNonNullable() - Method in class org.apache.spark.sql.expressions.UserDefinedFunction
-
Updates UserDefinedFunction to non-nullable.
- asNullable() - Method in class org.apache.spark.sql.types.ObjectType
- asRDDId() - Method in class org.apache.spark.storage.BlockId
- asSchema() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.ColumnsHelper
- assert_true(Column) - Static method in class org.apache.spark.sql.functions
-
Returns null if the condition is true, and throws an exception otherwise.
- assert_true(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns null if the condition is true; throws an exception with the error message otherwise.
- assertExceptionMsg(Throwable, String, boolean, ClassTag<E>) - Static method in class org.apache.spark.TestUtils
-
Asserts that exception message contains the message.
- assertNotSpilled(SparkContext, String, Function0<BoxedUnit>) - Static method in class org.apache.spark.TestUtils
-
Run some code involving jobs submitted to the given context and assert that the jobs did not spill.
- assertSpilled(SparkContext, String, Function0<BoxedUnit>) - Static method in class org.apache.spark.TestUtils
-
Run some code involving jobs submitted to the given context and assert that the jobs spilled.
- assignClusters(Dataset<?>) - Method in class org.apache.spark.ml.clustering.PowerIterationClustering
-
Run the PIC algorithm and returns a cluster assignment for each input vertex.
- assignedAddrs() - Method in interface org.apache.spark.resource.ResourceAllocator
-
Sequence of currently assigned resource addresses.
- Assignment(long, int) - Constructor for class org.apache.spark.mllib.clustering.PowerIterationClustering.Assignment
- Assignment$() - Constructor for class org.apache.spark.mllib.clustering.PowerIterationClustering.Assignment$
- assignments() - Method in class org.apache.spark.mllib.clustering.PowerIterationClusteringModel
- associationRules() - Method in class org.apache.spark.ml.fpm.FPGrowthModel
-
Get association rules fitted using the minConfidence.
- AssociationRules - Class in org.apache.spark.ml.fpm
- AssociationRules - Class in org.apache.spark.mllib.fpm
-
Generates association rules from a
RDD[FreqItemset[Item}
. - AssociationRules() - Constructor for class org.apache.spark.ml.fpm.AssociationRules
- AssociationRules() - Constructor for class org.apache.spark.mllib.fpm.AssociationRules
-
Constructs a default instance with default parameters {minConfidence = 0.8}.
- AssociationRules.Rule<Item> - Class in org.apache.spark.mllib.fpm
-
An association rule between sets of items.
- asTableCatalog() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.CatalogHelper
- asTableIdentifier() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.IdentifierHelper
- asTableIdentifier() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.MultipartIdentifierHelper
- AsTableIdentifier() - Constructor for class org.apache.spark.sql.connector.catalog.LookupCatalog.AsTableIdentifier
- AsTableIdentifier() - Method in interface org.apache.spark.sql.connector.catalog.LookupCatalog
- AsTableIdentifier$() - Constructor for class org.apache.spark.sql.connector.catalog.LookupCatalog.AsTableIdentifier$
- asTerms() - Static method in class org.apache.spark.ml.feature.Dot
- asTerms() - Static method in class org.apache.spark.ml.feature.EmptyTerm
- asTerms() - Method in interface org.apache.spark.ml.feature.Term
-
Default representation of a single Term as a part of summed terms.
- asTransform() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.BucketSpecHelper
- asTransforms() - Method in class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.PartitionTypeHelper
- AsyncEventQueue - Class in org.apache.spark.scheduler
-
An asynchronous queue for events.
- AsyncEventQueue(String, SparkConf, LiveListenerBusMetrics, LiveListenerBus) - Constructor for class org.apache.spark.scheduler.AsyncEventQueue
- AsyncRDDActions<T> - Class in org.apache.spark.rdd
-
A set of asynchronous RDD actions available through an implicit conversion.
- AsyncRDDActions(RDD<T>, ClassTag<T>) - Constructor for class org.apache.spark.rdd.AsyncRDDActions
- atan(String) - Static method in class org.apache.spark.sql.functions
- atan(Column) - Static method in class org.apache.spark.sql.functions
- atan2(double, String) - Static method in class org.apache.spark.sql.functions
- atan2(double, Column) - Static method in class org.apache.spark.sql.functions
- atan2(String, double) - Static method in class org.apache.spark.sql.functions
- atan2(String, String) - Static method in class org.apache.spark.sql.functions
- atan2(String, Column) - Static method in class org.apache.spark.sql.functions
- atan2(Column, double) - Static method in class org.apache.spark.sql.functions
- atan2(Column, String) - Static method in class org.apache.spark.sql.functions
- atan2(Column, Column) - Static method in class org.apache.spark.sql.functions
- atanh(String) - Static method in class org.apache.spark.sql.functions
- atanh(Column) - Static method in class org.apache.spark.sql.functions
- attempt() - Method in class org.apache.spark.status.api.v1.TaskData
- ATTEMPT() - Static method in class org.apache.spark.status.TaskIndexNames
- ATTEMPT_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.TaskData
- ATTEMPT_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper
- ATTEMPT_ID_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ApplicationAttemptInfo
- ATTEMPT_ID_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.StageData
- attemptId() - Method in class org.apache.spark.status.api.v1.ApplicationAttemptInfo
- attemptId() - Method in interface org.apache.spark.status.api.v1.BaseAppResource
- attemptId() - Method in class org.apache.spark.status.api.v1.StageData
- attemptNumber() - Method in class org.apache.spark.BarrierTaskContext
- attemptNumber() - Method in class org.apache.spark.scheduler.AskPermissionToCommitOutput
- attemptNumber() - Method in class org.apache.spark.scheduler.StageInfo
- attemptNumber() - Method in class org.apache.spark.scheduler.TaskInfo
- attemptNumber() - Method in class org.apache.spark.TaskCommitDenied
- attemptNumber() - Method in class org.apache.spark.TaskContext
-
How many times this task has been attempted.
- attempts() - Method in class org.apache.spark.status.api.v1.ApplicationInfo
- ATTEMPTS_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo
- AtTimestamp(Date) - Constructor for class org.apache.spark.streaming.kinesis.KinesisInitialPositions.AtTimestamp
- attr() - Method in class org.apache.spark.graphx.Edge
- attr() - Method in class org.apache.spark.graphx.EdgeContext
-
The attribute associated with the edge.
- attr() - Method in class org.apache.spark.graphx.impl.AggregatingEdgeContext
- attribute() - Method in class org.apache.spark.sql.sources.EqualNullSafe
- attribute() - Method in class org.apache.spark.sql.sources.EqualTo
- attribute() - Method in class org.apache.spark.sql.sources.GreaterThan
- attribute() - Method in class org.apache.spark.sql.sources.GreaterThanOrEqual
- attribute() - Method in class org.apache.spark.sql.sources.In
- attribute() - Method in class org.apache.spark.sql.sources.IsNotNull
- attribute() - Method in class org.apache.spark.sql.sources.IsNull
- attribute() - Method in class org.apache.spark.sql.sources.LessThan
- attribute() - Method in class org.apache.spark.sql.sources.LessThanOrEqual
- attribute() - Method in class org.apache.spark.sql.sources.StringContains
- attribute() - Method in class org.apache.spark.sql.sources.StringEndsWith
- attribute() - Method in class org.apache.spark.sql.sources.StringStartsWith
- Attribute - Class in org.apache.spark.ml.attribute
-
Abstract class for ML attributes.
- Attribute() - Constructor for class org.apache.spark.ml.attribute.Attribute
- AttributeFactory - Interface in org.apache.spark.ml.attribute
-
Trait for ML attribute factories.
- AttributeGroup - Class in org.apache.spark.ml.attribute
-
Attributes that describe a vector ML column.
- AttributeGroup(String) - Constructor for class org.apache.spark.ml.attribute.AttributeGroup
-
Creates an attribute group without attribute info.
- AttributeGroup(String, int) - Constructor for class org.apache.spark.ml.attribute.AttributeGroup
-
Creates an attribute group knowing only the number of attributes.
- AttributeGroup(String, Attribute[]) - Constructor for class org.apache.spark.ml.attribute.AttributeGroup
-
Creates an attribute group with attributes.
- AttributeKeys - Class in org.apache.spark.ml.attribute
-
Keys used to store attributes.
- AttributeKeys() - Constructor for class org.apache.spark.ml.attribute.AttributeKeys
- attributeNameSyntaxError(String) - Static method in class org.apache.spark.sql.errors.DataTypeErrors
- attributeNameSyntaxError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- attributes() - Method in class org.apache.spark.ml.attribute.AttributeGroup
-
Optional array of attributes.
- attributes() - Method in class org.apache.spark.scheduler.cluster.CoarseGrainedClusterMessages.RegisterExecutor
- attributes() - Method in class org.apache.spark.scheduler.cluster.ExecutorInfo
- attributes() - Method in class org.apache.spark.status.api.v1.ExecutorSummary
- ATTRIBUTES() - Static method in class org.apache.spark.ml.attribute.AttributeKeys
- ATTRIBUTES_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary
- attributesForTypeUnsupportedError(ScalaReflection.Schema) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- AttributeType - Class in org.apache.spark.ml.attribute
-
An enum-like type for attribute types:
AttributeType$.Numeric
,AttributeType$.Nominal
, andAttributeType$.Binary
. - AttributeType(String) - Constructor for class org.apache.spark.ml.attribute.AttributeType
- attrType() - Method in class org.apache.spark.ml.attribute.Attribute
-
Attribute type.
- attrType() - Method in class org.apache.spark.ml.attribute.BinaryAttribute
- attrType() - Method in class org.apache.spark.ml.attribute.NominalAttribute
- attrType() - Method in class org.apache.spark.ml.attribute.NumericAttribute
- attrType() - Static method in class org.apache.spark.ml.attribute.UnresolvedAttribute
- available() - Method in class org.apache.spark.io.NioBufferedFileInputStream
- available() - Method in class org.apache.spark.io.ReadAheadInputStream
- available() - Method in class org.apache.spark.storage.BufferReleasingInputStream
- availableAddrs() - Method in interface org.apache.spark.resource.ResourceAllocator
-
Sequence of currently available resource addresses.
- AvailableNow() - Static method in class org.apache.spark.sql.streaming.Trigger
-
A trigger that processes all available data at the start of the query in one or multiple batches, then terminates the query.
- Average() - Static method in class org.apache.spark.mllib.tree.configuration.EnsembleCombiningStrategy
- avg() - Method in class org.apache.spark.util.DoubleAccumulator
-
Returns the average of elements added to the accumulator.
- avg() - Method in class org.apache.spark.util.LongAccumulator
-
Returns the average of elements added to the accumulator.
- avg(String) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the average of the values in a group.
- avg(String...) - Method in class org.apache.spark.sql.RelationalGroupedDataset
-
Compute the mean value for each numeric columns for each group.
- avg(MapFunction<T, Double>) - Static method in class org.apache.spark.sql.expressions.javalang.typed
-
Deprecated.Average aggregate function.
- avg(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the average of the values in a group.
- avg(Seq<String>) - Method in class org.apache.spark.sql.RelationalGroupedDataset
-
Compute the mean value for each numeric columns for each group.
- avg(Function1<IN, Object>) - Static method in class org.apache.spark.sql.expressions.scalalang.typed
-
Deprecated.Average aggregate function.
- Avg - Class in org.apache.spark.sql.connector.expressions.aggregate
-
An aggregate function that returns the mean of all the values in a group.
- Avg(Expression, boolean) - Constructor for class org.apache.spark.sql.connector.expressions.aggregate.Avg
- avgEventRate() - Method in class org.apache.spark.status.api.v1.streaming.ReceiverInfo
- avgInputRate() - Method in class org.apache.spark.status.api.v1.streaming.StreamingStatistics
- avgLen() - Method in interface org.apache.spark.sql.connector.read.colstats.ColumnStatistics
- avgMetrics() - Method in class org.apache.spark.ml.tuning.CrossValidatorModel
- avgProcessingTime() - Method in class org.apache.spark.status.api.v1.streaming.StreamingStatistics
- avgSchedulingDelay() - Method in class org.apache.spark.status.api.v1.streaming.StreamingStatistics
- avgTotalDelay() - Method in class org.apache.spark.status.api.v1.streaming.StreamingStatistics
- avroIncompatibleReadError(String, String, String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- AvroMatchedField$() - Constructor for class org.apache.spark.sql.avro.AvroUtils.AvroMatchedField$
- AvroSchemaHelper(Schema, StructType, Seq<String>, Seq<String>, boolean) - Constructor for class org.apache.spark.sql.avro.AvroUtils.AvroSchemaHelper
- AvroUtils - Class in org.apache.spark.sql.avro
- AvroUtils() - Constructor for class org.apache.spark.sql.avro.AvroUtils
- AvroUtils.AvroMatchedField$ - Class in org.apache.spark.sql.avro
- AvroUtils.AvroSchemaHelper - Class in org.apache.spark.sql.avro
-
Helper class to perform field lookup/matching on Avro schemas.
- AvroUtils.RowReader - Interface in org.apache.spark.sql.avro
- awaitAnyTermination() - Method in class org.apache.spark.sql.streaming.StreamingQueryManager
-
Wait until any of the queries on the associated SQLContext has terminated since the creation of the context, or since
resetTerminated()
was called. - awaitAnyTermination(long) - Method in class org.apache.spark.sql.streaming.StreamingQueryManager
-
Wait until any of the queries on the associated SQLContext has terminated since the creation of the context, or since
resetTerminated()
was called. - awaitReady(Awaitable<T>, Duration) - Static method in class org.apache.spark.util.ThreadUtils
-
Preferred alternative to
Await.ready()
. - awaitResult(Future<T>, Duration) - Static method in class org.apache.spark.util.ThreadUtils
- awaitResult(Awaitable<T>, Duration) - Static method in class org.apache.spark.util.SparkThreadUtils
-
Preferred alternative to
Await.result()
. - awaitResult(Awaitable<T>, Duration) - Static method in class org.apache.spark.util.ThreadUtils
-
Preferred alternative to
Await.result()
. - awaitTermination() - Method in interface org.apache.spark.sql.streaming.StreamingQuery
-
Waits for the termination of
this
query, either byquery.stop()
or by an exception. - awaitTermination() - Method in class org.apache.spark.streaming.api.java.JavaStreamingContext
-
Deprecated.Wait for the execution to stop.
- awaitTermination() - Method in class org.apache.spark.streaming.StreamingContext
-
Deprecated.Wait for the execution to stop.
- awaitTermination(long) - Method in interface org.apache.spark.sql.streaming.StreamingQuery
-
Waits for the termination of
this
query, either byquery.stop()
or by an exception. - awaitTerminationOrTimeout(long) - Method in class org.apache.spark.streaming.api.java.JavaStreamingContext
-
Deprecated.Wait for the execution to stop.
- awaitTerminationOrTimeout(long) - Method in class org.apache.spark.streaming.StreamingContext
-
Deprecated.Wait for the execution to stop.
- axpy(double, Vector, Vector) - Static method in class org.apache.spark.ml.linalg.BLAS
-
y += a * x
- axpy(double, Vector, Vector) - Static method in class org.apache.spark.mllib.linalg.BLAS
-
y += a * x
B
- BACKUP_STANDALONE_MASTER_PREFIX() - Static method in class org.apache.spark.util.Utils
-
An identifier that backup masters use in their responses.
- balanceSlack() - Method in class org.apache.spark.rdd.DefaultPartitionCoalescer
- barrier() - Method in class org.apache.spark.BarrierTaskContext
-
:: Experimental :: Sets a global barrier and waits until all tasks in this stage hit this barrier.
- barrier() - Method in class org.apache.spark.rdd.RDD
-
:: Experimental :: Marks the current stage as a barrier stage, where Spark must launch all tasks together.
- BARRIER() - Static method in class org.apache.spark.RequestMethod
- BARRIER_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationNode
- BarrierCoordinatorMessage - Interface in org.apache.spark
- barrierStageWithDynamicAllocationError() - Static method in class org.apache.spark.errors.SparkCoreErrors
- barrierStageWithRDDChainPatternError() - Static method in class org.apache.spark.errors.SparkCoreErrors
- BarrierTaskContext - Class in org.apache.spark
-
:: Experimental :: A
TaskContext
with extra contextual info and tooling for tasks in a barrier stage. - BarrierTaskInfo - Class in org.apache.spark
-
:: Experimental :: Carries all task infos of a barrier task.
- base64(Column) - Static method in class org.apache.spark.sql.functions
-
Computes the BASE64 encoding of a binary column and returns it as a string column.
- BaseAppResource - Interface in org.apache.spark.status.api.v1
-
Base class for resource handlers that use app-specific data.
- baseOn(ParamMap) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Sets the given parameters in this grid to fixed values.
- baseOn(ParamPair<?>...) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Sets the given parameters in this grid to fixed values.
- baseOn(Seq<ParamPair<?>>) - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Sets the given parameters in this grid to fixed values.
- BaseReadWrite - Interface in org.apache.spark.ml.util
-
Trait for
MLWriter
andMLReader
. - BaseRelation - Class in org.apache.spark.sql.sources
-
Represents a collection of tuples with a known schema.
- BaseRelation() - Constructor for class org.apache.spark.sql.sources.BaseRelation
- baseRelationToDataFrame(BaseRelation) - Method in class org.apache.spark.sql.SparkSession
-
Convert a
BaseRelation
created for external data sources into aDataFrame
. - baseRelationToDataFrame(BaseRelation) - Method in class org.apache.spark.sql.SQLContext
- BaseRRDD<T,
U> - Class in org.apache.spark.api.r - BaseRRDD(RDD<T>, int, byte[], String, String, byte[], Broadcast<Object>[], ClassTag<T>, ClassTag<U>) - Constructor for class org.apache.spark.api.r.BaseRRDD
- BaseStreamingAppResource - Interface in org.apache.spark.status.api.v1.streaming
-
Base class for streaming API handlers, provides easy access to the streaming listener that holds the app's information.
- BasicBlockReplicationPolicy - Class in org.apache.spark.storage
- BasicBlockReplicationPolicy() - Constructor for class org.apache.spark.storage.BasicBlockReplicationPolicy
- basicCredentials(String, String) - Method in class org.apache.spark.streaming.kinesis.SparkAWSCredentials.Builder
-
Use a basic AWS keypair for long-lived authorization.
- basicSparkPage(HttpServletRequest, Function0<Seq<Node>>, String, boolean) - Static method in class org.apache.spark.ui.UIUtils
-
Returns a page with the spark css/js and a simple format.
- Batch - Interface in org.apache.spark.sql.connector.read
-
A physical representation of a data source scan for batch queries.
- BATCH_DURATION_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress
- BATCH_ID_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress
- BATCH_READ - Enum constant in enum class org.apache.spark.sql.connector.catalog.TableCapability
-
Signals that the table supports reads in batch execution mode.
- BATCH_WRITE - Enum constant in enum class org.apache.spark.sql.connector.catalog.TableCapability
-
Signals that the table supports append writes in batch execution mode.
- batchDuration() - Method in class org.apache.spark.sql.streaming.StreamingQueryProgress
- batchDuration() - Method in class org.apache.spark.status.api.v1.streaming.BatchInfo
- batchDuration() - Method in class org.apache.spark.status.api.v1.streaming.StreamingStatistics
- BATCHES() - Static method in class org.apache.spark.mllib.clustering.StreamingKMeans
- batchId() - Method in class org.apache.spark.sql.streaming.StreamingQueryProgress
- batchId() - Method in class org.apache.spark.status.api.v1.streaming.BatchInfo
- batchInfo() - Method in class org.apache.spark.streaming.scheduler.StreamingListenerBatchCompleted
- batchInfo() - Method in class org.apache.spark.streaming.scheduler.StreamingListenerBatchStarted
- batchInfo() - Method in class org.apache.spark.streaming.scheduler.StreamingListenerBatchSubmitted
- BatchInfo - Class in org.apache.spark.status.api.v1.streaming
- BatchInfo - Class in org.apache.spark.streaming.scheduler
-
:: DeveloperApi :: Class having information on completed batches.
- BatchInfo(Time, Map<Object, StreamInputInfo>, long, Option<Object>, Option<Object>, Map<Object, OutputOperationInfo>) - Constructor for class org.apache.spark.streaming.scheduler.BatchInfo
- batchInfos() - Method in class org.apache.spark.streaming.scheduler.StatsReportListener
- batchMetadataFileNotFoundError(Path) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- BatchStatus - Enum Class in org.apache.spark.status.api.v1.streaming
- batchTime() - Method in class org.apache.spark.status.api.v1.streaming.BatchInfo
- batchTime() - Method in class org.apache.spark.streaming.scheduler.BatchInfo
- batchTime() - Method in class org.apache.spark.streaming.scheduler.OutputOperationInfo
- BatchWrite - Interface in org.apache.spark.sql.connector.write
-
An interface that defines how to write the data to data source for batch processing.
- batchWriteCapabilityError(Table, String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- bbos() - Method in class org.apache.spark.storage.memory.SerializedValuesHolder
- bean(Class<T>) - Static method in class org.apache.spark.sql.Encoders
-
Creates an encoder for Java Bean of type T.
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.DB2Dialect
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.DerbyDialect
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.H2Dialect
- beforeFetch(Connection, Map<String, String>) - Method in class org.apache.spark.sql.jdbc.JdbcDialect
-
Override connection specific properties to run before a select is made.
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.MsSqlServerDialect
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.MySQLDialect
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.NoopDialect
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.OracleDialect
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.PostgresDialect
- beforeFetch(Connection, Map<String, String>) - Static method in class org.apache.spark.sql.jdbc.TeradataDialect
- BernoulliCellSampler<T> - Class in org.apache.spark.util.random
-
:: DeveloperApi :: A sampler based on Bernoulli trials for partitioning a data sequence.
- BernoulliCellSampler(double, double, boolean) - Constructor for class org.apache.spark.util.random.BernoulliCellSampler
- BernoulliSampler<T> - Class in org.apache.spark.util.random
-
:: DeveloperApi :: A sampler based on Bernoulli trials.
- BernoulliSampler(double, ClassTag<T>) - Constructor for class org.apache.spark.util.random.BernoulliSampler
- bestModel() - Method in class org.apache.spark.ml.tuning.CrossValidatorModel
- bestModel() - Method in class org.apache.spark.ml.tuning.TrainValidationSplitModel
- beta() - Method in class org.apache.spark.ml.evaluation.MulticlassClassificationEvaluator
-
The beta value, which controls precision vs recall weighting, used in
"weightedFMeasure"
,"fMeasureByLabel"
. - beta() - Method in class org.apache.spark.mllib.random.WeibullGenerator
- between(Object, Object) - Method in class org.apache.spark.sql.Column
-
True if the current column is between the lower bound and upper bound, inclusive.
- bin(String) - Static method in class org.apache.spark.sql.functions
-
An expression that returns the string representation of the binary value of the given long column.
- bin(Column) - Static method in class org.apache.spark.sql.functions
-
An expression that returns the string representation of the binary value of the given long column.
- Binarizer - Class in org.apache.spark.ml.feature
-
Binarize a column of continuous features given a threshold.
- Binarizer() - Constructor for class org.apache.spark.ml.feature.Binarizer
- Binarizer(String) - Constructor for class org.apache.spark.ml.feature.Binarizer
- binary() - Method in class org.apache.spark.ml.feature.CountVectorizer
- binary() - Method in class org.apache.spark.ml.feature.CountVectorizerModel
- binary() - Method in interface org.apache.spark.ml.feature.CountVectorizerParams
-
Binary toggle to control the output vector values.
- binary() - Method in class org.apache.spark.ml.feature.HashingTF
-
Binary toggle to control term frequency counts.
- binary() - Method in class org.apache.spark.sql.ColumnName
-
Creates a new
StructField
of type binary. - Binary() - Static method in class org.apache.spark.ml.attribute.AttributeType
-
Binary type.
- BINARY() - Static method in class org.apache.spark.sql.Encoders
-
An encoder for arrays of bytes.
- binaryArithmeticCauseOverflowError(short, String, short) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- BinaryAttribute - Class in org.apache.spark.ml.attribute
-
A binary attribute.
- BinaryClassificationEvaluator - Class in org.apache.spark.ml.evaluation
-
Evaluator for binary classification, which expects input columns rawPrediction, label and an optional weight column.
- BinaryClassificationEvaluator() - Constructor for class org.apache.spark.ml.evaluation.BinaryClassificationEvaluator
- BinaryClassificationEvaluator(String) - Constructor for class org.apache.spark.ml.evaluation.BinaryClassificationEvaluator
- BinaryClassificationMetricComputer - Interface in org.apache.spark.mllib.evaluation.binary
-
Trait for a binary classification evaluation metric computer.
- BinaryClassificationMetrics - Class in org.apache.spark.mllib.evaluation
-
Evaluator for binary classification.
- BinaryClassificationMetrics(RDD<? extends Product>, int) - Constructor for class org.apache.spark.mllib.evaluation.BinaryClassificationMetrics
- BinaryClassificationMetrics(RDD<Tuple2<Object, Object>>) - Constructor for class org.apache.spark.mllib.evaluation.BinaryClassificationMetrics
-
Defaults
numBins
to 0. - BinaryClassificationSummary - Interface in org.apache.spark.ml.classification
-
Abstraction for binary classification results for a given model.
- binaryColumn(String[]) - Static method in class org.apache.parquet.filter2.predicate.SparkFilterApi
- BinaryConfusionMatrix - Interface in org.apache.spark.mllib.evaluation.binary
-
Trait for a binary confusion matrix.
- binaryFiles(String) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Read a directory of binary files from HDFS, a local file system (available on all nodes), or any Hadoop-supported file system URI as a byte array.
- binaryFiles(String, int) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Read a directory of binary files from HDFS, a local file system (available on all nodes), or any Hadoop-supported file system URI as a byte array.
- binaryFiles(String, int) - Method in class org.apache.spark.SparkContext
-
Get an RDD for a Hadoop-readable dataset as PortableDataStream for each file (useful for binary data)
- binaryLabelValidator() - Static method in class org.apache.spark.mllib.util.DataValidators
-
Function to check if labels used for classification are either zero or one.
- BinaryLogisticRegressionSummary - Interface in org.apache.spark.ml.classification
-
Abstraction for binary logistic regression results for a given model.
- BinaryLogisticRegressionSummaryImpl - Class in org.apache.spark.ml.classification
-
Binary logistic regression results for a given model.
- BinaryLogisticRegressionSummaryImpl(Dataset<Row>, String, String, String, String, String) - Constructor for class org.apache.spark.ml.classification.BinaryLogisticRegressionSummaryImpl
- BinaryLogisticRegressionTrainingSummary - Interface in org.apache.spark.ml.classification
-
Abstraction for binary logistic regression training results.
- BinaryLogisticRegressionTrainingSummaryImpl - Class in org.apache.spark.ml.classification
-
Binary logistic regression training results.
- BinaryLogisticRegressionTrainingSummaryImpl(Dataset<Row>, String, String, String, String, String, double[]) - Constructor for class org.apache.spark.ml.classification.BinaryLogisticRegressionTrainingSummaryImpl
- BinaryRandomForestClassificationSummary - Interface in org.apache.spark.ml.classification
-
Abstraction for BinaryRandomForestClassification results for a given model.
- BinaryRandomForestClassificationSummaryImpl - Class in org.apache.spark.ml.classification
-
Binary RandomForestClassification for a given model.
- BinaryRandomForestClassificationSummaryImpl(Dataset<Row>, String, String, String, String) - Constructor for class org.apache.spark.ml.classification.BinaryRandomForestClassificationSummaryImpl
- BinaryRandomForestClassificationTrainingSummary - Interface in org.apache.spark.ml.classification
-
Abstraction for BinaryRandomForestClassification training results.
- BinaryRandomForestClassificationTrainingSummaryImpl - Class in org.apache.spark.ml.classification
-
Binary RandomForestClassification training results.
- BinaryRandomForestClassificationTrainingSummaryImpl(Dataset<Row>, String, String, String, String, double[]) - Constructor for class org.apache.spark.ml.classification.BinaryRandomForestClassificationTrainingSummaryImpl
- binaryRecords(String, int) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Load data from a flat binary file, assuming the length of each record is constant.
- binaryRecords(String, int, Configuration) - Method in class org.apache.spark.SparkContext
-
Load data from a flat binary file, assuming the length of each record is constant.
- binaryRecordsStream(String, int) - Method in class org.apache.spark.streaming.api.java.JavaStreamingContext
-
Deprecated.Create an input stream that monitors a Hadoop-compatible filesystem for new files and reads them as flat binary files with fixed record lengths, yielding byte arrays
- binaryRecordsStream(String, int) - Method in class org.apache.spark.streaming.StreamingContext
-
Deprecated.Create an input stream that monitors a Hadoop-compatible filesystem for new files and reads them as flat binary files, assuming a fixed length per record, generating one byte array per record.
- BinarySample - Class in org.apache.spark.mllib.stat.test
-
Class that represents the group and value of a sample.
- BinarySample(boolean, double) - Constructor for class org.apache.spark.mllib.stat.test.BinarySample
- binarySummary() - Method in class org.apache.spark.ml.classification.LogisticRegressionModel
-
Gets summary of model on training set.
- binarySummary() - Method in class org.apache.spark.ml.classification.RandomForestClassificationModel
-
Gets summary of model on training set.
- BinaryType - Class in org.apache.spark.sql.types
-
The data type representing
Array[Byte]
values. - BinaryType - Static variable in class org.apache.spark.sql.types.DataTypes
-
Gets the BinaryType object.
- BinaryType() - Constructor for class org.apache.spark.sql.types.BinaryType
- bind(StructType) - Method in interface org.apache.spark.sql.connector.catalog.functions.UnboundFunction
-
Bind this function to an input type.
- Binomial$() - Constructor for class org.apache.spark.ml.regression.GeneralizedLinearRegression.Binomial$
- BinomialBounds - Class in org.apache.spark.util.random
-
Utility functions that help us determine bounds on adjusted sampling rate to guarantee exact sample size with high confidence when sampling without replacement.
- BinomialBounds() - Constructor for class org.apache.spark.util.random.BinomialBounds
- bins() - Method in interface org.apache.spark.sql.connector.read.colstats.Histogram
- BisectingKMeans - Class in org.apache.spark.ml.clustering
-
A bisecting k-means algorithm based on the paper "A comparison of document clustering techniques" by Steinbach, Karypis, and Kumar, with modification to fit Spark.
- BisectingKMeans - Class in org.apache.spark.mllib.clustering
-
A bisecting k-means algorithm based on the paper "A comparison of document clustering techniques" by Steinbach, Karypis, and Kumar, with modification to fit Spark.
- BisectingKMeans() - Constructor for class org.apache.spark.ml.clustering.BisectingKMeans
- BisectingKMeans() - Constructor for class org.apache.spark.mllib.clustering.BisectingKMeans
-
Constructs with the default configuration
- BisectingKMeans(String) - Constructor for class org.apache.spark.ml.clustering.BisectingKMeans
- BisectingKMeansModel - Class in org.apache.spark.ml.clustering
-
Model fitted by BisectingKMeans.
- BisectingKMeansModel - Class in org.apache.spark.mllib.clustering
-
Clustering model produced by
BisectingKMeans
. - BisectingKMeansModel(ClusteringTreeNode) - Constructor for class org.apache.spark.mllib.clustering.BisectingKMeansModel
- BisectingKMeansModel.SaveLoadV1_0$ - Class in org.apache.spark.mllib.clustering
- BisectingKMeansModel.SaveLoadV2_0$ - Class in org.apache.spark.mllib.clustering
- BisectingKMeansModel.SaveLoadV3_0$ - Class in org.apache.spark.mllib.clustering
- BisectingKMeansParams - Interface in org.apache.spark.ml.clustering
-
Common params for BisectingKMeans and BisectingKMeansModel
- BisectingKMeansSummary - Class in org.apache.spark.ml.clustering
-
Summary of BisectingKMeans.
- bit_and(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the bitwise AND of all non-null input values, or null if none.
- bit_count(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the number of bits that are set in the argument expr as an unsigned 64-bit integer, or NULL if the argument is NULL.
- bit_get(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Returns the value of the bit (0 or 1) at the specified position.
- bit_length(Column) - Static method in class org.apache.spark.sql.functions
-
Calculates the bit length for the specified string column.
- bit_or(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the bitwise OR of all non-null input values, or null if none.
- bit_xor(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns the bitwise XOR of all non-null input values, or null if none.
- bitmap_bit_position(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the bit position for the given input column.
- bitmap_bucket_number(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the bucket number for the given input column.
- bitmap_construct_agg(Column) - Static method in class org.apache.spark.sql.functions
-
Returns a bitmap with the positions of the bits set from all the values from the input column.
- bitmap_count(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the number of set bits in the input bitmap.
- bitmap_or_agg(Column) - Static method in class org.apache.spark.sql.functions
-
Returns a bitmap that is the bitwise OR of all of the bitmaps from the input column.
- bitSize() - Method in class org.apache.spark.util.sketch.BloomFilter
-
Returns the number of bits in the underlying bit array.
- bitwise_not(Column) - Static method in class org.apache.spark.sql.functions
-
Computes bitwise NOT (~) of a number.
- bitwiseAND(Object) - Method in class org.apache.spark.sql.Column
-
Compute bitwise AND of this expression with another expression.
- bitwiseNOT(Column) - Static method in class org.apache.spark.sql.functions
-
Deprecated.Use bitwise_not. Since 3.2.0.
- bitwiseOR(Object) - Method in class org.apache.spark.sql.Column
-
Compute bitwise OR of this expression with another expression.
- bitwiseXOR(Object) - Method in class org.apache.spark.sql.Column
-
Compute bitwise XOR of this expression with another expression.
- BLACKLISTED_IN_STAGES_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary
- blacklistedInStages() - Method in class org.apache.spark.status.api.v1.ExecutorSummary
-
Deprecated.use excludedInStages instead. Since 3.1.0.
- BLAS - Class in org.apache.spark.ml.linalg
-
BLAS routines for MLlib's vectors and matrices.
- BLAS - Class in org.apache.spark.mllib.linalg
-
BLAS routines for MLlib's vectors and matrices.
- BLAS() - Constructor for class org.apache.spark.ml.linalg.BLAS
- BLAS() - Constructor for class org.apache.spark.mllib.linalg.BLAS
- BLOCK_NAME_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.RDDPartitionInfo
- BlockData - Interface in org.apache.spark.storage
-
Abstracts away how blocks are stored and provides different ways to read the underlying block data.
- blockDoesNotExistError(BlockId) - Static method in class org.apache.spark.errors.SparkCoreErrors
- blockedByLock() - Method in class org.apache.spark.status.api.v1.ThreadStackTrace
- blockedByThreadId() - Method in class org.apache.spark.status.api.v1.ThreadStackTrace
- BlockEvictionHandler - Interface in org.apache.spark.storage.memory
- BlockGeneratorListener - Interface in org.apache.spark.streaming.receiver
-
Listener object for BlockGenerator events
- blockHaveBeenRemovedError(String) - Static method in class org.apache.spark.errors.SparkCoreErrors
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.GetBlockStatus
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.GetLocations
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.GetLocationsAndStatus
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.GetRDDBlockVisibility
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.MarkRDDBlockAsVisible
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.RemoveBlock
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.ReplicateBlock
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.UpdateBlockInfo
- blockId() - Method in class org.apache.spark.storage.BlockManagerMessages.UpdateRDDBlockTaskInfo
- blockId() - Method in class org.apache.spark.storage.BlockUpdatedInfo
- blockId() - Method in interface org.apache.spark.streaming.receiver.ReceivedBlockStoreResult
- BlockId - Class in org.apache.spark.storage
-
:: DeveloperApi :: Identifies a particular Block of data, usually associated with a single file.
- BlockId() - Constructor for class org.apache.spark.storage.BlockId
- blockIds() - Method in class org.apache.spark.storage.BlockManagerMessages.GetLocationsMultipleBlockIds
- BlockInfoWrapper - Class in org.apache.spark.storage
- BlockInfoWrapper(BlockInfo, Lock) - Constructor for class org.apache.spark.storage.BlockInfoWrapper
- BlockInfoWrapper(BlockInfo, Lock, Condition) - Constructor for class org.apache.spark.storage.BlockInfoWrapper
- BlockLocationsAndStatus(Seq<BlockManagerId>, BlockStatus, Option<String[]>) - Constructor for class org.apache.spark.storage.BlockManagerMessages.BlockLocationsAndStatus
- BlockLocationsAndStatus$() - Constructor for class org.apache.spark.storage.BlockManagerMessages.BlockLocationsAndStatus$
- blockManager() - Method in class org.apache.spark.SparkEnv
- blockManagerAddedFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- blockManagerAddedToJson(SparkListenerBlockManagerAdded, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- BlockManagerHeartbeat(BlockManagerId) - Constructor for class org.apache.spark.storage.BlockManagerMessages.BlockManagerHeartbeat
- BlockManagerHeartbeat$() - Constructor for class org.apache.spark.storage.BlockManagerMessages.BlockManagerHeartbeat$
- blockManagerId() - Method in class org.apache.spark.scheduler.SparkListenerBlockManagerAdded
- blockManagerId() - Method in class org.apache.spark.scheduler.SparkListenerBlockManagerRemoved
- blockManagerId() - Method in class org.apache.spark.storage.BlockManagerMessages.BlockManagerHeartbeat
- blockManagerId() - Method in class org.apache.spark.storage.BlockManagerMessages.GetPeers
- blockManagerId() - Method in class org.apache.spark.storage.BlockManagerMessages.GetReplicateInfoForRDDBlocks
- blockManagerId() - Method in class org.apache.spark.storage.BlockManagerMessages.RegisterBlockManager
- blockManagerId() - Method in class org.apache.spark.storage.BlockManagerMessages.UpdateBlockInfo
- blockManagerId() - Method in class org.apache.spark.storage.BlockUpdatedInfo
- BlockManagerId - Class in org.apache.spark.storage
-
:: DeveloperApi :: This class represent a unique identifier for a BlockManager.
- BlockManagerId() - Constructor for class org.apache.spark.storage.BlockManagerId
- blockManagerIdCache() - Static method in class org.apache.spark.storage.BlockManagerId
-
The max cache size is hardcoded to 10000, since the size of a BlockManagerId object is about 48B, the total memory cost should be below 1MB which is feasible.
- blockManagerIdFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- blockManagerIdToJson(BlockManagerId, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- BlockManagerMessages - Class in org.apache.spark.storage
- BlockManagerMessages() - Constructor for class org.apache.spark.storage.BlockManagerMessages
- BlockManagerMessages.BlockLocationsAndStatus - Class in org.apache.spark.storage
-
The response message of
GetLocationsAndStatus
request. - BlockManagerMessages.BlockLocationsAndStatus$ - Class in org.apache.spark.storage
- BlockManagerMessages.BlockManagerHeartbeat - Class in org.apache.spark.storage
- BlockManagerMessages.BlockManagerHeartbeat$ - Class in org.apache.spark.storage
- BlockManagerMessages.DecommissionBlockManager$ - Class in org.apache.spark.storage
- BlockManagerMessages.DecommissionBlockManagers - Class in org.apache.spark.storage
- BlockManagerMessages.DecommissionBlockManagers$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetBlockStatus - Class in org.apache.spark.storage
- BlockManagerMessages.GetBlockStatus$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetExecutorEndpointRef - Class in org.apache.spark.storage
- BlockManagerMessages.GetExecutorEndpointRef$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetLocations - Class in org.apache.spark.storage
- BlockManagerMessages.GetLocations$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetLocationsAndStatus - Class in org.apache.spark.storage
- BlockManagerMessages.GetLocationsAndStatus$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetLocationsMultipleBlockIds - Class in org.apache.spark.storage
- BlockManagerMessages.GetLocationsMultipleBlockIds$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetMatchingBlockIds - Class in org.apache.spark.storage
- BlockManagerMessages.GetMatchingBlockIds$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetMemoryStatus$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetPeers - Class in org.apache.spark.storage
- BlockManagerMessages.GetPeers$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetRDDBlockVisibility - Class in org.apache.spark.storage
- BlockManagerMessages.GetRDDBlockVisibility$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetReplicateInfoForRDDBlocks - Class in org.apache.spark.storage
- BlockManagerMessages.GetReplicateInfoForRDDBlocks$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetShufflePushMergerLocations - Class in org.apache.spark.storage
- BlockManagerMessages.GetShufflePushMergerLocations$ - Class in org.apache.spark.storage
- BlockManagerMessages.GetStorageStatus$ - Class in org.apache.spark.storage
- BlockManagerMessages.IsExecutorAlive - Class in org.apache.spark.storage
- BlockManagerMessages.IsExecutorAlive$ - Class in org.apache.spark.storage
- BlockManagerMessages.MarkRDDBlockAsVisible - Class in org.apache.spark.storage
- BlockManagerMessages.MarkRDDBlockAsVisible$ - Class in org.apache.spark.storage
- BlockManagerMessages.RegisterBlockManager - Class in org.apache.spark.storage
- BlockManagerMessages.RegisterBlockManager$ - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveBlock - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveBlock$ - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveBroadcast - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveBroadcast$ - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveExecutor - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveExecutor$ - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveRdd - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveRdd$ - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveShuffle - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveShuffle$ - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveShufflePushMergerLocation - Class in org.apache.spark.storage
- BlockManagerMessages.RemoveShufflePushMergerLocation$ - Class in org.apache.spark.storage
- BlockManagerMessages.ReplicateBlock - Class in org.apache.spark.storage
- BlockManagerMessages.ReplicateBlock$ - Class in org.apache.spark.storage
- BlockManagerMessages.StopBlockManagerMaster$ - Class in org.apache.spark.storage
- BlockManagerMessages.ToBlockManagerMaster - Interface in org.apache.spark.storage
- BlockManagerMessages.ToBlockManagerMasterStorageEndpoint - Interface in org.apache.spark.storage
- BlockManagerMessages.TriggerHeapHistogram$ - Class in org.apache.spark.storage
-
Driver to Executor message to get a heap histogram.
- BlockManagerMessages.TriggerThreadDump$ - Class in org.apache.spark.storage
-
Driver to Executor message to trigger a thread dump.
- BlockManagerMessages.UpdateBlockInfo - Class in org.apache.spark.storage
- BlockManagerMessages.UpdateBlockInfo$ - Class in org.apache.spark.storage
- BlockManagerMessages.UpdateRDDBlockTaskInfo - Class in org.apache.spark.storage
- BlockManagerMessages.UpdateRDDBlockTaskInfo$ - Class in org.apache.spark.storage
- BlockManagerMessages.UpdateRDDBlockVisibility - Class in org.apache.spark.storage
- BlockManagerMessages.UpdateRDDBlockVisibility$ - Class in org.apache.spark.storage
- blockManagerRemovedFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- blockManagerRemovedToJson(SparkListenerBlockManagerRemoved, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- BlockMatrix - Class in org.apache.spark.mllib.linalg.distributed
-
Represents a distributed matrix in blocks of local matrices.
- BlockMatrix(RDD<Tuple2<Tuple2<Object, Object>, Matrix>>, int, int) - Constructor for class org.apache.spark.mllib.linalg.distributed.BlockMatrix
-
Alternate constructor for BlockMatrix without the input of the number of rows and columns.
- BlockMatrix(RDD<Tuple2<Tuple2<Object, Object>, Matrix>>, int, int, long, long) - Constructor for class org.apache.spark.mllib.linalg.distributed.BlockMatrix
- blockName() - Method in class org.apache.spark.status.api.v1.RDDPartitionInfo
- blockName() - Method in class org.apache.spark.status.LiveRDDPartition
- blockNotFoundError(BlockId) - Static method in class org.apache.spark.errors.SparkCoreErrors
- BlockNotFoundException - Exception in org.apache.spark.storage
- BlockNotFoundException(String) - Constructor for exception org.apache.spark.storage.BlockNotFoundException
- BlockReplicationPolicy - Interface in org.apache.spark.storage
-
::DeveloperApi:: BlockReplicationPrioritization provides logic for prioritizing a sequence of peers for replicating blocks.
- BlockReplicationUtils - Class in org.apache.spark.storage
- BlockReplicationUtils() - Constructor for class org.apache.spark.storage.BlockReplicationUtils
- blocks() - Method in class org.apache.spark.mllib.linalg.distributed.BlockMatrix
- blockSize() - Method in class org.apache.spark.ml.classification.MultilayerPerceptronClassificationModel
- blockSize() - Method in class org.apache.spark.ml.classification.MultilayerPerceptronClassifier
- blockSize() - Method in interface org.apache.spark.ml.param.shared.HasBlockSize
-
Param for block size for stacking input data in matrices.
- blockSize() - Method in class org.apache.spark.ml.recommendation.ALS
- blockSize() - Method in class org.apache.spark.ml.recommendation.ALSModel
- BlockStatus - Class in org.apache.spark.storage
- BlockStatus(StorageLevel, long, long) - Constructor for class org.apache.spark.storage.BlockStatus
- blockStatusFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- blockStatusQueryReturnedNullError(BlockId) - Static method in class org.apache.spark.errors.SparkCoreErrors
- blockStatusToJson(BlockStatus, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- blockUpdatedInfo() - Method in class org.apache.spark.scheduler.SparkListenerBlockUpdated
- BlockUpdatedInfo - Class in org.apache.spark.storage
-
:: DeveloperApi :: Stores information about a block status in a block manager.
- BlockUpdatedInfo(BlockManagerId, BlockId, StorageLevel, long, long) - Constructor for class org.apache.spark.storage.BlockUpdatedInfo
- blockUpdatedInfoFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- blockUpdatedInfoToJson(BlockUpdatedInfo, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- blockUpdateFromJson(JsonNode) - Static method in class org.apache.spark.util.JsonProtocol
- blockUpdateToJson(SparkListenerBlockUpdated, JsonGenerator) - Static method in class org.apache.spark.util.JsonProtocol
- bloomFilter(String, long, double) - Method in class org.apache.spark.sql.DataFrameStatFunctions
-
Builds a Bloom filter over a specified column.
- bloomFilter(String, long, long) - Method in class org.apache.spark.sql.DataFrameStatFunctions
-
Builds a Bloom filter over a specified column.
- bloomFilter(Column, long, double) - Method in class org.apache.spark.sql.DataFrameStatFunctions
-
Builds a Bloom filter over a specified column.
- bloomFilter(Column, long, long) - Method in class org.apache.spark.sql.DataFrameStatFunctions
-
Builds a Bloom filter over a specified column.
- BloomFilter - Class in org.apache.spark.util.sketch
-
A Bloom filter is a space-efficient probabilistic data structure that offers an approximate containment test with one-sided error: if it claims that an item is contained in it, this might be in error, but if it claims that an item is not contained in it, then this is definitely true.
- BloomFilter() - Constructor for class org.apache.spark.util.sketch.BloomFilter
- BloomFilter.Version - Enum Class in org.apache.spark.util.sketch
- bmAddress() - Method in class org.apache.spark.FetchFailed
- bool_and(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns true if all values of
e
are true. - bool_or(Column) - Static method in class org.apache.spark.sql.functions
-
Aggregate function: returns true if at least one value of
e
is true. - BOOLEAN() - Static method in class org.apache.spark.sql.Encoders
-
An encoder for nullable boolean type.
- booleanColumn(String[]) - Static method in class org.apache.parquet.filter2.predicate.SparkFilterApi
- BooleanParam - Class in org.apache.spark.ml.param
-
Specialized version of
Param[Boolean]
for Java. - BooleanParam(String, String, String) - Constructor for class org.apache.spark.ml.param.BooleanParam
- BooleanParam(Identifiable, String, String) - Constructor for class org.apache.spark.ml.param.BooleanParam
- BooleanType - Class in org.apache.spark.sql.types
-
The data type representing
Boolean
values. - BooleanType - Static variable in class org.apache.spark.sql.types.DataTypes
-
Gets the BooleanType object.
- BooleanType() - Constructor for class org.apache.spark.sql.types.BooleanType
- BooleanTypeExpression - Class in org.apache.spark.sql.types
- BooleanTypeExpression() - Constructor for class org.apache.spark.sql.types.BooleanTypeExpression
- boost(RDD<org.apache.spark.ml.feature.Instance>, RDD<org.apache.spark.ml.feature.Instance>, BoostingStrategy, boolean, long, String, Option<org.apache.spark.ml.util.Instrumentation>) - Static method in class org.apache.spark.ml.tree.impl.GradientBoostedTrees
-
Internal method for performing regression using trees as base learners.
- BoostingStrategy - Class in org.apache.spark.mllib.tree.configuration
-
Configuration options for
GradientBoostedTrees
. - BoostingStrategy(Strategy, Loss, int, double, double) - Constructor for class org.apache.spark.mllib.tree.configuration.BoostingStrategy
- bootstrap() - Method in class org.apache.spark.ml.classification.RandomForestClassificationModel
- bootstrap() - Method in class org.apache.spark.ml.classification.RandomForestClassifier
- bootstrap() - Method in class org.apache.spark.ml.regression.RandomForestRegressionModel
- bootstrap() - Method in class org.apache.spark.ml.regression.RandomForestRegressor
- bootstrap() - Method in interface org.apache.spark.ml.tree.RandomForestParams
-
Whether bootstrap samples are used when building trees.
- Both - Enum constant in enum class org.apache.spark.graphx.impl.EdgeActiveness
-
Both vertices must be active.
- Both() - Static method in class org.apache.spark.graphx.EdgeDirection
-
Edges originating from *and* arriving at a vertex of interest.
- boundaries() - Method in class org.apache.spark.ml.regression.IsotonicRegressionModel
-
Boundaries in increasing order for which predictions are known.
- boundaries() - Method in class org.apache.spark.mllib.regression.IsotonicRegressionModel
- BoundedDouble - Class in org.apache.spark.partial
-
A Double value with error bars and associated confidence.
- BoundedDouble(double, double, double, double) - Constructor for class org.apache.spark.partial.BoundedDouble
- BoundFunction - Interface in org.apache.spark.sql.connector.catalog.functions
-
Represents a function that is bound to an input type.
- BreezeUtil - Class in org.apache.spark.ml.ann
-
In-place DGEMM and DGEMV for Breeze
- BreezeUtil() - Constructor for class org.apache.spark.ml.ann.BreezeUtil
- broadcast(Dataset<T>) - Static method in class org.apache.spark.sql.functions
-
Marks a DataFrame as small enough for use in broadcast joins.
- broadcast(T) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Broadcast a read-only variable to the cluster, returning a
Broadcast
object for reading it in distributed functions. - broadcast(T, ClassTag<T>) - Method in class org.apache.spark.SparkContext
-
Broadcast a read-only variable to the cluster, returning a
Broadcast
object for reading it in distributed functions. - Broadcast<T> - Class in org.apache.spark.broadcast
-
A broadcast variable.
- Broadcast(long, ClassTag<T>) - Constructor for class org.apache.spark.broadcast.Broadcast
- BROADCAST() - Static method in class org.apache.spark.storage.BlockId
- BroadcastBlockId - Class in org.apache.spark.storage
- BroadcastBlockId(long, String) - Constructor for class org.apache.spark.storage.BroadcastBlockId
- broadcastCleaned(long) - Method in interface org.apache.spark.CleanerListener
- BroadcastFactory - Interface in org.apache.spark.broadcast
-
An interface for all the broadcast implementations in Spark (to allow multiple broadcast implementations).
- broadcastId() - Method in class org.apache.spark.CleanBroadcast
- broadcastId() - Method in class org.apache.spark.storage.BlockManagerMessages.RemoveBroadcast
- broadcastId() - Method in class org.apache.spark.storage.BroadcastBlockId
- broadcastManager() - Method in class org.apache.spark.SparkEnv
- bround(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the value of the column
e
rounded to 0 decimal places with HALF_EVEN round mode. - bround(Column, int) - Static method in class org.apache.spark.sql.functions
-
Round the value of
e
toscale
decimal places with HALF_EVEN round mode ifscale
is greater than or equal to 0 or at integral part whenscale
is less than 0. - btrim(Column) - Static method in class org.apache.spark.sql.functions
-
Removes the leading and trailing space characters from
str
. - btrim(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Remove the leading and trailing
trim
characters fromstr
. - bucket(int, String...) - Static method in class org.apache.spark.sql.connector.expressions.Expressions
-
Create a bucket transform for one or more columns.
- bucket(int, Column) - Static method in class org.apache.spark.sql.functions
-
A transform for any type that partitions by a hash of the input column.
- bucket(int, NamedReference[]) - Static method in class org.apache.spark.sql.connector.expressions.LogicalExpressions
- bucket(int, NamedReference[], NamedReference[]) - Static method in class org.apache.spark.sql.connector.expressions.LogicalExpressions
- bucket(Column, Column) - Static method in class org.apache.spark.sql.functions
-
A transform for any type that partitions by a hash of the input column.
- bucketBy(int, String, String...) - Method in class org.apache.spark.sql.DataFrameWriter
-
Buckets the output by the given columns.
- bucketBy(int, String, Seq<String>) - Method in class org.apache.spark.sql.DataFrameWriter
-
Buckets the output by the given columns.
- bucketByAndSortByUnsupportedByOperationError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- bucketByUnsupportedByOperationError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- BucketedRandomProjectionLSH - Class in org.apache.spark.ml.feature
-
This
BucketedRandomProjectionLSH
implements Locality Sensitive Hashing functions for Euclidean distance metrics. - BucketedRandomProjectionLSH() - Constructor for class org.apache.spark.ml.feature.BucketedRandomProjectionLSH
- BucketedRandomProjectionLSH(String) - Constructor for class org.apache.spark.ml.feature.BucketedRandomProjectionLSH
- BucketedRandomProjectionLSHModel - Class in org.apache.spark.ml.feature
-
Model produced by
BucketedRandomProjectionLSH
, where multiple random vectors are stored. - BucketedRandomProjectionLSHParams - Interface in org.apache.spark.ml.feature
-
Params for
BucketedRandomProjectionLSH
. - bucketingColumnCannotBePartOfPartitionColumnsError(String, Seq<String>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- Bucketizer - Class in org.apache.spark.ml.feature
-
Bucketizer
maps a column of continuous features to a column of feature buckets. - Bucketizer() - Constructor for class org.apache.spark.ml.feature.Bucketizer
- Bucketizer(String) - Constructor for class org.apache.spark.ml.feature.Bucketizer
- bucketLength() - Method in class org.apache.spark.ml.feature.BucketedRandomProjectionLSH
- bucketLength() - Method in class org.apache.spark.ml.feature.BucketedRandomProjectionLSHModel
- bucketLength() - Method in interface org.apache.spark.ml.feature.BucketedRandomProjectionLSHParams
-
The length of each hash bucket, a larger bucket lowers the false negative rate.
- bucketSortingColumnCannotBePartOfPartitionColumnsError(String, Seq<String>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- BucketSpecHelper(BucketSpec) - Constructor for class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.BucketSpecHelper
- buffer() - Method in class org.apache.spark.storage.memory.SerializedMemoryEntry
- bufferEncoder() - Method in class org.apache.spark.ml.feature.StringIndexerAggregator
- bufferEncoder() - Method in class org.apache.spark.sql.expressions.Aggregator
-
Specifies the
Encoder
for the intermediate value type. - BufferReleasingInputStream - Class in org.apache.spark.storage
-
Helper class that ensures a ManagedBuffer is released upon InputStream.close() and also detects stream corruption if streamCompressedOrEncrypted is true
- BufferReleasingInputStream(InputStream, ShuffleBlockFetcherIterator, BlockId, int, BlockManagerId, boolean, boolean, Option<CheckedInputStream>) - Constructor for class org.apache.spark.storage.BufferReleasingInputStream
- bufferSchema() - Method in class org.apache.spark.sql.expressions.UserDefinedAggregateFunction
-
Deprecated.A
StructType
represents data types of values in the aggregation buffer. - build() - Method in class org.apache.spark.ml.tuning.ParamGridBuilder
-
Builds and returns all combinations of parameters specified by the param grid.
- build() - Method in class org.apache.spark.resource.ResourceProfileBuilder
- build() - Method in interface org.apache.spark.sql.connector.read.ScanBuilder
- build() - Method in interface org.apache.spark.sql.connector.write.DeltaWriteBuilder
- build() - Method in interface org.apache.spark.sql.connector.write.RowLevelOperationBuilder
-
Returns a
RowLevelOperation
that controls how Spark rewrites data for DELETE, UPDATE, MERGE commands. - build() - Method in interface org.apache.spark.sql.connector.write.WriteBuilder
-
Returns a logical
Write
shared between batch and streaming. - build() - Method in class org.apache.spark.sql.jdbc.DB2Dialect.DB2SQLQueryBuilder
- build() - Method in class org.apache.spark.sql.jdbc.JdbcSQLQueryBuilder
-
Build the final SQL query that following dialect's SQL syntax.
- build() - Method in class org.apache.spark.sql.jdbc.MsSqlServerDialect.MsSqlServerSQLQueryBuilder
- build() - Method in class org.apache.spark.sql.jdbc.MySQLDialect.MySQLSQLQueryBuilder
- build() - Method in class org.apache.spark.sql.jdbc.OracleDialect.OracleSQLQueryBuilder
- build() - Method in class org.apache.spark.sql.types.MetadataBuilder
-
Builds the
Metadata
instance. - build() - Method in class org.apache.spark.status.protobuf.StoreTypes.AccumulableInfo.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationAttemptInfo.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfoWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfoWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.AppSummary.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.CachedQuantile.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetrics.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorResourceRequest.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorStageSummary.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorStageSummaryWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummaryWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetricDistributions.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetrics.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.JobDataWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.MemoryMetrics.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetricDistributions.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetrics.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.PairStrings.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.PoolData.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ProcessSummary.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ProcessSummaryWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDDataDistribution.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationEdge.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationNode.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDPartitionInfo.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfoWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceInformation.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceProfileInfo.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceProfileWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.RuntimeInfo.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetrics.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetrics.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetrics.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SinkProgress.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SourceProgress.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphEdge.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNodeWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SpeculationStageSummary.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SpeculationStageSummaryWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLPlanMetric.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.StageDataWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.StateOperatorProgress.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamBlockData.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryData.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgressWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetrics.Builder
- build() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskResourceRequest.Builder
- build() - Method in interface org.apache.spark.storage.memory.MemoryEntryBuilder
- build() - Method in class org.apache.spark.streaming.kinesis.SparkAWSCredentials.Builder
-
Returns the appropriate instance of
SparkAWSCredentials
given the configured parameters. - build(DecisionTreeModel, int) - Method in class org.apache.spark.ml.tree.EnsembleModelReadWrite.EnsembleNodeData$
-
Create
EnsembleModelReadWrite.EnsembleNodeData
instances for the given tree. - build(Node, int) - Method in class org.apache.spark.ml.tree.DecisionTreeModelReadWrite.NodeData$
-
Create
DecisionTreeModelReadWrite.NodeData
instances for this node and all children. - build(Expression) - Method in class org.apache.spark.sql.connector.util.V2ExpressionSQLBuilder
- builder() - Static method in class org.apache.spark.sql.SparkSession
-
Creates a
SparkSession.Builder
for constructing aSparkSession
. - Builder() - Constructor for class org.apache.spark.sql.SparkSession.Builder
- Builder() - Constructor for class org.apache.spark.streaming.kinesis.SparkAWSCredentials.Builder
- buildErrorResponse(Response.Status, String) - Static method in class org.apache.spark.ui.UIUtils
- buildFilter(Seq<Expression>, Seq<Attribute>) - Method in interface org.apache.spark.sql.columnar.CachedBatchSerializer
-
Builds a function that can be used to filter batches prior to being decompressed.
- buildFilter(Seq<Expression>, Seq<Attribute>) - Method in class org.apache.spark.sql.columnar.SimpleMetricsCachedBatchSerializer
- buildForBatch() - Method in interface org.apache.spark.sql.connector.write.WriteBuilder
-
Deprecated.use
WriteBuilder.build()
instead. - buildForStreaming() - Method in interface org.apache.spark.sql.connector.write.WriteBuilder
-
Deprecated.use
WriteBuilder.build()
instead. - buildLocationMetadata(Seq<Path>, int) - Static method in class org.apache.spark.util.Utils
-
Convert a sequence of
Path
s to a metadata string. - buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.AccumulableInfo.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationAttemptInfo.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfo.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationEnvironmentInfoWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfo.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ApplicationInfoWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.AppSummary.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.CachedQuantile.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetrics.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorMetricsDistributions.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorPeakMetricsDistributions.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorResourceRequest.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorStageSummary.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorStageSummaryWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummary.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ExecutorSummaryWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetricDistributions.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.InputMetrics.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.JobData.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.JobDataWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.MemoryMetrics.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetricDistributions.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.OutputMetrics.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.PairStrings.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.PoolData.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ProcessSummary.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ProcessSummaryWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDDataDistribution.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationClusterWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationEdge.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationGraphWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationNode.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDPartitionInfo.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfo.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RDDStorageInfoWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceInformation.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceProfileInfo.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ResourceProfileWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.RuntimeInfo.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetricDistributions.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShufflePushReadMetrics.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetricDistributions.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleReadMetrics.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetricDistributions.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetrics.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SinkProgress.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SourceProgress.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphClusterWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphEdge.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNode.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphNodeWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SparkPlanGraphWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SpeculationStageSummary.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SpeculationStageSummaryWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLExecutionUIData.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.SQLPlanMetric.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.StageData.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.StageDataWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.StateOperatorProgress.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamBlockData.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryData.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgress.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.StreamingQueryProgressWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskData.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskDataWrapper.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetricDistributions.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskMetrics.Builder
- buildPartial() - Method in class org.apache.spark.status.protobuf.StoreTypes.TaskResourceRequest.Builder
- buildPlanNormalizationRules(SparkSession) - Method in class org.apache.spark.sql.SparkSessionExtensions
- buildPools() - Method in interface org.apache.spark.scheduler.SchedulableBuilder
- buildReaderUnsupportedForFileFormatError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- buildScan() - Method in interface org.apache.spark.sql.sources.TableScan
- buildScan(String[]) - Method in interface org.apache.spark.sql.sources.PrunedScan
- buildScan(String[], Filter[]) - Method in interface org.apache.spark.sql.sources.PrunedFilteredScan
- buildScan(Seq<Attribute>, Seq<Expression>) - Method in interface org.apache.spark.sql.sources.CatalystScan
- buildTreeFromNodes(DecisionTreeModelReadWrite.NodeData[], String) - Static method in class org.apache.spark.ml.tree.DecisionTreeModelReadWrite
-
Given all data for all nodes in a tree, rebuild the tree.
- BYTE() - Static method in class org.apache.spark.api.r.SerializationFormats
- BYTE() - Static method in class org.apache.spark.sql.Encoders
-
An encoder for nullable byte type.
- BytecodeUtils - Class in org.apache.spark.graphx.util
-
Includes an utility function to test whether a function accesses a specific attribute of an object.
- BytecodeUtils() - Constructor for class org.apache.spark.graphx.util.BytecodeUtils
- ByteExactNumeric - Class in org.apache.spark.sql.types
- ByteExactNumeric() - Constructor for class org.apache.spark.sql.types.ByteExactNumeric
- BYTES_READ() - Method in class org.apache.spark.InternalAccumulator.input$
- BYTES_READ_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.CachedQuantile
- BYTES_READ_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.InputMetricDistributions
- BYTES_READ_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.InputMetrics
- BYTES_WRITTEN() - Method in class org.apache.spark.InternalAccumulator.output$
- BYTES_WRITTEN() - Method in class org.apache.spark.InternalAccumulator.shuffleWrite$
- BYTES_WRITTEN_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.CachedQuantile
- BYTES_WRITTEN_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.OutputMetricDistributions
- BYTES_WRITTEN_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.OutputMetrics
- BYTES_WRITTEN_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.ShuffleWriteMetrics
- bytesRead() - Method in class org.apache.spark.status.api.v1.InputMetricDistributions
- bytesRead() - Method in class org.apache.spark.status.api.v1.InputMetrics
- bytesToString(long) - Static method in class org.apache.spark.util.Utils
-
Convert a quantity in bytes to a human-readable string such as "4.0 MiB".
- bytesToString(BigInt) - Static method in class org.apache.spark.util.Utils
- byteStringAsBytes(String) - Static method in class org.apache.spark.util.Utils
-
Convert a passed byte string (e.g. 50b, 100k, or 250m) to bytes for internal use.
- byteStringAsGb(String) - Static method in class org.apache.spark.util.Utils
-
Convert a passed byte string (e.g. 50b, 100k, or 250m, 500g) to gibibytes for internal use.
- byteStringAsKb(String) - Static method in class org.apache.spark.util.Utils
-
Convert a passed byte string (e.g. 50b, 100k, or 250m) to kibibytes for internal use.
- byteStringAsMb(String) - Static method in class org.apache.spark.util.Utils
-
Convert a passed byte string (e.g. 50b, 100k, or 250m) to mebibytes for internal use.
- bytesWritten() - Method in class org.apache.spark.status.api.v1.OutputMetricDistributions
- bytesWritten() - Method in class org.apache.spark.status.api.v1.OutputMetrics
- bytesWritten() - Method in class org.apache.spark.status.api.v1.ShuffleWriteMetrics
- bytesWritten(long) - Method in interface org.apache.spark.util.logging.RollingPolicy
-
Notify that bytes have been written
- ByteType - Class in org.apache.spark.sql.types
-
The data type representing
Byte
values. - ByteType - Static variable in class org.apache.spark.sql.types.DataTypes
-
Gets the ByteType object.
- ByteType() - Constructor for class org.apache.spark.sql.types.ByteType
- ByteTypeExpression - Class in org.apache.spark.sql.types
- ByteTypeExpression() - Constructor for class org.apache.spark.sql.types.ByteTypeExpression
C
- cache() - Method in class org.apache.spark.api.java.JavaDoubleRDD
-
Persist this RDD with the default storage level (
MEMORY_ONLY
). - cache() - Method in class org.apache.spark.api.java.JavaPairRDD
-
Persist this RDD with the default storage level (
MEMORY_ONLY
). - cache() - Method in class org.apache.spark.api.java.JavaRDD
-
Persist this RDD with the default storage level (
MEMORY_ONLY
). - cache() - Method in class org.apache.spark.graphx.Graph
-
Caches the vertices and edges associated with this graph at the previously-specified target storage levels, which default to
MEMORY_ONLY
. - cache() - Method in class org.apache.spark.graphx.impl.EdgeRDDImpl
-
Persists the edge partitions using
targetStorageLevel
, which defaults to MEMORY_ONLY. - cache() - Method in class org.apache.spark.graphx.impl.GraphImpl
- cache() - Method in class org.apache.spark.graphx.impl.VertexRDDImpl
-
Persists the vertex partitions at
targetStorageLevel
, which defaults to MEMORY_ONLY. - cache() - Method in class org.apache.spark.mllib.linalg.distributed.BlockMatrix
-
Caches the underlying RDD.
- cache() - Method in class org.apache.spark.rdd.RDD
-
Persist this RDD with the default storage level (
MEMORY_ONLY
). - cache() - Method in class org.apache.spark.sql.Dataset
-
Persist this Dataset with the default storage level (
MEMORY_AND_DISK
). - cache() - Method in class org.apache.spark.streaming.api.java.JavaDStream
-
Persist RDDs of this DStream with the default storage level (MEMORY_ONLY_SER)
- cache() - Method in class org.apache.spark.streaming.api.java.JavaPairDStream
-
Persist RDDs of this DStream with the default storage level (MEMORY_ONLY_SER)
- cache() - Method in class org.apache.spark.streaming.dstream.DStream
-
Persist RDDs of this DStream with the default storage level (MEMORY_ONLY_SER)
- CACHED_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationNode
- CACHED_PARTITIONS() - Static method in class org.apache.spark.ui.storage.ToolTips
- CachedBatch - Interface in org.apache.spark.sql.columnar
-
Basic interface that all cached batches of data must support.
- CachedBatchSerializer - Interface in org.apache.spark.sql.columnar
-
Provides APIs that handle transformations of SQL data associated with the cache/persist APIs.
- CacheId - Class in org.apache.spark.storage
- CacheId(String, String, String) - Constructor for class org.apache.spark.storage.CacheId
- cacheNodeIds() - Method in class org.apache.spark.ml.classification.DecisionTreeClassificationModel
- cacheNodeIds() - Method in class org.apache.spark.ml.classification.DecisionTreeClassifier
- cacheNodeIds() - Method in class org.apache.spark.ml.classification.GBTClassificationModel
- cacheNodeIds() - Method in class org.apache.spark.ml.classification.GBTClassifier
- cacheNodeIds() - Method in class org.apache.spark.ml.classification.RandomForestClassificationModel
- cacheNodeIds() - Method in class org.apache.spark.ml.classification.RandomForestClassifier
- cacheNodeIds() - Method in class org.apache.spark.ml.regression.DecisionTreeRegressionModel
- cacheNodeIds() - Method in class org.apache.spark.ml.regression.DecisionTreeRegressor
- cacheNodeIds() - Method in class org.apache.spark.ml.regression.GBTRegressionModel
- cacheNodeIds() - Method in class org.apache.spark.ml.regression.GBTRegressor
- cacheNodeIds() - Method in class org.apache.spark.ml.regression.RandomForestRegressionModel
- cacheNodeIds() - Method in class org.apache.spark.ml.regression.RandomForestRegressor
- cacheNodeIds() - Method in interface org.apache.spark.ml.tree.DecisionTreeParams
-
If false, the algorithm will pass trees to executors to match instances with nodes.
- cacheSize() - Method in interface org.apache.spark.SparkExecutorInfo
- cacheSize() - Method in class org.apache.spark.SparkExecutorInfoImpl
- cacheTable(String) - Method in class org.apache.spark.sql.catalog.Catalog
-
Caches the specified table in-memory.
- cacheTable(String) - Method in class org.apache.spark.sql.SQLContext
-
Caches the specified table in-memory.
- cacheTable(String, StorageLevel) - Method in class org.apache.spark.sql.catalog.Catalog
-
Caches the specified table with the given storage level.
- calculate(double[], double) - Static method in class org.apache.spark.mllib.tree.impurity.Entropy
-
information calculation for multiclass classification
- calculate(double[], double) - Static method in class org.apache.spark.mllib.tree.impurity.Gini
-
information calculation for multiclass classification
- calculate(double[], double) - Method in interface org.apache.spark.mllib.tree.impurity.Impurity
-
information calculation for multiclass classification
- calculate(double[], double) - Static method in class org.apache.spark.mllib.tree.impurity.Variance
-
information calculation for multiclass classification
- calculate(double, double, double) - Static method in class org.apache.spark.mllib.tree.impurity.Entropy
-
variance calculation
- calculate(double, double, double) - Static method in class org.apache.spark.mllib.tree.impurity.Gini
-
variance calculation
- calculate(double, double, double) - Method in interface org.apache.spark.mllib.tree.impurity.Impurity
-
information calculation for regression
- calculate(double, double, double) - Static method in class org.apache.spark.mllib.tree.impurity.Variance
-
variance calculation
- calculateAmountAndPartsForFraction(double) - Static method in class org.apache.spark.resource.ResourceUtils
- calculateNumberOfPartitions(long, int, int) - Method in class org.apache.spark.ml.feature.Word2VecModel.Word2VecModelWriter$
-
Calculate the number of partitions to use in saving the model.
- CalendarInterval - Class in org.apache.spark.unsafe.types
-
The class representing calendar intervals.
- CalendarInterval(int, int, long) - Constructor for class org.apache.spark.unsafe.types.CalendarInterval
- CalendarIntervalType - Class in org.apache.spark.sql.types
-
The data type representing calendar intervals.
- CalendarIntervalType - Static variable in class org.apache.spark.sql.types.DataTypes
-
Gets the CalendarIntervalType object.
- CalendarIntervalType() - Constructor for class org.apache.spark.sql.types.CalendarIntervalType
- call() - Method in interface org.apache.spark.api.java.function.Function0
- call() - Method in interface org.apache.spark.sql.api.java.UDF0
- call(Iterator<T>) - Method in interface org.apache.spark.api.java.function.ForeachPartitionFunction
- call(Iterator<T>) - Method in interface org.apache.spark.api.java.function.MapPartitionsFunction
- call(K, Iterator<V>) - Method in interface org.apache.spark.api.java.function.FlatMapGroupsFunction
- call(K, Iterator<V>) - Method in interface org.apache.spark.api.java.function.MapGroupsFunction
- call(K, Iterator<V>, GroupState<S>) - Method in interface org.apache.spark.api.java.function.FlatMapGroupsWithStateFunction
- call(K, Iterator<V>, GroupState<S>) - Method in interface org.apache.spark.api.java.function.MapGroupsWithStateFunction
- call(K, Iterator<V1>, Iterator<V2>) - Method in interface org.apache.spark.api.java.function.CoGroupFunction
- call(T) - Method in interface org.apache.spark.api.java.function.DoubleFlatMapFunction
- call(T) - Method in interface org.apache.spark.api.java.function.DoubleFunction
- call(T) - Method in interface org.apache.spark.api.java.function.FilterFunction
- call(T) - Method in interface org.apache.spark.api.java.function.FlatMapFunction
- call(T) - Method in interface org.apache.spark.api.java.function.ForeachFunction
- call(T) - Method in interface org.apache.spark.api.java.function.MapFunction
- call(T) - Method in interface org.apache.spark.api.java.function.PairFlatMapFunction
- call(T) - Method in interface org.apache.spark.api.java.function.PairFunction
- call(T) - Method in interface org.apache.spark.api.java.function.VoidFunction
- call(T1) - Method in interface org.apache.spark.api.java.function.Function
- call(T1) - Method in interface org.apache.spark.sql.api.java.UDF1
- call(T1, T2) - Method in interface org.apache.spark.api.java.function.FlatMapFunction2
- call(T1, T2) - Method in interface org.apache.spark.api.java.function.Function2
- call(T1, T2) - Method in interface org.apache.spark.api.java.function.VoidFunction2
- call(T1, T2) - Method in interface org.apache.spark.sql.api.java.UDF2
- call(T1, T2, T3) - Method in interface org.apache.spark.api.java.function.Function3
- call(T1, T2, T3) - Method in interface org.apache.spark.sql.api.java.UDF3
- call(T1, T2, T3, T4) - Method in interface org.apache.spark.api.java.function.Function4
- call(T1, T2, T3, T4) - Method in interface org.apache.spark.sql.api.java.UDF4
- call(T1, T2, T3, T4, T5) - Method in interface org.apache.spark.sql.api.java.UDF5
- call(T1, T2, T3, T4, T5, T6) - Method in interface org.apache.spark.sql.api.java.UDF6
- call(T1, T2, T3, T4, T5, T6, T7) - Method in interface org.apache.spark.sql.api.java.UDF7
- call(T1, T2, T3, T4, T5, T6, T7, T8) - Method in interface org.apache.spark.sql.api.java.UDF8
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9) - Method in interface org.apache.spark.sql.api.java.UDF9
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10) - Method in interface org.apache.spark.sql.api.java.UDF10
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11) - Method in interface org.apache.spark.sql.api.java.UDF11
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12) - Method in interface org.apache.spark.sql.api.java.UDF12
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13) - Method in interface org.apache.spark.sql.api.java.UDF13
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14) - Method in interface org.apache.spark.sql.api.java.UDF14
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15) - Method in interface org.apache.spark.sql.api.java.UDF15
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, T16) - Method in interface org.apache.spark.sql.api.java.UDF16
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, T16, T17) - Method in interface org.apache.spark.sql.api.java.UDF17
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, T16, T17, T18) - Method in interface org.apache.spark.sql.api.java.UDF18
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, T16, T17, T18, T19) - Method in interface org.apache.spark.sql.api.java.UDF19
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, T16, T17, T18, T19, T20) - Method in interface org.apache.spark.sql.api.java.UDF20
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, T16, T17, T18, T19, T20, T21) - Method in interface org.apache.spark.sql.api.java.UDF21
- call(T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11, T12, T13, T14, T15, T16, T17, T18, T19, T20, T21, T22) - Method in interface org.apache.spark.sql.api.java.UDF22
- call(T, T) - Method in interface org.apache.spark.api.java.function.ReduceFunction
- call_function(String, Column...) - Static method in class org.apache.spark.sql.functions
-
Call a SQL function.
- call_function(String, Seq<Column>) - Static method in class org.apache.spark.sql.functions
-
Call a SQL function.
- call_udf(String, Column...) - Static method in class org.apache.spark.sql.functions
-
Call an user-defined function.
- call_udf(String, Seq<Column>) - Static method in class org.apache.spark.sql.functions
-
Call an user-defined function.
- callSite() - Method in class org.apache.spark.storage.RDDInfo
- CALLSITE_FIELD_NUMBER - Static variable in class org.apache.spark.status.protobuf.StoreTypes.RDDOperationNode
- callUDF(String, Column...) - Static method in class org.apache.spark.sql.functions
-
Call an user-defined function.
- callUDF(String, Seq<Column>) - Static method in class org.apache.spark.sql.functions
-
Deprecated.Use call_udf. Since .
- cancel() - Method in class org.apache.spark.ComplexFutureAction
- cancel() - Method in interface org.apache.spark.FutureAction
-
Cancels the execution of this action.
- cancel() - Method in class org.apache.spark.SimpleFutureAction
- cancelAllJobs() - Method in class org.apache.spark.api.java.JavaSparkContext
-
Cancel all jobs that have been scheduled or are running.
- cancelAllJobs() - Method in class org.apache.spark.SparkContext
-
Cancel all jobs that have been scheduled or are running.
- cancelJob(int) - Method in class org.apache.spark.SparkContext
-
Cancel a given job if it's scheduled or running.
- cancelJob(int, String) - Method in class org.apache.spark.SparkContext
-
Cancel a given job if it's scheduled or running.
- cancelJobGroup(String) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Cancel active jobs for the specified group.
- cancelJobGroup(String) - Method in class org.apache.spark.SparkContext
-
Cancel active jobs for the specified group.
- cancelJobsWithTag(String) - Method in class org.apache.spark.api.java.JavaSparkContext
-
Cancel active jobs that have the specified tag.
- cancelJobsWithTag(String) - Method in class org.apache.spark.SparkContext
-
Cancel active jobs that have the specified tag.
- cancelStage(int) - Method in class org.apache.spark.SparkContext
-
Cancel a given stage and all jobs associated with it.
- cancelStage(int, String) - Method in class org.apache.spark.SparkContext
-
Cancel a given stage and all jobs associated with it.
- cancelTasks(int, boolean, String) - Method in interface org.apache.spark.scheduler.TaskScheduler
- canCreate(String) - Method in interface org.apache.spark.scheduler.ExternalClusterManager
-
Check if this cluster manager instance can create scheduler components for a certain master URL.
- canDeleteWhere(Predicate[]) - Method in interface org.apache.spark.sql.connector.catalog.SupportsDelete
- canDeleteWhere(Predicate[]) - Method in interface org.apache.spark.sql.connector.catalog.SupportsDeleteV2
-
Checks whether it is possible to delete data from a data source table that matches filter expressions.
- canDeleteWhere(Filter[]) - Method in interface org.apache.spark.sql.connector.catalog.SupportsDelete
-
Checks whether it is possible to delete data from a data source table that matches filter expressions.
- canEqual(Object) - Static method in class org.apache.spark.ExpireDeadHosts
- canEqual(Object) - Static method in class org.apache.spark.metrics.DirectPoolMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.GarbageCollectionMetrics
- canEqual(Object) - Static method in class org.apache.spark.metrics.JVMHeapMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.JVMOffHeapMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.MappedPoolMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.OffHeapExecutionMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.OffHeapStorageMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.OffHeapUnifiedMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.OnHeapExecutionMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.OnHeapStorageMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.OnHeapUnifiedMemory
- canEqual(Object) - Static method in class org.apache.spark.metrics.ProcessTreeMetrics
- canEqual(Object) - Static method in class org.apache.spark.ml.feature.Dot
- canEqual(Object) - Static method in class org.apache.spark.ml.feature.EmptyTerm
- canEqual(Object) - Static method in class org.apache.spark.Resubmitted
- canEqual(Object) - Static method in class org.apache.spark.scheduler.AllJobsCancelled
- canEqual(Object) - Method in class org.apache.spark.scheduler.cluster.ExecutorInfo
- canEqual(Object) - Static method in class org.apache.spark.scheduler.JobSucceeded
- canEqual(Object) - Static method in class org.apache.spark.scheduler.ResubmitFailedStages
- canEqual(Object) - Static method in class org.apache.spark.scheduler.StopCoordinator
- canEqual(Object) - Static method in class org.apache.spark.sql.jdbc.MySQLDialect
- canEqual(Object) - Static method in class org.apache.spark.sql.jdbc.OracleDialect
- canEqual(Object) - Static method in class org.apache.spark.sql.jdbc.TeradataDialect
- canEqual(Object) - Static method in class org.apache.spark.sql.sources.AlwaysFalse
- canEqual(Object) - Static method in class org.apache.spark.sql.sources.AlwaysTrue
- canEqual(Object) - Static method in class org.apache.spark.sql.types.BinaryType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.BooleanType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.BooleanTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.ByteType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.ByteTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.CalendarIntervalType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.DateType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.DateTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.DayTimeIntervalType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.DoubleType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.DoubleTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.FloatType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.FloatTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.IntegerType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.IntegerTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.LongType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.LongTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.NullType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.ShortType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.ShortTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.StringType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.StringTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.TimestampNTZType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.TimestampType
- canEqual(Object) - Static method in class org.apache.spark.sql.types.TimestampTypeExpression
- canEqual(Object) - Static method in class org.apache.spark.sql.types.YearMonthIntervalType
- canEqual(Object) - Static method in class org.apache.spark.StopMapOutputTracker
- canEqual(Object) - Static method in class org.apache.spark.streaming.kinesis.DefaultCredentials
- canEqual(Object) - Static method in class org.apache.spark.streaming.scheduler.AllReceiverIds
- canEqual(Object) - Static method in class org.apache.spark.streaming.scheduler.GetAllReceiverInfo
- canEqual(Object) - Static method in class org.apache.spark.streaming.scheduler.StopAllReceivers
- canEqual(Object) - Static method in class org.apache.spark.Success
- canEqual(Object) - Static method in class org.apache.spark.TaskResultLost
- canEqual(Object) - Static method in class org.apache.spark.TaskSchedulerIsSet
- canEqual(Object) - Static method in class org.apache.spark.UnknownReason
- canEqual(Object) - Method in class org.apache.spark.util.MutablePair
- canHandle(String) - Method in class org.apache.spark.sql.jdbc.AggregatedDialect
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.DB2Dialect
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.DerbyDialect
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.H2Dialect
- canHandle(String) - Method in class org.apache.spark.sql.jdbc.JdbcDialect
-
Check if this dialect instance can handle a certain jdbc url.
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.MsSqlServerDialect
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.MySQLDialect
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.NoopDialect
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.OracleDialect
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.PostgresDialect
- canHandle(String) - Static method in class org.apache.spark.sql.jdbc.TeradataDialect
- canHandle(Driver, Map<String, String>) - Method in class org.apache.spark.sql.jdbc.JdbcConnectionProvider
-
Checks if this connection provider instance can handle the connection initiated by the driver.
- cannotAcquireMemoryToBuildLongHashedRelationError(long, long) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotAcquireMemoryToBuildUnsafeHashedRelationError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotAddMultiPartitionsOnNonatomicPartitionTableError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotAllocateMemoryToGrowBytesToBytesMapError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotAlterTableWithAlterViewError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotAlterViewWithAlterTableError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotBroadcastTableOverMaxTableBytesError(long, long) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotBroadcastTableOverMaxTableRowsError(long, long) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotBuildHashedRelationLargerThan8GError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotBuildHashedRelationWithUniqueKeysExceededError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCastError(DataType, DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCastFromNullTypeError(DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotChangeDecimalPrecisionError(Decimal, int, int, SQLQueryContext) - Static method in class org.apache.spark.sql.errors.DataTypeErrors
- cannotChangeDecimalPrecisionError(Decimal, int, int, SQLQueryContext) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotChangeStorageLevelError() - Static method in class org.apache.spark.errors.SparkCoreErrors
- cannotCleanReservedNamespacePropertyError(String, ParserRuleContext, String) - Static method in class org.apache.spark.sql.errors.QueryParsingErrors
- cannotCleanReservedTablePropertyError(String, ParserRuleContext, String) - Static method in class org.apache.spark.sql.errors.QueryParsingErrors
- cannotClearOutputDirectoryError(Path) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotClearPartitionDirectoryError(Path) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCloneOrCopyReadOnlySQLConfError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCompareCostWithTargetCostError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotConvertCatalystTypeToProtobufTypeError(Seq<String>, String, DataType, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotConvertCatalystValueToProtobufEnumTypeError(Seq<String>, String, String, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotConvertDataTypeToParquetTypeError(StructField) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotConvertOrcTimestampNTZToTimestampLTZError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotConvertOrcTimestampToTimestampNTZError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotConvertProtobufTypeToCatalystTypeError(String, DataType, Throwable) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotConvertProtobufTypeToSqlTypeError(String, Seq<String>, String, DataType) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotConvertSqlTypeToProtobufError(String, DataType, Throwable) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateArrayWithElementsExceedLimitError(long, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCreateColumnarReaderError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCreateDatabaseWithSameNameAsPreservedDatabaseError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateJDBCNamespaceUsingProviderError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateJDBCNamespaceWithPropertyError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateJDBCTableUsingLocationError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateJDBCTableUsingProviderError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateJDBCTableWithPartitionsError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCreateParquetConverterForDataTypeError(DataType, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCreateParquetConverterForDecimalTypeError(DecimalType, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCreateParquetConverterForTypeError(DecimalType, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCreateStagingDirError(String, IOException) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotCreateTableWithBothProviderAndSerdeError(Option<String>, Option<SerdeInfo>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateTempViewUsingHiveDataSourceError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateViewNotEnoughColumnsError(TableIdentifier, Seq<String>, LogicalPlan) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotCreateViewTooManyColumnsError(TableIdentifier, Seq<String>, LogicalPlan) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotDeleteTableWhereFiltersError(Table, Predicate[]) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotDropBuiltinFuncError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotDropDefaultDatabaseError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotDropMultiPartitionsOnNonatomicPartitionTableError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotDropNonemptyDatabaseError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotDropNonemptyNamespaceError(Seq<String>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotEvaluateExpressionError(Expression) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotExecuteStreamingRelationExecError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotFetchTablesOfDatabaseError(String, Exception) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotFindCatalogToHandleIdentifierError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotFindCatalystTypeInProtobufSchemaError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotFindColumnError(String, String[]) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotFindColumnInRelationOutputError(String, LogicalPlan) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotFindConstructorForTypeError(String) - Method in interface org.apache.spark.sql.errors.ExecutionErrors
- cannotFindConstructorForTypeError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotFindDescriptorFileError(String, Throwable) - Method in interface org.apache.spark.sql.errors.CompilationErrors
- cannotFindDescriptorFileError(String, Throwable) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotFindEncoderForTypeError(String) - Method in interface org.apache.spark.sql.errors.ExecutionErrors
- cannotFindEncoderForTypeError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotFindPartitionColumnInPartitionSchemaError(StructField, StructType) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotFindProtobufFieldInCatalystError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotGenerateCodeForExpressionError(Expression) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotGenerateCodeForIncomparableTypeError(String, DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotGetEventTimeWatermarkError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotGetJdbcTypeError(DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotGetOuterPointerForInnerClassError(Class<?>) - Method in interface org.apache.spark.sql.errors.ExecutionErrors
- cannotGetOuterPointerForInnerClassError(Class<?>) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotGetSQLConfInSchedulerEventLoopThreadError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotHaveCircularReferencesInBeanClassError(Class<?>) - Method in interface org.apache.spark.sql.errors.ExecutionErrors
- cannotHaveCircularReferencesInBeanClassError(Class<?>) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotHaveCircularReferencesInClassError(String) - Method in interface org.apache.spark.sql.errors.ExecutionErrors
- cannotHaveCircularReferencesInClassError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotInstantiateAbstractCatalogPluginClassError(String, String, Exception) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotInterpolateClassIntoCodeBlockError(Object) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotLoadClassNotOnClassPathError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotLoadClassWhenRegisteringFunctionError(String, FunctionIdentifier) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotLoadUserDefinedTypeError(String, String) - Static method in class org.apache.spark.sql.errors.DataTypeErrors
- cannotLoadUserDefinedTypeError(String, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotMergeClassWithOtherClassError(String, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotMergeDecimalTypesWithIncompatibleScaleError(int, int) - Static method in class org.apache.spark.sql.errors.DataTypeErrors
- cannotMergeDecimalTypesWithIncompatibleScaleError(int, int) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotMergeIncompatibleDataTypesError(DataType, DataType) - Static method in class org.apache.spark.sql.errors.DataTypeErrors
- cannotMergeIncompatibleDataTypesError(DataType, DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotModifyValueOfSparkConfigError(String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotModifyValueOfStaticConfigError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotMutateReadOnlySQLConfError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotOperateOnHiveDataSourceFilesError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotOverwritePathBeingReadFromError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotOverwriteTableThatIsBeingReadFromError(TableIdentifier) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotParseDecimalError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotParseIntervalError(String, Throwable) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotParseJsonArraysAsStructsError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotParseJSONFieldError(JsonParser, JsonToken, DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotParseJSONFieldError(String, String, JsonToken, DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotParseStatisticAsPercentileError(String, NumberFormatException) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotParseStringAsDataTypeError(JsonParser, JsonToken, DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotParseStringAsDataTypeError(String, String, DataType) - Method in interface org.apache.spark.sql.errors.ExecutionErrors
- cannotParseValueTypeError(String, String, SqlBaseParser.TypeConstructorContext) - Static method in class org.apache.spark.sql.errors.QueryParsingErrors
- cannotPassTypedColumnInUntypedSelectError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotPurgeAsBreakInternalStateError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotReadFilesError(Throwable, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotReadFooterForFileError(Path, Exception) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotRecognizeHiveTypeError(ParseException, String, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotRefreshBuiltInFuncError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotRefreshTempFuncError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotRemovePartitionDirError(Path) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotRemoveReservedPropertyError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotRenameTableAcrossSchemaError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotRenameTableWithAlterViewError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotRenameTempViewToExistingTableError(TableIdentifier) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotRenameTempViewWithDatabaseSpecifiedError(TableIdentifier, TableIdentifier) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotReplaceMissingTableError(Identifier) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotReplaceMissingTableError(Identifier, Option<Throwable>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotResolveAttributeError(String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotResolveColumnGivenInputColumnsError(String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotResolveColumnNameAmongAttributesError(String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotResolveStarExpandGivenInputColumnsError(String, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotResolveWindowReferenceError(String, SqlBaseParser.WindowClauseContext) - Static method in class org.apache.spark.sql.errors.QueryParsingErrors
- cannotRestorePermissionsForPathError(FsPermission, Path) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotRetrieveTableOrViewNotInSameDatabaseError(Seq<QualifiedTableName>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotRunSubmitMapStageOnZeroPartitionRDDError() - Static method in class org.apache.spark.errors.SparkCoreErrors
- cannotSafelyMergeSerdePropertiesError(Map<String, String>, Map<String, String>, Set<String>) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotSaveBlockOnDecommissionedExecutorError(BlockId) - Static method in class org.apache.spark.errors.SparkCoreErrors
- cannotSaveIntervalIntoExternalStorageError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotSetJDBCNamespaceWithPropertyError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotSetTimeoutDurationError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotSetTimeoutTimestampError() - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotSpecifyBothJdbcTableNameAndQueryError(String, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotSpecifyDatabaseForTempViewError(TableIdentifier) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotSpecifyWindowFrameError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotTerminateGeneratorError(UnresolvedGenerator) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotTranslateExpressionToSourceFilterError(Expression) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotTranslateNonNullValueForFieldError(int) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotUnsetJDBCNamespaceWithPropertyError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotUseAllColumnsForPartitionColumnsError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotUseDataTypeForPartitionColumnError(StructField) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotUseIntervalTypeInTableSchemaError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotUseInvalidJavaIdentifierAsFieldNameError(String, WalkedTypePath) - Method in interface org.apache.spark.sql.errors.ExecutionErrors
- cannotUseInvalidJavaIdentifierAsFieldNameError(String, WalkedTypePath) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- cannotUseMapSideCombiningWithArrayKeyError() - Static method in class org.apache.spark.errors.SparkCoreErrors
- cannotUsePreservedDatabaseAsCurrentDatabaseError(String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotWriteDataToRelationsWithMultiplePathsError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotWriteNotEnoughColumnsToTableError(String, Seq<String>, Seq<Attribute>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- cannotWriteTooManyColumnsToTableError(String, Seq<String>, Seq<Attribute>) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- canonicalName() - Method in interface org.apache.spark.sql.connector.catalog.functions.BoundFunction
-
Returns the canonical name of this function, used to determine if functions are equivalent.
- canonicalName() - Method in class org.apache.spark.sql.connector.expressions.aggregate.UserDefinedAggregateFunc
- canonicalName() - Method in class org.apache.spark.sql.connector.expressions.UserDefinedScalarFunc
- CanonicalRandomVertexCut$() - Constructor for class org.apache.spark.graphx.PartitionStrategy.CanonicalRandomVertexCut$
- canOnlyZipRDDsWithSamePartitionSizeError() - Static method in class org.apache.spark.errors.SparkCoreErrors
- canOverwrite(Predicate[]) - Method in interface org.apache.spark.sql.connector.write.SupportsOverwrite
- canOverwrite(Predicate[]) - Method in interface org.apache.spark.sql.connector.write.SupportsOverwriteV2
-
Checks whether it is possible to overwrite data from a data source table that matches filter expressions.
- canOverwrite(Filter[]) - Method in interface org.apache.spark.sql.connector.write.SupportsOverwrite
-
Checks whether it is possible to overwrite data from a data source table that matches filter expressions.
- canRenameConflictingMetadataColumns() - Method in interface org.apache.spark.sql.connector.catalog.SupportsMetadataColumns
-
Determines how this data source handles name conflicts between metadata and data columns.
- canUpCast(DataType, DataType) - Static method in class org.apache.spark.sql.types.UpCastRule
-
Returns true iff we can safely up-cast the
from
type toto
type without any truncating or precision lose or possible runtime failures. - capabilities() - Method in class org.apache.spark.sql.connector.catalog.DelegatingCatalogExtension
- capabilities() - Method in interface org.apache.spark.sql.connector.catalog.Table
-
Returns the set of capabilities for this table.
- capabilities() - Method in interface org.apache.spark.sql.connector.catalog.TableCatalog
- cardinality() - Method in class org.apache.spark.util.sketch.BloomFilter
- cardinality(Column) - Static method in class org.apache.spark.sql.functions
-
Returns length of array or map.
- cartesian(JavaRDDLike<U, ?>) - Method in interface org.apache.spark.api.java.JavaRDDLike
-
Return the Cartesian product of this RDD and another one, that is, the RDD of all pairs of elements (a, b) where a is in
this
and b is inother
. - cartesian(RDD<U>, ClassTag<U>) - Method in class org.apache.spark.rdd.RDD
-
Return the Cartesian product of this RDD and another one, that is, the RDD of all pairs of elements (a, b) where a is in
this
and b is inother
. - CaseInsensitiveStringMap - Class in org.apache.spark.sql.util
-
Case-insensitive map of string keys to string values.
- CaseInsensitiveStringMap(Map<String, String>) - Constructor for class org.apache.spark.sql.util.CaseInsensitiveStringMap
- caseSensitive() - Method in class org.apache.spark.ml.feature.StopWordsRemover
-
Whether to do a case sensitive comparison over the stop words.
- cast(String) - Method in class org.apache.spark.sql.Column
-
Casts the column to a different data type, using the canonical string representation of the type.
- cast(DataType) - Method in class org.apache.spark.sql.Column
-
Casts the column to a different data type.
- Cast - Class in org.apache.spark.sql.connector.expressions
-
Represents a cast expression in the public logical expression API.
- Cast(Expression, DataType) - Constructor for class org.apache.spark.sql.connector.expressions.Cast
- castingCauseOverflowError(Object, DataType, DataType) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- castingCauseOverflowError(String, DataType, DataType) - Static method in class org.apache.spark.sql.errors.DataTypeErrors
- castingCauseOverflowErrorInTableInsert(DataType, DataType, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- castPartitionSpec(String, DataType, SQLConf) - Static method in class org.apache.spark.sql.util.PartitioningUtils
- catalog() - Method in class org.apache.spark.sql.catalog.Database
- catalog() - Method in class org.apache.spark.sql.catalog.Function
- catalog() - Method in class org.apache.spark.sql.catalog.Table
- catalog() - Method in class org.apache.spark.sql.SparkSession
- Catalog - Class in org.apache.spark.sql.catalog
-
Catalog interface for Spark.
- Catalog() - Constructor for class org.apache.spark.sql.catalog.Catalog
- CatalogAndIdentifier() - Constructor for class org.apache.spark.sql.connector.catalog.LookupCatalog.CatalogAndIdentifier
- CatalogAndIdentifier() - Method in interface org.apache.spark.sql.connector.catalog.LookupCatalog
- CatalogAndIdentifier$() - Constructor for class org.apache.spark.sql.connector.catalog.LookupCatalog.CatalogAndIdentifier$
- CatalogAndMultipartIdentifier() - Method in interface org.apache.spark.sql.connector.catalog.LookupCatalog
- CatalogAndNamespace() - Constructor for class org.apache.spark.sql.connector.catalog.LookupCatalog.CatalogAndNamespace
- CatalogAndNamespace() - Method in interface org.apache.spark.sql.connector.catalog.LookupCatalog
- CatalogAndNamespace$() - Constructor for class org.apache.spark.sql.connector.catalog.LookupCatalog.CatalogAndNamespace$
- CatalogExtension - Interface in org.apache.spark.sql.connector.catalog
-
An API to extend the Spark built-in session catalog.
- catalogFailToCallPublicNoArgConstructorError(String, String, Exception) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- catalogFailToFindPublicNoArgConstructorError(String, String, Exception) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- CatalogHelper(CatalogPlugin) - Constructor for class org.apache.spark.sql.connector.catalog.CatalogV2Implicits.CatalogHelper
- catalogManager() - Method in interface org.apache.spark.sql.connector.catalog.LookupCatalog
- CatalogMetadata - Class in org.apache.spark.sql.catalog
-
A catalog in Spark, as returned by the
listCatalogs
method defined inCatalog
. - CatalogMetadata(String, String) - Constructor for class org.apache.spark.sql.catalog.CatalogMetadata
- CatalogNotFoundException - Exception in org.apache.spark.sql.connector.catalog
- CatalogNotFoundException(String) - Constructor for exception org.apache.spark.sql.connector.catalog.CatalogNotFoundException
- CatalogNotFoundException(String, Throwable) - Constructor for exception org.apache.spark.sql.connector.catalog.CatalogNotFoundException
- catalogOperationNotSupported(CatalogPlugin, String) - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- CatalogPlugin - Interface in org.apache.spark.sql.connector.catalog
-
A marker interface to provide a catalog implementation for Spark.
- catalogPluginClassNotFoundError(String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- catalogPluginClassNotFoundForCatalogError(String, String, Exception) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- catalogPluginClassNotImplementedError(String, String) - Static method in class org.apache.spark.sql.errors.QueryExecutionErrors
- Catalogs - Class in org.apache.spark.sql.connector.catalog
- Catalogs() - Constructor for class org.apache.spark.sql.connector.catalog.Catalogs
- catalogString() - Method in class org.apache.spark.sql.types.ArrayType
- catalogString() - Static method in class org.apache.spark.sql.types.BinaryType
- catalogString() - Static method in class org.apache.spark.sql.types.BooleanType
- catalogString() - Static method in class org.apache.spark.sql.types.ByteType
- catalogString() - Static method in class org.apache.spark.sql.types.CalendarIntervalType
- catalogString() - Method in class org.apache.spark.sql.types.DataType
-
String representation for the type saved in external catalogs.
- catalogString() - Static method in class org.apache.spark.sql.types.DateType
- catalogString() - Static method in class org.apache.spark.sql.types.DoubleType
- catalogString() - Static method in class org.apache.spark.sql.types.FloatType
- catalogString() - Static method in class org.apache.spark.sql.types.IntegerType
- catalogString() - Static method in class org.apache.spark.sql.types.LongType
- catalogString() - Method in class org.apache.spark.sql.types.MapType
- catalogString() - Static method in class org.apache.spark.sql.types.NullType
- catalogString() - Static method in class org.apache.spark.sql.types.ShortType
- catalogString() - Static method in class org.apache.spark.sql.types.StringType
- catalogString() - Method in class org.apache.spark.sql.types.StructType
- catalogString() - Static method in class org.apache.spark.sql.types.TimestampNTZType
- catalogString() - Static method in class org.apache.spark.sql.types.TimestampType
- catalogString() - Method in class org.apache.spark.sql.types.UserDefinedType
- CatalogV2Implicits - Class in org.apache.spark.sql.connector.catalog
-
Conversion helpers for working with v2
CatalogPlugin
. - CatalogV2Implicits() - Constructor for class org.apache.spark.sql.connector.catalog.CatalogV2Implicits
- CatalogV2Implicits.BucketSpecHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.CatalogHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.ColumnsHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.FunctionIdentifierHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.IdentifierHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.MultipartIdentifierHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.NamespaceHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.PartitionTypeHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.TableIdentifierHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Implicits.TransformHelper - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Util - Class in org.apache.spark.sql.connector.catalog
- CatalogV2Util() - Constructor for class org.apache.spark.sql.connector.catalog.CatalogV2Util
- CatalystScan - Interface in org.apache.spark.sql.sources
-
::Experimental:: An interface for experimenting with a more direct connection to the query planner.
- Categorical() - Static method in class org.apache.spark.mllib.tree.configuration.FeatureType
- categoricalCols() - Method in class org.apache.spark.ml.feature.FeatureHasher
-
Numeric columns to treat as categorical features.
- categoricalFeaturesInfo() - Method in class org.apache.spark.mllib.tree.configuration.Strategy
- CategoricalSplit - Class in org.apache.spark.ml.tree
-
Split which tests a categorical feature.
- categories() - Method in class org.apache.spark.mllib.tree.model.DecisionTreeModel.SaveLoadV1_0$.SplitData
- categories() - Method in class org.apache.spark.mllib.tree.model.Split
- categoryMaps() - Method in class org.apache.spark.ml.feature.VectorIndexerModel
- categorySizes() - Method in class org.apache.spark.ml.feature.OneHotEncoderModel
- cause() - Method in exception org.apache.spark.sql.AnalysisException
- cause() - Method in exception org.apache.spark.sql.streaming.StreamingQueryException
- CausedBy - Class in org.apache.spark.util
-
Extractor Object for pulling out the root cause of an error.
- CausedBy() - Constructor for class org.apache.spark.util.CausedBy
- cbrt(String) - Static method in class org.apache.spark.sql.functions
-
Computes the cube-root of the given column.
- cbrt(Column) - Static method in class org.apache.spark.sql.functions
-
Computes the cube-root of the given value.
- ceil() - Method in class org.apache.spark.sql.types.Decimal
- ceil(String) - Static method in class org.apache.spark.sql.functions
-
Computes the ceiling of the given value of
e
to 0 decimal places. - ceil(Column) - Static method in class org.apache.spark.sql.functions
-
Computes the ceiling of the given value of
e
to 0 decimal places. - ceil(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Computes the ceiling of the given value of
e
toscale
decimal places. - ceiling(Column) - Static method in class org.apache.spark.sql.functions
-
Computes the ceiling of the given value of
e
to 0 decimal places. - ceiling(Column, Column) - Static method in class org.apache.spark.sql.functions
-
Computes the ceiling of the given value of
e
toscale
decimal places. - censorCol() - Method in class org.apache.spark.ml.regression.AFTSurvivalRegression
- censorCol() - Method in class org.apache.spark.ml.regression.AFTSurvivalRegressionModel
- censorCol() - Method in interface org.apache.spark.ml.regression.AFTSurvivalRegressionParams
-
Param for censor column name.
- centerMatrix() - Method in class org.apache.spark.ml.clustering.KMeansAggregator
- chainl1(Function0<Parsers.Parser<T>>, Function0<Parsers.Parser<Function2<T, T, T>>>) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- chainl1(Function0<Parsers.Parser<T>>, Function0<Parsers.Parser<U>>, Function0<Parsers.Parser<Function2<T, U, T>>>) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- chainr1(Function0<Parsers.Parser<T>>, Function0<Parsers.Parser<Function2<T, U, U>>>, Function2<T, U, U>, U) - Static method in class org.apache.spark.ml.feature.RFormulaParser
- changePrecision(int, int) - Method in class org.apache.spark.sql.types.Decimal
-
Update precision and scale while keeping our value the same, and return true if successful.
- channel() - Method in interface org.apache.spark.shuffle.api.WritableByteChannelWrapper
-
The underlying channel to write bytes into.
- channelRead0(ChannelHandlerContext, byte[]) - Method in class org.apache.spark.api.r.RBackendAuthHandler
- char_length(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the character length of string data or number of bytes of binary data.
- character_length(Column) - Static method in class org.apache.spark.sql.functions
-
Returns the character length of string data or number of bytes of binary data.
- charOrVarcharTypeAsStringUnsupportedError() - Static method in class org.apache.spark.sql.errors.DataTypeErrors
- charOrVarcharTypeAsStringUnsupportedError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- CharType - Class in org.apache.spark.sql.types
- CharType(int) - Constructor for class org.apache.spark.sql.types.CharType
- charTypeMissingLengthError(String, SqlBaseParser.PrimitiveDataTypeContext) - Static method in class org.apache.spark.sql.errors.QueryParsingErrors
- checkAndGetK8sMasterUrl(String) - Static method in class org.apache.spark.util.Utils
-
Check the validity of the given Kubernetes master URL and return the resolved URL.
- checkColumnNameDuplication(Seq<String>, boolean) - Static method in class org.apache.spark.sql.util.SchemaUtils
-
Checks if input column names have duplicate identifiers.
- checkColumnNameDuplication(Seq<String>, Function2<String, String, Object>) - Static method in class org.apache.spark.sql.util.SchemaUtils
-
Checks if input column names have duplicate identifiers.
- checkColumnType(StructType, String, DataType, String) - Static method in class org.apache.spark.ml.util.SchemaUtils
-
Check whether the given schema contains a column of the required data type.
- checkColumnTypes(StructType, String, Seq<DataType>, String) - Static method in class org.apache.spark.ml.util.SchemaUtils
-
Check whether the given schema contains a column of one of the require data types.
- checkDataColumns(RFormula, Dataset<?>) - Static method in class org.apache.spark.ml.r.RWrapperUtils
-
DataFrame column check.
- checkFileExists(String, Configuration) - Static method in class org.apache.spark.streaming.util.HdfsUtils
-
Check if the file exists at the given path.
- checkHost(String) - Static method in class org.apache.spark.util.Utils
-
Checks if the host contains only valid hostname/ip without port NOTE: Incase of IPV6 ip it should be enclosed inside []
- checkHostPort(String) - Static method in class org.apache.spark.util.Utils
- checkIntegers(Dataset<?>, String) - Method in interface org.apache.spark.ml.recommendation.ALSModelParams
-
Attempts to safely cast a user/item id to an Int.
- checkNumericType(StructType, String, String) - Static method in class org.apache.spark.ml.util.SchemaUtils
-
Check whether the given schema contains a column of the numeric data type.
- checkOffHeapEnabled(SparkConf, long) - Static method in class org.apache.spark.util.Utils
-
return 0 if MEMORY_OFFHEAP_ENABLED is false.
- checkpoint() - Method in interface org.apache.spark.api.java.JavaRDDLike
-
Mark this RDD for checkpointing.
- checkpoint() - Method in class org.apache.spark.graphx.Graph
-
Mark this Graph for checkpointing.
- checkpoint() - Method in class org.apache.spark.graphx.impl.EdgeRDDImpl
- checkpoint() - Method in class org.apache.spark.graphx.impl.GraphImpl
- checkpoint() - Method in class org.apache.spark.graphx.impl.VertexRDDImpl
- checkpoint() - Method in class org.apache.spark.rdd.HadoopRDD
- checkpoint() - Method in class org.apache.spark.rdd.RDD
-
Mark this RDD for checkpointing.
- checkpoint() - Method in class org.apache.spark.sql.Dataset
-
Eagerly checkpoint a Dataset and return the new Dataset.
- checkpoint(boolean) - Method in class org.apache.spark.sql.Dataset
-
Returns a checkpointed version of this Dataset.
- checkpoint(String) - Method in class org.apache.spark.streaming.api.java.JavaStreamingContext
-
Deprecated.Sets the context to periodically checkpoint the DStream operations for master fault-tolerance.
- checkpoint(String) - Method in class org.apache.spark.streaming.StreamingContext
-
Deprecated.Set the context to periodically checkpoint the DStream operations for driver fault-tolerance.
- checkpoint(Duration) - Method in interface org.apache.spark.streaming.api.java.JavaDStreamLike
-
Enable periodic checkpointing of RDDs of this DStream.
- checkpoint(Duration) - Method in class org.apache.spark.streaming.dstream.DStream
-
Enable periodic checkpointing of RDDs of this DStream
- checkpointCleaned(long) - Method in interface org.apache.spark.CleanerListener
- checkpointDirectoryHasNotBeenSetInSparkContextError() - Static method in class org.apache.spark.errors.SparkCoreErrors
- Checkpointed() - Static method in class org.apache.spark.rdd.CheckpointState
- checkpointFailedToSaveError(int, Path) - Static method in class org.apache.spark.errors.SparkCoreErrors
- CheckpointingInProgress() - Static method in class org.apache.spark.rdd.CheckpointState
- checkpointInterval() - Method in class org.apache.spark.ml.classification.DecisionTreeClassificationModel
- checkpointInterval() - Method in class org.apache.spark.ml.classification.DecisionTreeClassifier
- checkpointInterval() - Method in class org.apache.spark.ml.classification.GBTClassificationModel
- checkpointInterval() - Method in class org.apache.spark.ml.classification.GBTClassifier
- checkpointInterval() - Method in class org.apache.spark.ml.classification.RandomForestClassificationModel
- checkpointInterval() - Method in class org.apache.spark.ml.classification.RandomForestClassifier
- checkpointInterval() - Method in class org.apache.spark.ml.clustering.LDA
- checkpointInterval() - Method in class org.apache.spark.ml.clustering.LDAModel
- checkpointInterval() - Method in interface org.apache.spark.ml.param.shared.HasCheckpointInterval
-
Param for set checkpoint interval (>= 1) or disable checkpoint (-1).
- checkpointInterval() - Method in class org.apache.spark.ml.recommendation.ALS
- checkpointInterval() - Method in class org.apache.spark.ml.regression.DecisionTreeRegressionModel
- checkpointInterval() - Method in class org.apache.spark.ml.regression.DecisionTreeRegressor
- checkpointInterval() - Method in class org.apache.spark.ml.regression.GBTRegressionModel
- checkpointInterval() - Method in class org.apache.spark.ml.regression.GBTRegressor
- checkpointInterval() - Method in class org.apache.spark.ml.regression.RandomForestRegressionModel
- checkpointInterval() - Method in class org.apache.spark.ml.regression.RandomForestRegressor
- checkpointInterval() - Method in class org.apache.spark.mllib.tree.configuration.Strategy
- checkpointLocationNotSpecifiedError() - Static method in class org.apache.spark.sql.errors.QueryCompilationErrors
- checkpointRDDBlockIdNotFoundError(RDDBlockId) - Static method in class org.apache.spark.errors.SparkCoreErrors
- checkpointRDDHasDifferentNumberOfPartitionsFromOriginalRDDError(int, int, int, int) - Static method in class org.apache.spark.errors.SparkCoreErrors
- CheckpointReader - Class in org.apache.spark.streaming
- CheckpointReader() - Constructor for class org.apache.spark.streaming.CheckpointReader
- CheckpointState - Class in org.apache.spark.rdd
-
Enumeration to manage state transitions of an RDD through checkpointing
- CheckpointState() - Constructor for class org.apache.spark.rdd.CheckpointState
- checkSchemaColumnNameDuplication(DataType, boolean) - Static method in class org.apache.spark.sql.util.SchemaUtils
-
Checks if an input schema has duplicate column names.
- checkSchemaColumnNameDuplication(StructType, Function2<String, String, Object>) - Static method in class org.apache.spark.sql.util.SchemaUtils
-
Checks if an input schema has duplicate column names.
- checkSingleVsMultiColumnParams(Params, Seq<Param<?>>, Seq<Param<?>>) - Static method in class org.apache.spark.ml.param.ParamValidators
-
Utility for Param validity checks for Transformers which have both single- and multi-column support.
- checkSpeculatableTasks(long) - Method in interface org.apache.spark.scheduler.Schedulable
- checkState(boolean, Function0<String>) - Static method in class org.apache.spark.strea