Package org.apache.spark.ui

Class UIUtils

Object
org.apache.spark.ui.UIUtils
public class UIUtils extends Object
Utility functions for generating XML pages with spark content.

  • Constructor Summary

    Constructors
    Constructor
    Description
    UIUtils()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static String
    actionFormMethod(boolean actionsViaGetEnabled)
    The method for the forms behind the UI's state-changing buttons (job and stage kill).
    static scala.collection.immutable.Seq<scala.xml.Node>
    basicSparkPage(jakarta.servlet.http.HttpServletRequest request, scala.Function0<scala.collection.immutable.Seq<scala.xml.Node>> content, String title, boolean useDataTables)
    Returns a page with the spark css/js and a simple format.
    static jakarta.ws.rs.core.Response
    buildErrorResponse(jakarta.ws.rs.core.Response.Status status, String msg)
     
    static scala.collection.immutable.Seq<scala.xml.Node>
    commonHeaderNodes(jakarta.servlet.http.HttpServletRequest request)
     
    static scala.collection.immutable.Seq<scala.xml.Node>
    dataTablesHeaderNodes(jakarta.servlet.http.HttpServletRequest request)
     
    static org.glassfish.jersey.internal.util.collection.MultivaluedStringMap
    decodeURLParameter(jakarta.ws.rs.core.MultivaluedMap<String,String> params)
    Decode URLParameter if URL is encoded by YARN-WebAppProxyServlet.
    static String
    decodeURLParameter(String urlParam)
    Decode URLParameter if URL is encoded by YARN-WebAppProxyServlet.
    static scala.collection.immutable.Seq<scala.xml.Node>
    detailsUINode(boolean isMultiline, String message)
     
    static scala.Tuple2<Object,scala.collection.immutable.Map<String,Object>>[]
    durationDataPadding(scala.Tuple2<Object,Map<String,Long>>[] values)
    There may be different duration labels in each batch.
    static String
    encodeLogParam(String value)
    URL-encode a single request-supplied value before embedding it in a log page query string.
    static scala.collection.immutable.Seq<scala.xml.Node>
    errorMessageCell(String errorMessage)
     
    static scala.Tuple2<String,Object>
    errorSummary(String errorMessage)
    This function works exactly the same as utils.errorSummary(javascript), it shall be
    static String
    formatBatchTime(long batchTime, long batchInterval, boolean showYYYYMMSS, TimeZone timezone)
    If batchInterval is less than 1 second, format batchTime with milliseconds.
    static String
    formatDate(long timestamp)
     
    static String
    formatDate(Date date)
     
    static String
    formatDuration(long milliseconds)
     
    static String
    formatDurationVerbose(long ms)
    Generate a verbose human-readable string representing a duration such as "5 second 35 ms"
    static String
    formatImportJavaScript(jakarta.servlet.http.HttpServletRequest request, String sourceFile, scala.collection.immutable.Seq<String> methods)
     
    static String
    formatNumber(double records)
    Generate a human-readable string representing a number (e.g.
    static int
    getTimeZoneOffset()
     
    static scala.collection.immutable.Seq<scala.xml.Node>
    headerSparkPage(jakarta.servlet.http.HttpServletRequest request, String title, scala.Function0<scala.collection.immutable.Seq<scala.xml.Node>> content, org.apache.spark.ui.SparkUITab activeTab, scala.Option<String> helpText, boolean showVisualization, boolean useDataTables)
    Returns a spark page with correctly formatted headers
    static <T> scala.collection.immutable.Seq<scala.xml.Node>
    listingTable(scala.collection.immutable.Seq<String> headers, scala.Function1<T,scala.collection.immutable.Seq<scala.xml.Node>> generateDataRow, scala.collection.Iterable<T> data, boolean fixedWidth, scala.Option<String> id, scala.collection.immutable.Seq<String> headerClasses, boolean stripeRowsWithCss, boolean sortable, scala.collection.immutable.Seq<scala.Option<String>> tooltipHeaders)
    Returns an HTML table constructed by generating a row for each object in a sequence.
    static String
    logPageOnloadScript(String logParams, long curLogLength, long startByte, long endByte, long logLength, long byteLength)
    Build the inline window.onload = initLogPage(...) bootstrap script shared by the log pages (history / master / worker / driver).
    static org.apache.spark.internal.Logging.LogStringContext
    LogStringContext(scala.StringContext sc)
     
    static scala.xml.NodeSeq
    makeDescription(String desc, String basePathUri, boolean plainText)
    Returns HTML rendering of a job or stage description.
    static String
    makeHref(boolean proxy, String id, String origHref)
    Return the correct Href after checking if master is running in the reverse proxy mode or not.
    static scala.collection.immutable.Seq<scala.xml.Node>
    makeProgressBar(int started, int completed, int failed, int skipped, scala.collection.immutable.Map<String,Object> reasonToNumKilled, int total)
     
    static org.slf4j.Logger
    org$apache$spark$internal$Logging$$log_()
     
    static void
    org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)
     
    static String
    prependBaseUri(jakarta.servlet.http.HttpServletRequest request, String basePath, String resource)
     
    static scala.collection.Seq<scala.xml.Node>
    showDagVizForJob(int jobId, scala.collection.Seq<org.apache.spark.ui.scope.RDDOperationGraph> graphs)
    Return a "DAG visualization" DOM element that expands into a visualization for a job.
    static scala.collection.immutable.Seq<scala.xml.Node>
    showDagVizForStage(int stageId, scala.Option<org.apache.spark.ui.scope.RDDOperationGraph> graph)
    Return a "DAG visualization" DOM element that expands into a visualization for a stage.
    static String
    TABLE_CLASS_NOT_STRIPED()
     
    static String
    TABLE_CLASS_STRIPED()
     
    static String
    TABLE_CLASS_STRIPED_SORTABLE()
     
    static scala.collection.immutable.Seq<scala.xml.Node>
    tooltip(String text, String position)
     
    static String
    uiRoot(jakarta.servlet.http.HttpServletRequest request)
     
    static scala.collection.immutable.Seq<scala.xml.Node>
    vizHeaderNodes(jakarta.servlet.http.HttpServletRequest request)
     

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • UIUtils

      public UIUtils()

  • Method Details

    • TABLE_CLASS_NOT_STRIPED

      public static String TABLE_CLASS_NOT_STRIPED()

    • TABLE_CLASS_STRIPED

      public static String TABLE_CLASS_STRIPED()

    • TABLE_CLASS_STRIPED_SORTABLE

      public static String TABLE_CLASS_STRIPED_SORTABLE()

    • formatDate

      public static String formatDate(Date date)

    • formatDate

      public static String formatDate(long timestamp)

    • formatDuration

      public static String formatDuration(long milliseconds)

    • formatDurationVerbose

      public static String formatDurationVerbose(long ms)
      Generate a verbose human-readable string representing a duration such as "5 second 35 ms"

    • formatBatchTime

      public static String formatBatchTime(long batchTime, long batchInterval, boolean showYYYYMMSS, TimeZone timezone)
      If batchInterval is less than 1 second, format batchTime with milliseconds. Otherwise, format batchTime without milliseconds.

      Parameters:
      batchTime - the batch time to be formatted
      batchInterval - the batch interval
      showYYYYMMSS - if showing the yyyy/MM/dd part. If it's false, the return value will be only HH:mm:ss or HH:mm:ss.SSS depending on batchInterval
      timezone - only for test
      Returns:
      (undocumented)

    • formatNumber

      public static String formatNumber(double records)
      Generate a human-readable string representing a number (e.g. 100 K)

    • uiRoot

      public static String uiRoot(jakarta.servlet.http.HttpServletRequest request)

    • prependBaseUri

      public static String prependBaseUri(jakarta.servlet.http.HttpServletRequest request, String basePath, String resource)

    • commonHeaderNodes

      public static scala.collection.immutable.Seq<scala.xml.Node> commonHeaderNodes(jakarta.servlet.http.HttpServletRequest request)

    • vizHeaderNodes

      public static scala.collection.immutable.Seq<scala.xml.Node> vizHeaderNodes(jakarta.servlet.http.HttpServletRequest request)

    • dataTablesHeaderNodes

      public static scala.collection.immutable.Seq<scala.xml.Node> dataTablesHeaderNodes(jakarta.servlet.http.HttpServletRequest request)

    • headerSparkPage

      public static scala.collection.immutable.Seq<scala.xml.Node> headerSparkPage(jakarta.servlet.http.HttpServletRequest request, String title, scala.Function0<scala.collection.immutable.Seq<scala.xml.Node>> content, org.apache.spark.ui.SparkUITab activeTab, scala.Option<String> helpText, boolean showVisualization, boolean useDataTables)
      Returns a spark page with correctly formatted headers

    • basicSparkPage

      public static scala.collection.immutable.Seq<scala.xml.Node> basicSparkPage(jakarta.servlet.http.HttpServletRequest request, scala.Function0<scala.collection.immutable.Seq<scala.xml.Node>> content, String title, boolean useDataTables)
      Returns a page with the spark css/js and a simple format. Used for scheduler UI.

    • listingTable

      public static <T> scala.collection.immutable.Seq<scala.xml.Node> listingTable(scala.collection.immutable.Seq<String> headers, scala.Function1<T,scala.collection.immutable.Seq<scala.xml.Node>> generateDataRow, scala.collection.Iterable<T> data, boolean fixedWidth, scala.Option<String> id, scala.collection.immutable.Seq<String> headerClasses, boolean stripeRowsWithCss, boolean sortable, scala.collection.immutable.Seq<scala.Option<String>> tooltipHeaders)
      Returns an HTML table constructed by generating a row for each object in a sequence.

    • makeProgressBar

      public static scala.collection.immutable.Seq<scala.xml.Node> makeProgressBar(int started, int completed, int failed, int skipped, scala.collection.immutable.Map<String,Object> reasonToNumKilled, int total)

    • showDagVizForStage

      public static scala.collection.immutable.Seq<scala.xml.Node> showDagVizForStage(int stageId, scala.Option<org.apache.spark.ui.scope.RDDOperationGraph> graph)
      Return a "DAG visualization" DOM element that expands into a visualization for a stage.

    • showDagVizForJob

      public static scala.collection.Seq<scala.xml.Node> showDagVizForJob(int jobId, scala.collection.Seq<org.apache.spark.ui.scope.RDDOperationGraph> graphs)
      Return a "DAG visualization" DOM element that expands into a visualization for a job.

    • tooltip

      public static scala.collection.immutable.Seq<scala.xml.Node> tooltip(String text, String position)

    • actionFormMethod

      public static String actionFormMethod(boolean actionsViaGetEnabled)
      The method for the forms behind the UI's state-changing buttons (job and stage kill). Those are always forms, carrying the per-UI CSRF token and the target id as hidden fields; only the method follows spark.ui.actionsViaGetEnabled. Submitted with GET, the browser turns the fields into the query string, the same request a plain link would make, which also works through proxies that do not forward POST, such as the YARN ResourceManager/AM proxy (SPARK-6846). SparkUI.initialize accepts GET on the endpoints under the same flag, so the form and the endpoint always agree, and the inline onclick handler prompts for confirmation before the submit either way.
      Parameters:
      actionsViaGetEnabled - (undocumented)
      Returns:
      (undocumented)

    • makeDescription

      public static scala.xml.NodeSeq makeDescription(String desc, String basePathUri, boolean plainText)
      Returns HTML rendering of a job or stage description. It will try to parse the string as HTML and make sure that it only contains anchors with root-relative links. Otherwise, the whole string will rendered as a simple escaped text.

      Note: In terms of security, only anchor tags with root relative links are supported. So any attempts to embed links outside Spark UI, other tags like &lt;script&gt;, or inline scripts like onclick will cause in the whole description to be treated as plain text.

      Parameters:
      desc - the original job or stage description string, which may contain html tags.
      basePathUri - with which to prepend the relative links; this is used when plainText is false.
      plainText - whether to keep only plain text (i.e. remove html tags) from the original description string.
      Returns:
      the HTML rendering of the job or stage description, which will be a Text when plainText is true, and an Elem otherwise.

    • decodeURLParameter

      public static String decodeURLParameter(String urlParam)
      Decode URLParameter if URL is encoded by YARN-WebAppProxyServlet. Due to YARN-2844: WebAppProxyServlet cannot handle urls which contain encoded characters Therefore we need to decode it until we get the real URLParameter.
      Parameters:
      urlParam - (undocumented)
      Returns:
      (undocumented)

    • decodeURLParameter

      public static org.glassfish.jersey.internal.util.collection.MultivaluedStringMap decodeURLParameter(jakarta.ws.rs.core.MultivaluedMap<String,String> params)
      Decode URLParameter if URL is encoded by YARN-WebAppProxyServlet.
      Parameters:
      params - (undocumented)
      Returns:
      (undocumented)

    • getTimeZoneOffset

      public static int getTimeZoneOffset()

    • encodeLogParam

      public static String encodeLogParam(String value)
      URL-encode a single request-supplied value before embedding it in a log page query string. This keeps characters such as '&', '=', '#' and whitespace from changing the structure of the subsequent /log requests (parameter injection / query truncation).
      Parameters:
      value - (undocumented)
      Returns:
      (undocumented)

    • logPageOnloadScript

      public static String logPageOnloadScript(String logParams, long curLogLength, long startByte, long endByte, long logLength, long byteLength)
      Build the inline window.onload = initLogPage(...) bootstrap script shared by the log pages (history / master / worker / driver). Individual parameter values in logParams must already be URL-encoded via encodeLogParam(java.lang.String) so the query string is unambiguous; the assembled string is additionally escaped for the surrounding JavaScript string literal so request-supplied values cannot break out of the inline &lt;script&gt; block.
      Parameters:
      logParams - (undocumented)
      curLogLength - (undocumented)
      startByte - (undocumented)
      endByte - (undocumented)
      logLength - (undocumented)
      byteLength - (undocumented)
      Returns:
      (undocumented)

    • makeHref

      public static String makeHref(boolean proxy, String id, String origHref)
      Return the correct Href after checking if master is running in the reverse proxy mode or not.
      Parameters:
      proxy - (undocumented)
      id - (undocumented)
      origHref - (undocumented)
      Returns:
      (undocumented)

    • buildErrorResponse

      public static jakarta.ws.rs.core.Response buildErrorResponse(jakarta.ws.rs.core.Response.Status status, String msg)

    • durationDataPadding

      public static scala.Tuple2<Object,scala.collection.immutable.Map<String,Object>>[] durationDataPadding(scala.Tuple2<Object,Map<String,Long>>[] values)
      There may be different duration labels in each batch. So we need to mark those missing duration label as '0d' to avoid UI rending error.
      Parameters:
      values - (undocumented)
      Returns:
      (undocumented)

    • detailsUINode

      public static scala.collection.immutable.Seq<scala.xml.Node> detailsUINode(boolean isMultiline, String message)

    • errorSummary

      public static scala.Tuple2<String,Object> errorSummary(String errorMessage)
      This function works exactly the same as utils.errorSummary(javascript), it shall be
      Parameters:
      errorMessage - (undocumented)
      Returns:
      (undocumented) remained the same whichever changed

    • errorMessageCell

      public static scala.collection.immutable.Seq<scala.xml.Node> errorMessageCell(String errorMessage)

    • formatImportJavaScript

      public static String formatImportJavaScript(jakarta.servlet.http.HttpServletRequest request, String sourceFile, scala.collection.immutable.Seq<String> methods)

    • org$apache$spark$internal$Logging$$log_

      public static org.slf4j.Logger org$apache$spark$internal$Logging$$log_()

    • org$apache$spark$internal$Logging$$log__$eq

      public static void org$apache$spark$internal$Logging$$log__$eq(org.slf4j.Logger x$1)

    • LogStringContext

      public static org.apache.spark.internal.Logging.LogStringContext LogStringContext(scala.StringContext sc)