Lightning-fast cluster computing

Apache Spark Community

Have Questions?

StackOverflow

For usage questions and help (e.g. how to use this Spark API), it is recommended you use the StackOverflow tag apache-spark as it is an active forum for Spark users’ questions and answers.

Some quick tips when using StackOverflow:

  • Prior to asking submitting questions, please:
  • Please follow the StackOverflow code of conduct
  • Always use the apache-spark tag when asking questions
  • Please also use a secondary tag to specify components so subject matter experts can more easily find them. Examples include: pyspark, spark-dataframe, spark-streaming, spark-r, spark-mllib, spark-ml, spark-graphx, spark-graphframes, spark-tensorframes, etc.
  • Please do not cross-post between StackOverflow and the mailing lists
  • No jobs, sales, or solicitation is permitted on StackOverflow

Mailing Lists

For broad, opinion based, ask for external resources, debug issues, bugs, contributing to the project, and scenarios, it is recommended you use the user@spark.apache.org mailing list.

Some quick tips when using email:

  • Prior to asking submitting questions, please:
  • Tagging the subject line of your email will help you get a faster response, e.g. [Spark SQL]: Does Spark SQL support LEFT SEMI JOIN?
  • Tags may help identify a topic by:
    • Component: Spark Core, Spark SQL, ML, MLlib, GraphFrames, GraphX, TensorFrames, etc
    • Level: Beginner, Intermediate, Advanced
    • Scenario: Debug, How-to
  • For error logs or long code examples, please use GitHub gist and include only a few lines of the pertinent code / log within the email.
  • No jobs, sales, or solicitation is permitted on the Apache Spark mailing lists.

Chat Rooms

Chat rooms are great for quick questions or discussions on specialized topics. The following chat rooms are not officially part of Apache Spark; they are provided for reference only.

  • Spark with Scala is for questions and discussions related to using Spark with the Scala programming language.

Events and Meetups

Conferences

Meetups

Spark Meetups are grass-roots events organized and hosted by leaders and champions in the community around the world. Check out http://spark.meetup.com to find a Spark meetup in your part of the world. Below is a partial list of Spark meetups.

If you'd like your meetup added, email user@spark.apache.org.

Issue Tracker

The project tracks bugs and new features on JIRA. If you'd like, you can also subscribe to issues@spark.apache.org to receive emails about new issues, and commits@spark.apache.org to get emails about commits.

Powered By

Our wiki has a list of projects and organizations powered by Spark.

Project History

Spark started as a research project at the UC Berkeley AMPLab in 2009, and was open sourced in early 2010. Many of the ideas behind the system are presented in various research papers.

After being released, Spark grew a developer community on GitHub and moved to Apache in 2013. A wide range of contributors now develop the project (over 400 developers from 100 companies).