Getting Started

This page summarizes the basic steps required to setup and get started with PySpark. There are more guides shared with other languages such as Quick Start in Programming Guides at the Spark documentation.

There are live notebooks where you can try PySpark out without any other step:

• Live Notebook: DataFrame

• Live Notebook: pandas API on Spark

The list below is the contents of this quickstart page:

• Installation
  • Python Version Supported
  • Using PyPI
  • Using Conda
  • Manually Downloading
  • Installing from Source
  • Dependencies
• Quickstart: DataFrame
  • DataFrame Creation
  • Viewing Data
  • Selecting and Accessing Data
  • Applying a Function
  • Grouping Data
  • Getting Data in/out
  • Working with SQL
• Quickstart: Pandas API on Spark
  • Object Creation
  • Missing Data
  • Operations
  • Grouping
  • Plotting
  • Getting data in/out

PySpark Documentation Installation