Python with Spark How-tos — Anaconda Enterprise 4.4 documentation

Anaconda Enterprise: Documentation

# Python with Spark How-tos

These how-tos will show you how to run Python tasks on a Spark cluster using the PySpark module. These how-tos will also show you how to interact with data stored within HDFS on the cluster.

While these how-tos are not dependent on each other and can be accomplished in any order it is recommended that you begin with the Overview of Spark, YARN and HDFS first.

- Overview of Spark, YARN and HDFS
- How to Run a Spark Standalone Job
- How to Run with the YARN resource manager
- How to perform a word count on text data in HDFS
- How to do Natural Language Processing
- How to do Image Processing with GPUs