• Online, Self-Paced
Course Description

In this course, you will learn about various ways to monitor Spark applications such as web UIs, metrics, and other monitoring tools. You will also learn about memory tuning.

Learning Objectives

Monitoring Spark Applications

  • start the course
  • access the web user interface
  • use the Spark environment configuration parameters
  • use JSON to query monitoring tools for Spark
  • set JVM fractional memory amounts for Spark

Tuning Spark Applications

  • modify speculation controls for Spark tasks
  • describe data serialization and the role it plays in the performance of Spark applications
  • describe memory management and consumption
  • determine executor memory allocation
  • describe garbage collection tuning
  • set the level of parallelism
  • use the broadcast functionality
  • use query execution plan explainer
  • implement data compression on parquet storage

Practice: Monitoring Spark Applications

  • monitor Spark applications

Framework Connections

The materials within this course focus on the Knowledge Skills and Abilities (KSAs) identified within the Specialty Areas listed below. Click to view Specialty Area details within the interactive National Cybersecurity Workforce Framework.