# Spark Batch Processing - Data Partitioning, Performance Optimization, and Iceberg Tables (Day 1 Lab)

> In this lab video, the presenter demonstrates how to execute Spark code and explore data partitioning. Key topics include running cells, monitoring the kernel's status, troubleshooting, the importance of using 'collect', the effects of terminating Spark sessions, and the differences between global…

- Web page: https://julie.techexpert.io/lesson/spark-batch-day-1-lab-v2
- Program: [Data Engineering Boot Camp V2 Combined Track](https://julie.techexpert.io/program/data-engineering-boot-camp-v2-combined-track)
- Module: Week 4: Batch Pipelines with Apache Spark V2
- Access: Requires enrollment in Data Engineering Boot Camp V2 Combined Track
- Length: 45 min video
- Skills: Python, Apache Spark, Apache Iceberg
- Academy: Julies academy to catch all other programs

## About this lesson

In this lab video, the presenter demonstrates how to execute Spark code and explore data partitioning. Key topics include running cells, monitoring the kernel's status, troubleshooting, the importance of using 'collect', the effects of terminating Spark sessions, and the differences between global sort and partition sort and their impact on performance. Additionally, the video covers writing data to iceberg tables and analyzing data set sizes.

## Navigation

- Previous lesson: [Applying Advanced SQL for Analytical Insights - Repeatable Analyses and State Change Tracking (Day 1 Lecture)](https://julie.techexpert.io/lesson/analytical-patterns-day-1-lecture-v2.md)
- Next lesson: [Applying KPIs and Experimentation - Understanding User Behavior and Driving Business Growth (Day 1 Lecture)](https://julie.techexpert.io/lesson/kpi-metrics-day-1-lecture-v2.md)