Como aprendi Apache Spark: revisitando uma jornada pela Engenharia de Dados

The author is launching a series to revisit and reconstruct old study materials on Apache Spark from the perspective of a Data Engineering and AI professional. The content explores how Spark emerged to solve real limitations of distributed processing, specifically contrasting it with the previous dominance of Hadoop and MapReduce. The series examines the evolution from disk-heavy batch processing to modern ecosystems involving Data Lakes, Lakehouses, Delta Lake, Apache Iceberg, and platforms like Databricks.