Exploiting Schema Inference in Apache Spark
Learn how Spark schema inference can be extracted to JSON and saved for later use. Advanced method of working with complex data in Spark.
Unit Testing Apache Spark Structured Streaming using MemoryStream
Writing Apache Spark unit tests, using MemoryStream. Unit testing Structured Streaming jobs in Apache Spark using built-in classes.
Delta Lake and data lakes – getting started
A gentle introduction to data lakes and the most prominent solution used – Delta Lake. Theory and practice on cutting edge Big Data technology.
3 best real-time processing frameworks
A concise comparison of Apache Spark, Apache Flink and Apache Storm data stream processing frameworks
