Project
IoT monitoring with Spark Structured Streaming
A smart-farm monitoring system using Scala, Apache Spark Structured Streaming and Kafka.
- Context
- Public streaming project documented on Medium and LinkedIn/GitHub.
- Problem
- IoT data needs continuous ingestion, processing and monitoring patterns rather than only batch reporting.
- Solution
- Built a streaming pipeline using Kafka and Spark Structured Streaming, with Scala for processing logic.
- Impact
- Shows practical streaming experience and the ability to work beyond notebook-only data workflows.
Stack
ScalaApache SparkStructured StreamingKafkaIoT
Links
This project connects Rafael’s data engineering profile with distributed streaming systems and typed JVM-based processing.