Explanation
Slots are the communication interface for executors and are used for receiving commands and sending results to the driver.
Wrong, executors communicate with the driver directly.
Slots are dynamically created and destroyed in accordance with an executor's workload.
No, Spark does not actively create and destroy slots in accordance with the workload. Per executor, slots are made available in accordance with how many cores per executor (property
spark.executor.cores) and how many CPUs per task (property spark.task.cpus) the Spark configuration calls for.
A slot is always limited to a single core.
No, a slot can span multiple cores. If a task would require multiple cores, it would have to be executed through a slot that spans multiple cores.
In Spark documentation, "core" is often used interchangeably with "thread", although "thread" is the more accurate word. A single physical core may be able to make multiple threads available. So, it
is better to say that a slot can span multiple threads.
To optimize I/O performance, Spark stores data on disk in multiple slots.
No – Spark stores data on disk in multiple partitions, not slots.
More info: Spark Architecture | Distributed Systems Architecture
Submit