What causes Spark Session Terminated Unexpectedly?

An executor ran out of memory during a large shuffle, join, or collect operation. The Fabric capacity was under pressure and preempted the Spark session to free resources. The Spark session exceeded its configured idle timeout while waiting between long-running cells. A native library or JNI call caused a JVM crash in one of the executors. Network issues between Spark driver and executors caused heartbeat failures, triggering session termination

How do I fix Spark Session Terminated Unexpectedly?

Check the Spark UI (accessible from the run history) for executor logs and OOM (OutOfMemoryError) messages.. Reduce the data volume processed in a single operation — repartition large datasets or filter before joining.. Increase executor memory in the Spark session configuration: set spark.executor.memory to a higher value.. Use spark.sql.shuffle.partitions to increase the number of shuffle partitions for large aggregations.. Add .cache() strategically to avoid recomputing large transformations, but remove unused caches to free memory.. If the session timed out during an idle period between cells, reduce idle gaps or increase the session timeout setting.

High severityfabric

Power BI Refresh Error:
Spark Session Terminated Unexpectedly

What does this error mean?

The Spark session in Microsoft Fabric was terminated before the job completed — executor crashes, out-of-memory conditions, capacity preemption, or a session idle timeout.

Common causes

1An executor ran out of memory during a large shuffle, join, or collect operation
2The Fabric capacity was under pressure and preempted the Spark session to free resources
3The Spark session exceeded its configured idle timeout while waiting between long-running cells
4A native library or JNI call caused a JVM crash in one of the executors
5Network issues between Spark driver and executors caused heartbeat failures, triggering session termination

How to fix it

1Check the Spark UI (accessible from the run history) for executor logs and OOM (OutOfMemoryError) messages.
2Reduce the data volume processed in a single operation — repartition large datasets or filter before joining.
3Increase executor memory in the Spark session configuration: set spark.executor.memory to a higher value.
4Use spark.sql.shuffle.partitions to increase the number of shuffle partitions for large aggregations.
5Add .cache() strategically to avoid recomputing large transformations, but remove unused caches to free memory.
6If the session timed out during an idle period between cells, reduce idle gaps or increase the session timeout setting.

Frequently asked questions

How do I find out if the session was terminated due to OOM or capacity preemption?

Check Spark driver logs for 'OutOfMemoryError' or 'GC overhead limit exceeded' for OOM. If the session was killed externally, it was likely capacity preemption — check Fabric capacity metrics in the admin portal.

Official documentation: microsoft-fabric