/ Home

PySpark

Java Setup

sudo apt install default-jdk
sudo apt install default-jre

echo "export JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64" >> ~/.zshrc

Change zsh to bash if required and ensure version of java on your system is Correct

PySpark Setup

pip install pyspark

Run Basic PySpark Script to Test