Step 1: Install CloudPanel Via Ansible (Optional)

$ apt update && apt -y upgrade && apt -y install curl wget git ansible

$ git clone https://github.com/saimintech/Ansible_Playbooks.git /home/Ansible_Playbooks/

$ cd /home/Ansible_Playbooks/
$ ansible-playbook -i hosts.ini cloudpanel_install.yaml

Step 2: Install Apache Airflow

  1. Install the Python package manager, and virtual environment.

     $ sudo apt-get install -y python3-pip python3-venv

  2. Create a new project directory.

     $ mkdir airflow-project

  3. Change to the directory.

     $ cd airflow-project

  4. Create a new virtual environment.

     $ python3 -m venv airflow-env

  5. Activate the virtual environment.

     $ source airflow-env/bin/activate

    Your terminal prompt should change as below:

     (airflow-env) user@example:~/airflow-project$

  6. Using pip, install Airflow.

     $ pip install apache-airflow

  7. Initialize a new SQLite database to create the Airflow meta-store that Airflow needs to run.

     $ airflow db init

  8. Create the administrative user and password used to access Airflow.

     $ airflow users create --role Admin --username admin --email admin --firstname admin --lastname admin --password my-password

 

Step 3: Install Node.js 18 from package

  1. Obtain the Node.js source: 

    curl -sL https://deb.nodesource.com/setup_18.x | sudo bash -

  2. Install the 16.x version of Node.js:

    sudo apt-get install -y nodejs

  3. Verify that the correct version of Node.js is installed:

    node -v

  4. Ensure that the most current version of NPM is installed:

    sudo npm install -g npm@latest

  5. Check the NPM version:


    npm -v

Step 4: Install PM2 

  1. Install PM2:

    npm install pm2@latest -g

  2. Start Airflow Scheduler & Webserver

    pm2 start --name "airflow-scheduler" /bin/bash --no-daemon -- -c "source /root/airflow-project/airflow-env/bin/activate && airflow scheduler"

    pm2 start --name "airflow-webserver" /bin/bash --no-daemon -- -c "source /root/airflow-project/airflow-env/bin/activate && airflow webserver -p 8080"

  3. Save PM2 Process List and Set Up Startup

    pm2 save

    pm2 startup
  4. Check PM2 Logs for Errors
pm2 logs airflow-scheduler

pm2 logs airflow-webserver