Skip to content
Snippets Groups Projects
Select Git revision
  • cc4e73af44f586e540b02078d14bf6cadf1f1cee
  • master default protected
  • ci-bullseye
  • wip/bigtop-3.0.0
  • bio3
  • feature/certificates2
6 results

README.md

Blame
    • README.md 992 B

    Info

    Build Hadoop cluster in OpenStack with Terraform.

    Primary goal of this project is to build Hadoop cluster. But the most part is generic - Hadoop deployment can be replaced by implementing different deployment type (providing user-data and optionally implementing plugin for orchestration).

    Requirements

    Locally installed:

    Hadoop image

    To setup Hadoop on single machine, launch:

    /usr/local/sbin/hadoop-setup.sh

    Hadoop image can be used also to build Hadoop cluster. It contains pre-downloaded and pre-installed Hadoop packages and dependencies, so this will speed things up.

    Cluster

    Build cluster:

    1. check variables.tf

    2. launch setup script

      ./launch.sh

    The launch.sh script is doing this:

    ./terraform apply
./terraform output -json > config.json
./orchestrate.py

    The orchestration script has multiple steps and dry-run option. See ./orchestrate.py --help.