Skip to content
Snippets Groups Projects
Select Git revision
  • 073d68358c4a3fb089631fa83b3d135824108cbd
  • master default protected
  • ci-bullseye
  • wip/bigtop-3.0.0
  • bio3
  • feature/certificates2
6 results

terraform

  • Clone with SSH
  • Clone with HTTPS
  • Info

    Build Hadoop cluster in OpenStack with Terraform.

    Primary goal of this project is to build Hadoop cluster. But the most part is generic - Hadoop deployment can be replaced by implementing different deployment type (providing user-data and optionally implementing plugin for orchestration).

    Requirements

    Locally installed:

    Hadoop image

    To setup Hadoop on single machine, launch:

    /usr/local/sbin/hadoop-setup.sh

    Hadoop image can be used also to build Hadoop cluster. It contains pre-downloaded and pre-installed Hadoop packages and dependencies, so this will speed things up.

    Cluster

    Build cluster:

    1. check variables.tf

    2. launch setup script

      ./launch.sh

    The launch.sh script is doing this:

    ./terraform apply
    ./terraform output -json > config.json
    ./orchestrate.py

    The orchestration script has multiple steps and dry-run option. See ./orchestrate.py --help.