Skip to content
Snippets Groups Projects
Commit e7ff5739 authored by Jaromír Hradil's avatar Jaromír Hradil
Browse files

Merge branch 'eosc-openstack' into 'main' 

Sample notebook for EOSC

See merge request !1
parents 186e89ce aaf49a57
No related branches found
No related tags found
1 merge request!1Sample notebook for EOSC
Show whitespace changes
Inline Side-by-side
figures/network_topology.png 0 → 100644
+ 0
− 0
View file @ e7ff5739
figures/network_topology.png

59.9 KiB

figures/network_topology_vm.png 0 → 100644
+ 0
− 0
View file @ e7ff5739
figures/network_topology_vm.png

60.3 KiB

figures/scheme.png 0 → 100644
+ 0
− 0
View file @ e7ff5739
figures/scheme.png

43.5 KiB

openstack_eosc.ipynb 0 → 100644
+ 1207
− 0
View file @ e7ff5739
%% Cell type:code id:0ce0f97a-a87b-4655-8f85-cfabb07172f7 tags:
``` python
!pip install python-openstackclient python-keystoneclient ipywidgets paramiko
```
%% Output
Collecting python-openstackclient
Using cached python_openstackclient-8.0.0-py3-none-any.whl.metadata (6.4 kB)
Collecting python-keystoneclient
Using cached python_keystoneclient-5.6.0-py3-none-any.whl.metadata (3.7 kB)
Requirement already satisfied: ipywidgets in /opt/conda/lib/python3.12/site-packages (8.1.6)
Collecting paramiko
Using cached paramiko-3.5.1-py3-none-any.whl.metadata (4.6 kB)
Collecting pbr!=2.1.0,>=2.0.0 (from python-openstackclient)
Using cached pbr-6.1.1-py2.py3-none-any.whl.metadata (3.4 kB)
Requirement already satisfied: cryptography>=2.7 in /opt/conda/lib/python3.12/site-packages (from python-openstackclient) (44.0.2)
Collecting cliff>=3.5.0 (from python-openstackclient)
Using cached cliff-4.10.0-py3-none-any.whl.metadata (1.9 kB)
Collecting iso8601>=0.1.11 (from python-openstackclient)
Using cached iso8601-2.1.0-py3-none-any.whl.metadata (3.7 kB)
Collecting openstacksdk>=3.3.0 (from python-openstackclient)
Using cached openstacksdk-4.5.0-py3-none-any.whl.metadata (12 kB)
Collecting osc-lib>=2.3.0 (from python-openstackclient)
Using cached osc_lib-4.0.0-py3-none-any.whl.metadata (3.6 kB)
Collecting oslo.i18n>=3.15.3 (from python-openstackclient)
Using cached oslo.i18n-6.5.1-py3-none-any.whl.metadata (2.1 kB)
Collecting python-cinderclient>=3.3.0 (from python-openstackclient)
Using cached python_cinderclient-9.7.0-py3-none-any.whl.metadata (19 kB)
Requirement already satisfied: requests>=2.27.0 in /opt/conda/lib/python3.12/site-packages (from python-openstackclient) (2.32.3)
Collecting stevedore>=2.0.1 (from python-openstackclient)
Using cached stevedore-5.4.1-py3-none-any.whl.metadata (2.3 kB)
Collecting debtcollector>=1.2.0 (from python-keystoneclient)
Using cached debtcollector-3.0.0-py3-none-any.whl.metadata (2.3 kB)
Collecting keystoneauth1>=3.4.0 (from python-keystoneclient)
Using cached keystoneauth1-5.11.0-py3-none-any.whl.metadata (4.0 kB)
Collecting oslo.config>=5.2.0 (from python-keystoneclient)
Using cached oslo_config-9.8.0-py3-none-any.whl.metadata (2.8 kB)
Collecting oslo.serialization>=2.18.0 (from python-keystoneclient)
Using cached oslo.serialization-5.7.0-py3-none-any.whl.metadata (2.1 kB)
Collecting oslo.utils>=3.33.0 (from python-keystoneclient)
Using cached oslo_utils-9.0.0-py3-none-any.whl.metadata (2.2 kB)
Requirement already satisfied: packaging>=20.4 in /opt/conda/lib/python3.12/site-packages (from python-keystoneclient) (24.2)
Requirement already satisfied: comm>=0.1.3 in /opt/conda/lib/python3.12/site-packages (from ipywidgets) (0.2.2)
Requirement already satisfied: ipython>=6.1.0 in /opt/conda/lib/python3.12/site-packages (from ipywidgets) (9.1.0)
Requirement already satisfied: traitlets>=4.3.1 in /opt/conda/lib/python3.12/site-packages (from ipywidgets) (5.14.3)
Requirement already satisfied: widgetsnbextension~=4.0.14 in /opt/conda/lib/python3.12/site-packages (from ipywidgets) (4.0.14)
Requirement already satisfied: jupyterlab_widgets~=3.0.14 in /opt/conda/lib/python3.12/site-packages (from ipywidgets) (3.0.14)
Collecting bcrypt>=3.2 (from paramiko)
Using cached bcrypt-4.3.0-cp39-abi3-manylinux_2_34_x86_64.whl.metadata (10 kB)
Collecting pynacl>=1.5 (from paramiko)
Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
Collecting autopage>=0.4.0 (from cliff>=3.5.0->python-openstackclient)
Using cached autopage-0.5.2-py3-none-any.whl.metadata (7.9 kB)
Collecting cmd2>=1.0.0 (from cliff>=3.5.0->python-openstackclient)
Using cached cmd2-2.5.11-py3-none-any.whl.metadata (17 kB)
Collecting PrettyTable>=0.7.2 (from cliff>=3.5.0->python-openstackclient)
Using cached prettytable-3.16.0-py3-none-any.whl.metadata (33 kB)
Requirement already satisfied: PyYAML>=3.12 in /opt/conda/lib/python3.12/site-packages (from cliff>=3.5.0->python-openstackclient) (6.0.2)
Requirement already satisfied: cffi>=1.12 in /opt/conda/lib/python3.12/site-packages (from cryptography>=2.7->python-openstackclient) (1.17.1)
Requirement already satisfied: wrapt>=1.7.0 in /opt/conda/lib/python3.12/site-packages (from debtcollector>=1.2.0->python-keystoneclient) (1.17.2)
Requirement already satisfied: decorator in /opt/conda/lib/python3.12/site-packages (from ipython>=6.1.0->ipywidgets) (5.2.1)
Requirement already satisfied: ipython-pygments-lexers in /opt/conda/lib/python3.12/site-packages (from ipython>=6.1.0->ipywidgets) (1.1.1)
Requirement already satisfied: jedi>=0.16 in /opt/conda/lib/python3.12/site-packages (from ipython>=6.1.0->ipywidgets) (0.19.2)
Requirement already satisfied: matplotlib-inline in /opt/conda/lib/python3.12/site-packages (from ipython>=6.1.0->ipywidgets) (0.1.7)
Requirement already satisfied: pexpect>4.3 in /opt/conda/lib/python3.12/site-packages (from ipython>=6.1.0->ipywidgets) (4.9.0)
Requirement already satisfied: prompt_toolkit<3.1.0,>=3.0.41 in /opt/conda/lib/python3.12/site-packages (from ipython>=6.1.0->ipywidgets) (3.0.50)
Requirement already satisfied: pygments>=2.4.0 in /opt/conda/lib/python3.12/site-packages (from ipython>=6.1.0->ipywidgets) (2.19.1)
Requirement already satisfied: stack_data in /opt/conda/lib/python3.12/site-packages (from ipython>=6.1.0->ipywidgets) (0.6.3)
Collecting os-service-types>=1.2.0 (from keystoneauth1>=3.4.0->python-keystoneclient)
Using cached os_service_types-1.7.0-py2.py3-none-any.whl.metadata (1.7 kB)
Requirement already satisfied: typing-extensions>=4.12 in /opt/conda/lib/python3.12/site-packages (from keystoneauth1>=3.4.0->python-keystoneclient) (4.13.2)
Collecting dogpile.cache>=0.6.5 (from openstacksdk>=3.3.0->python-openstackclient)
Using cached dogpile_cache-1.4.0-py3-none-any.whl.metadata (5.5 kB)
Requirement already satisfied: jmespath>=0.9.0 in /opt/conda/lib/python3.12/site-packages (from openstacksdk>=3.3.0->python-openstackclient) (1.0.1)
Requirement already satisfied: jsonpatch!=1.20,>=1.16 in /opt/conda/lib/python3.12/site-packages (from openstacksdk>=3.3.0->python-openstackclient) (1.33)
Requirement already satisfied: platformdirs>=3 in /opt/conda/lib/python3.12/site-packages (from openstacksdk>=3.3.0->python-openstackclient) (4.3.7)
Requirement already satisfied: psutil>=3.2.2 in /opt/conda/lib/python3.12/site-packages (from openstacksdk>=3.3.0->python-openstackclient) (5.9.8)
Collecting requestsexceptions>=1.2.0 (from openstacksdk>=3.3.0->python-openstackclient)
Using cached requestsexceptions-1.4.0-py2.py3-none-any.whl.metadata (1.2 kB)
Collecting netaddr>=0.7.18 (from oslo.config>=5.2.0->python-keystoneclient)
Using cached netaddr-1.3.0-py3-none-any.whl.metadata (5.0 kB)
Collecting rfc3986>=1.2.0 (from oslo.config>=5.2.0->python-keystoneclient)
Using cached rfc3986-2.0.0-py2.py3-none-any.whl.metadata (6.6 kB)
Requirement already satisfied: msgpack>=0.5.2 in /opt/conda/lib/python3.12/site-packages (from oslo.serialization>=2.18.0->python-keystoneclient) (1.1.0)
Requirement already satisfied: tzdata>=2022.4 in /opt/conda/lib/python3.12/site-packages (from oslo.serialization>=2.18.0->python-keystoneclient) (2025.2)
Requirement already satisfied: pyparsing>=2.1.0 in /opt/conda/lib/python3.12/site-packages (from oslo.utils>=3.33.0->python-keystoneclient) (3.2.3)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.12/site-packages (from pbr!=2.1.0,>=2.0.0->python-openstackclient) (78.1.0)
Requirement already satisfied: charset_normalizer<4,>=2 in /opt/conda/lib/python3.12/site-packages (from requests>=2.27.0->python-openstackclient) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.12/site-packages (from requests>=2.27.0->python-openstackclient) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.12/site-packages (from requests>=2.27.0->python-openstackclient) (2.4.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.12/site-packages (from requests>=2.27.0->python-openstackclient) (2025.1.31)
Requirement already satisfied: pycparser in /opt/conda/lib/python3.12/site-packages (from cffi>=1.12->cryptography>=2.7->python-openstackclient) (2.22)
Collecting pyperclip>=1.8 (from cmd2>=1.0.0->cliff>=3.5.0->python-openstackclient)
Using cached pyperclip-1.9.0-py3-none-any.whl
Requirement already satisfied: wcwidth>=0.2.10 in /opt/conda/lib/python3.12/site-packages (from cmd2>=1.0.0->cliff>=3.5.0->python-openstackclient) (0.2.13)
Requirement already satisfied: parso<0.9.0,>=0.8.4 in /opt/conda/lib/python3.12/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets) (0.8.4)
Requirement already satisfied: jsonpointer>=1.9 in /opt/conda/lib/python3.12/site-packages (from jsonpatch!=1.20,>=1.16->openstacksdk>=3.3.0->python-openstackclient) (3.0.0)
Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.12/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets) (0.7.0)
Requirement already satisfied: executing>=1.2.0 in /opt/conda/lib/python3.12/site-packages (from stack_data->ipython>=6.1.0->ipywidgets) (2.1.0)
Requirement already satisfied: asttokens>=2.1.0 in /opt/conda/lib/python3.12/site-packages (from stack_data->ipython>=6.1.0->ipywidgets) (3.0.0)
Requirement already satisfied: pure_eval in /opt/conda/lib/python3.12/site-packages (from stack_data->ipython>=6.1.0->ipywidgets) (0.2.3)
Using cached python_openstackclient-8.0.0-py3-none-any.whl (1.1 MB)
Using cached python_keystoneclient-5.6.0-py3-none-any.whl (397 kB)
Using cached paramiko-3.5.1-py3-none-any.whl (227 kB)
Using cached bcrypt-4.3.0-cp39-abi3-manylinux_2_34_x86_64.whl (284 kB)
Using cached cliff-4.10.0-py3-none-any.whl (84 kB)
Using cached debtcollector-3.0.0-py3-none-any.whl (23 kB)
Using cached iso8601-2.1.0-py3-none-any.whl (7.5 kB)
Using cached keystoneauth1-5.11.0-py3-none-any.whl (344 kB)
Using cached openstacksdk-4.5.0-py3-none-any.whl (1.8 MB)
Using cached osc_lib-4.0.0-py3-none-any.whl (93 kB)
Using cached oslo_config-9.8.0-py3-none-any.whl (131 kB)
Using cached oslo.i18n-6.5.1-py3-none-any.whl (46 kB)
Using cached oslo.serialization-5.7.0-py3-none-any.whl (25 kB)
Using cached oslo_utils-9.0.0-py3-none-any.whl (134 kB)
Using cached pbr-6.1.1-py2.py3-none-any.whl (108 kB)
Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
Using cached python_cinderclient-9.7.0-py3-none-any.whl (256 kB)
Using cached stevedore-5.4.1-py3-none-any.whl (49 kB)
Using cached autopage-0.5.2-py3-none-any.whl (30 kB)
Using cached cmd2-2.5.11-py3-none-any.whl (152 kB)
Using cached dogpile_cache-1.4.0-py3-none-any.whl (62 kB)
Using cached netaddr-1.3.0-py3-none-any.whl (2.3 MB)
Using cached os_service_types-1.7.0-py2.py3-none-any.whl (24 kB)
Using cached prettytable-3.16.0-py3-none-any.whl (33 kB)
Using cached requestsexceptions-1.4.0-py2.py3-none-any.whl (3.8 kB)
Using cached rfc3986-2.0.0-py2.py3-none-any.whl (31 kB)
Installing collected packages: requestsexceptions, pyperclip, rfc3986, PrettyTable, pbr, netaddr, iso8601, debtcollector, cmd2, bcrypt, autopage, stevedore, pynacl, oslo.i18n, os-service-types, paramiko, oslo.utils, oslo.config, keystoneauth1, dogpile.cache, cliff, python-cinderclient, oslo.serialization, openstacksdk, python-keystoneclient, osc-lib, python-openstackclient
Successfully installed PrettyTable-3.16.0 autopage-0.5.2 bcrypt-4.3.0 cliff-4.10.0 cmd2-2.5.11 debtcollector-3.0.0 dogpile.cache-1.4.0 iso8601-2.1.0 keystoneauth1-5.11.0 netaddr-1.3.0 openstacksdk-4.5.0 os-service-types-1.7.0 osc-lib-4.0.0 oslo.config-9.8.0 oslo.i18n-6.5.1 oslo.serialization-5.7.0 oslo.utils-9.0.0 paramiko-3.5.1 pbr-6.1.1 pynacl-1.5.0 pyperclip-1.9.0 python-cinderclient-9.7.0 python-keystoneclient-5.6.0 python-openstackclient-8.0.0 requestsexceptions-1.4.0 rfc3986-2.0.0 stevedore-5.4.1
Note: you may need to restart the kernel to use updated packages.
%% Cell type:code id:c3718348-c7df-44d3-9811-66b7fa7ec7fb tags:
``` python
"""
Import the required libraries
"""
import time
import sys
import paramiko
from keystoneauth1.session import Session
from keystoneauth1.identity.v3.oidc import OidcAccessToken
from keystoneclient.v3.client import Client
from openstack.connection import Connection
from openstack.exceptions import SDKException
import ipywidgets as widgets
from IPython.display import display, clear_output, Markdown
```
%% Cell type:markdown id:f1d44b71-9709-467e-92d4-c1faa31a93e7 tags:
# Deploying Your First Virtual Machine on PSNC OpenStack
Welcome to this hands-on tutorial for launching and configuring your first Virtual Machine (VM) in the PSNC OpenStack cloud environment. This guide will walk you through each step of the deployment process—from authentication to full VM provisioning and access.
📚 **Further reading:** [PSNC OpenStack Virtual Machines](https://docs.psnc.pl/display/EOSCUserGuides/Virtual+Machines)
%% Cell type:markdown id:02aedafa-9276-4350-b60c-53fe216514c9 tags:
# 🔐 Connect to OpenStack
## Set up OpenStack session with OIDC token authentication
To authenticate with OpenStack using OIDC (OpenID Connect), you'll need to provide the following parameters:
- **`auth_url`**: The URL of the OpenStack authentication server.
- **`protocol`**: The protocol used for authentication, such as `openid`.
- **`identity_provider`**: The name of the trusted external Identity Provider (IdP) that authenticates your identity.
- **`access_token`**: The token you receive from the IdP after successfully logging in.
This token proves your identity to OpenStack.
> ⚙️ These parameters are used to initialize an authenticated session with OpenStack through the `openstack.connect()` function.
%% Cell type:code id:5bc6fac0-c5fa-44a7-9d29-38ea5164e4bc tags:
``` python
def load_access_token(token_file_path="/var/run/secrets/oidc/access_token"):
"""Reads access token from specified file"""
try:
return open(token_file_path, "r", encoding='utf-8').read()
except IOError:
print("Reading from access token file failed.", file=sys.stderr)
return None
cloud_creds = OidcAccessToken(auth_url="https://api.cloud.psnc.pl:5000/v3/",
identity_provider="aai.open-science-cloud.ec.europa.eu",
protocol="openid",
access_token=load_access_token()
)
cloud_session = Session(auth=cloud_creds)
```
%% Cell type:markdown id:c27a6b45-cd62-446c-bccf-9cba425e54ab tags:
## OpenStack Keystone Client
The Keystone client is used to interact with OpenStack's identity service and enables to:
- Access OpenStack components not exposed in the primary API.
- Retrieve detailed information about your projects
Make sure to authenticate on https://eu-1.iaas.open-science-cloud.ec.europa.eu/ page
%% Cell type:code id:b871dc55-172a-43da-81d7-b7fa796a7c2c tags:
``` python
# Get list of accessible projects
keystone_client = Client(session=cloud_session)
my_projects = keystone_client.auth.projects()
```
%% Cell type:code id:9a58d8c5-7dcd-4549-9b10-0456ccd66b30 tags:
``` python
# pylint: disable=invalid-name, global-statement, unused-argument, protected-access
scoped_cloud_session = None
scoped_openstack_connection = None
clear_output(wait=True) # Clear previous output first
display(Markdown("## Scope to one of your projects you are entitled to access"))
project_dropdown = widgets.Dropdown(
options=[(project.name, idx) for idx, project in enumerate(my_projects)],
description="Project:",
layout=widgets.Layout(width='50%')
)
connect_button = widgets.Button(description="Connect to project", button_style="success")
output = widgets.Output()
def on_connect_clicked(b):
"""Handle click event to connect to the selected OpenStack project."""
with output:
clear_output()
idx = project_dropdown.value
selected_project = my_projects[idx]
print(f"🔌 Connecting to project: {selected_project.name} (ID: {selected_project.id}, \
Domain: {selected_project.domain_id})")
scoped_cloud_creds = OidcAccessToken(
auth_url="https://api.cloud.psnc.pl:5000/v3/",
identity_provider="aai.open-science-cloud.ec.europa.eu",
protocol="openid",
project_id=selected_project.id,
project_domain_id=selected_project.domain_id,
access_token=load_access_token()
)
global scoped_cloud_session, scoped_openstack_connection
scoped_cloud_session = Session(auth=scoped_cloud_creds)
scoped_openstack_connection = Connection(session=scoped_cloud_session)
print("✅ Connection established successfully!")
connect_button._click_handlers.callbacks.clear()
connect_button.on_click(on_connect_clicked)
display(project_dropdown, connect_button, output)
```
%% Output
## Scope to one of your projects you are entitled to access
%% Cell type:code id:7c50e961-b797-4e1c-a91b-e44487324c0c tags:
``` python
print("Your current project id:", scoped_openstack_connection.current_project_id)
```
%% Output
Your current project id: db6b4eb2d6fe454191e8d39b088564cd
%% Cell type:markdown id:5a8c4205-e524-44ca-96fb-d7577bc74d90 tags:
# Starting up your own VM
## 🗝️ Create an SSH Key Pair for VM Access
- If you already have a key, skip this step and provide the key name in the `create_server()` method.
- You can:
- Upload your **own** public key using `public_key="ssh-rsa ..."`
- Or let **OpenStack generate one** for you (as we’ll do below).
- Remember to save the private key securely!
%% Cell type:code id:5e4b3a58-687b-471f-8c7f-a6db86bde820 tags:
``` python
KEY_NAME = "key"
new_keypair = scoped_openstack_connection.create_keypair(KEY_NAME)
#Generating key-pair using existing public key
#scoped_openstack_connection.create_keypair("mykey2", public_key="ssh-rsa ....")
#If you let OpenStack generate your key-pair you will need to save your
#private and public keys for later use
with open(f"{KEY_NAME}.pem", "w", encoding="utf-8") as priv_file:
priv_file.write(new_keypair.private_key)
print(f"Private key saved to {KEY_NAME}.pem")
with open(f"{KEY_NAME}.pub", "w", encoding="utf-8") as pub_file:
pub_file.write(new_keypair.public_key)
print(f"Public key saved to {KEY_NAME}.pub")
```
%% Output
Private key saved to key.pem
Public key saved to key.pub
%% Cell type:markdown id:809ce42b-b70f-4dfb-82a9-cb505477663c tags:
## Create a Security Group for Your VM
Before creating a virtual machine, it's important to define what kind of network traffic it should allow.
This is done using **Security Groups** in OpenStack.
In this example, we:
- Create a new security group
- Allow incoming **SSH** (port 22) traffic to enable remote access
- Allow incoming **ICMP** traffic so the VM can be pinged
- Allow incoming **HTTP** (port 80) traffic for hosting web services
_Note: OpenStack automatically allows all outgoing traffic (egress) from your VM._
%% Cell type:code id:aceddf23-baeb-4e86-8100-52beec7241a4 tags:
``` python
# Create a new security group
new_security_group = scoped_openstack_connection.create_security_group(
name="My VM Security Group",
description="Allows SSH, HTTP and ICMP traffic"
)
# Allow incoming SSH (port 22)
scoped_openstack_connection.create_security_group_rule(
new_security_group.id,
port_range_min=22,
port_range_max=22,
protocol="tcp",
direction="ingress",
description="Allow SSH"
)
# Allow incoming HTTP (port 80)
scoped_openstack_connection.network.create_security_group_rule(
security_group_id=new_security_group.id,
port_range_min=80,
port_range_max=80,
protocol="tcp",
direction="ingress",
description="Allow HTTP"
)
# Allow incoming ICMP (ping)
scoped_openstack_connection.create_security_group_rule(
new_security_group.id,
protocol="icmp",
direction="ingress",
description="Allow ICMP"
)
print(f"✅ Security group '{new_security_group.name}' created and configured.")
```
%% Output
✅ Security group 'My VM Security Group' created and configured.
%% Cell type:markdown id:94a9ee51-c6d7-4c35-b969-5097bb777f1c tags:
## 🔍 Select Resources for Your VM
Before launching a VM, you need to choose the following:
- **Image (OS)**: The base operating system your VM will use.
- **Flavor (CPU/RAM/Disk)**: Determines how much compute power (vCPUs), memory (RAM), and disk your VM will have.
📖 [Flavor details at PSNC](https://docs.cloud.psnc.pl/en/general/regions/bst-pl2-region/)
- **Network**: The virtual public network the VM will connect to, including internal/external IP options.
📖 [PSNC Network Design Overview](https://docs.psnc.pl/display/EOSCUserGuides/PSNC+Network+Design)
> 🛠️ Once selected, these options will be used to launch your VM using the OpenStack backend.
%% Cell type:code id:5e29662e-1576-48f8-9ae2-f9e2be0ffc60 tags:
``` python
def format_size_gb(size_bytes):
"""Convert a size in bytes to a human-readable string in GB."""
return f"{size_bytes / (1024 ** 3):.2f} GB"
def format_image_options(images):
"""Format a list of image objects for display in a dropdown."""
return [
f"{image.name} (ID: {image.id}, Size: {format_size_gb(image.size)})"
for image in images
]
def format_generic_options(resources, label_attr='name', id_attr='id'):
"""Format a list of OpenStack resources into label (ID) format."""
return [f"{getattr(res, label_attr)} (ID: {getattr(res, id_attr)})" for res in resources]
available_images = list(scoped_openstack_connection.compute.images())
flavors = list(scoped_openstack_connection.compute.flavors())
networks = list(scoped_openstack_connection.network.networks())
display(Markdown("### Available Images"))
image_dropdown = widgets.Dropdown(
options=format_image_options(available_images),
description='Image:',
layout=widgets.Layout(width='85%')
)
display(image_dropdown)
display(Markdown("### Available Flavors"))
flavor_dropdown = widgets.Dropdown(
options=format_generic_options(flavors),
description='Flavor:',
layout=widgets.Layout(width='85%')
)
display(flavor_dropdown)
display(Markdown("### Available Networks"))
network_dropdown = widgets.Dropdown(
options=format_generic_options(networks),
description='Network:',
layout=widgets.Layout(width='85%')
)
display(network_dropdown)
```
%% Output
### Available Images
### Available Flavors
### Available Networks
%% Cell type:code id:c10e65f1-1e6e-44f4-8e86-aa6e5452d10c tags:
``` python
def get_selected_name(dropdown_value):
"""Extract the name part from a dropdown value string formatted as 'Name (ID: id)'."""
return dropdown_value.split(" (ID: ")[0]
def get_selected_id(dropdown_value):
"""Extract the ID part from a dropdown value string formatted as 'Name (ID: id)'."""
return dropdown_value.split(" (ID: ")[1].split(")")[0]
selected_image = get_selected_name(image_dropdown.value)
selected_flavor = get_selected_name(flavor_dropdown.value)
selected_network = get_selected_name(network_dropdown.value)
selected_network_id = get_selected_id(network_dropdown.value)
```
%% Cell type:code id:616210a6-a2f4-4732-bdbe-b37684d9dd74 tags:
``` python
print(f"Selected configuration: {selected_network}, {selected_flavor} and {selected_image}")
```
%% Output
Selected configuration: PSNC-EXT-PUB1-EDU, M1-NVME-2vCPU-8R-50D and ubuntu-24.04-x86_64-server-cloudimg-20241105
%% Cell type:markdown id:92b4ca77-ecc9-44f0-8464-5548b604e518 tags:
## 🌐 Set Up a Private Network Environment
You will also need to set up a **private network**, a **subnet**, and a **router** to manage traffic between your VMs and external networks. This process ensures that your VMs have both internal and external connectivity as required.
> Follow the steps below to configure your network:
%% Cell type:markdown id:e91b0bbe-8561-4bbd-87f7-927e53c35f68 tags:
<img src="figures/scheme.png" alt="Network scheme" width="400" height="200"/>
%% Cell type:markdown id:c3b75537-fb1b-448d-ad03-d637abb68b11 tags:
### Create Private Network
%% Cell type:code id:f9a37e64-ba85-43b3-b234-9ac09e743951 tags:
``` python
try:
print("Creating private network...")
network = scoped_openstack_connection.network.create_network(
name="My private network",
admin_state_up=True,
port_security_enabled=True
)
print(f"Created network: {network.name} (ID: {network.id})")
except SDKException as e:
print(f"Network creation failed: {e}")
raise
```
%% Output
Creating private network...
Created network: My private network (ID: 8207ddc7-4002-40d7-b9b5-cfedf6247024)
%% Cell type:markdown id:cb79e628-f76c-442c-b7f0-72e4d74a98ac tags:
### Create Internal Network (subnet)
%% Cell type:code id:6a36a9da-14fc-41be-92a7-6e4b140c1d47 tags:
``` python
try:
print("Creating subnet...")
subnet = scoped_openstack_connection.network.create_subnet(
network_id=network.id,
name="My internal network",
ip_version=4,
cidr="192.168.1.0/24",
)
print(f"Created subnet: {subnet.name} (ID: {subnet.id})")
except SDKException as e:
print(f"Subnet creation failed: {e}")
raise
```
%% Output
Creating subnet...
Created subnet: My internal network (ID: 444e1046-e893-4381-b092-b7645bd96347)
%% Cell type:markdown id:b6c347fc-c1a2-433e-9b71-4e20f93b9ac9 tags:
### Create Router and connect with network
%% Cell type:code id:1720eb87-7fb2-420b-bac4-2b9d8f5c0d31 tags:
``` python
try:
print(f"Creating router with external network: {selected_network}")
router = scoped_openstack_connection.network.create_router(
name="My router",
external_gateway_info={"network_id": selected_network_id}
)
print(f"Created router: {router.name} (ID: {router.id})")
# Connect private subnet to router
scoped_openstack_connection.network.add_interface_to_router(
router.id,
subnet_id=subnet.id
)
print(f"Connected subnet {subnet.name} to router {router.name}")
except SDKException as e:
print(f"Router setup failed: {e}")
raise
```
%% Output
Creating router with external network: PSNC-EXT-PUB1-EDU
Created router: My router (ID: cbde5d42-2001-4a9c-a1aa-fa2ef97e31f6)
Connected subnet My internal network to router My router
%% Cell type:markdown id:3d1ab4c3-f811-459a-917b-4664f01253f7 tags:
![Network Topology](figures/network_topology.png)
> **Diagram**: OpenStack networking setup — private network connected with the external public network (e.g., `PSNC-EXT-PUB1-EDU`) by router.
%% Cell type:markdown id:7483a3d0-7894-46f6-b202-128ff92b7276 tags:
## 🚀 Launch VM
%% Cell type:markdown id:7f29a0cd-897c-4c93-81d7-edf8d24f7240 tags:
> Create a new volume for VM with a size of 10 GB using your OpenStack connection
%% Cell type:code id:1329314a-9aad-4bee-97b4-f4fe2022e42a tags:
``` python
new_volume = scoped_openstack_connection.create_volume(10)
```
%% Cell type:markdown id:9d8c4f19-da3b-4faa-99fb-1d79f194c637 tags:
### Launch VM and run a **cloud-init** script to:
- Update system packages
- Install **NGINX**
- Set up a simple homepage
%% Cell type:code id:4cd699a0-ddd6-4356-b828-95673e15e405 tags:
``` python
# VM parameters
CLOUD_INIT_SCRIPT = """#!/bin/bash
apt-get update
apt-get upgrade -y
apt-get install -y nginx
echo 'Hello from my VM' > /var/www/html/index.html
"""
VM_NAME = "new_vm"
server = scoped_openstack_connection.create_server(
name=VM_NAME,
image=selected_image,
flavor=selected_flavor,
network=network, # Use new private network
userdata=CLOUD_INIT_SCRIPT,
key_name="key",
volumes=[new_volume],
security_groups=[new_security_group.name]
)
# Waiting until all is ready
time.sleep(170)
```
%% Cell type:markdown id:915cde8d-f668-44a1-913d-24436a93e4d4 tags:
> It will take some time for `CLOUD_INIT_SCRIPT` to finish.
%% Cell type:code id:47ac6e4e-ee8d-413b-a7bf-a7f1562d01e7 tags:
``` python
# Refreshing VM object to get the current state
new_vm = scoped_openstack_connection.compute.find_server(server.id)
```
%% Cell type:markdown id:eeae3963-20cf-4283-a40c-c6ab54ebea7d tags:
### Configure floating IP associated with the VM for an external access
To enable external access to your VM, create a floating IP and associate it with your external network:
%% Cell type:code id:b61224ce-446d-42f7-b3d6-12e204ef26b5 tags:
``` python
new_fip = scoped_openstack_connection.create_floating_ip(network=selected_network)
```
%% Cell type:code id:d547d657-15d5-4c9f-86f8-2ffd6f2f3466 tags:
``` python
floating_ip_address = new_fip.floating_ip_address
print(f"Floating IP: {floating_ip_address}")
```
%% Output
Floating IP: 62.3.174.177
%% Cell type:markdown id:99447853-3a5e-445d-a690-8e034ef6156a tags:
> Next step is to add new FIP to new VM to get access through SSH.
%% Cell type:code id:8b06239e-fae7-464e-a887-5ce9c36b124c tags:
``` python
# add_ip_list requires actual address string
# or list of strings of address
scoped_openstack_connection.add_ip_list(new_vm, new_fip.floating_ip_address)
```
%% Output
openstack.compute.v2.server.Server(id=e9ff03f2-2c79-4fed-a0bd-0db03a0d77c6, name=new_vm, status=ACTIVE, tenant_id=db6b4eb2d6fe454191e8d39b088564cd, user_id=0814e963545daece81ed1e1bda7b90ca33b58889e97e843aeb29ac634d15ae84, metadata={}, hostId=00645010378491da449ee53dea83e56eb7bb1376bc695df081a5f6ec, image={'id': '7816da3d-cc63-4f01-bc2a-832bf2391eb8', 'links': [{'rel': 'bookmark', 'href': 'https://claudius.cloud.psnc.pl:8774/images/7816da3d-cc63-4f01-bc2a-832bf2391eb8'}]}, flavor={'vcpus': 2, 'ram': 8192, 'disk': 50, 'ephemeral': 0, 'swap': 0, 'original_name': 'M1-NVME-2vCPU-8R-50D', 'extra_specs': {'aggregate_instance_extra_specs:nvme': 'general-purpose', 'quota:disk_read_bytes_sec': '524288000', 'quota:disk_read_iops_sec': '50000', 'quota:disk_write_bytes_sec': '524288000', 'quota:disk_write_iops_sec': '10000'}}, created=2025-05-20T08:54:21Z, updated=2025-05-20T08:55:13Z, addresses={'My private network': [{'version': 4, 'addr': '192.168.1.176', 'OS-EXT-IPS:type': 'fixed', 'OS-EXT-IPS-MAC:mac_addr': 'fa:16:3e:f4:dd:5d'}]}, accessIPv4=, accessIPv6=, links=[{'rel': 'self', 'href': 'https://claudius.cloud.psnc.pl:8774/v2.1/servers/e9ff03f2-2c79-4fed-a0bd-0db03a0d77c6'}, {'rel': 'bookmark', 'href': 'https://claudius.cloud.psnc.pl:8774/servers/e9ff03f2-2c79-4fed-a0bd-0db03a0d77c6'}], OS-DCF:diskConfig=MANUAL, progress=0, OS-EXT-AZ:availability_zone=BST0K10, pinned_availability_zone=BST0K10, config_drive=, key_name=key, OS-SRV-USG:launched_at=2025-05-20T08:55:13.000000, OS-SRV-USG:terminated_at=None, security_groups=[{'name': 'My VM Security Group'}], OS-EXT-STS:task_state=None, OS-EXT-STS:vm_state=active, OS-EXT-STS:power_state=1, os-extended-volumes:volumes_attached=[{'id': 'e5144f9b-0047-419c-861a-42a30d06dfb5', 'delete_on_termination': False}], locked=False, locked_reason=None, description=None, tags=[], trusted_image_certificates=None, OS-EXT-SRV-ATTR:hostname=new-vm, server_groups=[], location=Munch({'cloud': 'api.cloud.psnc.pl', 'region_name': None, 'zone': 'BST0K10', 'project': Munch({'id': 'db6b4eb2d6fe454191e8d39b088564cd', 'name': None, 'domain_id': None, 'domain_name': None})}))
%% Cell type:markdown id:9ac6fe06-da82-493b-a0ba-e6a4fd898a18 tags:
![Network Topology](figures/network_topology_vm.png)
> **Diagram**: After successful launching of VM, our network topology is extended with new created instance
%% Cell type:markdown id:ffd09a73-4a84-440e-8e8c-117b6b4163fc tags:
### 🔌 Connect to Your VM via SSH
%% Cell type:code id:43fbe01e-163a-44b1-ba1d-e7589562a918 tags:
``` python
print(f"Connecting to VM at {floating_ip_address}...")
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
ssh.connect(
hostname=floating_ip_address,
username="ubuntu",
key_filename="key.pem" # "/path/to/key.pem"
)
ssh.close()
except Exception as e:
print(f"SSH failed: {e}")
raise
print(f"Try accessing: http://{floating_ip_address}")
print(f"Access VM via ssh: ssh -i mykey.pem ubuntu@{floating_ip_address}")
```
%% Output
Connecting to VM at 62.3.174.177...
Try accessing: http://62.3.174.177
Access VM via ssh: ssh -i mykey.pem ubuntu@62.3.174.177
%% Cell type:markdown id:dd588470-3fc1-4b41-9bba-c929743fa9c9 tags:
# Finish session
%% Cell type:markdown id:89a921c0-8591-447a-b08a-ade26b7256ab tags:
## Detach floating IP from current server
%% Cell type:code id:3cf9a669-0dba-42aa-be26-e9f804c35e48 tags:
``` python
scoped_openstack_connection.detach_ip_from_server(new_vm, new_fip.id)
```
%% Output
True
%% Cell type:markdown id:29e3db6c-5b15-4b32-ad7b-8356436733e7 tags:
## Backup your attached volume
%% Cell type:code id:95feded0-22be-464c-876e-39f1015a773e tags:
``` python
attached_volume = new_vm.attached_volumes[0]
# OpenStack recommends to suspend VM and then backup volume.
# You can use `force` parameter to bypass it.
scoped_openstack_connection.create_volume_backup(attached_volume.id, name="My Backup", force=True)
```
%% Output
openstack.block_storage.v3.backup.Backup(name=My Backup, volume_id=6395a1f4-644e-4a6a-9d98-7892dba997b0, description=None, force=True, is_incremental=False, snapshot_id=None, id=65fa853d-9dfd-4b67-85e8-c4e333a3107c, links=[{'rel': 'self', 'href': 'https://claudius.cloud.psnc.pl:8776/v3/db6b4eb2d6fe454191e8d39b088564cd/backups/65fa853d-9dfd-4b67-85e8-c4e333a3107c'}, {'rel': 'bookmark', 'href': 'https://claudius.cloud.psnc.pl:8776/db6b4eb2d6fe454191e8d39b088564cd/backups/65fa853d-9dfd-4b67-85e8-c4e333a3107c'}], status=available, size=10, object_count=205, availability_zone=BST, container=cinder-backup-s3-claudius, created_at=2025-04-14T08:45:35.000000, updated_at=2025-04-14T08:47:24.000000, fail_reason=None, has_dependent_backups=False, data_timestamp=2025-04-14T08:45:35.000000, metadata={}, location=Munch({'cloud': 'api.cloud.psnc.pl', 'region_name': None, 'zone': 'BST', 'project': Munch({'id': 'db6b4eb2d6fe454191e8d39b088564cd', 'name': None, 'domain_id': None, 'domain_name': None})}))
%% Cell type:markdown id:878db2a7-dbd3-41c8-b629-6ca2954381c6 tags:
## Release allocated resources
%% Cell type:code id:301cbef1-ec92-439f-8f2d-fca92e2c33a9 tags:
``` python
scoped_openstack_connection.delete_server(new_vm.id)
scoped_openstack_connection.delete_floating_ip(new_fip)
```
%% Output
True
%% Cell type:code id:dc283305-7e0e-4967-84e9-d1573b45136e tags:
``` python
# If you do not wish to use generated key-pair anymore
# you can simply delete it
scoped_openstack_connection.delete_keypair("key")
```
%% Output
True
%% Cell type:code id:c642ccf6-83d8-4e8e-9d3d-0774b16706fc tags:
``` python
# Detach router interface first
scoped_openstack_connection.network.remove_interface_from_router(
router,
subnet_id=subnet.id
)
scoped_openstack_connection.network.delete_router(router)
scoped_openstack_connection.network.delete_security_group(new_security_group)
scoped_openstack_connection.network.delete_network(network)
```
%% Cell type:code id:189c00ae-378d-4647-8226-bb5e1e5e0c03 tags:
``` python
```
%% Cell type:code id:31a05bda-47a1-4d00-abe5-3514cff6a7cb tags:
``` python
```
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment