Category: Openstack Field Guide

Series of How-To articles to create projects/instances and cloud resources in Openstack and how these resources interact with existing campus services

Sharing files with Openstack and swift

Say you have a file be it a pdf, word document or even a large iso. You can use Openstack to temporarily share this file with others.

This assumes you are authenticated with Openstack API already.(Openstack Client Access)

Make sure you have a tempurl key generated and attached to your account (this only has to be done once and not every time you want to share a file)

#generate a key with uuidgen

$ uuidgen
$ 6b4139c0-3eda-4e3f-8bcf-97b0f19014b9
$ swift post -m 'Temp-URL-KEY:6b4139c0-3eda-4e3f-8bcf-97b0f19014b9'

The rest needs to be done each time you want to share a file or renew a url:
#First check account to collect data.

$swift stat

#From the output you will need the “Account” and “Meta Temp-Url-Key”
Account: KEY_6e9ed9c765b24951a7ca132a9acc87e4
Containers: 2
Objects: 1
Bytes: 4497342464
Containers in policy "standard-replica": 2
Objects in policy "standard-replica": 1
Bytes in policy "standard-replica": 4497342464
Meta Temp-Url-Key: 6c5d1c06-3f72-4d35-a806-3a90fbcdc1e5
Accept-Ranges: bytes
X-Trans-Id: tx0fdfd888549d41828a34e-005d691977
X-Account-Project-Domain-Id: default
X-Openstack-Request-Id: tx0fdfd888549d41828a34e-005d691977
X-Timestamp: 1564430023.61286
Content-Type: application/json; charset=utf-8

#Create a container or folder to place the file in:

$swift post iso

#Upload your file or object to the container

$swift upload iso /home/user/rhel7.iso

#Next, create a tempurl (filling in the brackets with the information we retrieved from the "swift stat" command above).

$swift tempurl 3600 /v1/{Account}/iso/rhel7.iso {Meta Temp-Url-Key}

/v1/KEY_6e9ed9c765b24951a7ca132a9acc87e4/iso/rhel7.iso?temp_url_sig=da4d84647bbe41f0a7fc2e0b723d5a8cc190999d&temp_url_expires=1567172915

#The resulting output should be appended to the swift endpoint. In this instance it would look something like:

https://swift.storm.gatech.edu/v1/KEY_6e9ed9c765b24951a7ca132a9acc87e4/iso/rhel7.iso?temp_url_sig=da4d84647bbe41f0a7fc2e0b723d5a8cc190999d&temp_url_expires=1567172915

Note: The url you generate will only be available for the amount of time (in seconds) you provided in the $swift tempurl command above. After which you'll need to generate another url.

Flavors SOP

Beside the basic configuration options such as cpu, ram and disk, the following options need to be set to ensure project resource security.
First the flavor should be private and access granted to your project(s)
To do so on a command line the options are:

--private --property filter_tenant_id='insert project id here' --project 'insert project id here'

These should be included with the rest of the options you wish to set for your flavor.

In the dashboard, while creating the flavor add your project via the ‘Flavor Access” tab. This will imply that the flavor will be private.

A heat template resource would look like:

project_flavor_1:
type: OS::Nova::Flavor
properties:
ram: 2000
disk: 20
vcpus: 1
name:
list_join: ['-',[{get_param: project_name}, 'm1.tiny']]
extra_specs:
filter_tenant_id: 'insert tenant id here'
is_public: False
tenants:
- 'insert tenant id here'

Volume Types

Because this infrastructures supports multiple departments, there are several volume backends.  The default volume backends for persistent volume and ephemeral instances are not the fastest.  To use volume types available to you, can do:

$openstack volume type list
+--------------------------------------+-----------+-----------+
| ID | Name | Is Public |
+--------------------------------------+-----------+-----------+
| dea91be1-971b-44d6-91dc-f9a6e91c7634 | ecs-ceph | False |

When you create a volume,

From the commandline add the following option to specify the volume backend you want:

'--type ecs-ceph'

From the dashboard gui, when creating a volume there is a ‘ volume type ‘ drop down box to select the type you have access to.

A sample heat resource that specifies a volume type:

sandbox_volume:
type: OS::Cinder::Volume
properties:
image: gt-rhel7server
name: arenarium
size: 20
volume_type: ecs-ceph

Backups

Images:

 

Images are not backed up.  You should make a backup of these images yourself or practice infrastructure as code and be able to easily recreate them if need be.

Volumes:
To backup a volume you’ll need to flag it for archive. The following script will help:

#!/bin/bash

export OS_CLOUD=<cloud name>

/usr/bin/openstack server set --property archive=true $1

Run script:

$ archive_flag.sh <instance name or id>

Storage Pools

Since Openstack is based on a shared filesystem backend, we need to divide the storage in to specific chunks or “Pools” based on the tasks we need these in order to accomplish said task.

VMS (Virtual Machine’S’):

Used to create ephemeral disks for instances to use.  Anything in the VMS pools gets deleted once the instance connected to it is deleted.

Images:

Mostly used for operating systems used to create instances.

Volumes:

Persistent Data disks.  Disks in this pool will only be deleted if specifically told to do so by the operator.  Warning:  See Delete Volume on instance deleted: