cloud:intro
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
cloud:intro [2019/12/06 15:12] – chudler | cloud:intro [2020/05/14 09:25] – move the naq chudler | ||
---|---|---|---|
Line 1: | Line 1: | ||
=== SCOPE OF THIS DOCUMENT === | === SCOPE OF THIS DOCUMENT === | ||
- | This guide will cover a common subset of tasks that a user would need to perform to have a set of clustered computer instances and associated resources, isolated from others, and accessible to a project for any general purpose, both long-term and short. We are catering heavily to short-term usage, perhaps lasting a few quarters. | + | This guide covers the common subset of tasks that users would need to perform to have a set of clustered computer instances and associated resources, isolated from others, and accessible to a project for any general purpose, both long-term and short. |
- | Some things that are not here but perhaps should be covered elsewhere | ||
- | |||
- | * Theory of operations (everything is by example) | ||
- | * Tasks accomplished from the Web Interface | ||
- | * Background and History | ||
- | * Organizational Policy, such as who can do what. | ||
- | * Deployment Architecture | ||
- | * Limitations | ||
- | * User management, Group management, and similar concepts | ||
- | * Good Practices (because they are nascent, at best) | ||
- | * Cloud init, Fog, Terraform, Heat, and other operational tools | ||
- | * Security | ||
- | * Backup and Restore | ||
- | |||
- | The list is not comprehensive. | ||
=== A WORD ABOUT SECURITY === | === A WORD ABOUT SECURITY === | ||
- | The security of the virtual computers that you launch is __your responsibility__. With this software you are able to create | + | The security of the virtual computers that you launch is __your responsibility__. With this software you are able to create insecure configurations that will be hacked, with no hope of recovering. This can put all of us at risk. The highest risk that you will encounter is to expose your computers to the Internet. Beware of the dangers associated with doing so! |
=== INTRODUCTION AND NOTES === | === INTRODUCTION AND NOTES === | ||
- | This cluster can spring into being computer resources, easily, and without | + | This cluster can spring into being computer resources without |
* L2 and L3 Network | * L2 and L3 Network | ||
* Router (SNAT and DNAT devices, etc) | * Router (SNAT and DNAT devices, etc) | ||
* Compute, and all that it entails such as RAM, CPU, Disk, ... | * Compute, and all that it entails such as RAM, CPU, Disk, ... | ||
- | * Block Storage (volume mounts) | + | * Additional |
* Security groups (firewall service) | * Security groups (firewall service) | ||
- | Some want | ||
- | * Distributed/ | ||
- | The cloud can also manage resources on your behalf that are traditionally handled by a human operator. This is less common, and amounts to cognitive debt that you may eventually have to pay: | + | The cloud can also manage resources on your behalf that are traditionally handled by a human operator: |
* Load balancer | * Load balancer | ||
* NFS | * NFS | ||
- | * Rancher Kubernetes | + | |
+ | | ||
+ | * Container Runtime | ||
- | == Web Access and Certificates == | + | More notes can be found at [[ cloud:naq | Frequently Asked Questions ]] |
- | The cloud is named **Overcloud**. The web interface at [[https:// | + | == Web Access |
- | NOTE: Our cloud DNS service might not meet your needs. Please test it anyway if you know how (TODO: document) | + | The cloud is named **Overcloud**. The web interface uses a non-public certificate authority and can be reached at [[https:// |
=== PROJECTS === | === PROJECTS === | ||
Openstack requires Users and their cloud resources to belong to a Project. Users have pre-defined Roles within that Project, such as Member or Admin. The Role, Project, and User together constitute in-context access control. So, when a user is in a certain Project, that User can read, modify, destroy the cloud resources in that Project, or even create new resources. All actions depend on the precise Role of the User in the project. Non-members of a project are not able to do anything with cloud resources of the project to which they are a non-member, including View Access. Users can belong to any number of Projects, and with potentially different Roles. Project Admins can modify the memberships of their own Projects. However, there is __not__ a Role known as Owner. | Openstack requires Users and their cloud resources to belong to a Project. Users have pre-defined Roles within that Project, such as Member or Admin. The Role, Project, and User together constitute in-context access control. So, when a user is in a certain Project, that User can read, modify, destroy the cloud resources in that Project, or even create new resources. All actions depend on the precise Role of the User in the project. Non-members of a project are not able to do anything with cloud resources of the project to which they are a non-member, including View Access. Users can belong to any number of Projects, and with potentially different Roles. Project Admins can modify the memberships of their own Projects. However, there is __not__ a Role known as Owner. | ||
- | |||
== Your Default Project == | == Your Default Project == | ||
Line 81: | Line 65: | ||
Use your favorite package manager on your own computer. Pip is preferred because the upstream packages it for themselves and it is in pure python. The general CS infrastructure will become a managed client for you to use in the near future (e.g., linux.cs.uchicago.edu). However, our experience has been that the software installs cleanly and is free from dependency problems. | Use your favorite package manager on your own computer. Pip is preferred because the upstream packages it for themselves and it is in pure python. The general CS infrastructure will become a managed client for you to use in the near future (e.g., linux.cs.uchicago.edu). However, our experience has been that the software installs cleanly and is free from dependency problems. | ||
- | Try: < | + | Try: < |
== PRELIMINARY SETUP == | == PRELIMINARY SETUP == | ||
Line 145: | Line 129: | ||
You are free to use the Network called __cloud__, if you don't need your hosts to be L2 isolated from other people, and you would like to proceed directly to creating servers. | You are free to use the Network called __cloud__, if you don't need your hosts to be L2 isolated from other people, and you would like to proceed directly to creating servers. | ||
Using the __cloud__ network cuts down your complexity significantly, | Using the __cloud__ network cuts down your complexity significantly, | ||
- | < | + | |
+ | < | ||
+ | openstack network list | ||
+ | </ | ||
[EDITOR NOTE: This section should be isolated from the main body] | [EDITOR NOTE: This section should be isolated from the main body] | ||
Line 152: | Line 139: | ||
Should you want to create a network of your own that your hosts will be on, not all of these options are necessary | Should you want to create a network of your own that your hosts will be on, not all of these options are necessary | ||
- | < | + | < |
+ | openstack network create mynet \ | ||
+ | | ||
+ | | ||
+ | | ||
+ | </ | ||
Now create a subnet for your network. This is mandatory for launching instances in the network that you just created. | Now create a subnet for your network. This is mandatory for launching instances in the network that you just created. | ||
Line 162: | Line 154: | ||
You are now advised that there is no " | You are now advised that there is no " | ||
- | < | + | < |
+ | openstack subnet create | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | </ | ||
After creating your own network and subnet(s), a router is also needed. However, a router is **not** needed if your instances only talk to each other. The router will take the gateway of your subnet automatically, | After creating your own network and subnet(s), a router is also needed. However, a router is **not** needed if your instances only talk to each other. The router will take the gateway of your subnet automatically, | ||
- | < | ||
- | openstack router add subnet mysubnet</ | ||
- | With the router created and attached to your subnet, develop it further. First, you need to obtain a free IP address on the UC Campus. We call this network __campus37__. | ||
- | < | ||
- | The output of the command is shown below. Take not of the IP Address: | ||
< | < | ||
- | +---------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | + | openstack router create |
- | | Field | Value | | + | </ |
- | +---------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | + | < |
- | | created_at | + | openstack router add subnet myrouter mysubnet |
- | | description | + | </ |
- | | dns_domain | + | |
- | | dns_name | + | |
- | | fixed_ip_address | + | |
- | | floating_ip_address | 128.135.37.244 | + | |
- | | floating_network_id | f6a5f729-d5bf-4fa7-9cd9-e4ed23c7d48f | + | |
- | | id | 7110ea40-8c32-4f99-8454-9a091bcd4623 | + | |
- | | location | + | |
- | | name | 128.135.37.244 | + | |
- | | port_details | + | |
- | | port_id | + | |
- | | project_id | + | |
- | | qos_policy_id | + | |
- | | revision_number | + | |
- | | router_id | + | |
- | | status | + | |
- | | subnet_id | + | |
- | | tags | [] | | + | |
- | | updated_at | + | |
- | +---------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+</ | + | |
- | Only you will be able to use this address until you destroy | + | With the router created and attached |
- | Give this address to your Router, on a new interface. | + | After this command, the router will have one leg in your subnet and one leg in the public campus network (and internet). |
+ | |||
+ | Only you will be able to use this address until you destroy it. **DONT ever take more than you need and free this resource as soon as you project ends.** | ||
< | < | ||
- | openstack router set --fixed-ip subnet=$(openstack subnet show --format value --column id public37), | + | openstack router set myrouter \ |
+ | | ||
+ | | ||
</ | </ | ||
Line 218: | Line 199: | ||
Like other openstack activities, creating a server has __many__ complex options and scenarios. This is a simple and ordinary depiction, creating one server | Like other openstack activities, creating a server has __many__ complex options and scenarios. This is a simple and ordinary depiction, creating one server | ||
- | < | + | |
+ | < | ||
+ | openstack server create | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | </ | ||
The command executed asynchronously, | The command executed asynchronously, | ||
- | < | + | < |
- | openstack server show myserver</ | + | openstack server list --name myserver |
+ | </ | ||
+ | < | ||
+ | openstack server show myserver | ||
+ | </ | ||
Here's an example for creating 10 of them, as promised (only the change at the end of the command) | Here's an example for creating 10 of them, as promised (only the change at the end of the command) | ||
- | < | + | < |
+ | openstack server create | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | myserver | ||
+ | </ | ||
Here's a nasty thing I use to determine what the security group is for a server (it can be determined also by looking at security groups directly) [ITS BRITTLE, BEWARE] | Here's a nasty thing I use to determine what the security group is for a server (it can be determined also by looking at security groups directly) [ITS BRITTLE, BEWARE] | ||
- | < | + | < |
+ | SEC_GROUP=$(openstack port list \ | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | ||
+ | | sed ' | ||
+ | </ | ||
If I learned the security group successfully, | If I learned the security group successfully, | ||
- | < | + | < |
+ | openstack security group rule create | ||
+ | | ||
+ | | ||
+ | | ||
+ | </ | ||
In actual fact, all of the servers you create will be in the same security group. The above was attempting to suggest effective use of the tools, in combination. | In actual fact, all of the servers you create will be in the same security group. The above was attempting to suggest effective use of the tools, in combination. | ||
Line 242: | Line 261: | ||
You could also use the web interface to access the console, but that's not quite the same. | You could also use the web interface to access the console, but that's not quite the same. | ||
As before, in the Network Gear section, get a campus IP address from our pool. | As before, in the Network Gear section, get a campus IP address from our pool. | ||
- | < | ||
- | At last, you can ssh into 128.135.37.XX. It is important for you to realize that your __local__ server IP does not change (no new interface is given to the instance). Instead, the router on the subnet simply performs DNAT on behalf of the clients. Here's another possibility: | ||
- | < | ||
- | **Now** | + | Where do you want to create |
+ | < | ||
+ | openstack network list | ||
+ | </ | ||
+ | Use the network from the previous command: | ||
+ | < | ||
+ | openstack floating ip create < | ||
+ | </ | ||
+ | You now have an IP you can use: | ||
+ | < | ||
+ | openstack server add floating ip myserver < | ||
+ | </ | ||
- | == A WORD ABOUT CLOUD INIT == | + | Note that the command |
- | Your author uses cloud init extensively and does not imagine a life without it. It is optional. The file used in these examples is available on request, but you should develop your own if you use it at all. | + | |
+ | < | ||
+ | openstack server $action $subresource $more_options | ||
+ | </ | ||
- | == FAQ, NAQ | + | At last, you can ssh into 128.135.37.XX. It is important for you to realize that your __local__ server IP does not change |
+ | < | ||
- | Q: Why does it use a self-signed certificate? | + | **Now** your server |
- | A: This is a loose end that might be addressed in the future. Let us know if it overburdens you. Note: we are unlikely | + | |
- | Q: What are all of the services enabled? | + | This section added a floating ip address directly to the server. You must realize that a router was needed on the subnet for that to happen. We had created the router earlier for the purpose of SNAT, and had we not done that, this command would have failed. This means that if you are not doing SNAT, you should create |
- | A: | + | |
- | * cinder-backup | + | |
- | * heat | + | |
- | * barbican | + | |
- | * mistral | + | |
- | * ironic (we *do* support baremetal instances!) | + | |
- | * octavia | + | |
- | * sahara | + | |
- | * manila | + | |
- | * ganesha | + | |
- | * ceph (that is, ceph as a service. Your instances can be ceph clients in a software-defined manor if you wish, directly accessing RBD) | + | |
- | * MDS/ | + | |
- | * swift | + | |
- | * designate | + | |
- | * DVR/HA | + | |
- | * Full Blown OVN, as you desire | + | |
- | * nova | + | |
- | * placement (standalone) | + | |
- | * glance | + | |
- | * gnocchi (as-a-service. Hundreds | + | |
- | Q: What about containers (docker)? | + | == A WORD ABOUT CLOUD INIT == |
- | A: We do not provide any support for them directly. We expect that you will want to manage this from inside the VMs that you create. We have no plans to deploy Magnum right now. | + | Your author uses cloud init extensively |
- | + | ||
- | Q: My servers are in ERROR state!!! | + | |
- | A: If the servers had been running previously, this is bad and may not be recoverable. Talk to us ASAP about anything that you know. We'll troubleshoot. The upside is that the servers might be gone but the volumes and anything else associated with them (ports, etc) can be attached to entirely new servers, as is often done in clouds. | + | |
- | + | ||
- | Q: How fast is it? | + | |
- | A: We haven' | + | |
- | + | ||
- | Q: Is my data safe? | + | |
- | A: PROBABLY NOT AT THIS STAGE! | + | |
- | + | ||
- | Q: How does the SSH key injection work? | + | |
- | A: Cloud-init. You won't be able to do this unless | + | |
- | + | ||
- | Q: What operating systems are supported? | + | |
- | A: We are prepared to run any workload | + | |
- | Q: What about limitations? | ||
- | A: The following quotas are set on your account and projects, only for the safety of the cloud. We will lift these easily if you need it (values are unspecified here, as yet, sorry): | ||
- | * gigabytes | ||
- | * volumes | ||
- | * secgroups | ||
- | * secgroup-rules | ||
- | * server-groups | ||
- | * ram | ||
- | * instances | ||
- | * fixed-ips | ||
- | * server-group-members | ||
- | * cores | ||
- | * per-volume-gigabytes | ||
- | * backup-gigabytes | ||
- | * snapshots | ||
- | * volumes | ||
- | * backups | ||
- | * subnetpools | ||
- | * ports | ||
- | * subnets | ||
- | * networks | ||
- | * floating-ips | ||
- | * routers | ||
== URLS == | == URLS == |
/var/lib/dokuwiki/data/pages/cloud/intro.txt · Last modified: 2021/04/15 17:50 by chudler