OpenStack Cloud Metering in Archimate – Part 10

Previous

This blog on Openstack modelled using Archimate follows on from the previous Blog on OpenStack Cloud Orchestration in Archimate – Part 9.

Ceilometer = Telemetry-Montoring-Collecting

Ceilometer project is for monitoring and collecting data regarding to resource usage in the cloud infrastructure.

Mission: Ceilometer is to provide an infrastructure to collect any information needed regarding OpenStack projects. It was designed so that rating engines could use this single source to transform events into billable items which we label as “metering”. As the project started to come to life, collecting an increasing number of meters across multiple projects, the OpenStack community started to realize that a secondary goal could be added to Ceilometer: become a standard way to meter, regardless of the purpose of the collection. This data can then be pushed to any set of targets using provided publishers mentioned in Publishers section.

If you divide a billing process into a 3 step process, as is commonly done in the telco industry, the steps are:

1. Metering
2. Rating
3. Billing

Ceilometer’s initial goal was, and still is, strictly limited to step one ‘Metering’. This is a choice made from the beginning not to go into rating or billing, as the variety of possibilities seemed too large for the project to ever deliver a solution that would fit everyone’s needs, from private to public clouds.  Ceilometer is a component of the Telemetry project. Its data can be used to provide customer billing, resource tracking, and alarming capabilities across all OpenStack core components.

Ceilometer = Telemetry-Montoring-Collecting Structural Logical Architecture

Previously, ceilometer used mysql as for backend storage. Currently, it is passed off to another Application Collaboration called gnocchi that does the backend job using mongodb. Ceilometer also has a Command Line Interface (CLI).

Each of Ceilometer’s services are designed to scale horizontally. Additional workers and nodes can be added depending on the expected load. Ceilometer offers two core services:

• polling agent – daemon designed to poll OpenStack services and build Meters. Polling agents, which is the less preferred method, will poll some API or other tool to collect information at a regular interval. The polling approach is less preferred due to the load it can impose on the API services.
• notification agent – daemon designed to listen to notifications on message queue, convert them to Events and Samples, and apply pipeline actions.  The Notification agent which takes messages generated on the notification bus and transforms them into Ceilometer samples or events. This is the preferred method of data collection. If you are working on some OpenStack related project and are using the Oslo library, you are kindly invited to come and talk to one of the project members to learn how you could quickly add instrumentation for your project. The heart of the system is the notification daemon (agent-notification) which monitors the message queue for data sent by other OpenStack components such as Nova, Glance, Cinder, Neutron, Swift, Keystone, and Heat, as well as Ceilometer internal communication.

Ceilometer = Telemetry-Montoring-Collecting Behavioural Services Architecture

These are the REST & CLI Services offered by Ceilometer for Monitoring used to be found here: https://docs.openstack.org/developer/ceilometer/webapi/v2.html. They are now deprecated as the storage elements have moved over to Aodh [Alarms], Gnocchi [Metrics] and Panko [Events].

Archimate Model Download

This file is shared under the GNU General Public License (GPL). You are free to download it and use it as you like. You can either make use of your own Archmate Tool and download it in .archmate format, or use your own tool that support the Open Groups open Exchange File format. These files are both compressed into a .zip file.

References

• https://Openstack.org
• UDemy.com – OpenStack essentials course
• https://www.slideshare.net/mirantis/openstack-architecture-43160012
• https://www.slideshare.net/prk1980/cloud-orchestration-major-tools-comparision
• https://www.slideshare.net/openstack/intro-grizzlyarchv1-19109550
• https://www.slideshare.net/Cloudenablers/openstack-heat-how-autoscaling-works-52244215
• https://www.slideshare.net/openstackindia/openstack-heat-51867693
• http://www.sparkmycloud.com/blog/telemetry-service-in-openstack/

OpenStack Cloud Orchestration in Archimate – Part 9

Previous

his blog on Openstack modelled using Archimate follows on from the previous Blog on OpenStack Cloud Identity in Archimate – Part 8.

Heat = Orchestration

Mission: Heat as the OpenStack Orchestration program is to create a human- and machine-accessible service for managing the entire lifecycle of infrastructure and applications within OpenStack clouds. Why Heat? It makes Clouds Rise!

Heat is the main project in the OpenStack Orchestration program. It implements an orchestration engine to launch multiple composite cloud applications based on templates in the form of text files that can be treated like code. A native Heat template format is evolving, but Heat also endeavours to provide compatibility with the AWS CloudFormation template format, so that many existing CloudFormation templates can be launched on OpenStack. Heat provides both an OpenStack-native ReST API and a CloudFormation-compatible Query API.

Heat = Orchestration Structural Logical Architecture

• A Heat template describes the infrastructure for a cloud application in a text file that is readable and writable by humans, and can be checked into version control, diffed, etc. Infrastructure resources that can be described include: servers, floating ips, volumes, security groups, users, etc.
• Heat also provides an autoscaling service that integrates with Telemetry, so you can include a scaling group as a resource in a template.
• Templates can also specify the relationships between resources (e.g. this volume is connected to this server). This enables Heat to call out to the OpenStack APIs to create all of your infrastructure in the correct order to completely launch your application.
• Heat manages the whole lifecycle of the application – when you need to change your infrastructure, simply modify the template and use it to update your existing stack. Heat knows how to make the necessary changes. It will delete all of the resources when you are finished with the application, too.
• Heat primarily manages infrastructure, but the templates integrate well with software configuration management tools such as Puppet and Chef. The Heat team is working on providing even better integration between infrastructure and software.

This Architecture appears to be in a state of constant refactoring and enhancement, so follow the Architecture here: https://wiki.openstack.org/wiki/Heat/Vision

Heat = Orchestration Behavioural Services Architecture

Orchestration services to create and manage the life-cycle of cloud applications are described like this:

• How do we model the cloud application deployment (YAML/JSON)? Heat provides Template (Openstack:HOT & AWS Cloud Formation:CFN)
• How do we configure the cloud application? Heat provides Software-Configuration
• How do we customize the cloud application? Heat provides Parameters & Environments
• How do we create and manage the cloud application? Heat provides Stack
• How do we model the cloud application element such as instances, volumes, etc ? Heat provides Resource (as resource plug-in)
• How do we manage the cloud application scalability? Heat provides Auto-scaling
• How do we track the progress of life-cycle operations ? Heat provides Events

Next

The next blog in the series is OpenStack Cloud Metering in Archimate – Part 10

OpenStack Cloud Identity in Archimate – Part 8

Previous

This blog on Openstack modelled using Archimate follows on from the previous Blog on Openstack Cloud Object Storage in Archimate – Part 7

Keystone = Identity

Keystone is an OpenStack service that provides API client authentication, service discovery, and distributed multi-tenant authorization by implementing OpenStack’s Identity API. It supports LDAP, OAuth, OpenID Connect, SAML and SQL. It offeres Role Based Access Control (RBAC).

Keystone = Identity Structural Logical Architecture

Keystone provides a single point of integration for OpenStack policy, catalog, token and authentication.
Keystone handles API requests as well as providing configurable keystone (service & admin APIs) catalog, policy, token and identity services.
Standard backends include LDAP or SQL, as well as Key Value OpenStack token backend, catalog backend, policy backend and identity backend stores (KVS).
Identity Service Most people will use this as a point of customization for their current authentication services.

Keystone = Identity Behavioural Services Architecture

These are the REST & CLI Services offered by Keystone for Identity. For each Service below there are one or many sub-services or commands that can be found here:  https://developer.openstack.org/api-ref/identity/v3/ 

1. User gets Token from Keystone.
2. Token includes the list of user Projects and Roles in them.
3. User calls the Service specifying the Token.
4. Service interprets the Roles:
   1. Service consults its policy.json file.
   2. Policy.json specifies the list of available rules.
   3. Policy.json specifies which rules are enforced for operations and resources.

Next

The next blog in the series is OpenStack Cloud Orchestration in Archimate – Part 9.

OpenStack Cloud Object Storage in Archimate – Part 7

Previous

This blog on Openstack modelled using Archimate follows on from the previous Blog on OpenStack Cloud Images in Archimate – Part 6

Swift = Object Storage

The OpenStack Object Store project, known as Swift, offers cloud storage software so that you can store and retrieve lots of data with a simple API. It’s built for scale and optimized for durability, availability, and concurrency across the entire data set. Swift is ideal for storing unstructured data that can grow without bound.

Swift is a highly available, distributed, eventually consistent object/blob store. Organizations can use Swift to store lots of data efficiently, safely, and cheaply. It’s built for scale and optimized for durability, availability, and concurrency across the entire data set. Swift is ideal for storing unstructured data that can grow without bound.

Swift = Object Storage Structural Logical Architecture

The Object Storage system organizes data in a hierarchy, as follows:

• Account. Represents the top-level of the hierarchy.  Your service provider creates your account and you own all resources in that account. The account defines a namespace for containers. A container might have the same name in two different accounts. In the OpenStack environment, account is synonymous with a project or tenant.
• Container. Defines a namespace for objects. An object with the same name in two different containers represents two different objects. You can create any number of containers within an account. In addition to containing objects, you can also use the container to control access to objects by using an access control list (ACL). You cannot store an ACL with individual objects. In addition, you configure and control many other features, such as object versioning, at the container level. You can bulk-delete up to 10,000 containers in a single request. You can set a storage policy on a container with predefined names and definitions from your cloud provider.
• Object. Stores data content, such as documents, images, and so on. You can also store custom metadata with an object.

Swift = Object Storage Behavioural Services Architecture

These are the REST & CLI Services offered by Swift for Object storage. For each Service below there are one or many sub-services or commands that can be found here: https://developer.openstack.org/api-ref/object-storage/index.html 

Next

The next blog in the series is OpenStack Cloud Identity in Archimate – Part 8

OpenStack Cloud Images in Archimate – Part 6

Previous

This blog on Openstack modelled using Archimate follows on from the previous Blog on OpenStack Cloud Block Storage in Archimate – Part 5.

Glance = Images

Mission: The Image service (glance) provides a service where users can upload and discover data assets that are meant to be used with other services. This currently includes images and metadata definitions.

What is a virtual machine image? A virtual machine image is a single file which contains a virtual disk that has a bootable operating system installed on it. Virtual machine images come in different formats: Raw, AKI, AMI, ARI, ISO, OVF, QCOW2, UEC Tarball, VDI, VHD, VHDX, VMDK.

Glance image services include discovering, registering, retrieving and format converting virtual machine (VM) images. Glance has a RESTful API that allows querying of VM image metadata as well as retrieval of the actual image.

Glance = Images Structural Logical Architecture

Glance image services include discovering, registering, and retrieving virtual machine images. Glance has a RESTful API that allows querying of VM image metadata as well as retrieval of the actual image. VM images made available through Glance can be stored in a variety of locations from simple filesystems to object-storage systems like the OpenStack Swift project.

Glance = Images Behavioural Services Architecture

These are the REST & CLI Services offered by Glance for Images. For each Service below there are one or many sub-services or commands that can be found here: https://developer.openstack.org/api-ref/image/v2/index.html

Next

The next blog in the series is OpenStack Cloud Object Storage in Archimate – Part 7