AWS Tagger

Image credits : jdhancock

Tagging in AWS is often not considered useful by many users. Tagging of resources in cloud and DC not only helps us identify resources but it can also do multiple other wonders that one might have never thought about. We don't tag resources in cloud for many reasons, laziness being the topmost reason.

Lets see why tagging is important:

1. Identification and Isolation: Tagging allows identification of resources as to what purpose a specific resource may have been created for. It also allows you to separate resources from each other. e.g. separating different environments.
2.  Automation: When you tag resources with certain values you can ensure that your automation scripts only addresses certain intended resources and not all. e.g execute security patches on certain systems that need to be compliant.
3. Costing: You can identify based on tags as to which resource is costly and also make business decisions based on the results received.
4. Define ownership: You can also understand based on proper tags as to who are the stakeholders for a certain resource or group of resources.
5. Versioning: Sometimes when you need certain resources to be preserved based on its state, you may also versionize them based on tagging. Although AWS provides versioning mechanism for a few services, it may not be applicable to all of them.

 In many organizations although the importance of tagging is understood a lot later. Until then its too late to start tagging and it becomes almost always a manual process to tag all the resources. Or you may need to write complex programs to identify systems and tag them as per your requirement. Thankfully, AWS Tagger comes to rescue if you have a requirement to tag your AWS resources. You may also bulk tag them to avoid a lot of manual work. So how do we do this.

Its a 3 step process to Bulk tag resources:

1. Collection : This is a simple process. Here all you need to do is, collect all the resources in a file. Hereafter you may process this data. AWS Tagger heavily depends of resource ID's of all the resources you create. Resource ID's are further used to implement all the tags. To get the resource ID's for all the resources, simply login to your AWS account and navigate to https://resources.console.aws.amazon.com/r/tags . On this page, you are given a field to enter the region for the resource you want to choose and all choose the types of resource. Choose "All resource types" here and click on "Find Resources" button. Click on the "Download" button to download the CSV data generated.
2. Identification and filtering: I recommend this step particularly to filter the data so that AWS Tagger can act on individual resources. Here you may use your excel skills to separate data based on resource types.
3. Tagging: Once the resources are separated, you may start executing AWS Tagger scripts as per the documentation provided on their Github page.

Automation for VMware vCloud Director using Chef's knife-vcloud - Part-II

Version 1.2.0

 Some right reserved by Phil Wiffen

For some reason with the previous repo I could not see the list of all vApps. Only some of it (a mixture of both chef node and non chef nodes) were seen. So I went ahead with another version of knife-vcloud plugin available which solved my problem to a large extent.

Plugin is available at https://github.com/astratto/knife-vcloud

Configuration used:

• CentOS 6.5
• Chef 11.8.2
• knife-vcloud 1.2.0

Following steps were used to complete the automation process:

Installation is fairly simple

gem install knife-vcloud
gem list | grep vcloud
- See if after entering the above command you see the gem knife-cloud. If yes the setup was successful. If no something went wrong.

cd ~./chef
vim knife.rb

Configuration is almost automated:

knife vc configure

You will be prompted for vcloud_url, login and password. After entering the details check that the details you entered are reflected in the knife.rb file.

knife[:vcloud_url] = 'https://vcloud.server.org'
knife[:vcloud_org_login] = 'vcloud_organization'
knife[:vcloud_user_login] = 'vcloud_user'
knife[:vcloud_password] =

Note: The organization was not updated for me, and it kept giving authorization failure for quite sometime. If you see that the organization is not updated automatically, please update it manually in the knife.rb file.

The subsequent commands would also change for the detailed listing. Although the documentation at many instances says that the name of VM or vApp should suffice to pull up the required details, note that at many instances you will be required to enter the ID and not just the name.

To see the list of catalog items

[root@chefworkstation ~]# knife vc catalog show All_ISOs
Description: All ISO Dumps
Name                                           ID                                          
CentOS-6.3                                          WhAtEvEr-Id-tO-bE-SeEn1       
CentOS-6.4_x64                                   WhAtEvEr-Id-tO-bE-SeEn2        
Ubuntu-copy                                           WhAtEvEr-Id-tO-bE-SeEn3        

To see details of the organization

[root@chefworkstation ~]# knife vc org show MYORG
CATALOGS                                                                 
Name                                  ID                                 
All_ISOs                                  WhAtEvEr-Id-tO-bE-SeEn4
Master Catalog                        WhAtEvEr-Id-tO-bE-SeEn5
                                                                         
VDCs                                                                     
Name                                  ID                                 
MyorgVDC-Tier1     WhAtEvEr-Id-tO-bE-SeEn6
MyorgVDC-Tier2        WhAtEvEr-Id-tO-bE-SeEn7
MyorgVDC-Tier3        WhAtEvEr-Id-tO-bE-SeEn8

NETWORKS                                                                 
Name                                  ID                                 
MyorgNet-Router                   WhAtEvEr-Id-tO-bE-SeEn9

TASKLISTS                                                                
Name                                  ID                                 
                        WhAtEvEr-Id-tO-bE-SeEn10

To create a new vApp:

[root@chefworkstation ~]# knife vc vapp create MyorgVDC-Tier1 chefnode2 "Just Created node2" WhAtEvEr-Id-tO-bE-SeEn
vApp creation...
Summary: Status: error - time elapsed: 52.012 seconds
WARNING: ATTENTION: Error code 400 - The following IP/MAC addresses have already been used by running virtual machines: MAC addresses: 10:20:30:40:50:0f IP addresses: 192.168.0.20 Use the Fence vApp option to use same MAC/IP. Fencing allows identical virtual machines in different vApps to be powered on without conflict, by isolating the MAC and IP addresses of the virtual machines.
vApp created with ID: WhAtEvEr-Id-tO-bE-SeEn

Note: that there are certain problems that were corrected later.

To show the deatils of created vApp:

[root@chefworkstation ~]# knife vc vapp show WhAtEvEr-Id-tO-bE-SeEn1
Note: --vdc not specified, assuming VAPP is an ID
Name: chefnode2
Description: Just Created node2
Status: stopped
IP: 192.168.0.12
Networks
MyorgNet-Router
   Gateway      Netmask        Fence Mode  Parent Network       Retain Network
      192.168.0.1  255.255.255.0  bridged     MyorgNet-Router  false        
      VMs
      Name    Status   IPs           ID                                    Scoped ID                          
      centos  stopped  192.168.0.12  WhAtEvEr-Id-tO-bE-SeEn  WhAtEvEr-Id-tO-bE-SeEn

To show the vm specific details:

[root@chefworkstation ~]# knife vc vm show WhAtEvEr-Id-tO-bE-SeEn --vapp MyvApp_Chef
Note: --vapp and --vdc not specified, assuming VM is an ID
VM Name: centos
OS Name: CentOS 4/5/6 (64-bit)
Status: stopped
Cpu                                          
Number of Virtual CPUs  1 virtual CPU(s)     

Memory                                       
Memory Size             2048 MB of memory    

Disks                                        
Hard disk 1             16384 MB             
Hard disk 2             16384 MB             

Networks                                     
MyorgNet-Router                          
Index                 0                    
Ip                    192.168.0.12         
External ip                                
Is connected          true                 
Mac address           10:20:30:40:50:0f    
Ip allocation mode    MANUAL               

Guest Customizations                         
Enabled                 false                
Admin passwd enabled    true                 
Admin passwd auto       false                
Admin passwd                                 
Reset passwd required   false                
Computer name           centos

  

To set new info to the vm:

[root@chefworkstation ~]# knife vc vm set info --name ChefNewNode WhAtEvEr-Id-tO-bE-SeEn --vapp MyvApp_Chef centos
Note: --vapp and --vdc not specified, assuming VM is an ID
Renaming VM from centos to ChefNewNode
Summary: Status: success - time elapsed: 7.09 seconds

To update other info:

[root@chefworkstation ~]# knife vc vm set info --ram 512 WhAtEvEr-Id-tO-bE-SeEn --vapp MyvApp_Chef
Note: --vapp and --vdc not specified, assuming VM is an ID
VM setting RAM info...
Summary: Status: success - time elapsed: 9.843 seconds

To edit network info:

[root@chefworkstation ~]# knife vc vm network edit WhAtEvEr-Id-tO-bE-SeEn MyorgNet-Router --net-ip 192.168.0.117 --ip-allocation-mode MANUAL
Note: --vapp and --vdc not specified, assuming VM is an ID
Forcing parent network to itself
VM network configuration...
Guest customizations must be applied to a stopped VM, but it's running. Can I STOP it? (Y/N) Y
Stopping VM...
Summary: Status: success - time elapsed: 7.092 seconds
VM network configuration for MyorgNet-Router...
Summary: Status: success - time elapsed: 6.783 seconds
Forcing Guest Customization to apply changes...
Summary: Status: success - time elapsed: 22.639 seconds

To show the changes made:

[root@chefworkstation ~]# knife vc vm show WhAtEvEr-Id-tO-bE-SeEn
Note: --vapp and --vdc not specified, assuming VM is an ID
VM Name: ChefNewNode
OS Name: CentOS 4/5/6 (64-bit)
Status: running

Cpu                                          
Number of Virtual CPUs  1 virtual CPU(s)     

Memory                                       
Memory Size             512 MB of memory     

Disks                                        
Hard disk 1             16384 MB             
Hard disk 2             16384 MB             

Networks                                     
MyorgNet-Router                          

Index                 0                    
Ip                    192.168.0.117        
External ip                                
Is connected          true                 
Mac address           10:20:30:40:50:0f    
Ip allocation mode    MANUAL               

Guest Customizations                         
Enabled                 true                 
Admin passwd enabled    true                 
Admin passwd auto       false                
Admin passwd                                 
Reset passwd required   false                

Computer name           centos

Reference Links:

• https://github.com/astratto/knife-vcloud

Automation for VMware vCloud Director using Chef's knife-vcloud

 Some right reserved by jdhancock

Plugin is available at https://github.com/opscode/knife-vcloud

Configuration used:

• CentOS 6.5
• Chef 11.8.2
• knife-vcloud 1.0.0

Following steps were used to complete the automation process:

cd ~
git clone https://github.com/opscode/knife-vcloud.git
cd knife-vcloud/
bundle install
gem build knife-vcloud.gemspec
gem install knife-vcloud-1.0.0.gem
gem list | grep vcloud
- See if after entering the above command you see the gem knife-cloud. If yes the setup was successful. If no something went wrong.

cd ~./chef
vim knife.rb

- add the following details to the last line of this file (Note: username is @ i.e organisation name supplied) :
knife[:vcloud_username] = "username@orgname"
knife[:vcloud_password] = "##########"
knife[:vcloud_host] = "xxx.xxxxxxxxxxxxx.com"

[ESC]:wq

knife vcloud server list
- Should list all the existing servers

You can also create your own server using "knife vcloud server create" with additional parameters with caution.

e.g

knife vcloud server create --vcpus 2 -m 1024 -I TestServer -A 'roshan' -K "MyPassword" -r 'role[webserver]' --network myNetword-id

Good Luck!!

Reference Links:

• https://github.com/opscode/knife-vcloud
• http://blogs.clogeny.com/category/tutorials/
• http://www.getchef.com/blog/chefconf-talks/writing-a-knife-plugin-for-your-shiny-new-vmware-vcloud-director-based-cloud-chirag-jog/

Public, Private and Hybrid Cloud

A lot has been said, heard and read about Cloud. There so many ways that the cloud gets filtered further. In my previous blog we discussed about Saas, Paas and Iaas. More and more companies are looking for cloud as the solution for their business needs. We shall further discuss 3 important types of cloud.

1. Public Cloud
2. Private Cloud
3. Hybrid Cloud

Some rights reserved by FutUndBeidl

 Public Cloud : Public cloud is considered to be a standard cloud computing model where there is a direct interaction with the users of cloud. It also called as 'shared cloud'. All applications, infrastructures or storage are made directly available to the users. It could be a Pay as You Go service or free as well. Types of public clouds include all Saas, Pass and Iaas platforms. The primary benefit is that it is accessible from anywhere anytime. Public cloud is an ultimate choice when you have lots of users for you application. For e.g an email application like Google, a social network like Facebook. A collaboration is needed among developers over a Paas or for employees to work remotely public cloud is the best choice. This cloud may or may not be managed by the providers but usually it is. Also, it can be scaled very easily as per our needs.
For e.g Iaas based service like Dropbox you can add and remove space dynamically as per your choice.

Some rights reserved by FutUndBeidl

Private Cloud : Here the services and Applications are not exposed to general public and are instead kept private. Highest level of security and control is maintained in these kind of architectures. These services often run behind a firewall and are also called as 'Enterprise clouds'. Advantage is security and resources can be shared among groups. There's a choice to make for H/W and S/W with private clouds and the ability is greatly dependent on what is being used. Its used mostly by companies dealing with high level confidential data.
Many companies are now opting for Enterprise cloud

Hybrid Cloud : Even though many organizations make the use of private as well as public cloud as per their need, there could be vendors looking for functionality of both a private as well as a public cloud. This is achieved with a Hybrid Cloud. At times there are companies that want their data to be secure as well as still are required to communicate to the customers over the network. Many of such companies choose a Hybrid Cloud. Here basically you can set access permissions for which applications need to be publicly accessible and which of them should not be and needs to be in private cloud. 

(To be contd..)