Continuous Integration with Jenkins

Continuous Integration:

What's the deal? As per a tutorial that I came across a few months ago, its a process in which all the development work is integrated at some predefined time interval and the resulting work is tested and built automatically.

So the basic functionality of Jenkins is to monitor a version control system and if changes are encountered in it, it will start the build system and monitor it. Build system could be e.g Apache ant or Maven. Jenkins can monitor entire build process and it has functionality to report the monitoring and send out notifications as per the results encountered. Plugins could be added to Jenkins for building and testing many kind of applications.

To use Jenkins, you will need,
An accessible source code repository, e.g github, with your code checked in.
A working build script e.g maven script checked into the repository. Jenkins can be started from a command line or also from a web application server.

Installation of Jenkins:
Jenkins provides debian/ubuntu packages that installs it. Post installation, you can start jenkins with:
Download Jenkins.war file and install using the following command:
java -jar jenkins.war --httpPort=9000

sudo service jenkins start or
/etc/init.d/jenkins start

Running jenkins locally will start in on http://localhost:8080

At times you may also want to create a virtual host for Jenkins to run. You can create a virtual host on ubuntu by adding a file in /etc/apache2/sites-available/jenkins

Following details need to be added in this file:

<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName jenkins.local
ServerAlias ci
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPreserveHost on
ProxyPass / http://localhost:8080/
</VirtualHost>

You will then need to enable this virtual host by the following command
sudo a2ensite jenkins

To disable this site at any instance you can use the command
sudo dissite jenkins

Once the site is accessible on your localhost, you will need to



--------------------------------------------------------------------------------
meetup
for installation goto jenkins-ci.org
latest versus LTS try to go for LTS and not latest. latest might give you good plugins and fast processing etc. but it may break
installation can also be done using the war file java -jar jenkins.war

Configure Jenkins Server

Install a Patched Ruby Interpreter With Rbenv and Ruby-build for 2.0.0-p247

Installation of Ruby 2.0.0-p247 recently had some issues with Openssl package for Centos 6.5. I had to patch the version to get it running. Following script was written later that worked to automate the patch later.

 
#!/bin/sh
rm ~/.rbenv/cache/* -rf
mkdir /tmp/build
wget https://raw.github.com/sstephenson/ruby-build/master/share/ruby-build/2.0.0-p247 
cp 2.0.0-p247 /tmp/build/ 
# download and patch the ruby sources
wget http://ftp.ruby-lang.org/pub/ruby/2.0/ruby-2.0.0-p247.tar.gz
tar xvzf ruby-2.0.0-p247.tar.gz
cd ruby-2.0.0-p247
curl https://gist.githubusercontent.com/spkane/8059362/raw/01585dcf6b33254124566f4521a3946e6f26e0a9/ruby-2.0.0-p247-openssl-el65.patch | patch -p1
cd ..
tar -cvzf ruby-2.0.0-p247-openssl.tar.gz ruby-2.0.0-p247
# download and patch the ruby-build version definition
sed 's|"2.0.0-p247.*|"2.0.0-p247-openssl.tar.gz" "file:///tmp/ruby-build/2.0.0-p247"|' < 2.0.0-p247 > 2.0.0-p247
#install the patched version
rbenv install /tmp/build/2.0.0-p247
rbenv rehash

Integrating Docker with Chef

Ever since Docker was introduced to me, the first thing that came to my mind was, Docker is a replacement for Chef. Docker does almost everything that Chef did and provides me with a light weight solution. While Chef just configures the system. Well I was wrong. I read many articles, heard many speeches and came to a conclusion that, it was not Chef vs Docker, it was Chef with Docker. When they work together, you get the most powerful tool to deliver fast, light and efficiently.

I conducted a demo at Docker-Bangalore meetup #4 to show how integration with Chef and Docker works. More information on the meetup page.

What is Chef?

• Configuration Management Tool.
• Helps write Infrastructure as a code.
• Users write code called ‘recipes’ to help manage and configure servers and applications.
• Runs in client/server as well as stand-alone configuration called chef-solo.
• Ensures each resource is properly configured and is in the desired state.
• More on my blog Learning Chef.

What is Docker?

• Light weight Linux container that packages an application and all its dependencies in a virtual container.
• The container can be packed, shipped and the application can run on any Linux machine on public cloud, private cloud or bare metal.
• Does not include OS.
• Layers the changes made in the container just like a version control system, which allows you to reach any previous state within no time.
• More on my blog Docker.

Why use Chef+Docker, What are the issues with Vagrant+Chef?

• If each project is in its own VM, resource usage is prohibited. 
• All the projects in a Single VM, management will be difficult, different versions for same software for different projects. 
• Building and Rebuilding takes long time, partial builds or rebuilds are difficult. Unless explicitly snapshot is created at various stages. 
• If you rely on external dependencies(which we do), full rebuilds can fail  due to one or more broken dependencies. 
• If your stack has multiple components (web, db, cache, etc) and they are installed in 1 Vagrant VM, the resulting setup differs from production. Or you could use multiple VMs… but point # 1 holds.

Chef Pros:

• Great at provisioning (knife bootstrap and plugins)
• Configuring services – Writing your infrastructure as a code.
• Testing your Infrastructure

Chef Cons:

• Packaging applications
• Deployments and rollbacks
• Dynamic service delivery

Docker Pros:

• Packaging applications
• Deployments and rollbacks
• Dynamic service delivery

Docker Cons:

• Managing persistent storage
• Complex networking
• Monolithic services

How to get Chef and Docker work together?

The key is understanding the Developers and Operations workflow and making them work together.

Developers own:

• Code and Libraries
• Build and Test Automation
• System Packages
• Runtime Configuration
• Release Management
• Logs monitoring
• Horizontal Scaling

Operations own the Platform:

• Hosts
• Routers
• Monitoring
• Logs
• Security
• Backing Services

Basically,

Developers take the control of the containers
&
Everything outside containers for Operations

Setting up Chef and Docker to work together:

• Install Chef workstation with Omnibus Installer.
• Create an account on manage.opscode.com or create your own chef-server.
• Create a vagrant node or a cloud instance.
• Site install docker community cookbook with site install.
• Create your own cookbook which depends on docker.
• Upload all cookbooks.
• Bootstrap the node.
• Login and verify of the node has the required images.

References:

• Brain Flad's chef-docker cookbook
• Gabriel Monroy's ChefConf presentation
• StackOverflow, Google groups and Quora