n00b-alert-3: Setting up Symfony2

Now that we have our development stack in place, we are going to install Symfony2, which is the PHP framework we will be using to develop our new system.

Installing Git

For version control, collaboration and backup of our codebase, we will be using Git, with our repositories hosted at GitHub. As I am working on Windows, I have chosen to use Git Extensions. Download and install this. Then we’re good to clone our respository from GitHub.

Clone repository

Find the respsitory and then copy the SSH URL from near the top of the page. Paste this into the Repository to clone field. Then select the Destination folder and also the subdirectory which we want to create for the repository. If there are any submodules which are referenced from the base repository, make sure that “Initialize all submodules” is checked, so we can pull all of them down from GitHub too.

Load SSH key

In order to authenticate with GitHub, we need to generate a public/private key with PuTTYGen and then upload the public key to the SSH Keys section of our GitHub account settings. Save the Private key locally then use this in the “Load SSH Key” part when importing the repository.

Now click the clone button to pull all the code down from GitHub and complete the local cloning of the repository.

Download Symfony2 dependancies with Composer

As part of our stack, the webserver running Zend Server has the basic Symfony2 framework installed. However, this is not much use, without all of the dependencies which make it fully functional.

If Symfony2 is not installed, go and follow the instructions on their download page. Download without vendors, so that we can have more fine grained control here with our dependancy management later on.

Log into the webserver via SSH:

As you can see we then need to run a Composer update in order to bring everything up to date:

Now we’re ready to start using (learning) Symfony2.

n00b-alert-2: Provisioning the Development Servers with Chef Solo

Following my previous post about setting up the ability to spawn multiple *nix (Ubuntu) development VMs with Cygwin, VirtualBox and Vagrant, this tutorial is for Chef Solo, to provision the VMs with their databases, servers and programming environments we need. Although this is essentially a duplicate post of Dan’s, I find it best to put everything down myself, to help re-inforce the steps.

Setting up the Chef library

According to Dan, Chef will try to load all the cookbooks, regardless of whether it uses them or not. Therefore to keep things lean, it is best to only use the relevant cookbooks at a project level. To facilitate this, I will create a Chef library which I can then copy and paste the relevant cookbooks from into each project.

Now, there isn’t much in here, so I am going to set up my first project and use this to configure Chef Solo. Thereafter, I will copy these files across into the Chef Library, so that when I create further projects, I can take the Chef files I want from the library.

Creating the first Chef project

The Vagrantfile has now been made and we are ready to copy over the Chef files that we need from the Library. Although these files are currently empty, we will propogate them and then copy them back into the Library. When making future instances, we only need to copy them across from the Library to the Project and off we go. By copying across the files, we preserve the directory structure, to avoid us getting lost.

At this point is is probably best to import the project folder into your IDE, in my case Aptana. I can then edit the config files from the comfort of the IDE and not the command line.

Configuring Vagrant

Configuring Chef

Firstly the recipe we will be using. In this case, we are using it to install Zend Server.

Next we need to configure the Role that our server will be using, in this case we are calling it WebServer. Although the name is generic, we can here specify which webserver we wish to set up, currently Zend Server, which installs with Apache2. Should we wish to completely change the web server completely, we only need to add a new recipe for the new server and then change the configuration of the Role.

Now everything is configured for a basic PHP webserver using Zend Server.

Booting the VM

Cd to the directory with the Vagrant file

We hit a problem

Oh dear, something went wrong. Upon investigation it is the nightmare to understand first two execute codes in the recipe. We need to use not_if instead of only_if:

The logic is as follows:

If zend cannot be found in list of apt-keys (i.e. grep returns 0) then run the command in the line above to wget and add the key. Grep returns nothing, i.e. 0 / false and the not_if only runs if it evaluates to false / true, whereas the only_if runs on true. Our error was saying if the Zend Key can be found, try to download it – this is obviously the opposite of what we want.

Ditto for the source of Zend server, but this is by grepping the apt sources list.

The confusion was the double negatives, but all now makes sense.

Re-provisioning the VM

As the VM is already running, we do not need to restart it, instead we can re-provision it, following the changes to our recipe.

If this gives you any trouble, then you may need to destroy the VM completely and then boot it again:

Now we can log in via SSH via the port 2222 forwarded to localhost.

Default username and password is vagrant.

To test the webserver, open http://10.0.0.2/ or if port forwarding is set up http://127.0.0.1:port (in our case 8081).

Configuring Apache2

Now edit the recipe to include:

finally to get it to work:

Now via my IDE, I can change my index.htm under the app folder, and it will change when I refresh the website http://10.0.0.2/ we can see the changes.

Filling the library

Finally, we can copy back the Recipes and Roles into the Chef Library we made earlier. If we wish to expand on these recipes, without affecting other instances, we can then take copies of these when we make new projects. Alternatively if we wish to just use them straight out of the box, we can symlink to these directories from the new projects. By symlinking we can easily re-provision multiple instances without the need to edit each of the recipes.

n00b-alert-1: Running Multiple Development Environments on Windows with Cygwin, VirtualBox and Vagrant

Technology seems to be moving so quickly; far too quickly for me to follow. Thanks to Dan, I’ve been able to ignore the evaluation of new shiny things, as he is keen to do this, whereas I am not. Occasionally he shows me something which raises my interest, even gets me excited. In this instance, it is the latter. Over the last few months Dan has tried and tested many different things, to see what suits our requirements best and we’ve now settled upon a number of things, which I will have to learn on my own following his departure in a week’s time. Therefore this last week is my crash course.

What we’re doing

I have in fact done this twice at home already, but a couple of weeks later trying to do it from my work PC is like starting from scratch. Therefore, Dan has invited me to his blog, to write some tutorials of my own, essentially for my own future consumption. This first one is about setting up multiple Virtual Machine instances for development purposes on my local PC.

What we need

Why Cygwin?

This could all be done via Windows, using CMD where necessary. However, as our server environments are *nix and as it’s what I am most familiar with from the command line, it is best to do this via Cygwin, to practice commands regardless of whether I am running these as Dev VMs or later down the line when rolling out to production.

Installation

  • Install Cygwin
    • I included the following options (use the search, rather than scrolling through the list):
      • nano
      • All SSL libraries – not sure which I need, but they don’t take up much disk space, so I included them all. Obviously I’ll be a little more conscious of what I am installing on the production box
      • All SSH libraries – ditto
    • I then set these alias in my ~/.bashrc file:
      • pico=nano – to use pico even though not installed
      • vagrant=vagrant.bat – to avoid typing the .bat which is necessary on Win but not *nix

      Then to refresh the profile run:

  • Install Oracle VirtualBox with default settings
  • Install Vagrant with default settings

Config

From here on we are in the Cygwin shell. I will run all the project folders from my D Drive, so I am making a folder there to store all my user project files. Then initialise Vagrant.

Testing the installation

Run through the Vagrant tutorial from this page, in order to re-familiarise myself with the Vagrant basics as well as testing that everything works correctly.

SSH from Cygwin

When trying to SSH into the new VM, as we are on Windows, we can either:

  1. Use PuTTY, by generating a PPK with PuttyGen
  2. Use the Cygwin shell SSH
    • Firstly chmod the insecure_private_key file to 700 – otherwise it will not import
       

      Thus mine is

    • SSH into the shell

      As you can see we’re in:

Upon success

That’s it for the moment. The next tutorial will be to learn provisioning the development instances with Chef Solo – Chef Server will be later down the line for the production servers.

Installing Zend Server and MongoDB using Chef and Vagrant

I am in my last two weeks at my current job, in this time frame I will be helping my colleague (Ingram) make sure he is prepared for my departure.

When ever I have used Vagrant I have used Puppet for provisioning but Ingram has other ideas and prefers Chef (he hasn’t use either before but has done some researching), I haven’t tried Chef before so thought it’s worth a try.

I am now a convert, I have nothing against Puppet, it does a perfectly good job but its declarative nature isn’t something i’m used to. Chef on the other hand using Ruby and its own Ruby like syntax which I am more comfortable with.

While writing this post I acted out each of the steps and zipped it all up for your consumption.
You can download it from http://www.snatchfrigate.com/wp-content/uploads/2012/07/example-vagrant-chef-zend-mongo.zip.

So kids, for this lesson I will cover the same ground as I did with the Puppet, Zend Server and MongoDB post but using Chef in place of Puppet.

I will be using Ubuntu for this example.

About Chef

Chef is an open-source systems integration framework. It’s been designed specifically for use in the cloud, it’s also very useful for provisioning development environments which we will be doing shortly.

Preparation

Lets create the file structure of our environment now so we know where everything goes the structure is as follows

Vagrant creates “Vagrantfile” on “vagrant init” in your current directory. We want to keep anything Vagrant or Chef related away from your application, this is why we created “projects/myproject/vagrant” and “cd” into it before running “vagrant init”.

projects/myproject/vagrant/Vagrantfile defines and configures our virtual machine(s) from a hardware perspective. For a better idea of what you can do with this file check out Vagrants documentation.

projects/myproject/vagrant/chef/roles/server.rb tells Cheff which recipes to run

projects/myproject/vagrant/chef/cookbooks/my_cookbook/recipes/server.rb is where the magic happens! You normally break the environment down into small recipes for each part of the environment.

Consider this, in your production environment you have two servers, one being your web server and another for your database server, you could create two recipes one for each server or you could separate your recipes down to package level.

Because you’ve made the effort creating these recipes you can mix them up via a role file.

Let’s say you want to virtualise your production environment for development, rather than virtualising the entire stack you could simply include all the packages on one VM simply by creating a new role file that ran all the recipes.

projects/myproject/vagrant/chef/cookbooks/my_cookbook/files/default/app.conf contains an Apache VirtualHost which has document root pointing to our “app” folder so you can have a play around with PHP and MongoDB after we have the environment up and running.

We will be creating one recipe for our example

Configuring our environment

projects/myproject/vagrant/chef/cookbooks/my_cookbook/recipes/server.rb is responsible for installing packages, starting services, creating users and so on. We could use other pre-existing cookbooks to achieve this but that won’t help you grasp what is going on, so we’re doing it the hard way ^^.

Just a quick note, paths can be relative in Vagrantfile which I have used above.

The ruby above is telling Chef to execute the following command providing certain conditions are met. Let’s do the same for the Zend Server repository

I need to go through this process one more time for MongoDB

We have our repositories and their keys installed, we need to update apt so we can install some packages

As you can see our recipe is doing a lot, it’s best to split them up for each package you want to install. For this example using one recipe will do fine.

Before our VM is ready to be booted we must create a role file for it, in this instance our role file is projects/myproject/vagrant/chef/roles/server.rb. The role files job is to specify the recipes to be used and any additional configuration information that may be needed.

Lets start the VM using the following command:

Please be patient, it can take a while to install Zend Server.

Once the VM has booted up and Chef has done its thing, you should be able to open http://10.0.0.2 in your browser and get a response. If you left forwarding for port 80 enabled, you will also be able to get a response from http://127.0.0.1

If you had any problems with Chef you can SSH into the VM and check the log files in more detail. The details you need to SSH into the VM are [email protected] on port 2222 with the password vagrant and in most cases the log file will be located in /tmp/vagrant-chef-1/chef-stacktrace.out. Try to fix the issue then execute the following command to re-run Chef.

Is everything working? Awesome! let’s configure Apache.

I am going to keep this file brief as configuring Apache is outside of this posts scope.

This file is no use to us at its present location, lets update our recipe to put the file somewhere useful.

We are now left with one thing to do in our recipe and that is to ensure our services are running

now re-execute

and we are done

when your finished with the VM you can execute either

That’s you have Zend Server, MongoDB and a VirtualHost ready for you to play with.

Please let me know if you find any issues or have anything to add to this post, it would much appearciated.

A great end to the month

Holding the stockade!I went to Apocolypse for some airsoft this weekend and had probably one of the best ending battles I’ve had whilst playing the game, Which is odd for this site.

Normally they have really good morning games and dire afternoon games but at this event it completely switched. The morning game was poor, we (playing Chechen terrorists) had to get out with our hostages, but limited to having one hostage leaving our HQ every 15 minutes which made it very easy for the opposing teams to keep us at bay. the real problem with this game was that the hostages were paying players that mostly did nothing for half the day.

After some lunch things became interesting, we now had to get a dirty bomb into the stockade, which we managed and set us up for the final game, holding the stockade till either the bomb went off or the one person assigned with disarming it does so.

This final game was long, we were mostly surrounded (only behind us was clear) and cover was limited. Several times the other team tried to get behind us to clear the area but we mostly held them off and the few that did make it were not carrying automatics, so the areas were easy to recover.

I was moving between areas (wherever there was action) and even got a few fancy slides in without falling over, one of them putting me in the right place to take out a sniper only to be hit by friendly fire after the kill.

It was an intense battle and definitely made up for the poor morning game.

Wheel update follow up

In my previous entry I demonstrated how to configure the router with our new Route class NamedVars. I thought it would be a good idea to explain how the route classes works.

First we will look at the parent class Wheel\Route\Route

Wheel\Route\Route.php

This abstract class has two abstract methods to be implemented.

isMatch
This method must return a bool (true or false) based on whether the path given matches the path specified in the route.

getAction
Should configure the action before it’s returned, the most that should be done here is configuring the parameters before they are passed to the action.

Now we have an idea of what needs to be implemented, we can take a look at the NamedVars class.

Wheel\Route\NamedVars.php

For this type of route I’ve had to override the configure method to generate a regular expression from the $path specified in the constructor. The regular expression generated will be used to match and extract values from the paths.

If the path given matching the path in the route, we will extract the values and merge them with any params already in the action before returning the action.

Wheel framework update

I have been working on my framework (Wheel) today. I wanted my routing system to be a bit smarter when handling dynamic URL’s and passing their arguments to the action.

I have gone for behaviour similar to what you’d find in Symfony2, as a result my actions and routes must have matching argument names.

Here is an example of config/router.php

With this route, paths such as “/hi/daniel/chalk” will be accepted. The difference between this and the regular expression routes is that the named variables will match arguments of the same name in your controllers action.

The class below is the controller for the route specified above. The named variables will be passed as the two arguments in the method shown below.

There are improvements to be made to this route class, which I will write about in a follow up article.

If you want to have a play with the framework go to http://projects.snatchfrigate.com you can grab a copy of the source (GIT) from their

Installing MongoDB and Zend Server using Puppet

I spent a large part of today researching the installation of MongoDB and Zend Server.

As I use Vagrant for my development environments, I thought it would be worth making a Puppet file I can reuse.

Rather than retrieving the source and building PHP and MongoDB, I have chosen to install repo’s so I can install the latest and greatest as and when they become available.

Just in case someone would find this useful, I have shared the Puppet file as a Gist on GitHub. You can find the Gist at https://gist.github.com/2781470


# Lets tell Puppet the order of our stages
stage { 
  'users':      before => Stage['repos'];
  'repos':      before => Stage['updates'];
  'updates':    before => Stage['packages'];
  'packages':   before => Stage['configure'];
  'configure':  before => Stage['services'];
  'services':   before => Stage['main'];
}

class services {
  #we want apache and mongo to be running when the server boots
  service { 
    'mongodb':
      ensure => running,
      enable => true;
    'apache2':
      ensure => running,
      enable => true
  }
}

class configure {
  # I know where apache conf's should go but for convinience im putting them with the project,
  # So i must include the folder in httpd.conf
  file { 
    '/etc/apache2/httpd.conf' :
      content => 'Include /home/vagrant/apache/conf/enabled/'
  }

  # Installing zend server doesn't install the php-cli so i will create a simlnik to zend servers binary
  exec { "php simlink":
    command => "/bin/ln -s /usr/local/zend/bin/php /usr/bin/php",
    unless  => "/usr/bin/test -L /usr/bin/php",
  }
}

class packages {
  package {
    "mysql-server-5.1":           ensure => "present"; # MySQL - you may still want to use it so here it is
    "git-core":                   ensure => "present"; # Git! - I use it for my projects.
    "zend-server-ce-php-5.3":     ensure => "present"; # Zend Server (CE) (remove ce- for the paid version)
    "php-5.3-mongo-zend-server":  ensure => "present"; # MongoDB extension - provided by Zend
    "mongodb-10gen":              ensure => "present"; # MongoDB
  }
}

class updates {
  # We must run apt-get update before we install our packaged because we installed some repo's
  exec { "apt-update":
    command => "/usr/bin/apt-get update -y -q",
    timeout => 0
  }
}

class repos {
  #lets install some repos
  exec { 
    # before i install repo's i install the keys
    "get-zend-key" :
      # install the key
      command => "/usr/bin/wget http://repos.zend.com/zend.key -O- | /usr/bin/apt-key add -",
      # only install the key if the key doesn't exist already
      unless  => "/usr/bin/apt-key list| /bin/grep -c zend";
    "get-mongo-key" :
      # same as key installation above
      command => "/usr/bin/apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10",
      unless  => "/usr/bin/apt-key list| /bin/grep -c 10gen";
    # now lets install some repo's
    "install-zend-repo":
      # we are echo'ing the repo into /etc/apt/sources.list
      command => "/bin/echo 'deb http://repos.zend.com/zend-server/deb server non-free' >> /etc/apt/sources.list",
      # tell puppet not to run the command if the repo already exists
      unless  => "/bin/grep 'http://repos.zend.com/zend-server/deb' -c /etc/apt/sources.list";
    "install-mongo-repo":
      #we are doing the same thing as we did with the other repo above.
      command => "/bin/echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' >> /etc/apt/sources.list",
      unless  => "/bin/grep 'http://downloads-distro.mongodb.org/repo/ubuntu-upstart' -c /etc/apt/sources.list";
  }

}

class users
{
  #add your users and groups here
  group { "puppet":
    ensure => "present",
  }
}

# Here we are linking our classes to stages
class { 
# class: stage => "stagename";
  users:      stage => "users";
  repos:      stage => "repos";
  updates:    stage => "updates";
  packages:   stage => "packages";
  configure:  stage => "configure";
  services:   stage => "services";
}

Please let me know if you found this useful.

Getting started with Rails

After years of avoiding it, I have decided to take on Ruby on Rails. I have never really used Ruby but I’m sure I can pick it up easily enough, Installing Ruby and Rails is another matter.

Installing Rails is painful! If Ruby and Rail are so great why is there such a barrier to get started?

I was trying to install them on Ubuntu (lucid) using apt-get only to find that something breaks rails often the cause is Rake. After destroying and rebuilding my VM on vagrant roughly 15 times, I gave up.

This morning a colleague of mine directed me to Ruby Version Manager (RVM), this definitely took a lot of the pain out of getting rails working. There was still one problem remaining, rake needs a javascript runtime. I was hoping RVM would have dealt with this but unfortunately not.

So I decided I will be installing node.js. Here are the commands I ran to get node installed.

After this you should be able to create your rails project AND use rake.