Drupal Planet

Drupal Sprints around the World

On the 22nd October Manchester saw it’s very first Drupal Sprint, this was run in parallel with London Sprints.

Regular sprints are now becoming a great way to help out with Drupal Core and Contrib. In the UK there are now two sprints covering London and Manchester. In India there are 3 user groups all sprinting regularly. No doubt there are some in the US and in other countries.

Sprints unlike regular user groups are an ideal place to flex your code knowledge and get your hands dirty. With initiatives such as the Drupal Ladder it’s proving that everyone can contribute in some fashion whether this be via code, documentation or testing. Within the first 30 minutes of Drupal Sprints Manchester we had our first commit to a contributed project (Omega 4) by a first time contributor.

We communicated with London via Google Hangouts where we updated each other on our progress, I know in the past that London has spoken with India an impressive feat considering the time difference.

It’s not out of the realms of possibility to organise a monthly world-wide sprint. The idea being that user groups all over the world organise a sprint for a specific date with the aim of working on core and contrib. I believe this will be possible with a minimal amount of effort as we can offload a lot of the organisation to each user group. Coordination of communication can be done through IRC. It would be a great publicity stunt as well, something that every country could benefit from especially in the run up to Drupal 8 release.

If anyone is interested in a world wide sprint comment below and lets organise something!

Update - Thanks for the extra information Berdir and YesCT. It's really cool to see the community come together for things like this. Subject to our venue being available Manchester will be ready to sprint on the 25th January.

A Call to Arms - Support your local Drupal Community

For me the the Drupal community is amazing, it's always felt like a family, one you can rely on to help and support you.

It's time for a call to arms, people need to support their local and regional Drupal user groups and events. A lot of these events are volunteer run, it takes time and energy to run meetup after meetup and big events. The people that organise and run these events are quite something, 9 to 5 days don't exist for them.

So this is where you come in, it's easy to help out:

  1. Volunteering - everyone needs volunteers, get in touch with your local usergroup or next camp and see if they need your help.
  2. Sponsorship - camps aren't free, they cost money and no one likes to talk about it. If your a company or individual then please consider sponsoring your local camp/meetup. It gets your name out there and some extra brownie points within the community.
  3. Spreading the word - advertise your local camp/meetup, tell your friends and colleagues, blog and tweet about it. It all helps.

I'm lucky to have worked with some great people on both NWDUG (North West Drupal User Group) and DrupalCamp NW 12 so I know how much the smallest of contribution can help.

If your in the UK and looking to support your local or regional group then take a look at DrupalCamp NW 2013 - http://camp2013.nwdrupal.org.uk/ - they're looking for sponsors and volunteers.

Drupal Sprints Manchester 26th October

I'm proud to announce Drupal Sprints Manchester (UK). Last year at Drupal Camp NW I spoke with various other northern Drupalers and decided that we should get together and attempt to emulate Drupal Sprints London.

The format is simple, Drupal developers / themers / builders get to together and work on Drupal core and contrib.

My aim is to provide a space where people can get together and get stuck in working on Drupal and giving back to the community in the form of code, all that's required is a laptop with an development stack and your brain.

The first meetup will be held on the 26th October at Techhub in Manchester more details can be found here - http://www.meetup.com/nwdrupal/events/14...

If you intend on coming then please sign up so we have a good idea of the numbers.

Please help spread the word!

Testing with Codeception and Drupal Projects

As part of my increased like of agile development and the control it gives you over your estimates and deliverables I've become increasingly aware of the horrible fact our code coverage sucks. There's no real way to sugar coat it we don't do proper testing, it's bugged me for years.

Technical Debt

I first read of technical debt over 5 years at my previous job, the basics of technical debt are that every hack/bodge job will bite you in the ass and you will spend more time fixing things in the long run that implementing it correctly. Now this is where agile comes in, with better estimations it allows you to factor in more unknowns, break down user stories into smaller more achievable tasks and ultimately give you a higher level of what is to be achieved.

With smaller tasks comes great responsibility

When you break down each user story into a task it gives you a clear set of deliverables, now both the user and developer are clear on what needs to be achieved. Now with this method there's no more "that was way too ambiguous", you should now exactly what has to be delivered. So on completion of that test how do you confidently say it's complete? There are two ways:

  1. Suck it and see, mark task as complete and let client figure it out
  2. Test the thing and test it hard and repeatedly.

Over and over and over and over again

So now you know what you have to deliver, you should test it. Here's where codeception comes in, it's a framework that allows you to quickly (and I mean quickly!) write acceptance testing for you application. It also integrates with other types of testing (e.g. unit tests) but lets start small.

With each task you should always create a test, each test should actually test functionality, remember don't test for the sake of it!

Getting down to it

  1. Install it - http://codeception.com/install
  2. Create your fist acceptance test:
    php codecept.phar generate:cept acceptance DrupalUserLogin
  3. Write your first test:

    $I = new WebGuy($scenario);
    $I->wantTo('Ensure Drupal Login Works');
    $I->amOnPage('/user');
    $I->fillField('edit-name', 'digital');
    $I->fillField('edit-pass', 'THIS IS MY REAL PASS');
    $I->click('edit-submit');
    $I->see('digital');

  4. Define your site (tests/acceptance.suite.yml):

class_name: WebGuy
modules:
    enabled: [PhpBrowser, WebHelper]
    config:
        PhpBrowser:
            url: 'drupal.org'

  1. Run the badger! -
    php codecept.phar run
  2. Bask in the glory of your first test.

There it is, dead easy to do! There's no reason you shouldn't implement this for every task in your sprint backlog.

Where to go from here?

After playing around with this for a few hours I seem to be doing the same thing. Each test requires me to login as I don't think each test environment shares session/cookies. I need to figure out a way of creating a codeception module which allows you to plug in a drupal testing user (ideally multiple so you can test each role) and then all the you have to do is call a function which executes the above steps to confirm your logged in before testing authenticated behaviour.

Something along the lines of:

$I->drupalLogin('editor');

Why will this work?

When you initially set out estimation of each story you should add an extra 5-10% for testing. This way you can cover the additional time taken to write these tests. Trust me it's worth it in the long run!

Update 20/05/13

My colleague Paul Byrne has posted a second part to this article with more information - http://paul.leafish.co.uk/articles/code/... - Check it out.

Must have Chrome tools for Drupal Admin

Here are some really handy tools for when you have to deal with Drupals admin interface

Autofill - https://chrome.google.com/webstore/detai...

Automatically fill out forms, great for testing node/add forms with specific sets of data, can store profiles for use on multiple sites.

Edit This Cookie - https://chrome.google.com/webstore/detai...

Excellent tool for checking out cookies on a site, useful when testing SSOs.

Link Clump - https://chrome.google.com/webstore/detai...

Right click and highlight a bunch of links then open them up in new tabs, more helpful than you'd think when having to test n number of pages during a migration.

HTTP Headers - https://chrome.google.com/webstore/detai...

Handy for checking whether pages are being served via Varnish of Drupal Cache. Also great for checking the IP of a server on a HA setup (if setup to output to headers).

Easy Check - https://chrome.google.com/webstore/detai...

Easiest way to manage Drupals permissions page, click on a checkbox and then drag your mouse and it will auto check all the ones you mouse over, no more clicking like a madman!

Using Jenkins CI and Drupal

While building my Drupal 7 profile I started playing around with Jenkins CI. My reasons for doing so were two fold:

  1. Code Quality I believe in code quality, there are multiple tools out there that will analyse code and report back using various different metrics. CI allows me to run these metrics on a repeatable basis, if my code changes then the quality changes. This makes me a better developer overall.
  2. Repeatable Tasks Jenkins can do anything you throw at it, why not throw at it all your boring repeatable tasks?

Installation

sudo apt-get install jenkins

Simple!

Configuration

After installing Jenkins read this. Out the box Jenkins is open to the world so now might be a good idea to lock it down a bit.

It's also worth clicking around the admin screens and getting to grips with it, there are a lot of options when setting things up. Checkout the available module list, chances are your going to want to use a few. Installing these are as simple and point and click.

Useful plugins

  • Git - of course this will come in really really handy
  • Copy Artifact Plugin - Jenkins allows you to create artifacts (a zipped up copy of all files built) this comes in really handy when using multiple build steps, no more cloning repos a couple of times.
  • Jenkins Doxygen Plug-in - does what is says on the tin, executes doxygen to create documentation.

Example Jobs

  1. Testing drush make files
  2. Testing profile install using drush si
  3. Running cron, mostly used for getting notifications of failures and audit trail
  4. Building doxygen documentation

Testing drush make files

Sometimes drush make files take a while to run, I have a project that contains a lot of modules, it's average build time is around 5 minutes. The project roughly needs rebuilding at least once or twice a week. So that's 10 minutes time spent waiting, now imaging if the build fails, this could be due to a couple of reasons (mostly bad release numbers) but if it fails that 5 minutes wasted. Testing this make allows me to make changes to the make file without worrying about it, I usually just

drush dl module
and update the make file manually for other developers to use. Overall this saves time and frustration over broken builds.

This is also a great time to use the copy artifact plugin, this will save you having to redownload drupal core and contrib multiple times throughout your tests. Be sure to exclude any .git repos otherwise copying the artifact will fail due to permission errors. Artifacts are only saved when a successful build.

Testing profile install

You can use drush to install profiles, this comes in really handy when your testing profiles in development. In my setup I have it chained so that every time a successful make file is built it then copies the files to a new workspace and runs

drush si
this then captures all the output and exits if there are any errors in the install.

Running cron

This is as simple as it sounds, you can setup a job to run cron on all your sites. This is a great tool as it provides an audit trail and notifications on any errors.

Building doxygen documentation

Assuming you have a doxygen setup file configured for you project you can get Jenkins to build the documentation for you. If you add this into your build chain it'll run whenever all the test you run are successful, this will save building docs for broken functionality.

Summary

Repeating time consuming jobs over and over again is boring and distracts away from the focus of a project, by using Jenkins in my development setup it's allowed me to focus on what's important and monitor my errors (we all make them) without having to worry about them. There is a lot more you can do with Jenkins this is only a taster. I'm hoping to present next Wednesday at NWDUG in Manchester. Over the next few weeks I'll expand on the example jobs above and post my setup and code used to achieve them.

Thanks to Stewart Robinson for giving me a really good code sample and point in the right direction

My first Drupal 7 Distribution and what I learnt

I've been using Tumblr for a few months now as a simple way of sharing the images I find interesting and beautiful (can be found at ayil.co.uk). I found it was lacking in a few features that would be nice, being a Drupal developer I thought hell why not try and recreate it in Drupal 7.

I've been here before.

I'm not new to profiles, I started playing around with them at the start of 2010 in Drupal 6 for a clients site. It seemed like a great way to get a site up and running quickly and cut out all the faffing about with installing modules and content types. It went pretty well but then I discovered Features and decided to move everything to them and haven't looked back (regardless of how many times I've wanted to stab my own eyes out with dependency hell).

The start of something.

So why not recreate Tumblr in Drupal 7? It would have all the best bits of Tumblr and all the best bits of Drupal. Half the stuff I wanted to do comes straight out of the box, especially now that cck is in core.

Queue the montage sequence.

For those not familiar with Tumblr you get 7 "content types", each of these have specific functionality such as posting a photo or standard text blog post. Creating the 7 content types was easy, a lot of point and clicking but achievable without any coding. Within a few hours I had all 7 content types mapped out and created within Drupal, it was very rough around the edges but it was there and most of all working (well almost).

Shiny new toys.

Features allowed me to export all the configuration into separate modules which were controlled by a central controller module (a concept I've used a few times and should probably write more about). Adding a dependency to this controller module in my distribution info file meant that on install it would cascade down and install all the content types and associated settings. My distribution info file also contained the bare minimum to set-up various menus and blocks.

Since this was a personal project I decided to throw in some responsive theming for good measures, I used the Omega theme to give me a starting point but since my skills lie in code the theme never really went far, I just have no passion for web design. I highly suggest you checkout omega tools as it's a great way of configuring the theme in the shortest amount of time and it even comes with some handy drush extensions.

Lights... Camera... ... is this thing on?

Since my use case was mostly media based I figured I'd check out the media module. This was where it all went wrong, it's a usability nightmare, I went round in circles trying to figure out how to configure it properly to do what I wanted. In the end another colleague ended up ripping out what I'd done and replace it with something that nearly worked. I found the modal window approach to be horrible (this coming from a guy who's spent the last year living in a modal hell), here are a few of the issues I found: 1. Upload size = 0, documented bug. 2. Remote embed for image shows video options. 3. Ignore above it doesn't work for video or image using the third party modules. 4. Edge case conflict with omega theme.

So now I'm left with a distribution that installs and works for the most part. If I was a good Samaritan I'd jump into the issue queue and help where I can but I don't have the time or the patience. I'm not familiar enough with the changes in Drupal 7 as all my work time is spent in Drupal 6 and I haven't had the opportunity to delve behind the scenes of 7 yet.

Making my life easier

I use drush a lot, it's probably one of the most used command-line tools in my arsenal at the moment. For those not familiar with drush make it's command that downloads modules and 3rd party libraries so you don't have to, it's a great time saver and well worth looking into. Drupal.org allows you to package a special version of a make file so that it will grab all the modules and 3rd party libraries (only white listed ones, a relatively new feature) and build it for you. Great idea in principle but the thing that annoyed me was the structure of the make file is fundamentally different between the normal type and drupal.org standard. This was a bit of pain for two reasons, the first is it broke my CI setup as a valid d.o make file can't contain "core" whereas normal drush make requires it. Secondly as I found out after converting my file is that sandbox distributions don't have this packaging setup available. Thankfully everything was tracked in git and 2 reverts later I was back to a normal make file.

Why even bother with a make file testing?

I've been burnt too many times before by developers not testing their make files, it's annoying to have to sit through a whole build process for 4-5 minutes only to have it failed because someone has either spelt the module name wrong or used an invalid release. Automating this test is a great way to save myself and others hassle down the line.

Dead and buried?

Far from it, every week I check the Media issue queue and various twitter postings to see the improvements being made on the module. It'll get there and when it does I'll continue to build my distribution towards some kind of stable release as I still believe that this is a great project.

Where is it?

It can be found here. Feel free to contribute suggestions/ideas/code. Out of the box you get a responsive site which emulates around 60% of the functionality of Tumblr, it's not pretty but it's functional.

What did I learn?

Bucket loads, I'm now familiar with the new Drupal admin interface, I have a deeper understanding of install profiles. I also built a Jenkins setup to test various different aspects of building and creation of the profile which I'll write another blog post about. Taking on a distribution is a large task but ultimately worth it.

Using ack with Drupal projects

I decided to move to Ack for searching as it has nice switches like --php and --nosql.

By default ack doesn't recognise .module, .install or .inc files so you have to add them. Create a .ackrc file in your home directory and add the following:

--type-set=drupal=.module,.install,.inc

DrupalCon London 2011 Wrapup

It's that time again now that I'm well and truly full of drupalflu.

This years pickings were slim to none, not that there wasn't anything great out there but I just didn't have the time to go out and grab any. My time was spent attending sessions and talking to potential new clients on our booth (which was awesome!).

I did however grab a conference t-shirt which is great, as someone who collects a lot of cool and interesting shirts this one was pretty amazing, so congrats to the designers.

This years conference was great, so many good sessions to attend and as always it was great to meet up with other drupal people including lots from #drupal-uk. I'm so pleased I got to see Batman Live I thought it was a great show and even after my skeptisism it was worth it.

I came away with so many ideas that I can't wait to put into fluition. I particuarly like drush deploy, looking forward to seeing that get a stable release. I attended a great BoF on Kit/Build Kit/Features, something that I've been using for a while now.

Next years european DrupalCon is in Munich... Beer Beer and more Beer!