Category Archives: Agile

Standardised Bug Reporting with SEERS

SEERS

When raising bugs and defects, it can be easy to miss out vital pieces of information without a strict policy to adhere to. Often the developers have to refer back to the tester that raised the bug and ask exactly what went wrong, what they expected to happen or in which environments the problem occurred. SEERS is a term I have coined to use as a standardised series of criteria for bugs and defects. SEERS stands for:

Screenshot

There should always be a screenshot of the problem (unless it is a non-visual bug). If possible some kind of obvious visual outline around the problem should be shown. As noted by Alun Coppack in the comments, if the error is behavioural we should annotate the screenshot to describe the problem – and if this is not practicable a screencast of the problem leaves no margin for miscommunication.

Environment

There should always be details of the environment(s) (browser, operating system, screen resolution, etc) that the problem occurs in.

Expected/Actual Behaviour

There should always be details on the expected behaviour and the actual behaviour.

Reproduction

There should always be accurate details on how to consistently reproduce the bug. As noted by Alun (again!), some errors can be temporal and the testers should be aware of this and take note of the time/date before they try to recreate the error.

Severity

There should always be details on the severity of the bug.

Severity

It is a common complaint that the recording of the severity of a bug or defect is inaccurate. An element being two pixels to the left in IE6 is not a huge problem, but is often rated as Critical. To that end, I list the severities we use below.

Blocker

A blocker means that our entire story or functionality is not working or unusable, even if following The Happy Path. The Happy Path is that where the user does exactly what we expect them to and does not stray from the norm at all – e.g. entering the correct login details and clicking “login”. If we do not fix this bug we cannot release. An example is a login box where clicking the login button does nothing.

Critical

A critical bug indicates that functionality is severely impaired. If we do not fix this bug we cannot release. An example is a login box where entering invalid credentials takes the user to an empty page. The Happy Path (entering correct credentials) works fine, but a very likely situation exists which could cause functionality to break.

Major

A major bug indicates that some functionality does not work as expected. Copy may be obscured, or specific situations may cause errors to occur. We should certainly fix this bug, but it would not be impossible to release without. An example is a login box where entering over 100 characters in the password field causes an error.

Minor

A minor bug indicates some kind of cosmetic issue or unlikely situation that causes an error. We should fix this bug if it is cost effective to do so and other more important tasks do not exist. An example is the login button being aligned incorrectly, or clicking the button 30 times causing an error.

Trivial

A trivial bug indicates a very minor cosmetic issue or very unlikely situation that causes an error. We may decide not to fix this bug. An example is IE6 and Firefox not looking exactly the same, although IE6 looks acceptable and functionality is not impaired.

Terminology

Let’s discuss what actually constitutes a bug or defect, although this is terminology that I use and your mileage may vary.

Bug

When a task is in the current iteration or sprint and we raise an issue against it, this is called a bug.

Defect

When an issue is found that does not relate to any current tasks, this is called a defect.

Graded Browsers

Finally, here is a list of the browsers we support and their grades. We expect grade A browsers to have complete functionality, and look close to perfect (as much is reasonable or practicable). Grade B browsers should look acceptable and work as expected (although potentially with less of the flashy but essentially unnecessary functionality). Grade C browsers should allow the user to read the content, and accomplish their tasks. It is important to mention that we do expect content to be accessible in absolutely any environment; it is the aesthetics and nice-to-have functionality that we are less concerned with.

Grade A

These browsers should work perfectly, look good and perform well.

  • Windows XP/Vista
    • IE8
    • IE7
    • Firefox 3.5
    • Opera (latest version)
    • Safari (latest version)
    • Chrome (latest version)
  • Mac OSX 10.5/10.6
    • Safari (latest version)
    • Firefox 3.5
    • Opera (latest version)

Grade B

These browsers should work acceptably (any unsupported functionality should be dealt with gracefully e.g. JS animations or Flash), look good (any unsupported rendering functionality should be dealt with gracefully e.g. rounded corners, alpha transparency or drop shadows) and perform reasonably well.

  • • Windows XP/Vista
    • IE6
    • Firefox 3
  • Mac OSX 10.5/10.6
    • Safari (previous version)
    • Firefox 3

Grade C

All other browsers should work acceptably (degrading gracefully where appropriate), have no obscured or illegible content and perform acceptably.

Notes

It is important to note that in my list IE6 has been relegated to Grade B (which is at odds with Yahoo!’s graded browser support, which I otherwise use as a base for my list). This is simply due to the development effort to class it as Grade A. Where it is easy to do, I still recommend striving for good aesthetics for IE6 – but (for any version of IE) rounded corners and drop-shadows are often simply not cost-effective. I would implement these with CSS3, and argue that the time saved (and performance increase) by doing this The Right Way gives us enough time to develop new features. I obviously do not expect testers to test every single Grade C browser in existence, but simply to be aware of what does and what does not class as a bug or defect.

Remember, your users do not compare browsers side by side like we do. They have no idea there are inconsistencies, and more often than not simply wouldn’t care even if they did. It is unfortunately impossible for a website to look the same in every browser or environment (without just using a huge image map, and even then a text-only browser like Lynx would not be able to render it) and there is nothing to gain in trying needlessly to overcome this.

Making web service APIs that play nicely

We are part of the way through writing stories for the next Silver Squid project and I thought I would share some of these to do with the basic requests and responses of our web service APIs.

Choosing Data Types

Something that I feel is important, is to allow the user of your APIs to send and receive data in the format of their choice. They may be trying to integrate your system with one that only speaks one language, or they might be better versed in some than others.

With that in mind, our latest project allows the user to set whether they are sending data as a normal form submission (necessary in this instance, to allow the user to send files), as XML or as JSON. We allow these to be set in HTTP headers, so that they are set before the system examines the sent data at all. The story is as follows:

As an API user
I want to be able to send HTTP headers defining my request and response data types
so that I can better integrate the API with my system

Acceptance criteria:

  • M: A requestDataType header can be sent – the values can be: form, xml or json. This should dictate the expected data and default to form: if not provided.
  • S: If requestDataType is set to a different value the error ERR_UNSUPPORTED_REQUESTDATATYPE should be thrown.
  • M: A responseDataType header can be sent – the values can be: xml or json. This should dictate the response data type, and default to xml if not provided.
  • S: If responseDataType is set to a different value the error ERR_UNSUPPORTED_RESPONSEDATATYPE should be thrown (in XML).

I have begun to employ the MoSCoW Method for acceptance criteria priorities. According to Wikipedia – these are defined as:

M
MUST have this.
S
SHOULD have this if at all possible.
C
COULD have this if it does not affect anything else.
W
WON’T have this time but WOULD like in the future.

It is very helpful to have a convention to follow for things like this, and this method seems as good as any other.

Consistent Errors

You’ll see that I have defined the errors that will be thrown, and specifically stated that when the Response Data Type cannot be inferred, to use XML.

I try to always be very specific about the error that will be thrown, and to throw those errors in a consistent manner. It feels natural to me to throw HTTP errors when our API fails, and for a while I tried to always throw the appropriate HTTP error code. I gave up on this, because there just is not always an appropriate error code. Now we always throw a 500 error, and provide a string describing the specific error, in the format requested by the user. The story in this project that defines this is:

As an API user
I want errors to be thrown in a consistent fashion
so that my system can detect these and respond appropriately

Acceptance Criteria:

  • M: When an error is thrown, the system should return an HTTP error code of 500.
  • M: When an error is thrown and responseDataType is set to xml (or missed out and thus defaulted), the error should take the following format:
    <?xml version="1.0" encoding="UTF-8"?>
    <rootNode>
    	<error>ERR_CODE</error>
    </rootNode>
  • M: When an error is thrown and responseDataType is set to json, the error should take the following format:
    {"error":"ERR_CODE"}
  • S: All error codes should begin with ERR_.
  • S: A default error of ERR_UNSPECIFIED_ERROR should be thrown when we can not further isolate the error.

This should make it easy to predict the error formats, and also make them easier to write. Knowing that all errors will begin with “ERR_” allows the user of our API to easily handle for all errors, and add more specific handlers when/if necessary, although checking for an error property or node is enough to detect that an error was thrown.

I hope this has been useful. How do you deal with errors and data types in your APIs?

How to do Agile Properly – Part 2: Design

Part 1 of this series

Designers naturally fight against the Agile process, and it is hard to blame them. The typical process defines that the designer provide little pieces of a design at a time. This doesn’t make sense – a design is a holistic piece of creativity in its own right. Imagine if you were asked to draw a house, but piece-by-piece. First, draw the chimney. Now a window box. Now a hill in the background. Now the front door. It would drive you crazy, and of course, whenever you sketched the next bit in, you would have to change the previous ones to keep the composition sensible.

In practise, trying to shoe-horn the designers into this process ends up (in places I have worked, at least) with the developers being halfway through implementing the design on the stories for the current iteration, and the designers being halfway through changing everything for the next one. I would catch a glimpse of the designer’s screen, and see that everything I had done so far was already out-of-date. This problem would progressively worsen with every iteration. It was disheartening, and inefficient.

This isn’t the designer’s fault. They are not used to the same restrictive processes that developers are, and they shouldn’t be. They need more room to be creative.

Colouring It In

Before any designs can be built, we need to have a good idea of what it is we are building. Brainstorming ideas and mad designs is all well and good – but this should certainly take place before any development begins. That said, whilst the stories are being written, it’s a great idea to be throwing together a few off-the-wall designs, to be treated as prototypes. This gives the designers a chance to work without restriction and really express themselves creatively. Some great ideas can come out of this process.

Once the stories are written, we can move on to wireframes. This is where our design starts to get some structure, and this what our developers will be able to initially start working with. There are lots of tools for creating wireframes – OmniGraffle, Visio, and so on. My tool of choice would definitely have to be Balsamiq Mockups.

It can be very easy to start treating a wireframe as a design. I’ve been in several meetings where it’s been suggested the font is too small, or the colours in the wireframes are wrong and so on. The point of a wireframe is to map out what elements we can expect to have in a design, relationships between pages, and their basic interation and positioning. The wonderful thing about Balsamiq Mockups is the result looks like a sketch. There is no way you could mistake it for a design, and this keeps conversations focussed on the important points. Here is an example I made in less than two minutes!

Balsamiq Mockups

Balsamiq Mockups is available as a standalone app, or as plug-ins to Jira and Confluence (my management tools of choice). It stores data in XML format, and the web-enabled versions of the software have very little disadvantages over the desktop app. I recently chatted with Giacomo ‘Peldi’ Guilizzoni (the company’s founder) over Skype, and he was a lovely and helpful guy. The desktop version is only $79 per user. The plug-ins include licenses for up to 3 desktop versions, and you can get a refund of the difference if you bought the desktop version first!

We’re Good To Go

So, now we have our stories, and our wireframes. It is at this point development would begin. We will concentrate on the design track first. Since we have valid wireframes, and we know what elements to expect on our page, our developers can get on with functionality, and good, clean POSH. The design does not have to exist at this point. I have found, at Silver Squid that developing without design first let’s the developer get the cleanest code, and the designer work without restriction.

Our first story might be:

As a Designer I want to create a base design so that I have a template to build further designs upon

For this the designer should get an idea of the design language of the site, the colour schemes, the typography, and so on. Of course, this is Agile and as such it can all change, but having somewhere to start is A Good Thing. It should be based upon the prototypes they built in the planning stage, which were hopefully well received!

Following that, stories could be something like:

As a Developer I want a design for navigation to exist so that I can apply it to the HTML to make the site more attractive

This means that the designer focusses on the navigation. The wireframes should have included basic expected behaviour – whether the navigation is dropdown, or static, etc. The acceptance criteria will list the expected states of the navigation, to ensure all scenarios are covered. For example:

  • Navigation Item
  • Active Navigation Item
  • Hovered Navigation Item
  • Navigation Item with Subnavigation (contracted)
  • Navigation Item with Subnavigation (expanded)
  • Subnavigation Item
  • Active Subnavigation Item
  • Hovered Subnavigation Item

There are a lot of potential states in a even slightly complex web app. Making sure these are all covered in the acceptance criteria leaves little room for error later. We should also allow in each story for amendments to the rest of the design to keep everything feeling coherent.

Don’t Forget Interaction & Animation

The latest and greatest web apps have sliding menus, fading modal dialogue boxes, pulsing buttons etc. Any dynamic movement like this will not come through in your static Photoshop files. It is important that these are decided by a qualified designer or UX guy, and that he works with the front-end developers to ensure it works as he expects it to. As much as possible should be noted along-side marked up designs. Often these touches get forgotten and the front-enders make it up themselves, which can be a huge mess of sliding, throbbing, fading, scaling madness.

As I’ve said before, these are practises that work for me! Everyone is different. How do you implement design in your Agile process?

In part 3, we will start breaking down the development process into its component parts, and show how the team works together to get things done.

How to do Agile Properly – Part 1: Preparation with Stories

Agile is A Good Thing. It gives us room to be flexible with our projects, and when implemented right can be a genuinely much more productive way to work. Even so, I see so many overspills from legacy processes that just hold us back, and people are reluctant to change. Today we have tools available that go so much further than “Track Changes” in a Word Document.

In this series, I will try to give my ideas about how an Agile project should work, from start to finish. It won’t be perfect (everyone does it differently) and my terminology might not match yours, but I hope it’s helpful. If you have comments, please, let me know. After all – even the process of Agile is agile itself!

Everything Starts With Stories

The first thing we need is a plan for our project: what we want it to do, how it should work and what we want to achieve.

I have made the assumption you already have a good idea of how your project will work – this is not a series about getting ideas or brainstorming.

For the sake of example, we’ll say we want to make a website that sells shoes. So, we’ve got a warehouse full of shoes. We’ve got people ready to send them off, and we’ve got a bank account to take the cash. But what is it we actually need to do to sell them online?

Stories (or User Stories) are a nice way of formalising our requirements. We just need a sentence, formed like:

As a(n) A I want (to) B so that C.

A is the person we are creating this requirement for, usually the User, but could also be the Developer, the Product Owner, the Tester, etc. It’s good to have this list formalised before-hand, so we’ll go with:

  • User
  • Developer
  • Product Owner

B is the actual requirement. This doesn’t need to be too specific; we capture the specifics in Acceptance Critera.
C is our justification for this requirement.

Here’s a good example:

As a User
I want to add a pair of shoes to my shopping basket
So that I can buy these shoes when I check-out.

This assumes we already have a page showing us shoes, but no way yet to buy them. Without yet going into the technicalities, we can now write Acceptance Critera.

Acceptance Criteria are a list of tasks which will either pass or fail, and inform us as to whether or not we have satisfied this story. These serve as a quick checklist to make sure the story was completed. We can prefix these with priorities, so that we know how imporant they are. Here’s a typical list of priorities:

  • Must – this criteria must be satisfied or we cannot release this project.
  • Should – this criteria should be satisfied, but we can release the project without it, and address it in the future.
  • Could – this criteria could be satisfied (it is nice-to-have) but we can release the project without it, and may never implement it if more important things come up.

With these in mind, we can now write acceptance critera for this story:

  • Must – An “Add to Basket” button is displayed alongside each pair of shoes
  • Must – When clicking the button, the pair is added to the basket
  • Should – The basket should visibly update immediately
  • Could – An animation could give a visual indicatation that the pair has been added to the basket

I recommend writing a “Project Story” so that you never lose sight of exactly what your project is supposed to achieve, before you get into the fine details. An example of our Project Story might be:

As a User
I want to browse through a catalogue of shoes and purchase them
So that I can satisfy my desires to be fashionable or protect my feet in various environments.

Acceptance Critera:

  • Must – The user can browse a catalogue of shoes
  • Must – The user can purchase any of these pairs of shoes

Now we’re in a good place to write the rest of the stories. We should cover how the browsing of the catalogue works, the checkout process, how we cope with items that are out of stock. We should also deal with Non-Functional Requirements like the kind of response time we expect for our site, and how much traffic it should be able to handle. At the end of this story-writing (aside from feeling like a novelist), we should have a comprehensive list of stories that detail what we can expect from our project.

Tools

We need ways to capture all of these stories, and manage them as we progress. Once upon a time, we would use Excel or Word, but that gives us a static file and isn’t great for sharing or collaboration. I would recommend using a wiki to achieve this. MediaWiki is a great option; it’s free and open source, and forms the base for Wikipedia. If you’ve got some cash splashing around though, the best option (I think) is Confluence. At $1200 for up to 25 users it’s not unreasonably priced, and supports all kinds of clever plug-ins and so on. It even has a pretty good WYSIWYG editor, so that you don’t terrify the business with mark-up. For a lot of this series, I will assume you are using Confluence, but will try to make it clear where functionality is Confluence-specific.

Once you have these stories in your wiki you can share them with the rest of the team, and bounce them back and forward until they seem pretty solid, and everyone has a good understanding of how the project will work from the user’s perspective.

In part 2, I will go into how the design team get involved, and look at wireframes, and designs.