At the practical exam for the Certified Agile Tester, there are 7 product types in which you need to show your mastery. Before my CAT exam I took some time to reflect on the essence of each of the products and how I could best show that I know how to test in an agile team. In this article I’ll list my notes on the 7 things to show your skills at the practical CAT exam. I strongly suggest that you take some time to do that as well.
- Sprint Plan
- Test Strategy
- Task Board
- Sprint Burn-down Chart
- Session Sheets
- Sprint Review
- Sprint Retrospective
When you’re done reading through my notes, please leave a comment about how you will deliver your favourite .
Because there are seven product types, it doesn’t mean that you’ll pass if you only deliver seven – it is said that it has been done, but whoever made it with only one of each must have been very exact and to the point. Remember that the more content you deliver, the higher your chances are for success because what ever rubbish you deliver just doesn’t count. You can never lose marks only gain them, so train your handwriting and produce as much as you can.
1. Sprint Plan
This is the first thing I did at the exam – even while the instructor and the invigilator where setting up the computers, I was scribbling away just to get it out of my head.
At the practical exam the SUT simulation is delivered in three drops that all need initial testing, regression testing, and eventually some re-testing of the defects that you will find. Apart from that you’ll spend your time on planning, analysing the material, and creating the products that you’ll deliver (i.e. each of these seven products)
It is crucial to set time boxes and to be strict on yourself to uphold them. Start practicing now: how much time will you spend on reading and commenting on this article? I found that my time scheduling slipped when testing the drops, so in retrospective, I’d like to have made smaller time boxes than I initially did (have a look at my initial plan below).
First I thought it was okay to slack a bit on the time boxes in the testing time, but it turned out to be where I needed to be the most strict.
Because my native language is Danish, I had more time for the exam because it was in English, so my sprint plan looked like this:
30 min: Planning
* Test Strategy
* Task board
* Initial Burndown
30 min: Drop 1
30 min: Drop 2
* Regression & Re-Testing
30 min: Drop 3
* Regression & Re-Testing
10 min: Review
* List defects (before the exam I planned to list passed and failed stories, but I ended up listing > those at the Sprint Retrospective)
20 min: Retrospective
* Listed the passed and failed stories. I.e. I was behind my planning so I didn’t get to test the third story properly. I compensated by explaining what parts of it that I did test and that it would go into the next sprint in a real life situation. (Before the exam I really did plan to explain how to mitigate any of the types of failures that I’ve seen in the exam.)
How much detail will you put into your Sprint Plan and how will you practice and enforce the time boxes that you set for the tasks? Share your thoughts in the comments.
2. Test Strategy
Start by making a bare-bone strategy with only the most basic statements that describe how you will test throughout the practical exam.
After each test session, or after you’re done testing a drop, take a minute to update the strategy by adding any methods that you’ve used during the session. Also make sure to take notice of which methods that you added earlier and have not used yet. This will be valuable information for the Sprint Retrospective later on.
These are examples on the basic methods that I wanted to use for my exam. Be sure to use your own favourites, and go through those listed in the manual, then take a look at the stories that you will need to test and add any that will make sense in your specific exam situation.
- Risk assessment: how to start with user stories that has the highest business value or user stories that have other depending stories (this one didn’t make sense in my situation because each drop had a specific story and the order was predefined with the highest business value first. It would however make a lot of sense to include this statement in a real world test strategy)
How tests will be conducted: which test activities, types, techniques, regression testing, re-testing, testing levels, etc.
- Exploratory testing
- Regression testing after each new drop
- Re-testing of defects reported in earlier drops
- A short description of the use of session sheets for documenting the test sessions (more why than how)
- How acceptance criteria are used to design tests that the Definition of Done (DoD) include that all acceptance criteria must pass before a user story can be accepted at the sprint review (I really struggled with this at the exam, because all of the stories had some AC that didn’t make it. My conclusion is that the exercise here was to make a DoD that made room for technical and testing debt and to set a threshold for what was acceptable for the product owner)
- That the tester will inspect both the front-end (UI) and the back-end (database)
- That any defects will be reported on the session sheets (i.e. not on separate sheets)
Take a look in the manual on what else a test strategy should contain. These notes are reflecting my style and I am looking forward to reading about yours in the comments.
3. Task Board
At the practical exam, there are no real opportunity to use a task board, so you are only asked to make a sketch of one. Make sure to clearly show how it would work in a real situation and include a legend and any additional explanations that you need to clearly explain how your Task Board would be used.
Because it is not actively used, it only eats your time , so you should be very careful not to spend too much time on the sketch.
What does your favourite Task Board look like? In what genius ways have your team’s task board evolved over time?
4. Sprint Burn-down Chart
Compared to the Task Board, the Sprint Burn-down Chart can actually be used, so here are my notes on how I did it at the exam and what I had planned.
In my first attempt at the exam, I found so many severe defects that I didn’t feel I could pass any of them. In hindsight, I should have passed some of the defects on to “the next sprint” as testing debt and passed the (some of the) stories with notes on that in the retrospective. At my second attempt, I only passed two out of three stories and my Burn-down Chart was more interesting to look at and not just a flat-line.
In your planning you will give each of the stories story points on a given scale (in both my cases it was 1 through 6), so you will simply plot the initial sum of the points at drop zero and draw a line for your expected velocity – i.e. when you have estimated the stories and create the Sprint Burn-down Chart. As you finish each drop, plot your progress in the chart.
5. Session Sheets
The Session Sheets are used throughout the practical exam. The difference from the rest of the products is that there is a special way to name your testing sheets so they can be traced to the user stories. Be sure to check the syntax in the practical exercises that you’ll get at the course.
At the exam I also got copies of the user stories with a field for my notes. I was not entirely sure what to do with it, so I simply made some notes related to the acceptance criteria that were listed on the story card.
Because the Session Sheets are the core products of the practical exam, I set up some rules for myself before the exam:
- Refer directly to Acceptance Criteria and make notes in the actions taken to test them.
- Not all AC are fulfilled in one code drop, so be specific on which AC are and can be tested in a certain session/drop.
- Name defects according to AC and any other odd behaviour should be noted as Issues (there are a field for defects and a field for issues below a larger field for session notes on each session sheet)
Make a note in the strategy that any AC that are not mentioned is passed.I crossed this out and made a note that the danger of this is that the AC will too easily be forgotten. Instead: Note which AC that are tested and which are left for later drops – even if they are so simple as “The site can be accessed”.
- When AC pass, create a regression test sheet per story per user story in Drop 2 and 3. On them make a list of the passed AC and check them off as they pass your regression test.
- If there is a regression, make a specific note on it and note a defect. This actually happened in my second drop. In my first attempt at the exam, I didn’t make any specific regression tests, so I hope this makes a difference.
- Only create regression test sheets for AC that previously passed and are not implicitly tested with other AC that specifically belong to the current drop. E.g. if an AC says “The site can be accessed”, it will be implicitly be regression tested with any new testing that you do.
The point of my rules was to be specific about what to make note of and what not to waste time on. In essence, my rules reflect my weaknesses and act as a declaration towards addressing those. Make sure to reflect on your own weaknesses and how you’ll keep them in check during the exam.
In the comments, share the one most important rule that you will set for yourself when taking session notes?
6. Sprint Review
As you can see in my Sprint Plan, the Sprint Review has the smallest time box. All I found necessary was to list the unfixed defects, related to the user stories. Then I added a note simulating a product owner’s decision taken on the customer’s behalf.
Any defects not directly related to acceptance criteria, but related to the additional requirement specifications will also be listed here if they were not fixed. Don’t add any details about the defects. I didn’t even write out the descriptions/titles of the defects, just the IDs that were also listed on the session sheets.
Bear in mind that the Sprint Review should simulate a demonstration of what a team has a accomplished during a sprint, so it is an honest report on what happened and what was delivered. The core value to both the customer and the team, in relation to the collaboration, is the honesty because it makes it clearer what can be expected in the next sprint. The same holds true at the exam where you need to show your professionalism regarding the state of the SUT and that you have done what was in your power, as the tester, to contribute to the product.
As with the other topics, make sure to read the manual and take some time to reflect on how your sprint review will turn out at the exam. If you already have a clear idea, put it in the comments.
7. Sprint Retrospective
The sprint retrospective sheet contain notes about how the process can be improved in the future. In the exam situation, it will should give rational ideas on how your behaviour in the exam process was and how it could be improved.
The retrospective can explain symptoms on what the simulated team can improve. E.g. if there are many security related defects, you can suggest to bring on a security expert to test the product. Likewise with any performance and user experience issues or defects you may find.
If you can see that a majority of the defects could have been prevented with code reviews, unit tests, clearer user stories and acceptance criteria, the Sprint Retrospective is where you should address those.
Most important is to address all the issues that are noted on the session sheets during the tests – and make sure to make notes on any issues like these, on the session sheets, when you find the issues.
Explain which issues are most important to the team and the customer and how the team would mitigate these.
Issues are e.g. if stories are too big, label them as epics and break them down into smaller stories. If acceptance criteria are unclear, explain that the testers should pair up with the product owner to make them testable.
If you have an additional example, post it in the comments and explain what a symptom could be and how the issue can be a problem in the future.