Live from Tallinn!! Nordic Testing Days – Day 1 (Rhythm of Testing)

On June 5, 2014, in Syndicated, by Association for Software Testing

Thursday, June 5 came earlier than expected.  It is AMAZING how bright the sun is at 04:00 – yeah – 4:00 AM!  MAN!  Good thing the curtains were closed!  Normally that is not too big of a deal particularly as I’m an early riser.  However, given the full day of tutorial on Wednesday, then a lengthy, leisurely Estonian dinner, followed by “a little more beer” at the conference center bar.

We are off and running!

Lean Coffee with a small but doughty gathering met to discuss items from Exploratory Regression testing, ideas from the Lean Software Testing tutorial Matt Heusser and I did yesterday. 
The conference is organized by volunteers – this is the third year – Yeah – 3 years and it is looking really, really good!
And now, we are starting the fun of Keynotes!

The opening keynote is by Jevgeni Kabanov on Testing in the Automation Age.

And here we go!

Jevgeni is a doctor – like PhD – and is the founder of GeekOut! another conference in Estonia.  He is also the Founder and CEO of ZeroTurnaround.

Now, for my American colleagues, Estonia is a country of 1.3 Million people in Eastern Europe – like waaaaay Eastern Europe.  The neighbors to the East are Russia – North, beyond the Bay of Finland is… Finland.  There are ferries that run between Tallinn, the capitol city of Estonia, and Helsinki, Finland.  Its a cool city with loads of medieval buildings, castles/city walls, churches and wifi – like wifi EVERYwhere.

And the Keynote begins –

ZeroTurnaround started with a single product – JRebel – which runs for Java, and provides continuous testing availability for Java developers.  The point is to speed the process that developers go through to deply to testing and make life easier for everyone.  Their estimation is that this product can save teams 1 month of effort a year, by making life easier.

They added LiveRebel a bit later.  LiveRebel provides Release Management for Continuous Delivery.  It allows for testing of releases and, if bad things happen, can rollback in the case of failure.

Then XRebel allows for interactive Java profiling/  This allows developers (and others) to see/understand what is happening behind the scenes. 

And in the Cost of Quality slide (following the XRebel summary) – the image is an atomic explosion.  Cool.

When things don’t work, you risk losing trust of your customers.  If your software fails, or generally does not deliver, then there are problems  Like – people want their money back.   Like people walk away.  Like people walk away and don’t come back and tell people your software… stinks.

To help them manage their quality, they automated testing.  They support 77 environments, 1,428 test suites and some 219,912 test results.  (OK. let’s see where this goes…)  They are running a “team” of 27 “machines” (servers?), 3,132 unique job configurations and runs 294 jobs/hour – which gives 8+K jobs a day and… yeah – you get the picture.

They are running Jenkins, Chef, Vagrant on Amazon EC2.  Oh, and a scripting language called Groovy.  The issue they have is that Amazon tends to get annoyed with them if they don’t give a “heads up” at least, or ask permission, to use more than a few thousand servers at a time.  Ummm – yeah – that is sucking up a fair portion of Amazon’s resource.

This gives them the ability to run significant numbers of “Acceptance tests” which they define as tests that fails before the release and succeeds/passes, after the release.  They run these nightly .

Captain Obvious says “Everybody Tests:Test Everything.”

(OK – that sets up some interesting point/counter-point for the closing keynote for the day “On Complete Testing” with Matt Heusser and… well, me.)

The thing is, per Jevgeni, for them – engineers spend 30 – 50% of their time writing tests. Since Functionality is seen as a liability, tests help to counter it.  This is an important point.  We can catch/trap items before they go live by running a massive array of test scenario combinations.

For them, “Everybody Tests” includes developers, sysadmins, customers and product managers.   And he makes an awesome point on his “CEO Tests” – in that people who don’t use the software all the time (or at least regularly) will sometimes find interesting and significant bugs by not following the rules.

Now – “Test Everything” – this includes challenging specs, considering usability, releases (obvious, no?) and processes.  Then there is something really interesting.  That is “test the business model” – does this make sense for us to do?  This is something that many organizations can learn something from!

One aspect to consider – testers to go undercover as customers, put in support requests instead of bugs – suddenly devs take it very seriously.  (Me: a little sneaky, but I get the gist.) 

Founders are often generalists – they are “equally bad at everything” – which is maybe more honest than the “equally good at everything” view that people often spout.  The consideration is that people will get as far as they can without a specialist.  By the time they realize they need one, they are probably a year or so behind the curve.  That is, you often find yourself in trouble and realize you need someone who is an expert/specialist later than you should have brought them in.

Two reasons to hire QA guys: productivity and pro-activity. Testers who are trained as testers and have specialized in testing tend to look at things differently than those who are not trained as testers.  (Me: less confirmation bias?)

The final point he makes is that the people around us are the type companies need.  When they started out, he took the view of automation will take care of things – now he needs testing specialists.

DING!  Time is up – and into question/answers –

Q/A – Interesting question – What is the ratio of the work your developers put in to writing test automation vs writing production code?  The answer was that all developers do both.  The breakdown is that roughly 30% to 50% of each developer’s time/effort will be to write automation code – the balance is writing production code.

A: (OK, so I missed the question, but the answer was a really good point) – For us the QA guys are part of the team – they participate in the same standups, they work with the developers, they learn how the developers think and they look at things differently.


I’ll be diving in to the Exploratory Testing Dojo being run by Huib Schoots.  The bummer of this for me is that Dan Billing will be presenting on Security Testing at the same time.  BUMMER!  I wanted to hear that one as well.  (Note – for me, that is the sign of a good conference program – where I find myself torn between which session I’m going to next.)

OH! I ran into Stephen Janaway – who is ALSO live blogging –

Right – so Huib has a MASSIVELY full room to learn about Exploratory Testing – in his ET Dojo.  I mean like – he has a full room – all the seats taken and a bunch of folks in chairs lined against the back wall.  He is using LeanKit as the tool – (available here: ) and it is available at the appstore and google play.

AND – with a room full of people exercising apps via wireless – the connection slows waaaaaaaaaaaaay down.  bummer.  And Huib wants me to not say nasty things about him…

Right – Huib is making a crucial point – TAKE NOTES!  LOTS OF NOTES! 

If you can’t track what you did, how will you remember when the boss asks you at the end of the day – or when you’re getting coffee in the middle of the day – “What did you do?”  You may be able to have a general idea, but specifics?  Probably not. 

ME:  This is where most people fail with ET in my experience!!!!  People don’t take notes detailed enough to be able to recreate what they did – EXACTLY – they can do it generally, but to be able to describe WHAT THEY DID?  Usually not.  It comes off as wishy-washy, and not very understandable. 


Comments are closed.

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!