Blog

RTI Report Report #2 (James Bach’s Blog)

On February 11, 2013, in Syndicated, by Association for Software Testing
0

There’s a reason I haven’t been blogging much. Every time I want to blog about something I remember that I should be working on finishing my report from the first Rapid Testing Intensive. Lots of reasons that’s been delayed (lots of work on the road plus my father being almost dead for weeks temporarily destroyed my concentration), but now I have something to say about it. There’s been progress.

The Rapid Testing Intensive Onsite happened in July on Orcas Island. This training event involved more than 100 testers, 20 of which were here on Orcas Island, and 80+ scattered around the world. We decided afterward that it was overambitious to do an onsite/online simultaneous seminar, but we did collect a lot of student work and other data. One of the things I promised was to make this public. A grand test report.

Turns out that’s a lot of work. I wrote software last week to analyze chatroom traffic (to help study how chatrooms might have helped remote testers work together). I also wrote code to extract comments and hundreds of attachments from our Confluence archive in order to gather the raw data for reporting. We are chipping away at the problem. Actually we did another RTI online just recently– a smaller scale event– and the report from that has already been delivered to the students.

In the process I found a bug in Confluence. Atlassian nicely allows you to backup your entire Confluence archive. But then its servers won’t let you actually download it, because they automatically terminate long downloads. Then when it terminated, I couldn’t try again because the download link disappeared and the system informed me that I could only have one download per day. I don’t know how any reasonable sized customer has ever been able to download their archive without using special tools. Their customer service people have offered to slice up the zip files, next time, but what I don’t understand is why they are offering a workaround to this instead of just fixing the problem. It’s a pretty important feature to get your backup.

Despite my troubles with Confluence, we are pretty happy with it– that’s down to great customer service.

Chatroom Analysis

Part of why I want to do testing intensives is to collect empirical data on exploratory testing. For instance, I’m curious about how a chatroom might help a group of remote testers collaborate. So we used HipChat for communication– an Atlassian product I am quite happy with– then mined and fixed up the message archive (more than 10,000 messages I extracted and reformatted and in some cases tweaked by hand). I then used NodeXL from Microsoft (yet another unreliable tool with wonderful capabilities from the boys in Redmond) to perform a network analysis of the directed chat messages.

The lines represent directed messages (typing in a chat room but using a person’s handle to get their attention). The fuchsia lines represent communication between students. Gray lines are communications to or from the staff. Blue nodes are teaching staff; green is admin; and black are students. From this I can see that there was a huge variation among the students in how they used the chatrooms. The offsite students used it a lot more, in general, as I would expect. A few students were heavy talkers, connecting with a lot of other students. Based on this analysis, I’ve invited Griffin Jones to be a peer advisor at the next RTI, since he was one of the most active talkers, helping to create a good social atmosphere.

I would really like to do a semantic analysis by tagging each message with a category. Were people being generally social or were they actually talking about testing? That analysis may have to wait, but at least we have the data.

 

Someone asked for a word cloud. I don’t find those very interesting, but here it is anyway.

 

 

 

 

Tagged with:
 

Comments are closed.


Looking for something?

Use the form below to search the site:


Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!