CAST 2016



“Your request for a grant from the AST has been approved!”, Speechless. Even now as I am writing this, I don’t quite know how to describe it. Happiness? No, it had to have been something far better than that. I’ll get back to you on that one. As an entry level software tester, CAST 2016 is the top of the line if you want to learn and connect with the leading minds in the industry. It’s like telling an aspiring singer that they are going to be given the opportunity to meet their favorite artists/composers. Fast forwarding 168 hours later and 2,988 miles away from home I found myself in Vancouver where the “Lollapalooza of Software Testing” was to take place. According to the schedule my day began with “Lean Coffee”. As a frequent customer at Starbucks I knew what a skinny mocha was but “Lean Coffee” was a new one for me and I wasn’t sure what to expect. I guess you can say that from the very moment I entered Simon Fraser University I was learning new things. So what did “Lean Coffee” turn out to be? Well it was a concept I was familiar with, however this would be the first time I would be putting this into practice as part of a team. Similar to agile testing, a group of a dozen or so people gathered around a collaborative chart that organized the topics they wanted to discuss. It resembled the average person’s to-do list. One of the topics that was brought up was how folks were going to communicate after the conference and someone said something which I totally agree with which was that communication is best when it’s in person. The very idea of going to the conference was out of my comfort zone; however, I don’t regret a minute of it. It was an amazing experience!










I’d say the overall theme amongst both tutorials that I attended was that one has options. In the first lecture, Mr. Bach started by explaining the core values and meaning of testing. He then went on to talk about what we considered to be difficult to test. I brought up the idea of color and how individual perception of color can impede testing as looking for identifying factors can differ from person to person. Is it a problem? The curveball question. A war of technicality. Yes, the purpose of testing is to find problems, however, the inability to identify colors is ironically in the “grey zone”. It can be a problem but that depends on one’s interpretation. In the second lecture, Ms. Charles shaped the discussion so that the attendees took control. People were given the opportunity to seek advice from fellow testers by speaking about situations where they’ve wanted to say no. One story in particular stuck out to me which was a case of skills becoming a burden. In this story the individual found that co workers relied too much on said person’s generosity and willingness to learn things outside of the job description. The advice for this situation in particular ranged from a “don’t continue to do this” to a “speak with your superiors”.














Now let’s talk about the keynotes that took place throughout the CAST conference. Day 2 was comprised of two conferences. The opening keynote speaker was Nicholas Carr. He raised the subject about how humans have become far too dependent on automation. A plane crash is how Mr. Carr chose to illustrate his view. One small error in the pilot software can cause a pilot to go into a frenzy because the individual may not be fully capable of controlling the plane under certain stressful conditions. Conversely, his next point was about human error, “When our brain is overtaxed we find distractions more distracting”. In essence, Mr. Carr proposed a wise new division of labor that does not seek replacement but a partnership between humans and computers. The following keynote on that Tuesday was conducted by Anne Marie Charrett who presented ways to change testing environments; not the software or hardware aspect, if not the setting in which testers work. An interesting diagram was displayed during this presentation which laid out the fundamentals of this unique test management approach. The diagram seemed to consist of personality and action guidelines. The next day featured a keynote by Sallyann Freudenberg which had quite the impact with the audience. It addressed a pressing issue, employment for those with special needs in the IT field. Ms. Freudenberg started off by sharing that she found herself to be somewhere on the autistic spectrum. Perhaps the most impactful part of this keynote was when the signs and symptoms of autism were shown because it made the audience wonder about the possibility of being somewhere on the spectrum as well. It was also quite intriguing to hear about the qualities of an autistic perspective in IT, some of which included keenness for detail, a more spacial take on tasks, and the ability to complete repetitive tasks.





Prior to the CAST conference, attendees were given the choice to create a schedule for the sessions they wanted to attend. As a novice conference-goer I decided upon an impromptu approach. Choosing between these sessions was rather difficult because it was as though I had been given a menu that only offered top-notch cuisine. “What developers taught me about testing” was the session that I thought was the most appealing out of the great lineup of sessions. Ms. Charrett spoke about how to open communication with developers in a seemingly hostile environment where everyone is obsessed with their work. She taught us how create a safety net by building up trust and relationships. This made me recall a quote by A. Den Heaver that she had referenced during her keynote which said, “When a flower doesn’t bloom you fix the environment in which it grows, not the flower”.


Remember that word I was looking for in the beginning? After giving it much thought, I decided that the most appropriate word was Ecstatic. What better word to describe how I felt after being given the opportunity to attend CAST 2016. Anyhow, time for overall impressions. Across the board I found that the conference’s content was very diverse. Concepts were presented from different angles, that allowed speakers to be both broad and deep in their approaches. Some took the approach focusing on personality and individual initiative whereas others gave presentations that stressed a methodical progression. What I took from this conference was not only technical changes for software testing but also the importance of collaboration skills. As I told Mr. Davis at the conference, I look forward to using what I learned in my future endeavors.

Jonathan Guzman

CAST 2016



“Your request for a grant from the AST has been approved!”, Speechless. Even now as I am writing this, I don’t quite know how to describe it. Happiness? No, it had to have been something far better than that. I’ll get back to you on that one. As an entry level software tester, CAST 2016 is the top of the line if you want to learn and connect with the leading minds in the industry. It’s like telling an aspiring singer that they are going to be given the opportunity to meet their favorite artists/composers. Fast forwarding 168 hours later and 2,988 miles away from home I found myself in Vancouver where the “Lollapalooza of Software Testing” was to take place. According to the schedule my day began with “Lean Coffee”. As a frequent customer at Starbucks I knew what a skinny mocha was but “Lean Coffee” was a new one for me and I wasn’t sure what to expect. I guess you can say that from the very moment I entered Simon Fraser University I was learning new things. So what did “Lean Coffee” turn out to be? Well it was a concept I was familiar with, however this would be the first time I would be putting this into practice as part of a team. Similar to agile testing, a group of a dozen or so people gathered around a collaborative chart that organized the topics they wanted to discuss. It resembled the average person’s to-do list. One of the topics that was brought up was how folks were going to communicate after the conference and someone said something which I totally agree with which was that communication is best when it’s in person. The very idea of going to the conference was out of my comfort zone; however, I don’t regret a minute of it. It was an amazing experience!










I’d say the overall theme amongst both tutorials that I attended was that one has options. In the first lecture, Mr. Bach started by explaining the core values and meaning of testing. He then went on to talk about what we considered to be difficult to test. I brought up the idea of color and how individual perception of color can impede testing as looking for identifying factors can differ from person to person. Is it a problem? The curveball question. A war of technicality. Yes, the purpose of testing is to find problems, however, the inability to identify colors is ironically in the “grey zone”. It can be a problem but that depends on one’s interpretation. In the second lecture, Ms. Charles shaped the discussion so that the attendees took control. People were given the opportunity to seek advice from fellow testers by speaking about situations where they’ve wanted to say no. One story in particular stuck out to me which was a case of skills becoming a burden. In this story the individual found that co workers relied too much on said person’s generosity and willingness to learn things outside of the job description. The advice for this situation in particular ranged from a “don’t continue to do this” to a “speak with your superiors”.














Now let’s talk about the keynotes that took place throughout the CAST conference. Day 2 was comprised of two conferences. The opening keynote speaker was Nicholas Carr. He raised the subject about how humans have become far too dependent on automation. A plane crash is how Mr. Carr chose to illustrate his view. One small error in the pilot software can cause a pilot to go into a frenzy because
the individual may not be fully capable of controlling the plane under certain stressful conditions. Conversely, his next point was about human error, “When our brain is overtaxed we find distractions more distracting”. In essence, Mr. Carr proposed a wise new division of labor that does not seek replacement but a partnership between humans and computers. The following keynote on that Tuesday was conducted by Anne Marie Charrett who presented ways to change testing environments; not the software or hardware aspect, if not the setting in which testers work. An interesting diagram was displayed during this presentation which laid out the fundamentals of this unique test management approach. The diagram seemed to consist of personality and action guidelines. The next day featured a keynote by Sallyann Freudenberg which had quite the impact with the audience. It addressed a pressing issue, employment for those with special needs in the IT field. Ms. Freudenberg started off by sharing that she found herself to be somewhere on the autistic spectrum. Perhaps the most impactful part of this keynote was when the signs and symptoms of autism were shown because it made the audience wonder about the possibility of being somewhere on the spectrum as well. It was also quite intriguing to hear about the qualities of an autistic perspective in IT, some of which included keenness for detail, a more spacial take on tasks, and the ability to complete repetitive tasks.





Prior to the CAST conference, attendees were given the choice to create a schedule for the sessions they wanted to attend. As a novice conference-goer I decided upon an impromptu approach. Choosing between these sessions was rather difficult because it was as though I had been given a menu that only offered top-notch cuisine. “What developers taught me about testing” was the session that I thought was the most appealing out of the great lineup of sessions. Ms. Charrett spoke about how to open communication with developers in a seemingly hostile environment where everyone is obsessed with their work. She taught us how create a safety net by building up trust and relationships. This made me recall a quote by A. Den Heaver that she had referenced during her keynote which said, “When a flower doesn’t bloom you fix the environment in which it grows, not the flower”.


Remember that word I was looking for in the beginning? After giving it much thought, I decided that the most appropriate word was Ecstatic. What better word to describe how I felt after being given the opportunity to attend CAST 2016. Anyhow, time for overall impressions. Across the board I found that the conference’s content was very diverse. Concepts were presented from different angles, that allowed speakers to be both broad and deep in their approaches. Some took the approach focusing on personality and individual initiative whereas others gave presentations that stressed a methodical progression. What I took from this conference was not only technical changes for software testing but also the importance of collaboration skills. As I told Mr. Davis at the conference, I look forward to using what I learned in my future endeavors.

Jonathan Guzman

Continuous Delivery within the .NET realm

Continuous what?

Well, if you browse the internet regularly, you will encounter two different terms that are used rather inconsistently: Continuous Delivery and Continuous Deployment. In my words, Continuous Delivery is a collection of various techniques, principles and tools that allow you to deploy a system into production with a single press of a button. Continuous Deployment takes that to the next level by completely automating the process of putting some code changes that were committed to source control into production, all without human intervention. These concepts are not trivial to implement and involve both technological innovations as well some serious organizational changes. In most projects involving the introduction of Continuous Delivery, an entire cultural shift is needed. This requires some great communication and coaching skills. But sometimes it helps to build trust within the organization by showing the power of technology. So let me use this post to highlight some tools and techniques that I use myself. 

What do you need?
As I mentioned, Continuous Delivery involves a lot more than just development effort. Nonetheless, these are a few of the practices I believe you need to be successful.

  • As much of your production code as possible must be covered by automated unit tests. One of the most difficult part of that is to determine the right scope of those tests. Practicing Test Driven Development (TDD), a test-first design methodology, can really help you with this. After trying both traditional unit testing as well as TDD, I can tell you that is really hard to add maintainable and fast unit tests after you’ve written your code.
  • If your system consists of multiple distributed subsystems that can only be tested after they’ve been deployed, then I would strongly recommend investing in acceptance tests. These ‘end-to-end’ tests should cover a single subsystem and use test stubs to simulate the interaction with the other systems.
  • Any manual testing should be banned. Period. Obviously I realize that this isn’t always possible due to legacy reasons. So if you can’t do that for certain parts of the system, document which part and do a short analysis on what is blocking you.
  • A release strategy as well as a branching strategy are crucial. Such a strategy defines the rules for shipping (pre-)releases, how to deal with hot-fixes, when to apply labels what version numbering schema to use.
  • Build artifacts such as DLLs or NuGet packages should be versioned automatically without the involvement of any development effort.
  • During the deployment, the administrator often has to tweak web/app.config settings such as database connections strings and other infrastructure-specific settings. This has to be automated as well, preferably by parametrizing deployment builds.
  • Build processes, if they exist at all, are quite often tightly integrated with build engines like Microsoft’s Team Build or JetBrain’s Team City. But many developers forget that the build script changes almost as often as the code itself. So in my opinion, the build script itself should be part of the same branching strategy that governs the code and be independent of the build product. This allows you to commit any changes needed to the build script together with the actual feature. An extra benefit of this approach is that developers can test the build process locally.
  • Nobody is more loathed by developers than DBAs. A DBA that needs to manually review and apply database schema changes is a frustrating bottleneck that makes true agile development impossible. Instead, use a technique where the system uses metadata to automatically update the database schema during the deployment.

What tools are available for this?

Within the .NET open-source community a lot of projects have emerged that have revolutionized the way we build software.

  • OWIN is an open standard to build components that expose some kind of HTTP end-point and that can be hosted everywhere. WebAPI, RavenDB and ASP.NET Core MVC are all OWIN based, which means you can build NuGet packages that expose HTTP APIs and host them in IIS, a Windows Service or even a unit test without the need to open a port at all. Since you have full control of the internal HTTP pipeline you can even add code to simulate network connectivity issues or high-latency networks.
  • Git is much more than a version control system. It changes the way developers work at a fundamental level. Many of the more recent tools such as those for automatic versioning and generating release notes have been made possible by Git. Git even triggered de-facto release strategies such as GitFlow and GitHubFlow that directly align with Continuous Delivery and Continuous Deployment. In addition to that, online services like GitHub and Visual Studio Team Services add concepts like Pull Requests that are crucial for scaling software development departments.
  • XUnit is a parallel executing unit test framework that will help you build software that runs well in highly concurrent systems. Just try to convert existing unit tests built using more traditional test frameworks like MSTest or Nunit to Xunit. It’ll surface all kinds of concurrency issues that you normally wouldn’t detect until you run your system in production under a high load.
  • Although manual testing of web applications should be minimized and superseded by JavaScript unit tests using Jasmine, you cannot entirely get rid of a couple of automated tests. These smoke tests can really help you to get a good feeling of the overall end-to-end behavior of the system. If this involves automated tests against a browser and you’ve build them using the Selenium UI automation framework, then BrowserStack would be the recommended online service. It allows you to test your web application against various browser versions and provides excellent diagnostic capabilities.
  • Composing complex systems from small components maintained by individual teams has been proven to be a very successful approach for scaling software development. MyGet offers (mostly free) online NuGet-based services that promotes teams to build, maintain and release their own components and libraries and distribute using NuGet, all governed by their own release calendar. In my opinion, this is a crucial part of preventing a monolith.
  • PSake is a PowerShell based make-inspired build system that allows you to keep your build process in your source code repository just like all your other code. Not only does this allow you to evolve your build process with new requirements and commit it together with the code changes, it also allows you to test your build in complete isolation. How cool is it to be able to test your deployment build from your local PC, isn’t it?
  • So if your code and your build process can be treated as first-class citizens, why can’t we do the same to your infrastructure? You can, provided you take the time to master PowerShell DSC and/or modern infrastructure platforms like TerraForm. Does your new release require a newer version of the .NET Framework (and you’re not using .NET Core yet)? Simply commit an updated DSC script and your deployment server is re-provisioned automatically.

Where do you start?

By now, it should be clear that introducing Continuous Delivery or Deployment isn’t for the faint of heart. And I didn’t even talk about the cultural aspects and the change management skills you need to have for that. On the other hand, the .NET realm is flooded with tools, products and libraries that can help you to move in the right direction. Provided I managed to show you some of the advantages, where do you start?

  • Switch to Git as your source control system. All of the above is quite possible without it, but using Git makes a lot of it a lot easier. Just try to monitor multiple branches and pull requests with Team Foundation Server based on a wildcard specification (hint: you can’t).
  • Start automating your build process using PSake or something alike. As soon as you have a starting point, it’ll become much easier to add more and more of the build process and have it grow with your code-base.
  • Identify all configuration and infrastructural settings that deployment engineers normally change by hand and add them to the build process as parameters that can be provided by the build engine. This is a major step in removing human errors.
  • Replace any database scripts with some kind of library like Fluent Migrator or the Entity Framework that allows you to update schema through code. By doing that, you could even decide to support downgrading the schema in case a (continuous) deployment fails.
  • Write so-called characterization tests around the existing code so that you have a safety net for the changes needed to facilitate continuous delivery and deployment.
  • Start the refactoring efforts needed to be able to automatically test more chunks of the (monolithical) system in isolation. Also consider extracting those parts into a separate source control project to facilitate isolated development, team ownership and a custom life cycle.
  • Choose a versioning and release strategy and strictly follow it. Consider automating the version number generation using something like GitVersion.

Let’s get started

Are you still building, packaging and deploying your projects manually? How much time have you lost trying to figure out what went wrong, only to find out you forgot some setting or important step along the way? If this sounds familiar, hopefully this post will help you to pick up some nice starting points. And if you still have question, don’t hesitate to contact me on twitter or by reaching out to me at TechDays 2016.

How do you do, Head of Testing? vol. 3

Quite OK, thanks for asking. I feel like this writing exercise helps me clear my head a bit. Many have learned by now that “all models are wrong but some are useful” – a saying attributed to the statistician George Box. In my quest for trying to understand an organization and its dynamics, I’ve come […]