Newsletters

Ten Steps to Building Software Test Automation That Works

Step 3: Analyze Existing QA and Development Processes

Analyze Existing QA and Development Processes

First and foremost, your existing situation should be evaluated in order to determine what needs to be modified or added to today's QA and Development processes. We advise you to put together a matrix or a questionnaire and engage in a number of short conversations with key people in your organization. Depending on your organization's size and structure, the levels and titles of these people may vary, but we generally recommend talking to:

  • VP of Engineering
  • Development Manager / individual contributors
  • QA Manger / individual contributors
  • Release Manager
  • Tech Support Manager / individual contributors

The answers to the questions below should be collected.

  1. How does your development cycle look?
    * Flip to the next section and see which of the diagrams is a better representation of your development cycle: "Diagram 1" or "Diagram 2"?
    (hold tight for next month's article or request the complete White Paper at the end of this section.)
  2. If the answer is the "Diagram 1," then most likely you will hear "yes" when you talk to the members of your QA and Development teams. Ask them the following:

    Does QA feel that:

    • They always find themselves at the end of the process?
    • The application is given to them for testing when all possible deadlines for development are blown?
    • There is never enough time for testing the release properly?
    • They are the ones who are being blamed for a late release?

    Do developers feel that:

    • The defects that are given to them to fix should have been caught much earlier in the development life cycle?
    • There are bugs coming from the field that by all means should have been caught during release?
    • They are being asked to redo their work many times for no obvious reason?
    • The problems that QA reports are very often impossible to re-create?
  3. Specific data (averages):
    1. Releases:
      • Frequency of releases (number of releases per year)
      • Duration of development cycle (from start of development to turning it over to QA)
      • Duration of testing cycle (from start of testing to giving it back to development)
      • Number of iterations (how many times the "b" and "c" above are repeated before the product is released)
      • Duration of the whole release cycle
      • Deadline slippage (average number of days / weeks a release is late)
      • Number of emergency releases / patches per year
    2. Builds:
      • Do you have an established version control system and process in place?
      • How often do you build your application (daily, weekly, monthly, every three months)?
    3. Resources:
      • Who is doing testing?
      • Do you have a dedicated QA team?
      • Do you have or plan to have a dedicated QA Automation team or resource?
      • Are your QA resources being asked to work in other areas not related to QA?
      • Are your Test Automation Resources being asked to do manual testing?
      • Are you using other team members (not QA - such as developers, tech support personnel) to test your application?
    4. Environment / Hardware:
      • Do you have a dedicated test system?
      • Is the above test environment being used exclusively for testing?
      • Does your testing ever happen on a live environment?
      • Do you have or plan to have a dedicated Test Automation System?
      • Is the Test Automation System ever used for anything other than test automation (for example - for manual testing)?
    5. Statistics: collect the following information on a consistent basis:
      • Bugs found during release:
        • Total Number
        • Effort to find them - QA (resource-days)
        • Effort to fix them - Development (resource-days)
      • Bugs from the field found after release:
        • Total Number
        • Effort to find them - QA & Tech Support Team (resource-days)
        • Effort to fix them - Development (resource-days)

The collected information will indicate symptoms of problems you can attack with effective adjustment of your processes. It will also show the direction to proceed. This information will be your benchmark to compare against when you are at the point where test automation is successfully implemented and starts bringing results. The consistent collection of data will increase your ability to justify your efforts and to calculate your project's ROI.

For a FREE ROI calculator developed by using data from a number of actual client cases, please visit the following link: qasignature.com/resources Stay tuned for the next topic: "Analyzing Existing QA and Development Processes."

Please e-mail your comments to cblaylock@qasignature.com or call 617 510-6545.

Click here for a PDF copy of the complete white paper:

Proof of Concept Service

To Companies Who Want To Build QA Automation - But Can't Get Started. Take qaSignature For A Test Drive.

An effective approach to getting started with test automation is the qaSignature Proof of Concept Service. Standard test automation produces limited results and is difficult and costly to maintain. The qaSignature methodology is different. We'll prove it. Give us your most difficult automation challenge. We will:

  • Recommend the optimal automation tool (or use yours)
  • Build the automation as a proof of concept and show you how our methodology works
  • Let you change the code, we will update the test and rerun
  • Show you the results in our easy to follow test logs
  • Prioritize areas for automation based on results from a cost benefit analysis
  • Give you a fixed price estimate for a fast start program

Here's what our clients had to say.

"The amount of automation that they were able to develop in a couple of days including fact finding was impressive. Their competition could not even begin the process without first learning the application."
Keith Hillyard, Custom Engineering SQA Manager, Kronos

"The qaSignature proof of concept is a "no brainer." They delivered much more than we expected. The process helped us map out our QA Automation Framework."
Frank F. Frazier Jr., Senior Program Manager, ZANTAZ

"The dedication, enthusiasm and passion of the qaSignature team was refreshing. I would recommend them to anyone who needs to develop QA Automation."
Paul Bradley, Systems Consultant, DAFCA Inc.

"The proof of concept demonstrated the applicability of applying the qaSignature QA Automation Methodology to our Agile Development Process."
Keith Hillyard, Custom Engineering SQA Manager, Kronos

"I was impressed by qaSignature's objectivity. They recommended the right tool for the job even though it wasn't their standard offering. They really know QA Automation."
Larry Leonard, Director of Development, SmartTime

Call us now @ 617 510-6545

Click on the link below for details:

Proof of Concept

«« Back to qaSignature Newsletter Archive