Successful Test Engineering and The ISTQB Programme for Professional Capability Development | The ISTQB Programme in Depth

The ISTQB Programme in Depth

The ISTQB programme has been developed with the needs of today’s complex systems and delivery methods firmly in mind. With over one million certifications to date, it is, without doubt, the leading global certification scheme in the field of software testing.

ISTQB® established, and continues to evolve, the internationally recognized Certified Tester scheme, which is a portfolio of certifications that develop, extend, and validate testing skills. The scheme is built around syllabi created by a global network of experts, the ISTQB® Glossary, sample exams, and the Testing Body of Knowledge (TBOK).

The Certified Tester Foundation Level certification provides testing professionals with essential terminology and a breadth of knowledge. It is the prerequisite to the other modules within the scheme which offer depth and specialization.

1. The “Core” stream modules are valid for any technology/ methodology/ application domain and build on the Foundation Level.
2. The “Agile” stream focuses on testing practices specifically for the Agile methodology.
3. The “Specialist” stream provides a deep dive into areas which may be based on quality characteristics, specific test approaches and test activities, or which cluster testing know-how for certain industry domains.

The Core Stream

The ‘Core Stream’ is designed to reflect modern test engineering principles, coupled with competencies that meet the needs of a Test Practice and varying development models, supported by capability from competent, multi-skilled individuals. The Core Stream provides mainstream competencies associated with test engineering in three different levels:

1. Foundation – the ‘get you going’ level;
2. Advanced for role specific competencies;
3. Expert for those working at senior levels.

The Foundation Level

The Foundation level is about getting people and organizations into the common language of test engineering. It is the qualification on which all ISTQB syllabi for every other certification in the programme is based. That is, it provides the common terms, terminology and processes for test engineering.

The Foundation level programme has traditionally been based upon two syllabi and qualifications:

1. Foundation Tester; and
2. Agile Tester.

Foundation Tester

Historically, this provided the basics of testing, processes, terms and terminology for projects that would typically be based around the waterfall, iterative and v-model life cycles. Holding this certificate allows people to progress to the Advanced and Specialist streams for further development.

However, the Foundation level has undergone a recent transformation with the release this year of a new syllabus at Version 4. It is a radical departure from the 2018 syllabus and now addresses much around agile testing. This is the course that provides students with the common terms and processes they will need through their careers, and which are industry-recognised as the de facto standard.

The syllabus addresses in detail:

•  The Fundamentals of Testing;
• Testing Throughout the Software Development Lifecycle;
• Static Testing;
• Test Analysis and Design;
• Managing the Test Activities;
• Test Tools.

The syllabus and teaching in Foundation at version 4 is shown on page 13. The exam is multiple choice, with at least 26 questions from 40 being answered correctly over 60-minutes. A copy of the syllabus can be downloaded here.

People sitting and passing an exam should expect the following business-based outcomes:

• Understand what testing is and why it is beneficial;
• Understand fundamental concepts of software testing;
• Identify the test approach and activities to be implemented depending on the context of testing;
• Assess and improve the quality of documentation;
• Increase the effectiveness and efficiency of testing;
• Align the test process with the software development lifecycle;
• Understand test management principles;
• Write and communicate clear and understandable defect reports;
• Understand the factors that influence the priorities and efforts related to testing;
• Work as part of cross-functional team;
• Know risks and benefits related to test automation;
• Identify essential skills required for testing;
• Understand the impact of risk on testing;
• Effectively report on test progress and quality.

Agile Tester

Agile is now mainstream, and students must be able to work within it. The course expands on the techniques and learning in the Foundation Course and the difference that Agile brings. This is an extension to the Foundation that addresses the needs of agile test engineers. People holding this certificate allows them to progress to the Agile stream for further development.

The syllabus addresses in detail:

• Agile Software Development;
• Fundamental Agile Testing Principles & Processes;
• Agile Training Methods, Techniques & Tools.

The syllabus and teachings are shown on page 15. The exam is multiple choice, with at least 26 questions from 40 being answered correctly over 60-minutes. A copy of the syllabus can be downloaded here.

People sitting and passing an exam should expect the following business-based outcomes:

• Collaborate in a cross-functional Agile team being familiar with principles and basic practices of Agile software development;
• Adapt existing testing experience and knowledge to Agile values and principles;
• Support the Agile team in planning test-related activities;
• Apply relevant methods and techniques for testing in an Agile project;
• Assist the Agile team in test automation activities;
• Assist business stakeholders in defining understandable and testable user stories, scenarios, requirements, and acceptance criteria as appropriate;
• Work and share information with other team members using effective communication styles and channels.

Important note for those wishing to progress to the Agile stream: People sitting a Foundation course and passing the exam at syllabus version 4 are now exempt from the requirement to sit an Agile Tester course and exam before advancing to the Agile stream. However, people who hold a Foundation certificate based upon the 2018 syllabus or earlier are required to sit the Agile Tester course or take a new foundation course at Version 4 before advancing to the Agile stream.

The Advanced Level

The Advanced Level supports people and organisations by taking them beyond the basics and principles established at Foundation Level. It is specifically for those who want to develop and specialise in testing in one of three role-based competencies, including:

• Advanced Test Analyst;
• Advanced Technical Test Analyst;
• Advanced Test Manager.

There are significant business-based outcomes at this level (more later), but people entering this stream must first have sat and passed a Foundation Tester exam at Version 4 or earlier. There is no interdependency between the certificates; although one of the Analyst certificates is recommended prior to embarking on the Test Manager Certificate.

Advanced Test Analyst

The ISTQB® Advanced Level Test Analyst (CTAL-TA) certification provides the skills needed to perform structured and thorough software testing across the software development life cycle. It goes into detail about the test analyst’s role and responsibilities in every step of a standard test process and expands on important test techniques.

The Advanced Level Test Analyst certification is aimed at people who wish to further develop their expertise in functional test analysis and test design techniques after 12-18 months of solid testing experience following passing Foundation. It addresses key subjects that include:

• The Test Analyst’s Tasks in the Test Process;
• The Test Analyst’s Tasks in Risk-Based Testing;
• Test Techniques;
• Testing Software Quality Characteristics;
• Reviews;
• Test Tools & Automation.

The syllabus is shown pictorially on page 18 and a copy can be downloaded here. The exam is multiple choice, with 40 questions offering a combined value of 80 points. The point pass mark is set at 52 over 2-hours.

People sitting and passing an exam should expect the following business-based outcomes:

• Perform the appropriate testing activities based on the software development life cycle being used;
• Determine the proper prioritization of the testing activities based on the information provided by the risk analysis;
• Select and apply appropriate test techniques to ensure that tests provide an adequate level of confidence, based on defined coverage criteria;
• Provide the appropriate level of documentation relevant to their testing activities;
• Determine the appropriate types of functional testing to be performed;
• Work effectively in a usability testing team;
• Effectively participate in requirements / user story reviews with stakeholders, applying knowledge of typical mistakes made in work products;
• Improve the efficiency of the test process with the use of tools.

Advanced Technical Test Analyst

The ISTQB® Advanced Level Technical Test Analyst (CTAL-TTA) provides a thorough introduction to the technical testing skills which are fundamental in many organizations today. These skills include risk-based testing, white box testing, static and dynamic analysis, non-functional testing, and test automation. Essentially, Technical Test Analyst concentrates on classes of testing that are not specifically intended for deep functional testing, but more on the operability of a system and its workings.

The Advanced Level Technical Test Analyst certification is suitable for anyone who is involved in testing as well as anyone interested in further developing their software testing knowledge. This includes people performing activities such as test analysis, test consulting, and software development. It addresses key subjects that include:

• Technical Test Analyst Tasks in Risk-Based Testing;
• White-Box Techniques;
• Static and Dynamic Analysis;
• Quality Characteristics for Technical Testing;
• Reviews
• Test Tools & Automation.

The syllabus is shown on page 20 and a copy can be downloaded here from the ISTQB website. The exam is multiple choice, with 45 questions of variable value that offer a combined value of 78 points. The point pass mark is set at 51 over 2-hours.

People sitting and passing an exam should expect the following business-based outcomes:

• Recognise and classify the typical risks associated with the performance, security, reliability, portability and maintainability of software systems;
• Provide technical elements to the planning, design and execution of tests for mitigating performance, security, reliability, portability and maintainability risks;
• Select and apply appropriate white-box test techniques to ensure that tests provide an adequate level of confidence, based on design coverage;
• Effectively participate in reviews with developers and software architects applying knowledge of typical defects in the code and architecture;
• Improve the quality characteristics of code and architecture by making use of different analysis techniques;
• Outline the costs and benefits to be expected from introducing particular types of test automation;
• Select appropriate tools to automate technical testing tasks;
• Understand the technical issues and concepts in applying test automation.

Advanced Test Manager

The ISTQB® Advanced Level Test Manager (CTAL-TM) certification provides the knowledge and competencies to take responsibility for managing all the testing activities across the software development lifecycle. It covers everything from how to design a suitable test approach for the project based on the organizational test strategy to building a test team or testing competencies to complete the necessary testing.

The Advanced Level Test Manager certification is aimed at people who have already achieved an advanced point in their careers in software testing and wish to develop further their expertise in test management. It addresses key subjects that include:

•  Testing Process;
• Test Management;
• Test Reviews;
• Defect Management;
• Improving the Test Process;
• Test Tools and Automation;
• People Skills & Team Composition.

The syllabus is shown on page 22 and a copy can be downloaded from the website here. The exam is multiple choice, with 65 questions of variable value that offer a combined value of 115 points. The point pass mark is set at 75 over 3-hours.

People sitting and passing an exam should expect the following business-based outcomes:

• Manage a testing project by implementing the mission, goals and testing processes established for the testing organization;
• Organize and lead risk identification and risk analysis sessions and use the results of such sessions for test estimation, planning, monitoring and control;
• Create and implement test plans consistent with organizational policies and test strategies;
• Continuously monitor and control the test activities to achieve project objectives;
• Assess and report relevant and timely test status to project stakeholders;
• Identify skills and resource gaps in their test team and participate in sourcing adequate resources;
• Identify and plan necessary skills development within their test team;
• Propose a business case for test activities which outlines the costs and benefits expected;
• Ensure proper communication within the test team and with other project stakeholders;
• Participate in and lead test process improvement initiatives.

The Expert Level

The Expert Level is aimed at the most senior test engineers who are tasked with operating strategically typically at group or organizational levels. They will be tasked with setting test engineering strategy, process, management and similar activities. People entering this stream will generally hold 10 or more years’ experience as a test engineer, with at least 2-years in specialist roles, such as Test Management, Process Definition, Maturity Assessment and other key tasks.

This level is significant for organizations in that people who qualify from it will be able to undertake Test Assessments, Establish Maturity of People and Process against industrial models and, importantly, set the frameworks for the entire test engineering process and its operation. These are key assets for an organization that will pay dividends many times over the cost of investment.

The Expert level comprises two-key subjects:

1. Test Management, which involves passing all three management exams to be awarded an Expert Test Management certificate;
   
   
2. Improving the Test Process, which involves passing Assessing Test Process and Implementing Test Process Improvement exams to be awarded an Expert certificate in Improving the Test Process.

Certificates awarded are valid for a period of 7-years, following which an assessment must be undertaken to retain the certificate.

People attempting to obtain a certificate at this level must first have sat and passed an Advanced Test Manager exam.

The Expert Improving the Test Process

The ‘Expert Improving the Test Process’ consists of two modules, both of contribute to the overall ‘Expert Improving the Test Process’ certificate. That is, a certificate is awarded only once both modules have been sat and a successful exam sat.

Expert Assessing the Test Process

Assessing the Test Process (CTEL-ITP-ATP) certification helps prepare individuals to fulfil the specific expert role within their organization or project, and more specifically to assess and advise on test process improvement. It covers the context of improvement, recognized models for improvement, analytic approaches, selection of an approach, and initiation of the process.

It addresses key subjects that include:

• The Basic Context of Improvement;
• Model Based Improvement;
• Analytical Based Improvement;
• Selecting the Approach for Test Process Improvement;
• Process for Improvement.

The syllabus is shown on page 25 and can downloaded here. The pass mark is 65% for the exam, which has two parts:

• Multiple choice over 45 minutes;
• Written response over 90-munites;

People awarded an overall Expert Improving the Test Process certificate should expect the following business-based outcomes:

•  Lead programme for improving the testing process within an organization or project and can identify and manage critical success factors;
• Take appropriate business-driven decisions on how to approach improvement to the test process;
• Assess the current status of a test process, propose stepwise improvements and show how these are linked to achieving business goals;
• Set up a strategic policy for improving the testing process and implement that policy;
• Analyse specific problems with the test process and propose effective solutions;
• Create a test improvement plan which meets business objectives;
• Develop organizational concepts for improvement of the test process which include required roles, skills and organizational structure;
• Establish a standard process for implementing improvement to the test process within an organization;
• Manage the introduction of changes to the test process, including co-operation with the sponsors of improvements;
• Understand and effectively manage the human issues associated with assessing the test process and implementing necessary changes.

This is part 1 of Improving the Test Process. For Part 2 of the CTEL-ITP certification, refer to Expert Implementing Test Process.

Expert Implementing Test Process Improvement

Implementing Test Process Improvement helps prepare individuals to fulfil the specific expert role within their organization or project, and more specifically to ensure that the implementation of test process improvements within their organization or project takes place effectively and stands the best chance of success.

It covers the improvement process, organization considerations, change management, and success factors.

It addresses key subjects that include:

• Process for Improvement;
• Organization, Roles & Skills;
• Managing Change;
• Critical Success Factors;
• Adapting to Different Life Cycle Models.

The syllabus is shown on page 27 and can be further downloaded here. The pass mark is 65% for the exam, which has two parts:

• Multiple choice over 45 minutes;
• Written response over 90-munites;

People awarded an overall Expert Improving the Test Process certificate should expect the following business-based outcomes:

• Lead programs for improving the testing process within an organization or project and can identify and manage critical success factors;
• Take appropriate business-driven decisions on how to approach improvement to the test process;
• Assess the current status of a test process, propose stepwise improvements and show how these are linked to achieving business goals;
• Set up a strategic policy for improving the testing process and implement that policy;
• Analyse specific problems with the test process and propose effective solutions;
• Create a test improvement plan which meets business objectives;
• Develop organizational concepts for improvement of the test process which include required roles, skills and organizational structure;
• Establish a standard process for implementing improvement to the test process within an organization;
• Manage the introduction of changes to the test process, including co-operation with the sponsors of improvements;
• Understand and effectively manage the human issues associated with assessing the test process and implementing necessary changes.

The Expert Test Management Certification

The ‘Expert Test Management’ consists of three modules, all of which contribute to the overall ‘Expert Test Management’ certificate. That is, a certificate is awarded only once all three modules have been sat and a successful exam sat.

It is a complex certificate that examines the Test Manager’s capability to operate across the organizational spectrum to set:

•  Frameworks for Test Management;
• Operating and executing large scale test programmes, such as major integrations following acquisitions, and similar;
• Managing the test team to meet organization needs and those of people in their teams to increase their knowledge, capability and competence to grow and climb their own career ladder.
• People passing all three exams are awarded an Expert Test Certificate that will be valid for 7-years, following which a further assessment is mandated to continue holding the certificate.

Expert Strategic Test Management

The ‘Strategic Test Management’ content within the overall, 3-part Test Management syllabus considers the purpose of testing or the test mission, and the relation between the test policy, the test strategy, and the test objectives.

It addresses the effectiveness, efficiency, and satisfaction metrics for the test policy objectives. Integrating tools across the organization and testing as part of a larger set of quality management activities necessary to deliver quality software products and services are included, as well as the management of external relationships, most especially the merging of test strategies with third party organizations.

This element of the qualification also looks at the test management considerations for different project lifecycle models.

It addresses key subjects that include:

• Test Missions, Policies, Strategies and Goals;
• Managing External Relationships;
• Managing Across the Organization;
• Testing Considerations for Domain and Project Factors;
• Evaluating Effectiveness and Efficiency.

The syllabus is shown on page 30 and can be further downloaded here. The pass mark is 65% for the exam, which has two parts:

• Multiple choice over 45 minutes;
• Written response over 90-minutes;

People awarded an overall Expert Test Management certificate should expect the following business-based outcomes:

• Lead the test management within an organization, project or program to identify and manage critical success factors with management commitment at CEO/Board level;
• Take appropriate business-driven decisions on a test management strategy and implement organization wide commitment and compliance based on quality KPIs;
• Assess the current status of the test management, propose stepwise improvements and show how these are linked to achieving business goals within the organizational context of test management (organization or project/program);
• Set up a strategic policy for improving the test management and the testing, and implement that policy in an organization;
• Analyse specific problems with the test management and its alignment with other roles or management areas in the project/organization, and propose effective solutions;
• Create a master test plan with matching governance dashboard to meet or exceed the business objectives of the organization or a project/program;
• Develop innovative concepts for test management (project) organizations which include required roles, skills, methodologies (tools) and organizational structure;
• Establish a standard process for implementing test management in an organization (project/program) with standardized delivery based on quality KPIs;
• Lead an organization to improve the test management process and manage the introduction of changes;
• Understand and effectively manage the human issues associated with test-project management and implement necessary changes.

Expert Operational Test Management

The ‘Operational Test Management’ content within the overall, 3-part Test Management syllabus considers the skills needed for the Test Manager to manage third party relationships including contractual, communication, integration, and verification of quality aspects.

It covers the ability to create, build, and maintain relationships with other managers and other teams, as well as project management skills, including project risk management, and organizing and moderating effective retrospective meetings. Reporting and interpreting test results by evaluation of exit criteria is elaborated upon, and metrics are described which play a crucial role in the management of the test process. Release considerations for the Test Manager are also highlighted.

It addresses key subjects that include:

• Managing External Relationships;
• Project Management Essentials;
• Test Project Evaluation and Reporting;
• Test Considerations for Domain and Project Factors;
• Evaluating Effectiveness and Efficiency.

The syllabus is shown on page 32 and can be further downloaded here. The pass mark is 65% for the exam, which has two parts:

• Multiple choice over 45 minutes;
• Written response over 90-minutes;

People awarded an overall Expert Test Management certificate should expect the following business-based outcomes:

• Lead the test management within an organization, project or program to identify and manage critical success factors with management commitment at CEO/Board level;
• Take appropriate business-driven decisions on a test management strategy and implement organization wide commitment and compliance based on quality KPIs;
• Assess the current status of the test management, propose stepwise improvements and show how these are linked to achieving business goals within the organizational context of test management (organization or project/program);
• Set up a strategic policy for improving the test management and the testing, and implement that policy in an organization;
• Analyze specific problems with the test management and its alignment with other roles or management areas in the project/organization, and propose effective solutions;
• Create a master test plan with matching governance dashboard to meet or exceed the business objectives of the organization or a project/program;
• Develop innovative concepts for test management (project) organizations which include required roles, skills, methodologies (tools) and organizational structure;
• Establish a standard process for implementing test management in an organization (project/program) with standardized delivery based on quality KPIs;
• Lead an organization to improve the test management process and manage the introduction of changes;
• Understand and effectively manage the human issues associated with test-project management and implement necessary changes.

Expert Managing the Test Team

The ‘Managing’ content within the overall, 3-part Test Management syllabus considers the people management skills that are a critical part of the Test Manager’s role and necessary to build, develop and lead test teams in organizations. This includes recruitment, setting goals and objectives, individual personalities and roles within teams, performance reviews, motivating and challenging the team and managing distributed teams. Being able to manage across the organization is another core part of the Test Manager’s role including being able to advocate, defend and promote the test team and its contribution to the organization, being able to communicate with stakeholders, advocating quality across the organization and being able to manage ethical issues.

There are only two primary subjects, this part of the Test Management certification considers strategic subjects including:

• Managing the Test Team;
  • Building the Test Team;
  • Developing the Test Team;
  • Leading the Test Team.
• Managing Across the Organization.
  • Advocating the Test Team;
  • Placement of the Test Team;
  • Stakeholder Communication;
  • Managing Ethical Issues.

The syllabus is shown on page 35 and can be further downloaded here. The pass mark is 65% for the exam, which has two parts:

• Multiple choice over 45 minutes;
• Written response over 90-minutes;

People awarded an overall Expert Test Management certificate should expect the following business-based outcomes:

• Lead the test management within an organization, project or program to identify and manage critical success factors with management commitment at CEO/Board level;
• Take appropriate business-driven decisions on a test management strategy and implement organization wide commitment and compliance based on quality KPIs;
• Assess the current status of the test management, propose stepwise improvements and show how these are linked to achieving business goals within the organizational context of test management (organization or project/program);
• Set up a strategic policy for improving the test management and the testing, and implement that policy in an organization;
• Analyze specific problems with the test management and its alignment with other roles or management areas in the project/organization, and propose effective solutions;
• Create a master test plan with matching governance dashboard to meet or exceed the business objectives of the organization or a project/program;
• Develop innovative concepts for test management (project) organizations which include required roles, skills, methodologies (tools) and organizational structure;
• Establish a standard process for implementing test management in an organization (project/program) with standardized delivery based on quality KPIs;
• Lead an organization to improve the test management process and manage the introduction of changes;
• Understand and effectively manage the human issues associated with test-project management and implement necessary changes.