Pass the ISTQB ISTQB AI Testing CT-AI Questions and answers with CertsForce

From the confusion matrix:

True Positives (TP) = 15

False Positives (FP) = 5

False Negatives (FN) = 15

True Negatives (TN) = 65

Accuracy= (TP + TN) / Total

= (15 + 65) / 100

=80%

Precision= TP / (TP + FP)

= 15 / (15 + 5)

= 15 / 20

=75%

Section3.2 – Functional Performance Criteriain the syllabus explains accuracy and precision exactly these ways when evaluating ML classification performance.

Option B is therefore the only correct pair of values.

The ISTQB CT-AI syllabus explains that AI-based decision-support systems benefit strongly fromback-to-back testingandmetamorphic testingwhen oracle problems exist or when limited regression tests are available. In this scenario, CleverPropose replaces an older advisory system.Back-to-back testing(Option A) is ideal because the outputs of the existing conventional system can serve as areference, enabling comparison against the new AI system. This is exactly what the syllabus recommends when AI is replacing a traditional deterministic system.

Metamorphic testing(Option C) is also appropriate, as stated in Section4.6 – Metamorphic Relations. With limited regression tests and complex decision logic, testers can define metamorphic relations such as “if customer income increases, risk rating should not worsen.” These relations allow validation even when exact expected outputs are unavailable.

Exploratory data analysis (Option D) is not a system testing technique. Pairwise testing (Option E) is not well suited for complex AI-based financial advice systems. Adversarial testing (Option B) is more relevant for security-critical or robustness evaluation, not primary system testing for advisory tools.

Thus,A and Care the correct and syllabus-supported choices.

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.

The syllabus defines back-to-back testing as a method to compare a modified AI system to the previous version, which is ideal in this scenario:

"Back-to-back testing is performed by comparing the outputs of two systems that are supposed to provide the same outputs, one being a known and trusted system and the other being the test system. This approach can be used to test ML systems after re-training to verify that improvements have not introduced regressions."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.3, page 67 of 99)

AI test environments have several unique characteristics that differentiate them from traditional test environments. Let’s evaluate each option:

A. Challenges resulting from low accuracy of the models.

Low accuracy is a common challenge in AI systems, especially during initial development and training phases. Ensuring the model performs accurately in varied and unpredictable scenarios is a critical aspect of AI testing.

B. The challenge of mimicking undefined scenarios generated due to self-learning.

AI systems, particularly those that involve machine learning, can generate undefined or unexpected scenarios due to their self-learning capabilities. Mimicking and testing these scenarios is a unique challenge in AI environments.

C. The challenge of providing explainability to the decisions made by the system.

Explainability, or the ability to understand and articulate how an AI system arrives at its decisions, is a significant and unique challenge in AI testing. This is crucial for trust and transparency in AI systems.

D. Challenges in the creation of scenarios of human handover for autonomous systems.

While important, the creation of scenarios for human handover in autonomous systems is not a characteristic unique to AI test environments. It is more related to the operational and deployment challenges of autonomous systems rather than the intrinsic technology-related characteristics of AI.

Given the above points, optionDis the correct answer because it describes a challenge related to operational deployment rather than a technology-related characteristic unique to AI test environments.

In a neural network, the activation value of a neuron is determined by a combination of inputs from the previous layer, the weights of the connections, and the bias at the neuron level. Here’s a detailed breakdown:

Inputs for Activation Value:

Activation Values of Neurons in the Previous Layer:These are the outputs from neurons in the preceding layer that serve as inputs to the current neuron.

Weights Assigned to the Connections:Each connection between neurons has an associated weight, which determines the strength and direction of the input signal.

Individual Bias at the Neuron Level:Each neuron has a bias value that adjusts the input sum, allowing the activation function to be shifted.

Calculation:

The activation value is computed by summing the weighted inputs from the previous layer and adding the bias.

Formula: z=∑(wi⋅ai)+bz = \sum (w_i \cdot a_i) + bz=∑(wi​⋅ai​)+b, where wiw_iwi​ are the weights, aia_iai​ are the activation values from the previous layer, and bbb is the bias.

The activation function (e.g., sigmoid, ReLU) is then applied to this sum to get the final activation value.

Why Option A is Correct:

Option A correctly identifies all components involved in computing the activation value: the individual bias, the activation values of the previous layer, and the weights of the connections.

Eliminating Other Options:

B. Activation values of neurons in the previous layer, and weights assigned to the connections between the neurons: This option misses the bias, which is crucial.

C. Individual bias at the neuron level, and weights assigned to the connections between the neurons: This option misses the activation values from the previous layer.

D. Individual bias at the neuron level, and activation values of neurons in the previous layer: This option misses the weights, which are essential.

A. Regression

Regression is used to predict a continuous value and is not suitable for grouping people based on similar tastes.

B. Association

Association is used to find relationships between variables in large datasets, often in the form of rules (e.g., market basket analysis). It does not directly group individuals but identifies patterns of co-occurrence.

C. Clustering

Clustering is an unsupervised learning method used to group similar data points based on their features. It is ideal for identifying groups of people with similar tastes without prior knowledge of the group labels. This technique will help the company segment its customer base effectively.

D. Classification

Classification is a supervised learning method used to categorize data points into predefined classes. It requires labeled data for training, which is not the case here as we want to identify groups without predefined labels.

Therefore, the correct answer isCbecause clustering is the most suitable method for grouping people with similar tastes for targeted product marketing​​.

The ISTQB CT-AI syllabus (Section4.3 – Test Environments for AI Systems) describes that, unlike conventional software testing, testing AI systems may requirespecialized toolsfor analyzing and explaining the decisions of ML models. This includes visualization tools, explainability frameworks, and diagnostic utilities to understand why the AI made a certain prediction. Since AI decisions may be non-transparent, the test environment must supportexplainability, making OptionBcorrect.

Option A is incorrect: the syllabus does not state that an autonomous AI system requires an autonomous test environment. Option C is incorrect because test environments mustnot learn; they must remain stable to avoid unpredictable testing conditions. Option D is incorrect because multi-agent systems often involve stochastic interactions, and determinism is neither required nor realistic.

Thus,Option Bis the syllabus-accurate choice.

A. This is an example of expert system bias.

Expert system bias refers to bias introduced by the rules or logic defined by experts in the system, not by the data distribution.

B. This is an example of sample bias.

Sample bias occurs when the training data is not representative of the overall population that the model will encounter in practice. In this case, the over-representation of ethnicity A (70%) compared to B, C, and D (30%) creates a sample bias, as the model may become biased towards better performance on ethnicity A.

C. This is an example of hyperparameter bias.

Hyperparameter bias relates to the settings and configurations used during the training process, not the data distribution itself.

D. This is an example of algorithmic bias.

Algorithmic bias refers to biases introduced by the algorithmic processes and decision-making rules, not directly by the distribution of training data.

Based on the provided information, optionB(sample bias) best describes the situation because the training data is skewed towards ethnicity A, potentially leading to biased model performance​​.

 Labelling Approaches:Among the options provided, pre-labeled datasets require the least time and effort because the data has already been labeled, eliminating the need for further manual or automated labeling efforts.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 4.5 Data Labelling for Supervised Learning, which discusses various approaches to data labeling, including pre-labeled datasets, and their associated time and effort requirements​​.