A0: Applied Exercise

4. INTERROGATE

Test the model carefully.

Ask both your systematic questions and collection-specific questions.

Treat the model’s first response as a controlled test, then use the continuing conversation to interrogate its answer. Keeping these stages separate helps distinguish what the model can infer from the original visual evidence from what it says after you guide, challenge, or correct it.

For Each Question

Always begin each independent test in a fresh conversation.

• Start a new conversation for each test question.
• Provide only the relevant image(s) and ask the question.
• Do not provide hints, corrections, or additional explanation beforehand.
• Record the model’s first response.
• For the next independent question, start another fresh conversation.

This helps prevent information from earlier questions from influencing later responses, and makes the tests easier to compare.

Follow-Up Interrogation

After recording the first response, continue the conversation and probe the model’s answer more closely.

Useful follow-up questions include:

• What visual evidence supports your answer?
• Which part of the image are you referring to?
• How certain are you?
• What are you assuming?
• Could there be another explanation?
• What information would you need to know for certain?
• Look again. Would you revise your answer?

Record what happens during the interrogation. Does the model:

• Correct itself?
• Maintain an incorrect answer?
• Become more precise?
• Contradict itself?
• Introduce unsupported claims?
• Acknowledge uncertainty or missing information?

Compare Visual Inputs

For selected systematic questions, ask the same question using different combinations of visual evidence.

For example:

• Context only
• System only
• Detail only
• System + Detail
• Context + System + Detail
• Multiple viewpoints of the same condition

Keep the wording of the question as consistent as possible. This allows you to examine how changing the visual evidence affects the model’s answer.

The following guidelines will help you prepare this part of the assignment:

5. EVALUATE

Decide what the AI actually did well or badly.

The model’s response is not the result by itself. Your task is to evaluate it against the available visual evidence and your own observations from EXAMINE, then look for patterns across multiple tests.

Remember that uncertainty is not necessarily a failure. A model that correctly recognises that it cannot determine something from the available evidence may be performing better than one that gives a confident but unsupported answer.

Evaluate Each Response

Consider:

• Is the response Correct / Partially Correct / Incorrect?
• Is the answer actually supported by the available visual evidence?
• Does the model acknowledge uncertainty when appropriate?
• Does it make assumptions or claims that cannot be verified from the evidence?
• Does its answer improve, deteriorate, or change during follow-up interrogation?

Compare Across Cases

Compare your systematic questions across:

• Different collections
• Different scales
• Different viewpoints
• Different combinations of visual input
• First responses and follow-up interrogation

Look for recurring patterns rather than isolated mistakes.

Identify:

• Where the model performs well
• Where it consistently struggles
• What kinds of visual information improve its performance
• When additional views or scales help
• When it makes unsupported inferences
• When it recognises — or fails to recognise — the limits of the available evidence

The aim is to understand patterns of capability and limitation, rather than simply count correct and incorrect answers.

The following guidelines will help you prepare this part of the assignment: