A0: Applied Exercise

2. EXAMINE

Look before asking AI.

Before showing anything to the LLM, examine each collection yourself and form your own initial interpretation.

The aim is not to become an expert in the artefact. Instead, notice what appears clear, what remains ambiguous, and what may require several views, scales, or additional knowledge to understand. These observations will help you formulate questions that test something meaningful.

Look For

• Important, repeated, or defining features
• Unusual or potentially problematic conditions
• Ambiguous relationships, overlaps, or occlusions
• Difficult spatial, geometric, or structural relationships
• Features that only become clear across different scales or viewpoints
• Information that appears to be missing

Ask Yourself

• What can I tell immediately from the available evidence?
• What requires several views or scales to understand?
• What am I uncertain about?
• What might be easy for the AI to recognise?
• What might the AI misunderstand or infer too confidently?
• What would require information beyond what is actually visible?

Before moving on:

• Record your own interpretation of the collection.
• Note important uncertainties or ambiguities.
• Identify conditions that may be worth testing.

The following guidelines will help you prepare this part of the assignment:

3. IDENTIFY

Turn observations into testable questions.

Your questions are the core of the investigation. Use your observations from EXAMINE to develop questions that probe what the model can see, relate, interpret, reason about, and recognise that it does not know.

Some questions should be repeated systematically across several collections so that you can compare the model’s behaviour. Others should respond to something distinctive or difficult in a particular artefact.

Types of Questions

A. Systematic Questions

Develop approximately 5 questions that can be applied across several of your collections.

Across the set, include at least:

• One question about relationships / connectivity
• One question requiring interpretation / reasoning
• One question about uncertainty / missing information

Try to move across different levels of visual understanding:

Seeing → Relating → Interpreting → Reasoning → Knowing the Limits

B. Collection-Specific Questions

For each collection, develop 1–2 additional questions that respond to something distinctive about that particular case.

Useful targets might include:

• An unusual connection or configuration
• An occluded or partially visible element
• An apparent hazard or problematic condition
• A difficult spatial or structural relationship
• A missing or interrupted element
• A condition that becomes clearer across scale or viewpoint
• Something you found difficult to interpret yourself

These questions do not need to work across every collection. Their purpose is to probe the particular challenges presented by that artefact.

General Tips

Make Questions Assessable

Where possible, ask questions for which you have a reasonable way of checking the answer.

• If you ask whether two scaffold members connect, another photograph or viewpoint should ideally reveal the condition.
• If you ask whether two rooms connect, check the plan yourself.
• If you ask about the location of an element, another scale or view may provide supporting evidence.

A question may also be deliberately unanswerable. In that case, you should understand why the available evidence is insufficient and what additional information would be needed.

Go Beyond Simple Description

Avoid relying only on questions such as “What is shown here?” Instead, formulate questions that reveal something about the model’s visual understanding.

For example:

• Which elements appear to connect?
• How is one part positioned relative to another?
• What might explain this condition?
• Can this conclusion actually be supported by the available evidence?
• What additional view or information would be needed to answer confidently?

The aim is not simply to catch the AI making mistakes. It is to identify where its visual understanding works, where it breaks down, and whether it recognises the limits of the available evidence.

The following guidelines will help you prepare this part of the assignment: