BEST PRACTICE

(1) Not being familiar with the contents of the assessment resource

Teachers must not think that they can just open the books and administer the assessment. If this was a multiple choice assessment it might be possible, but it is not. PROBE is an individual, behavioural assessment. The purpose, taxonomy, instructions, stories, questions, answers and notes must be understood.

(2) NOT ensuring that students read each text twice.

Studies have shown that students who read twice actually complete tests more quickly. They are able to get an overview with the first reading and a detailed view with the second. This enables them to relate the question words to the text material and locate it without a lengthy search.

(3) NOT referring to the answers as they are testing

Teachers need the answers open beside them every time they assess. Anything else is just guessing, slapdash and unprofessional.

(4) NOT referring to the notes or key words below the answers when testing

These appendices to the answers are very important. The sole purpose of the notes and the key words that accompany the answers is to help teachers decide when an answer is correct or not.

If this type of assessment is used without referring to the notes, it is all too easy to give marks to students who can articulate and sound as though they have carefully considered the text before giving an answer – when they are, in fact, quite wrong.

(5) NOT using one of the supplementary responses

A strength of the PROBE assessments is that it gives students a chance to clarify an answer.

"Can you tell me more?" should be asked if a student’s answer is limited yet the intention seems right. Of course the student’s response to this supplementary question must be sufficiently clear to be marked correct.

“Show me where it says that." should be said if the given answer ‘appears’ to be right but it needs to ascertained that it came from the text.

Supplementary responses should not be used when an answer is clearly wrong .

(6) LEADING students to the answers.

If they can't answer the question, then move on to the next one.
Do not say things like:
“The answer is in the top half of the story.”
“Would you like to look again?”
“Almost right. Think again and tell me about…”
By assisting students in this way is counterproductive to them getting the help they need.

(7) ACCEPTING answers that are not related to the text, or because they seem close enough.

The answers given have been carefully matched to the words or intent of the text. There are also examples of what not to accept as an answer. Sometimes an answer seems to be reasonable and plausible – but is not actually true to the text. Don’t get caught in the sympathy trap: close enough is not good enough.

(8) GIVING a half mark because an answer is partially correct.

PROBE is not designed for half marks. Each question has a value of one mark for a fully correct answer. There are no exceptions. What is expected as a correct answer is provided for you in the notes that accompany every answer. Do not guess or memorise these answers: have them with you.

NOTE: If you adapt the questions or answers in any way you will invalidate the assessment.

(9) NOT giving students a higher text when scoring decoding 96% comprehension 70%+

That is the minimum pass rate. Your student may score the minimum and then also pass the next text up – maybe with a higher score (this is not unknown). Take nothing for granted. You must keep going until they fail.

(10) NOT giving students a lower text when scoring below decoding 96% comprehension 70%

By failing an assessment does not mean that the student's reading age has been found. You must keep going until they pass. They may fail more than you think they will.

Some teachers have suggested that while they are assessing a student they could have a second student silent reading their (first) text while waiting for their turn.

Question: Should students independently read a text before working with the teacher?
Short answer: NO

Having a student read a text in isolation at an undefined time prior to the assessment is counterproductive to the purpose of this assessment. The first, (silent) reading is part of the assessment, not a separate activity. If teachers don't observe their students silently reading, they cannot get the full picture of their reading behaviours.

If PROBE is being used correctly, more than one text must be used to gain a result. Consequently, an assessment will take 20 minutes or more.

If, while you are busy assessing Student A with at least two to four texts (you don't know how many before you start), you set Student B to read a text he is later to be assessed on, what does this student do after the few minutes it takes to read the text?

- Worry the story (and questions) to death?
- Switch off?

If Student B is kept in a holding pattern, momentum is lost.
A large component of this assessment is the interaction between assessor and student. If a student is set to read the text in isolation, the assessor is devaluing the assessment preparation – the settling in, the reassurance, the instructions and overview.

As outlined in the Manual, PROBE is a controlled in-depth interview, designed to assist educators in identifying reading behaviours and difficulties and to address them by providing a programme of change.

Some students will display few or no behaviours that would cause concern, but there are many (more than most teachers may believe) who will. These students need assistance in more than decoding. They need a range of skills and strategies to become competent readers.

PROBE IS NOT A CONVEYOR BELT ASSESSMENT
It is a consideration of an individual student's ability to comprehend and an evaluation of how the result was arrived at. The observation of students' behaviours, even as they read silently is a vital part of the assessment.

Looking for much more than a numerical result.

PROBE is an acronym for Prose Reading Observation, Behaviour and Evaluation (of Comprehension).  The observation of reading behaviours during testing is vital and valuable for acquiring information on your students and developing teaching programs that will best suit them.

WHAT ARE OBSERVABLE READING BEHAVIOURS?
There is a range of behaviours that will give teachers an understanding of the student's strengths and weaknesses in reading. Relevant behaviours can be observed at different stages of the assessment:
• when reading silently (Option 1 first reading, Option 2 first & second readings)
• when reading aloud (Option 1 second reading)
• when answering the comprehension questions

Observable behaviours can be seen in decoding and in body language. We are concerned with behaviours that may be hindering comprehension.  If you, as the teacher, are to help your students comprehend better, you will need to know the possible causes of the problem. Behaviours may affect reading comprehension and therefore should be noted.
Familiarise yourself with these behaviours and their possible consequences.
BE AWARE: if students exhibit any of these behaviours, it does not automatically mean they don’t understand what they have just read.

OBSERVABLE BEHAVIOURS INCLUDE:

SUBVOCALISING
What exactly is sub-vocalisation?
Does it happen internally, by thinking the words, or externally by mouthing the word?

Oxford Online Dictionary: utter or form (words or sounds) with the lips silently or with barely audible sound, especially when talking to oneself, memorizing something, or reading. Merriam-Webster Online Dictionary: the act or process of inaudibly articulating speech with the speech organs.

So, contrary to popular thinking, sub-vocalisation is a physical, not a mental, activity.  Thinking words silently in your head as you read is not sub-vocalisation.
There is much debate about it.  Is it good or is it bad?
•  Does it help or hinder speed and comprehension?
•  Should it be encouraged or discouraged?
You may observe a student who mouths or whispers each word very quietly, or silently.  Subvocalisation may not always be that easy to detect.
- They may only have the slightest movement of their lips.
- They may nod their head very slightly at each word, or move their head as their eyes
move across a line and back to the next one.

- Even more difficult to detect is that their eye movements will be jerky, instead of sliding smoothly over the words.

Sub-vocalising may slow a student's reading speed by locking them in to reading word by word; concentrating on decoding may take them away from thinking about the text.  Consequently, comprehension results can be poor.

If a student who is subvocalising achieves 70% or more comprehension there still can be a problem: speed.

Because of the amount of time and energy that it takes, a slow reader is less likely to develop a strong interest in reading.  This will also negatively affect any future studies.

READING WORD BY WORD
When reading aloud, the student may feel that each word must be spoken correctly before they are allowed to move on.  Previous testing for decoding only may have demanded or encouraged this behaviour. A slow reading rate can often be linked to a lower comprehension rate.

REGRESSING
The student reads a word or group of words and then re-reads them.  If this occurs once or twice it is not significant, it is even normal.  But if there is a consistent pattern of going back and forth then the student is in danger of losing the essence of the text in an effort to decode each word correctly.

LOSS OF PLACE IN THE TEXT
This means the student has to stop and search the text to get back to where they were before they lost their place.  Momentum and cohesion are lost.

TRACKING
Tracking refers to running a finger along text as it is read.  This is not necessarily a negative behaviour; it can be most helpful.  However, if the finger is moving all over the place, or consistently back and forth, the student is losing track of text.

LOSS OF ATTENTION
Some students are easily distracted by what is going on around them and lose concentration. When their attention is divided by outside noises or movements, comprehension can be diminished or lost. While comprehension at a surface level may be retained, it is very difficult to get into the depths of a text in these circumstances.  Distracted behaviour may also indicate a difficulty in coping independently with the text or the content.

NOT REFERRING TO THE TEXT
Some students think that looking back at the text is cheating and that they must remember everything they have just read.  They look up, or down, around the room – anywhere but at the text.  Some have been so conditioned that they will close the book to put the text beyond temptation.

The outcome of this is that they do not read closely or think about deeper meaning of the text. Some will take a lot of persuading before they use the text.

Explain that while in some cases they may not be able to refer to the text, in the PROBE Reading Comprehension Assessment, it is most important that they do.  They will have to think beyond the literal or simple inference questions.  They will need to scan the whole text in some cases before a question can be answered.

Seating arrangement during an assessment can have an impact.

The seating arrangement during an assessment is an issue that needs some thought. The position of one in relation to the other may have an impact that has not been considered before.

Students and assessors may have preferred positions, but it is important for the assessor to consider the student and, if necessary, change the seating arrangement. At all times the assessor should consider what is best for the student.

Have the least number of possible distractors and stressors for the student. Some students prefer to maintain a degree of personal space. Some will lose focus if they are distracted by the activity or proximity of the assessor.

Think about your placement of the answer page. It is for you to refer to as you work, not the student. Make sure that you and the student can hear each other.

SITTING AT RIGHT ANGLES
THE ASSESSOR:
• may be able to see the reading age of a text, notes and answers.
• can make eye contact. to hear the student better
• can see everything that the student does. (could be intrusive)
THE STUDENT:
• is able to see the assessor
• can see how an assessor reacts to answers they give.
• can see when something is written, ticked or crossed.

SITTING OPPOSITE
THE ASSESSOR:
• may be able to hear the student better.
• can see everything that the student does. (could be intrusive)
THE STUDENT:
• is easily able to see the assessor. (could feel confronted)
• possibly see the reading age of a text, notes and answers.
• can easily make eye contact.
• can see how an assessor reacts to answers they give.
• can see when something is written, ticked or crossed.

SITTING BESIDE
THE ASSESSOR:
• cannot see everything that the student does.
THE STUDENT:
• is not easily able to see the assessor. (may feel more comfortable)
• cannot see the reading age of a text, notes and answers.
• cannot easily make eye contact.
• cannot easily see how an assessor reacts to answers they give.
• cannot easily see when something is written, ticked or crossed.

Best practice demands as few distractors & stressors for the student as possible.

At each assessment level there are two texts: one non-fiction and one fiction.  When conducting a 'full' PROBE assessment of their students some teachers elect to assess comprehension of just the fiction texts, or just the non-fiction texts.

Question: Is it best to test just fiction or just non-fiction?

Short answer: NO

Fiction and non-fiction results usually do not match.  It is common for differences in results to be six months to two years - in extreme cases, five years.  For a more complete picture of a student's comprehension skills both fiction and non-fiction must be assessed.

WHY THIS DISCREPANCY?
Usually, it is not because the student can't read the words.  The style of fiction and non-fiction writing differs greatly, and this difference is what contributes to the reader's mental images and understanding of the text.

Fiction stories generally involve language that is more metaphoric, more figurative and with more complex sequence shifts than non-fiction stories. Fiction is usually more conversational and requires readers to engage on a personal level.

The most obvious difference between fiction and non-fiction is that fiction is based on imagined events, noted as much for the way it is written as what it is about.

Non-fiction is based on real events, noted for the topic – what or who it is about. Examples of this difference can be found in any comparison of historical novels with the factual accounts of events of the same time. Anecdotal results from a wide range of schools over many years suggest that students' comprehension ability is greater in the writing style they are most familiar with.

So, if fiction is read either mostly, or to the exclusion of non-fiction, the fiction comprehension results tend to be higher. Fuller information enables a meaningful, targeted reading program to be implemented.

For these reasons, it is best that both text types are assessed.

While students or assessors may not like the style of the texts, or may not regard the content as interesting, it must be understood that they are not intended for enjoyment.

The texts are tools to help establish a student's reading comprehension ability. This is what matters and must be always kept in mind.