Episodic SOAP

Vision Research 90 (2013) 43–51
Contents lists available at SciVerse ScienceDirect

Vision Research

j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / v i s r e s
Measuring reading performance
0042-6989/$ – see front matter � 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.visres.2013.02.015

⇑ Address: Department of Visual Neuroscience, UCL Institute of Ophthalmology,
London EC1V 9EL, United Kingdom.

E-mail address: [email protected]
Gary S. Rubin ⇑
UCL Institute of Ophthalmology, London, United Kingdom
NIHR Moorfields Biomedical Research Centre, London, United Kingdom

a r t i c l e i n f o
Article history:
Available online 16 March 2013

Keywords:
Reading
Low vision
Visual impairment
Outcome measures
Clinical trials
a b s t r a c t

Despite significant changes in the treatment of common eye conditions like cataract and age-related mac-
ular degeneration, reading difficulty remains the most common complaint of patients referred for low
vision services. Clinical reading tests have been widely used since Jaeger introduced his test types in
1854. A brief review of the major developments in clinical reading tests is provided, followed by a discus-
sion of some of the main controversies in clinical reading assessment. Data for the Salisbury Eye Evalu-
ation (SEE) study demonstrate that standardised clinical reading tests are highly predictive of reading
performance under natural, real world conditions, and that discrepancies between self-reported reading
ability and measured reading performance may be indicative of people who are at a pre-clinical stage of
disability, but are at risk for progression to clinical disability.

If measured reading performance is to continue to increase in importance as a clinical outcome mea-
sure, there must be agreement on what should be measured (e.g. speed or comprehension) and how it
should be measured (e.g. reading silently or aloud). Perhaps most important, the methods for assessing
reading performance and the algorithms for scoring reading tests need to be optimised so that the reli-
ability and responsiveness of reading tests can be improved.

� 2013 Elsevier Ltd. All rights reserved.
1. Introduction

In the early 1990s we obtained data from 1000 consecutive pa-
tients referred for low vision evaluation at the Johns Hopkins Wil-
mer Eye Institute low vision service (Unpublished data). An intake
questionnaire asked each patient to indicate the primary reason for
seeking referral to low vision. The results are shown in Fig 1. The
most common reason for referral was difficulty reading, which ap-
plied to over 60% of patients. The second most common reason was
difficulty driving, applicable to only 5% of patients. Similar results
have been published for other populations (see, e.g. Elliott et al.,
1997).

Since 1990 there have been significant improvements in the
treatment of eye disease – most notably the introduction of anti-
VEGF therapy for neovascular (‘‘wet’’) AMD. Yet reading difficulty
continues to be a primary concern for patients referred for low vi-
sion services. In a small but detailed study of patient expectations
prior to low vision rehabilitation 14 of 15 patients with AMD re-
ported that reading difficulty was a primary concern (Crossland
et al., 2007). Although we are inclined to interpret these findings
as an indication of the importance of reading in everyday life, there
is another possibility – that patients with reading difficulty are re-
ferred to low vision services because low vision rehabilitation is
most likely to improve reading performance through the prescrip-
tion of magnifiers. Other problems such as driving or recognising
faces are more difficult to address with current technology and pa-
tients with these problems may not be referred.

But in support of the ’’reading is important’’ explanation it is
also worth noting that most commonly used questionnaires for
assessing the various aspects of vision disability include one or
more items on reading difficulty. Popular instruments such as
the ADVS (Mangione et al., 1992) VF-14 (Steinberg et al., 1994),
NEI-VFQ-25 (Mangione et al., 2001), Massof Activity Inventory
(Massof et al., 2005) and many others include an item about diffi-
culty reading newsprint, and entire questionnaires have been
developed just to evaluate reading performance such as the Read-
ing Behaviour Inventory (Goodrich et al., 2006). Moreover, mea-
sured reading performance is among the best predictors of
patient-reported visual ability (McClure et al., 2000) and vision-re-
lated quality of life (Hazel et al., 2000).

Reading performance has been used as the primary outcome
measure for several clinical trials on the effectiveness of low vision
rehabilitation (see Binns et al., 2012) and as a secondary outcome
measure for clinical trials of pharmaceutical and surgical treatment
of various eye diseases including laser photocoagulation (Macular
Photocoagulation Study Group, 1991), submacular surgery (Haw-
kins et al., 2004), anti VEGF (Tufail et al., 2010) treatments for
AMD, and comparison of intraocular lenses following cataract

http://crossmark.dyndns.org/dialog/?doi=10.1016/j.visres.2013.02.015&domain=pdf

http://dx.doi.org/10.1016/j.visres.2013.02.015

mailto:[email protected]

http://dx.doi.org/10.1016/j.visres.2013.02.015

http://www.sciencedirect.com/science/journal/00426989

http://www.elsevier.com/locate/visres

0 10 20 30 40 50 60 70

Reading
Driving

Near Tasks
Distance Tasks

Mobility
Faces

Intermed. Tasks
Glare

Writing
TV

Percent

Fig. 1. Chief complaints of 1000 consecutive low vision patients seen at Wilmer
Low Vision Service (unpublished data).

44 G.S. Rubin / Vision Research 90 (2013) 43–51
extraction (Akutsu et al., 1992). Although reading tests have a long
history and extensive literature, there are still several controversial
issues about reading ability as a clinical outcome measure. One
question is whether standardised tests of reading performance in
the lab informs us about reading performance under real-world
conditions. A second issue is the relationship between self-re-
ported reading ability and measured reading performance. If the
two are in close agreement do we need to measure performance
– can’t we just ask the patient? And if the two disagree what can
we learn from the discrepancy. Finally there are practical questions
about how to best measure reading performance. To help put these
issues into perspective, it is useful to begin with a brief history of
clinical uncireading tests developed for ophthalmic research.
2. A brief history of clinical reading tests

Space does not permit a comprehensive review of reading tests,
but the following brief history of these tests highlights some of the
key issues about reading assessment that still concern us.

Although clinical reading tests seem to be a relatively recent
development, the first known test, developed by Eduard von Jaeger
in 1854 (Runge, 2000), actually predated the introduction of Snel-
len’s visual acuity tests in the 1870s (Fig. 2).

The Jaeger test types were based on a graduated series of sen-
tence fragments of decreasing size. In the US, some of the most
popular clinical reading charts still specify letter size using the Jae-
ger J1, J2, etc. notation. The J notation has been criticised for lack of
consistency across manufacturers and for the failure to follow a
meaningful size progression (Jose & Atcherson, 1977). However
the original Jaeger texts followed a strict geometric progression,
foretelling the introduction of the Bailey–Lovie Near Reading Card
Fig. 2. Original Jaeger test types in German, French and English (from Runge
(2000)).
by over 125 years. When the Jaeger charts were first published in
the US using local typefaces they lost their original calibration.

A noteworthy development in clinical reading tests was the Slo-
an Continuous Text Read Cards, with text size specified in M units
(Sloan & Brown, 1963).

Actually, the M unit was promoted and used by Snellen and he
tried to convince Jaeger to specify his test types in M units. M nota-
tion designates the distance (in metres) at which the object sub-
tends 5 minarc. Therefore 1M print subtends 5 minarc at 1 m.
The Sloan reading cards present a short text passage at one size
per card (Fig. 3) The amount of text varies with letter size from a
few words at 20M to an entire paragraph at 1M. Though popular
in low vision clinics, M notation has not been widely adopted else-
where in clinical ophthalmology.

The next significant advance in reading assessment was the
introduction of the Bailey–Lovie Near Reading Card in 1980 (Bailey
& Lovie, 1980).

Bailey–Lovie cards present two to six unrelated words per line
and the size of the text decreases by a constant percentage from
line to line (Fig. 4) Letter size is represented in LogMAR units
(log10 of the minimum angle of resolution). Though sometimes
criticised because some of the words are quite long (up to 10 let-
ters) and difficult for poor readers, the Bailey–Lovie near cards
are still widely used for determining the magnification required
to read normal print sizes.

A rather unusual reading test, the Pepper Visual Skills for Read-
ing Test (VSRT) was published in 1986 (Baldasare et al., 1986) by
Watson and colleagues at Pennsylvania College of Optometry.
The VSRT progresses from well-spaced individual letters, to
crowded letters, digrams, trigrams, words and words arranged in
a paragraph style (Fig. 5). Unrelated words are used throughout.
The test is timed and scored by adding together the number of cor-
rect letters, digrams, trigrams, and words read, but the test is said
to measure print recognition and navigation skills rather than the
amount of magnification required.

Legge and colleagues introduced the MNREAD Test in 1989
(Legge et al., 1989a). Originally a computer-based test, MNREAD
was soon converted to printed cards (Fig. 6).

The original MNREAD Test consisted of both sentences and
groups of unrelated words rendered in a fixed letter size that sub-
tended 6� at a 20 cm viewing distance. The large print size was de-
signed to measure maximum reading speed rather than reading
Fig. 3. Louise Sloan’s continuous text reading cards with letter size specified in M
units (see text).

Fig. 4. Bailey–Lovie word reading card illustrating logMAR progression of letter
sizes.

Fig. 5. The Visual Skills for Reading Test (Pepper Test) progresses from single letters
to sequences of unrelated words.

Fig. 6. The MNREAD reading chart consists of standardised sentences displayed in a
wide range of letter sizes. The size decreases in a logarithmic fashion with smaller
letters on the reverse side of the chart (not shown).

Fig. 7. The Colenbrander mixed contrast reading card is composed of two-line
sentences that follow a logarithmic progression of letter sizes. Lines alternate
between high (>90%) and low (10%) contrast.

G.S. Rubin / Vision Research 90 (2013) 43–51 45
acuity. The large print cards were replaced by the MNREAD Acuity
Chart, which was designed to measure reading acuity and maxi-
mum reading speed (Mansfield et al., 1993; Mansfield, Legge, &
Bane, 1996). The MNREAD Acuity Chart consisted of a series of
60-character sentences displayed on two lines. The sentences de-
crease in size by 0.1 log unit from a maximum of 1.3 logMAR
(equivalent to 20/400 or 6/12 when viewed at 40 cm) to �0.5 log-
MAR (20/6 or 6//2). One advantage of using logMAR scaling of let-
ter size is that the range of print sizes (angular subtense) can be
extended by changing the viewing distance.

With the MNREAD Acuity Chart, reading acuity corresponds to
the smallest letter size that can be read and maximum reading rate
is the number of words read correctly per minute for the sentence
with the shortest reading time. A third parameter, critical print
size, is the smallest letter size that can be read at the maximum
speed and is an indication of the minimum magnification required
for best reading, Several variations on the methods of computing
maximum reading rate and critical print size have been proposed,
(Patel et al., 2011) and these will be discussed below.

Several of the more common reading tests are available in mul-
tiple languages. But one test was developed specifically for cross-
language comparisons. The International Reading Speed Texts (IR-
eST) are paragraphs of about 170 words (in the English version)
that are carefully equated across languages for word frequency
and syntactic complexity. Originally published in four European
languages, (Hahn et al., 2006) IReST was recently expanded to 17
languages with normative data for normally sighted young adults
(Trauzettel-Klosinski, 2012).

In addition to the reading tests described above, which use
short selections of high-contrast text, there are several special-pur-
pose reading tests that are also worth mentioning. Colenbrander
(Dexl et al., 2010) has developed a mixed contrast reading chart
with alternating lines of high and low (10%) contrast words (Fig. 7).

46 G.S. Rubin / Vision Research 90 (2013) 43–51
The lines decrease in letter size, similar to the Bailey–Lovie card
and the test is designed to screen for contrast and reading deficits
simultaneously.

A radically different mode of text presentation is used for the
RSVP test. The name stands for Rapid Serial Visual Presentation
and was first used in 1970 by Forster (1970) to study cognitive pro-
cessing during reading. With RSVP, single words are presented
sequentially at a fixed location on a video display. The sequence
is illustrated in Fig. 8. In 1994, we (Rubin & Turano, 1994) intro-
duced RSVP as a means to overcome difficulty generating efficient
saccadic eye movements when reading with a non-foveal preferred
retinal locus (PRL).

However we observed that people with intact central vision
read 2 to 4 times faster with RSVP compared to conventional static
presentation while those with central scotomas read only about
40% faster with RSVP (Rubin & Turano, 1994). Eye movement
recordings revealed that people with central scotomas still made
intra-word saccades when reading with RSVP, presumably because
their restricted visual span (Legge et al., 1997) made it difficult to
recognise a word with a single fixation. Nevertheless, RSVP contin-
ues to be used to isolate visual processing and reduce the influence
of eye movements during reading and to control where on the ret-
ina text is presented.

Possibly the newest clinical reading test is one designed by
Ramulu and colleagues (Ramulu et al., 2013) to evaluate sustained
reading. Until recently, all reading tests used relatively brief pas-
sages of text – usually no more than 200 words. However, a fre-
quent complaint of readers with low vision is that while they can
read a few words or sentences with appropriate magnification,
they cannot sustain reading for longer than a few minutes. The
new sustained reading test measures reading speed over 30 min
of silent reading using 7000-word stories followed by 16–20 com-
prehension questions. The sustained reading test has been shown
to be a valid and reliable measure of sustained reading perfor-
mance (Ramulu et al., 2013).

The Salzburg Reading Desk (Dexl et al., 2010)s takes a very dif-
ferent approach to measuring reading performance. Instead of pre-
senting text printed on a card or on paper, the SRD displays text on
a high-resolution computer monitor (Fig. 9).

One either side of the monitor are IR cameras that capture an
image of each pupil and use the distance between pupil centroids
to determine viewing distance with much greater accuracy than
can be done with a tape measure or knotted length of string. The
SRD also has voice detection to accurately measure the beginning
and end of a trial. The SRD can display letters, words, and short
paragraphs in random order and adjusted to the viewer’s preferred
letter size or to follow an adaptive staircase technique for efficient
measurement of reading acuity and critical print size. However,
Fig. 8. Demonstration of rapid serial visual presentation. Single words are
presented sequentially, centred on a fixed location. RSVP is used to measure
reading speed without the need for eye movements.
computer monitors need to be carefully calibrated to ensure that
the text is of appropriate luminance and contrast if one wishes to
generalise to reading printed text.

3. What do clinical reading tests tell us about reading in the real
world?

Clinical reading tests are thoroughly standardised and highly
artificial. The content is carefully controlled as are the lighting con-
ditions, viewing distance, letter size and contrast. But when we
read at home or while out shopping, all of these factors are allowed
to vary. Can we learn anything about real-world reading from
standardised laboratory tests?

The Salisbury Eye Evaluation (SEE) Study looked at this question
in some detail (West et al., 1997). One hundred participants were
selected at random from the original group of 2520 SEE study par-
ticipants living in Salisbury, MD. All were between the ages of 65
and 85. The participants had been to the SEE clinic to have their vi-
sion tested, to answer questionnaires about difficulty with daily
activities and to have their reading performance assessed with a
computer-based reading test. Short paragraphs (�100 words) were
displayed on the computer monitor for 15 s and the participant
read the words aloud. The time to read the text was measured with
a stopwatch, the number of words read correctly were counted and
reading speed in words/minute was computed. Letter size varied
from 0.1� (20/30 or 6/9) to 0.5� (20/120 or 6/36) in equal logarith-
mic steps.

For the home reading test, participants were asked to read
aloud a paragraph selected from a local newspaper. The participant
arranged the lighting, chose the viewing distance, and was free to
use any vision aids that were customarily used. The results are
shown in Fig. 10. The graph plots reading speed at hone as a func-
tion of reading speed for the largest print (0.5�) in the clinic.

The correlation is quite high (r = 0.87) but the regression line
(solid) deviates from the line of equality (dashed). The regression
equation.

Home reading rate ¼ clinic reading rate � 0:7 þ 24:7:

indicates that slower readers do better at home, where they can
make full use of whatever adaptations they are accustomed to
using. Faster readers do better in the clinic. The reason for this is un-
clear as we would expect fast readers to be less susceptible to envi-
ronmental factors such as lighting and show less benefit from the
high luminance and high contrast of the clinic test. But the same
Fig. 9. The Salzburg Reading Desk uses modern computer technology to present
text in random order while measuring reading distance with IR cameras and
reading speed with voice detection.

G.S. Rubin / Vision Research 90 (2013) 43–51 47
effect was observed for other visually demanding tasks such as find-
ing and dialling a phone number.
4. Do we need to measure reading performance? Can’t we just
ask the patient?

With the current prominence of patient-reported outcome
measures it is tempting to conclude that performance-based read-
ing tests are no longer necessary. All we need to do is ask the pa-
tient whether he/she has any difficulty reading. However, it has
been shown (Guralnik et al., 1989) that performance-based test
provide better discrimination in ability level than self report, are
earlier predictors of functional decline and disability and are less
influence by the participants’ sociodemographic, psychosocial,
and cognitive characteristics. Also, performance-based tests are
independent predictors of morbidity and mortality, even after tak-
ing self-report into account.

But how well do patient-reported reading difficulty and mea-
sured reading performance agree, and when they disagree does
this provide any interesting information about the patient or is it
just a reflection of the imprecision of our measurement tools?

Again we can look to the SEE study for some hints (Friedman
et al., 1999;). SEE included both patient-reported difficulty reading
via the Activities of Daily Vision Scale (ADVS) (Mangione et al.,
1992;) and the performance-based reading test described above.
The ADVS includes a question about difficulty reading newsprint
with response options of ‘‘no difficulty’’, ‘‘a little difficulty’’ ‘‘mod-
erate difficulty’’. ‘‘a lot of difficulty,’’ and ‘‘can’t do’’ (because of vi-
sion problems). Responses to the newsprint question were
compared to reading speeds for the text closest in size to news-
print (0.3�). We considered reading speeds greater than 80
words/minute as ‘‘functional’’ reading and reading speeds greater
than 160 words/minute as ‘‘fluent’’ (Carver, 1992;). 49.1% of SEE
participants reported no difficulty reading and read fluently by
our definition, while 3.7% reported at least moderate difficulty
reading and read at less than a functional level. In both cases, pa-
tient-reported reading difficulty is concordant with measured
reading speed. However, 6.4% were slow readers (less than func-
tional reading speed) while reporting no difficulty and 1.5% read
fluently while reporting at least moderate reading difficulty. For
the majority of participants’ self report is in agreement with their
Fig. 10. Comparison of reading rate under standardised laboratory conditions to
reading rate under natural conditions at home. Solid line is least squares regression
line. Dashed line indicates equality between lab and home.
measured performance (concordant, unmarked entries in Table 1).
But 7.9% show a significant discrepancy between self report and
measured reading speed (discordant, single asterisk) and a further
33.8% are mildly discordant (double asterisked entries in Table 1).
Some of the discrepancy undoubtedly reflects measurement error,
but an analysis of the characteristics of discordant readers (Fried-
man et al., 1999) suggests a more interesting explanation. When
we looked at the vision test results (acuity, contrast sensitivity,
glare sensitivity, stereoacuity, and visual fields), all showed a sim-
ilar pattern of results: visual function for discordant participants
was intermediate between results for fast concordant and slow
concordant readers. So, for example, distance acuity averaged
�0.04 logMAR (±04 S.E) for fast concordant readers (read fluently
and report no difficulty), 0.15 logMAR (±0.01) for slow discordant
readers (slow readers who report no difficulty) and 0.40 logMAR
(±).02 S.E.) for slow concordant reader (read slowly and report dif-
ficulty). Furthermore, 80% of discordant readers showed concor-
dance between measured performance and self report when
reading text of a larger print size.

Taken together, these results suggest that a discrepancy be-
tween performance-based tests and self report may be indicative
of patients who are at a transition between visual ability and dis-
ability where visual function has begun to decline but the person
is able to maintain (or at least thinks they can maintain) good per-
formance, possibly through modification of the task. In the geriat-
rics literature this is referred to as ‘‘preclinical’’ disability and is an
important predictor of future disability if left unattended (Fried
et al., 1991).

The association of visual acuity with concordance/discordance
described above does not mean that a simple test of letter acuity
will substitute for measuring reading performance. In a study of
40 patients with AMD, visual acuity was not correlated with read-
ing speed, even for text that was magnified to greater than the crit-
ical print size (r = 0.26, p > 0.1 (Rubin & Feely, 2009)).
5. How should we measure reading performance?

If we accept that clinical reading tests are informative about
everyday reading outside the clinic, and that the measurement of
reading performance provides additional information that is not
captured by self-report alone, then we must ask how should that
performance be measured? As the review above makes clear, there
are many different types of reading tests. It is natural to ask which
test is ‘‘the best.’’ However, the optimal test will depend on how it
is to be used. If an investigator wants to know whether a pharma-
ceutical treatment retains or restores vision, as measured by the
ability to read small print, then a test with multiple print sizes held
at a fixed distance (such as MNREAD) may be most suitable. But if
the investigator needs to evaluate how well a patient reads ordin-
ary text with available low vision aids then a test with longer pas-
sages of fixed print size (such as IReST) viewed from a distance that
is appropriate for the low vision aid may be more appropriate. Nev-
ertheless, there are certain well-accepted standards for comparing
Table 1
Comparison of Self-reported reading difficulty with measured reading speed.

Measured reading speed Self-reported difficulty reading
newsprint (%)

Moderate A little None

Slow (<80 words/min) 3.7 3.5 6.4�� Functional (80 6 words/min < 160) 2.1� 5.4 21.3� Fluent (P160 words/min) 1.5�� 6.9� 49.1 Unmarked values are concordant, in italics with double asterisks are strongly dis- cordant, and in italics with single asterisk are mildly discordant. 48 G.S. Rubin / Vision Research 90 (2013) 43–51 and selecting among tests. These are based on demonstration of the test’s validity (does the test measure what it is intended to measure?), reliability (are the measurements consistent and repeatable?) and responsiveness (is the test able to measure change?). Tests used for diagnostic purposes also need to be eval- uated for sensitivity and specificity, but since we are not proposing that reading tests be used to aid diagnosis, sensitivity and specific- ity are of less importance. None of the reading tests has been thoroughly evaluated for validity, reliability, and responsiveness in visually impaired read- ers. In most cases, the evaluation has been restricted to test–retest variability and often limited to readers with normal vision. Few studies have made direct comparisons between tests and compar- ing across studies is difficult when the testing conditions and sub- ject characteristics differ. Clearly, more data are needed to determine the psychometric properties of available reading tests. Despite 150 years of development and refinement of clinical reading tests, there are still several points of disagreement. The first is what should be measured? In developing the scoring algo- rithm for the MNREAD Test, Legge and colleagues (Mansfield, Legge, & Bane, 1996) defined three parameters: reading acuity (the smallest print that can be read, however slowly), maximum reading rate (the fastest reading rate regardless of print size) and critical print size (the smallest letter size that allows reading at the maximum rate). There is little controversy about reading acu- ity. Following Bailey’s recommendation for scoring letter acuity charts, reading acuity is scored by counting the number of words read correctly, until the participant no longer identify the text, and the count is converted to a LogMAR value that takes viewing distance into account. Maximum reading rate and critical print size are not so simple. There at least four methods for calculating max- imum reading rate and four for critical print size. The various methods are described and compared in a recent paper (Patel et al., 2011) and there is not space here for a thorough discussion of the pros and cons of each method. Briefly, most of the definitions rely on an underlying model for the shape of the reading rate vs. letter size function. This function is thought to rise rapidly from 0 words/minute at the reading acuity until it reaches a plateau at the maximum reading rate. The critical print size is at the ‘‘knee’’ between the rising part of the function and the plateau. Real data show that patients who have very poor vision may fail to reach a plateau and even for patients with good vision, it is sometimes dif- ficult to discern which points belong to the plateau., The uncer- tainty results, in part, from imprecision in the measurement of reading speed when using short, 60 character sentences. The reac- tion time of the experimenter when using a stop watch to time each sentence, pauses, false starts, time taken to self-correct read- ing errors, and other ‘‘glitches’’ by the reader, all lessen the preci- sion and repeatability of reading speed measurements. A study of the test–retest variability of the MNREAD Test with a group of AMD patients participating in a clinical trial of anti-VEGF therapy (Patel et al., 2011) reported coefficients of repeatability of 0.30 log- MAR for reading acuity, about 0.55 logMAR for critical print size, and more than 60 words/minute for maximum reading rate. The exact values depended on the definition of maximum reading rate and critical print size used. Another study conducted in a labora- tory setting with highly trained researchers and less fatigued pa- tients produced much better coefficients of repeatability …