Feedback from end-users of the service was anecdotal and
self-reported. Often end-users would
express their opinions about the value and usefulness of the service in email
messages or at the conclusion of an interactive online reference session.
Generally the end-users of InfoEyes
were very satisfied with and excited by this new service. They appreciated an online service that was
designed specifically for them. Several InfoEyes patrons became frequent repeat customers of the
service. They also were ardent
supporters.
The accessibility challenges associated with the InfoEyes service were varied. They were not completed resolved during the
beta phase. The text chat aspect of the InfoEyes service used an interface that refreshed the
screen every 15 seconds. This created
accessibility challenges for individuals who use screen reader software, such
as JAWS for Windows and WindowEyes. When a software program or website has an
embedded automatic screen refresh capability, the refreshed screen often is
interpreted by screen reader software programs as an entirely new screen. The screen reader software program will begin
reading aloud the contents of the screen each time it refreshes.
We also experienced some accessibility challenges with the
various enhanced reference service options we tested. For example, the text chat window within QuestionPoint Enhanced Communications was difficult or
impossible to read with screen reader software when the window was in its
default "docked" position within the overall QPE interface. The simple workaround to this accessibility
challenge was to ask each reference librarian on duty to "undock" the
text chat window whenever he or she answered a request to initiate a QPE
interaction. A blind programmer also was
hired to develop scripts so that JAWS would work with QuestionPoint
Enhanced. This solution, however,
required a blind end-user to now download both the QPE end-user client and the
set of JAWS scripts before he or she could even ask a question.
The voice-over-IP aspects of the various software options we
tested did not in-and-of-itself present major accessibility challenges. Most InfoEyes
patrons appreciated being able to hear the reference librarian's voice coming
over the Internet, even if the patron did not have a microphone to speak back. Although it initially seemed
counterintuitive, enhanced virtual reference interactions in which the
reference librarian was communicating primarily via VoIP
and the patron primarily via text chat seemed to work well and to the
satisfaction of both parties.
Patrons who used the InfoEyes
service during the beta test period were generally satisfied with the
service. Satisfaction parameters
included the thoroughness of the answers, the quality of the answers, and, for
email questions, the turnaround time.
The users who tried InfoEyes
during the beta testing phase and provided feedback tended to highly value the
service. For many individuals, InfoEyes was the first online reference and instruction service
designed specifically to meet their needs.
Some individuals specifically praised the value of the instructional
sessions, because it helped them become independent seekers and users of
information resources on the Internet and the Web.
The email reference module within QuestionPoint
worked quite well. Not only was it easy
for patrons to pose questions and for InfoEyes
service providers to respond, but it also was very easy to manage the overall
flow of email questions. InfoEyes service providers could easy monitor the progress
of responding to email questions.
From the perspective of the InfoEyes
project, the basic drawback of the basic text chat module of QuestionPoint is that the screen automatically refreshes
every 15 seconds. For a blind or
visually impaired individual using screen reader software, such as JAWS for
Windows or WindowEyes, each screen refresh is
interpreted as a new screen to be read.
This was very annoying to users.
After trying QuestionPoint
Enhanced for several weeks, the InfoEyes Advisory
Council decided to suspend active testing of this module with the InfoEyes service population. The VoIP
functionality performed inconsistently, often with long transmission
delays. The text chat box had to be
maximized and “undocked” by the reference librarian in order to improve the
accessibility of the contents of the text chat for end-users using screen
reader software. The end-users had to
download a plug-in to enable VoIP, co-browsing, and
application sharing.
As with QuestionPoint Enhanced Communcations, with iVocalize
end-users are required to download a plug-in in order to access the room. Downloading the iVocalize
plug-in, however, was fairly transparent to the user and generally easy to
implement. Although the process of
downloading the iVocalize plug-in was very easy, in a
few instances patrons expressed problems or concerns about having to download a
plug-in. Downloading the QuestionPoint enhanced end-user client was a more elaborate
process, requiring a user-initiated software installation.
The iVocalize software was very
accessible to blind and visually impaired users. The VoIP function
worked well and reliably, with little delay time. Patrons seemed to appreciate the VoIP function even if they did not have a microphone at
their end and had to resort to using text chat to communicate. The ability to co-browse with patrons
throughout the web, even into proprietary databases, was very useful in
numerous reference and online instruction interactions. The software can be configured so that the
URLs of all webpages visited are automatically added
to the text chat transcript. The
transcripts are easily saved.
It is possible to record the audio portion of any
interaction in an iVocalize online room. However, the InfoEyes
Advisory Council decided not to use this feature of the system, because we
wanted to protect patron privacy and confidentiality. To further ensure patron confidentiality, we
set up the InfoEyes iVocalize
room so that only two people—the reference librarian and the patron—could be in
the room at any one time.
The fact that iVocalize did not
directly interface with the QuestionPoint system
caused some service and administrative challenges. Because the iVocalize
software is not designed specifically for providing reference service, it also
lacked some features commonly found in VR software systems, such as the ability
to create and easily insert scripted messages.
The iVocalize software worked very
well for online end-user group training sessions. The instructor was able to co-browse into
password-protected databases for the purposes of demonstration and
instruction. Much peer-to-peer sharing
of tips occurred during these sessions, too.
The iVocalize software from Talking
Communities also was used to provide group training sessions for InfoEyes service providers.
The software proved very useful for this purpose.
All of the meetings of the InfoEyes
Advisory Council were held in a large iVocalize
online meeting room. Members of the
Advisory Council included sighted, partially sighted, and blind
individuals. The software worked well
for these meetings.
Numerous online orientation and training sessions were
provided for InfoEyes service providers by OCLC and
the Core Project Team. These sessions
were well-attended by staff members from the participating libraries. The mentor system that assigned a member of
the Core Project Team to each participating library seemed to facilitate
communication and encouraged continued commitment to the development, testing,
and deployment of the InfoEyes service.
Providing technical support to participating libraries and
patrons was a joint effort of OCLC, Talking Communities, and the Core Project
Team. Other than the core accessibility
challenges outlined above, generally technical issues were resolved quickly and
satisfactorily via this team approach.
Creating a “desk” schedule involving so many states and time
zones was a complex, arduous task, but overall the scheduling process worked
well. Time zone differences and the fact
that one participating state did not observe daylight savings time created some
confusion. The process of resolving last
minute reference desk substitutions was not as efficient as the overall
scheduling process.
During the beta testing period the Core Project Team sought external
funding for InfoEyes by applying for a major grant
from a federal agency. Shortly after the
conclusion of the beta phase of InfoEyes we learned
that our application had not been funded.
The governance structure and process have worked quite well,
and they already have received some substantial tests. For example, the decision to switch from the QuestionPoint Enhanced Communications software platform to iVocalize as the basis for the enhanced InfoEyes
service was not a trivial decision. The
decision to drop the basic text chat option also was a substantial
decision. The failure to attract
external funding on the first attempt could have severely damaged or killed the
project, but the participating libraries, working within the existing
governance and management structures, was able to determine a way to sustain InfoEyes on a minimal budget and with a well-distributed
in-kind contributions plan.
Statistical reports about the activity in the InfoEyes reference service, including the use of the free
trial FirstSearch databases, were compiled and
distributed each month. The cumulative
report for the first six months of activity is contained in Appendix A. The Advisory Council initially decided that
the standard monthly reports provided by QuestionPoint
were not sufficiently detailed for the purposes of a beta test, so each
reference transaction during the beta test period of reviewed and coded
appropriately.
Already in 2004 the program website (www.infoeyes.org) has gone through several major
revisions.
Maintaining the website has been difficult. More attention needs to be paid to ensuring
the out-of-date information is removed and that all current information is
accurate.
Summary
data about the usage of the InfoEyes website can be
found at http://www.infoeyes.org/stats.
Note: Please see
Appendix A for a complete summary of the usage of the InfoEyes
service during the beta testing phase.
·
Total
Sessions: The sum of all the discrete sessions recorded
in the set of closed interactions for the month being reported.
·
Countable
Sessions: Two situations caused some sessions not to be
counted:
1) If two or more sessions in close
temporal proximity to each other were clearly from the same patron with the
same information need, the cluster of sessions was counted as only one
session.
2) If a session clearly displayed a
failure to communicate (e.g., because of basic system problems,
or the patron never responded and expressed the need), it was not counted as a
session.
·
Real
Sessions: Sessions that appeared to involve patrons
with real information needs.
·
Interest
Sessions: Sessions that appeared to be from non-InfoEyes team members who were more interested in just
seeing or testing the software or service than in satisfying a
real information need. Often
these visitors were professional colleagues from other libraries.
·
Practice
Sessions: Sessions that appeared to be test and
practice sessions between two or more InfoEyes team
members.
·
Timestamping: Sessions are timestamped
when they commence. In the case of email
questions, that means the time they are submitted, not when they are answered
by an InfoEyes reference provider. The timestamp is
Note: The
InfoEyes service had a soft launch on
A total of 295 total sessions occurred during the first six
months of the InfoEyes service. Of those, 233 were defined as countable
sessions, using the criteria outlined above.
Of the 184 days during the six-month period, 102 (55.4
percent) contained at least one InfoEyes session. The average number of countable sessions per
active day for the six-month period was 2.28.
However, there was a noticeable downward monthly trend in the average
number of countable sessions per active day, starting in May with 5.14 and
concluding in August with 1.17.
Of the 233 countable sessions, 161 (69.1 percent) were
“real” (i.e., based on the logs, these appeared to be real information needs
expressed by real patrons), 25 (10.7 percent) were “interest” sessions (i.e.,
the person was interested mainly in learning more about InfoEyes),
41 (17.6 percent) were practice sessions involving two or more InfoEyes service providers, and 6 countable sessions (2.58
percent) were of unknown basic intention.
The day of the week during which the 233 countable sessions
were initiated was fairly evenly spread across the seven days of the week. Thursday was the most active day of the week,
during which 60 (25.75 percent) of the countable sessions were initiated, and
Sunday was the least active day of the week, when only 7 (3 percent) sessions
were initiated.
In terms of how the countable sessions spread out across the
24 hours of the day, there were two peaks of activity: from 10-11 in the morning, and from 3-4 in
the afternoon. This pattern of a late
morning and afternoon peak in demand appears to be fairly common among online
library services, remote use of online catalogs, etc. These hourly data, however, must be
interpreted with caution for several reasons.
First, all session activity for the email and basic text chat sessions
were time-stamped using a clock set to
Of the 295 total sessions initiated during the first six
months of the InfoEyes online reference service, 114
(38.64 percent) were enhanced sessions, 112 (37.97 percent) were email
questions, and 69 (23.39 percent) were basic text chat sessions. If, however, the March and April usage data
are removed, beginning in May and continuing through August the utilization of
basic text chat (just 16 total sessions) and enhanced (just 7 total sessions)
has been very low. The use of email as
an interaction mode has remained much more consistent across the entire first
six months of the service. Please bear
in mind, however, that no enhanced online reference service communication mode
was offered from April 29 until June 7 as the InfoEyes
service providers prepared to switch from using QuestionPoint
Enhanced to iVocalize from Talking Communities.
The average response time for all email questions was 17 and
two-thirds hours. The average basic text
chat session lasted a little over 9 minutes, and the average enhanced session
lasted slightly more than 4 and one-half minutes.
Of the 12 databases that were available for trial use, WorldCat (24 percent) received the most use, followed by ArticleFirst (21 percent) and Periodical Abstracts (20
percent). The trial passwords were
shared both with InfoEyes service providers as well
as with patrons who attended the group training sessions during the summer
months.
Several InfoEyes users reported
that the group training and orientation sessions on Google
and FirstSearch were fantastic. The sessions were held during June and
July. Separate clusters of sessions
covered introductory and intermediate Google
searching, as well and introductory and intermediate FirstSearch
searching. The sessions introduced small
groups of users to the overall InfoEyes service, and
actually created some activity with the one-on-one reference service.
InfoEyes was announced and
promoted through messages to email discussion groups, blog
postings, and printed and electronic newsletters distributed by the
participating libraries. Several
presentations at conferences, as well as one published article, served further
to raise awareness about InfoEyes.