Evaluating Internet End-to-end Performance: Overview of Test Methodology and Results

OBJECTIVE: An evaluation of Internet end-to-end performance was conducted for the purpose of better understanding the overall performance of Internet pathways typical of those used to access information in National Library of Medicine (NLM) databases and, by extension, other Internet-based biomedical information resources. DESIGN: The evaluation used a three-level test strategy: 1) user testing to collect empirical data on internet performance as perceived by users when accessing NLM Web-based databases, 2) technical testing to analyze the Internet paths between the NLM and the user's desktop computer terminal, and 3) technical testing between the NLM and the World Wide Web ("Web") server computer at the user's institution to help characterize the relative performance of Internet pathways. MEASUREMENTS: Time to download the front pages of NLM Web sites and conduct standardized searches of NLM databases, data transmission capacity between NLM and remote locations (known as the bulk transfer capacity BTC), "ping" round-trip time as an indication of the latency of the network pathways, and the network routing of the data transmissions (number and sequencing of hops). RESULTS: Based on 347 user tests spread over 16 locations, the median time per location to download the main NLM home page ranged from 2 to 59 seconds, and 1 to 24 seconds for the other NLM Web sites tested. The median time to conduct standardized searches and get search results ranged from 2 to 14 seconds for PubMed and 4 to 18 seconds for Internet Grateful Med. The overall problem rate was about 1 percent; that is, on the average, users experienced a problem once every 100 test measurements. The user terminal tests at five locations and Web host tests at 13 locations provided profiles of BTC, RTT, and network routing for both dial-up and fixed Internet connections. CONCLUSION: The evaluation framework provided a profile of typical Internet performance and insights into network performance and time-of-day/day-of-week variability. This profile should serve as a frame of reference to help identify and diagnose connectivity problems and should contribute to the evolving concept of Internet quality of service.