Assessing the Accuracy of Small Area Forecasts Continued from page 15 Because an ArcSDE-enabled geodatabase was used, multiple users could access a single feature class, which allowed rapid analysis of the data and the performance of data management tasks. At the end of each workday, the database was sorted for unique values, which identified data entry anomalies. A daily sort also provided a count of the number of records per analyst. This determined the rate of data entry per  The block could not be visually inspected due to issues with the satellite imagery (e.g., date of image not current, tree or cloud coverage masking structures, poor image resolution). Reviewers were asked to supply additional information about the block such as housing type (single family, mobile home, or multiunit structure), the presence of a new development, or the amount of developable land area remaining for future housing growth. Reviewers chose from among a predefined set of responses for each characteristic but could add notes in a comments field. Project Evolution and Highlights The project leader actively monitored the team from start to finish. This enabled developers to focus on quickly adapting applications to enhance their efficiency. Team feedback led to changes that resulted in a more robust GUI. The GUI was adjusted numerous times over the course of the project to improve productivity and ease of use, particularly by enabling the reviewers to input data with a minimal number of clicks. Rapid prototyping helped the team fine-tune the methodology, input data, application, and training to suit the areas being analyzed. During the early phases of the project, the Production Line Tool Set (PLTS) extension [now known as Esri Production Mapping] was used to prototype efficient workflows for block analysis. ArcGIS, with its customizable interface, allowed numerous refinements over the course of the project. Quality assurance was monitored each week. Some reviewers were able to modify code to streamline their own workflows, tripling throughput from 25 to 75 blocks reviewed per hour. These productivity gains allowed the project's total block review goal to be met sooner than expected, and more blocks were reviewed than had originally been planned for the project. Another unexpected benefit of this manual review process was the team's ability to adapt quickly to housing units unique to different areas (e.g., shotgun duplexes in New Orleans, retirement communities on golf courses, or mobile home parks). The rate at which features were analyzed increased every day of the project, and the quality of the information was monitored daily to ensure consistent application of counting and reporting procedures. Conclusion In the end, the team reviewed 2,596 block groups, or 112,033 blocks, in the time allotted for the project. Although this is only a fraction of the total of 208,000 block groups, the selection process focused on neighborhoods with the most significant change in the past decade. This excluded more than 90 percent of the block groups. Less than 9 percent of all block groups display a change of ±2.5 percent yearly. Thanks to the GUI and feedback from the review staff, the initial project milestone was reached ahead of schedule and below budget estimates. Ultimately, the team was twice as productive as had originally been estimated. By combining multiple authoritative sources of information on the map, the team was able to identify areas of agreement quickly among those sources. Given the project's purpose—assessing demographic data sources—just identifying disagreement among the sources furnished useful information. Most important, the project provided ground truth confirmation and improved the accuracy of Esri's 2010/2015 forecasts. For more information, contact Douglas Skuta at dskuta@esri.com. Two images, taken at different times, were spliced together. The newer image provides evidence of residential development and enabled a straightforward comparison of current and alternate estimates. analyst and helped in assigning each analyst to regions that best fit that analyst's skills. The entry form and multiuser environment combined to facilitate the well-organized, efficient analysis of the blocks under review. Another application, created to complement and enhance the productivity and accuracy of the visual review, allowed the reviewer to click on an area and choose either Google's Street View or Microsoft Bing Maps Bird's Eye tool to get an online, oblique view of an area without leaving the application. This Visual Basic script, assigned to a tool and added to the toolbar, automatically initiates a Web browser, inputs the URL link to the online imagery service, and provides the coordinates so that the user can inspect the area using detailed imagery from these sources. The Review Team This visual review was performed by a team of 13 temporary employees. Half the team worked full-time, and the balance worked part-time schedules. After the team members underwent an orientation to learn software and data procedures, they were also taught many tips and techniques to increase efficiency and enhance workflow volumes. After reviewing each block with respect to the satellite image overlays, reviewers chose one of four possible outcomes:  The residential structure count matched the 2009 housing estimate.  The residential structure count did not match the 2009 housingestimate.  The block boundaries were misaligned with the structures within the imagery, preventing a confirmation count. 16 ArcUser Fall 2010 www.esri.com