On Friday March 29, 2019 ACASE Associates and UAlbany PhD Students Panpan Yang and Monica De Tuya, along with co-author and ACASE Director Paul Zachos, will present their poster Practical Reliability for Teachers at the 14th Annual New Trends in Information Science Research (NTIR) Conference at The University of Albany. This research describes and evaluates a new approach to measuring the reliability of educational information that is practical and useful for supporting daily classroom practice as well providing a tool for social science researchers.

Reliability studies can be a valuable, practical tool for teachers and students as well as for social scientists. But, there are two shortcomings associated with conventional method of calculating reliability:

1. Measurement: Current methods of calculating reliability are too complex for those who are not versed in psychometrics (e.g., classroom teachers) to apply or even to interpret.

2. Information: Current methods of measuring reliability are tied to conventional tests scores derived from composite measures (aggregations of distinct and different learning outcomes). The result of this type of aggregation is that the salient information concerning these distinct learning outcomes is obscured.

The poster illustrates the power of inter-rater agreement (and the value of disagreement!) as a tool for monitoring the reliability of information on the attainment of learning outcomes.  It can serve as a generic tool for enhancing the quality of research information in the social sciences, and produces simple to interpret and useful information for improving daily classroom activities.