(P 222) DO WE ALL SEE IT THE SAME: AN INTER-RATER RELIABILITY ANALYSIS OF THE MUSCULOSKELETAL TUMOR REGISTRY DATA ELEMENTS

Objective: The Musculoskeletal Tumor Registry (MsTR) is designed to understand clinical data and patient-reported outcomes following treatment for musculoskeletal malignancies. The initial Sarcoma Module affords a level of detail regarding the nature of the disease and treatment not previously seen outside of single-institutional databases. However, the utility of the MsTR in providing research-quality data depends upon ensuring reliable and accurate data entry. To assess data integrity, the current analysis aims to: 1) understand data elements for which MSTS members do not have consensus regarding how best to code into the registry, and 2) to develop a data dictionary to advise as to how registry coding should be performed.

Methods: A series of surveys were conducted of MsTR Steering Committee Members and MsTR users to identify data elements for which uncertainty may exist as to the optimal registry coding. Once these were compiled, a survey was completed by 60 MSTS members involving the data entry for 35 clinical vignettes related to the areas of question. Consensus was defined as at least 90% agreement. A modified Delphi method was utilized to establish a consensus on a data dictionary to inform guidance as to how registry elements should be coded in previously ambiguous clinical scenarios.

Results: The MsTR member survey highlighted areas of inconsistency or confusion, particularly for 1) the range of percent necrosis on histopathology, 2) defining the longitudinal location of the tumor along the compartment, and 3) the appropriate categorization of tumor depth when the tumor was situated along the superficial aspect of the fascia, without invasion deep to the fascia.

Conclusion: Through a series of surveys and a consensus meeting, the MsTR utilized a modified Delphi approach to identify data elements for which inter-rater reliability of data entry was a concern. This allowed for development of a data dictionary to promote coding consistency between institutions contributing to the registry. This effort, and similar ones in the future, aim to optimize the accuracy, reliability, and utility of the MsTR in providing reproducible, research-quality data for this highly complex patient population.

Areas Without Initial Consensus