Conceptual and Methodological Profiling of Data: A Qualitative Metadata Process to Understand the Meaning and the Formation of Data Sources

Data science research often involves ingesting and linking disparate sources of secondary data. While these sources can often be cleaned and wrangled into a usable form for analysis, robust documentation on how variables are created, and their intrinsic meaning, might not always be readily apparent. Without such meaning applied, data can lack the context necessary to understand the best ways to use and analyze it and risk misinterpretation. We introduce conceptual and methodological profiling processes into the data science pipeline as a qualitative tool to help researchers derive additional meaning and understanding from their data. Conceptual and methodological profiling uses various taxonomies to categorize variables and produce metadata to inform about how variables were created or recorded and the concepts they represent. To help explicate these processes, we first broadly describe these approaches and their place in the data science pipeline, then present a real-world example applying these techniques in our research using disparate data sources from the U.S. Army. Lastly, we discuss how researchers can find agreement while conducting these qualitative processes. We hope that the processes outlined here will provide data scientists additional tools to know their data better and how best to use it.

Ratcliff N, Thurston J. Conceptual and Methodological Profiling of Data: A Qualitative Metadata Process to Understand the Meaning and the Formation of Data Sources, Technical Report. TR# 2023-284. Proceedings of the Biocomplexity Institute, University of Virginia; 2022 January.