Recently, there has been increased interest in career and technical education as a mechanism to create pathways to college and employment. This increased interest has occurred despite the fact that, aside from two studies on career academies, there is relatively little high-quality evidence about whether and how CTE provides educational and work-related benefits to students. In my new report with the Fordham Institute, Career and Technical Education in High School: Does It Improve Student Outcomes?, we capitalized on the willingness of state agencies to partner with us and share data as a way to answer these questions. Our ability to produce answers is related to the rich datasets from Arkansas that enabled us to translate this data and available computing power into actionable policy findings.
In the past, roughly one in five students took three or more high school courses in a field classified under career and technical education. But some recent evidence suggests that the number of students taking a larger share of CTE courses may have receded during the expansion of high-stakes, test-based accountability. Very little of the data accumulated in recent years has been examined to explain how major shifts in policy and educational practice may have impacted the provision, quality, and impact of CTE coursework, and how participating in this coursework influenced the outcomes of students.
Doing high-quality research in education policy (and many other arenas) requires detailed and usable data. And if we’ve learned anything from the last fifteen years, it is that relying solely on test scores as indicators of impact on students misses other important outcomes and elements of the educational process. This is particularly true in CTE research because part of the purpose of CTE is to broaden student awareness of and access to a larger number of education and career pathways. Rather than student performance on a state test required to hold schools (rather than individual students) accountable, it’s more important to know whether and where students earned a credential or went to college, as well as what wages they earn. However, the ability to track these outcomes is dependent on the capacity of otherwise disconnected agencies (K–12, higher education, and departments of labor) to share and link their data. Though it may seem intuitive that such sharing of data would already occur, this hasn’t been the case in the past. And even when partners were willing, differences in how data are stored and what information is collected about individuals (Social Security numbers in some areas, as opposed to other organization-assigned identifiers) has been a barrier to successfully building a data set that allows tracking of people over time.
In this report, I was fortunate to work with the Arkansas Research Center (ARC). Their established process of requesting and obtaining data, as well as their knowledge of the state policy context, were elements that led to the creation of this report. I was lucky. Such systems and data are hardly universal, and there is more work to be done before what we did in Arkansas can be replicated at scale, especially for CTE.
At present, there are five states that have effectively linked their K–12 student data to high-quality records of college enrollment and completion and labor market data. These include Alaska, Arkansas, Florida, Ohio, and Texas. Most other states have grants to build similar longitudinal data systems, but few are operational enough that labor market data are included. There are, however, at least ten other states that can provide access to well-integrated K–12 and college enrollment data; in at least a handful of those ten, there are processes that exist similar to Arkansas’s, through which researchers and state or district agencies can partner to answer relevant policy questions (e.g., the NYC Research Alliance, the Chicago Consortium, the University of Michigan partnership with its state, and Massachusetts’s partnerships with Harvard and others).
Yet even when coordinated, high-quality data exist, access can remain limited, despite the good intentions of the organizations that join such data. To gain access to it, researchers and policy makers must engage in appropriate agreements about confidentiality and protection of personal information. But these processes are not always well systematized, and permission to access data is not guaranteed even when there is a clear policy need. Importantly, the long processes that can exist to access data can make the length of time it takes to answer meaningful questions exceed interest in the question or policy that started the request!
Constraints to data access aren’t always imposed by state agencies either. In order to access data, there must be a formal agreement between the state and the researcher. Yet many universities are ill-equipped and lack the infrastructure to create data use agreements nimbly, especially since they often require negotiation and paperwork (not to mention a great deal of patience). Individual scholars have even less capacity to do so. The organizations best positioned to do this are research firms, but they may face incentives that cause them to seek questions and answers for projects that can be funded—a reality that universities do not face to the same degree. In Arkansas, for instance, the ARC took on the task of establishing the infrastructure for individual researchers to directly request and access the data, and also have the tools to establish data agreements with universities or other research organizations.
Creating such capacity is crucial to facilitating timely and meaningful research that capitalizes on rich available data on important policy contexts (like Arkansas). Furthermore, universities are partnering with potential researchers eager for data and often faced with the expectation (especially social scientists in education schools) of doing research that is connected to practice and policy. It is therefore crucial that we reduce transaction costs between researchers and policy-oriented organizations with data. We also must enhance the capacity of universities to establish required security protocols and data-sharing agreements. They can thus connect their untapped capacity for analysis to faculty, whose incentives to conduct research are potentially less expensive to meet than those of research firms. (The incentives of those firms, it should be noted, demand that projects have funding).
There has been a mounting number of concerns lately about data security and FERPA provisions, as well as related assertions that data cannot and should not be shared for research purposes. We can’t afford this sort of backslide. We have states that serve as wonderful examples of what is possible in terms of establishing good relationships strengthened by appropriate precautions—such as agreements that hold researchers liable for storing and using data responsibly. We need to scale these models, improve the understanding of the general counsels of state and local agencies, and create opportunities to connect researchers with the practitioners and policy makers who collect and store data.
Shaun M. Dougherty is an assistant professor of educational policy and leadership at the University of Connecticut’s Neag School of Education.
