Skip to main content

We are 1167 members and growing!

BACKGROUND PAPER

Building a Comprehensive Education Data System 

In California

A hearing of the 

Senate Select Committee on High School Graduation

Senator Darrell Steinberg, Chair

November 29, 2007

 

“…California is lagging most other states in developing education data systems capable of helping policymakers and others understand how schools are doing and which programs are effective for improving student attainment and achievement. The state has only in the last several years begun to move beyond the traditional approach to data collection: emphasizing discrete, disconnected data “silos” that address reporting and monitoring requirements but do not lend themselves to analyses that can guide policy and program improvement.”

-- Janet Hansen, Senior Researcher, the RAND Corporation 

 

Background: The suite of Getting Down to Facts (GDF) studies assessing public education in California covered many topics, but featured a common refrain: To improve schools, California needs better data. While states such as Florida, Ohio and Texas have built robust longitudinal data systems that track many different aspects of individual student, school and district performance, California has only inched toward a system that would inform policymakers and the public with a minimum of information. 

As now planned, California’s longitudinal student and teacher data systems – known as CALPADS (California Longitudinal Pupil Achievement Data System) and CALTIDES (California Teacher Integrated Data Education System) – encompass only those data elements required by the federal government as part of No Child Left Behind. In order to make sound decisions about education reform, California must think more broadly about its education data system and invest adequately in the development of a system that allows policymakers, parents and taxpayers to assess the performance of their schools.

Status Report: CDE anticipates that CALPADS and CALTIDES will be fully operational in 2009-10, although the department may be able to report some key data – such as one-year dropout rates – as early as next year. The California Student Information System (CSIS) has assigned a “unique student identifier” to all students in California public schools. School districts and other local education agencies are required to maintain those identifiers on an ongoing basis with the assistance of CSIS, which maintains the statewide student locator database. CSIS continues to refine the process to maintain the identifiers, which will be integrated into CALPADS. 

In addition, all K-12 educators working in positions requiring a CTC credential will also have a unique teacher identifier. The CTC plans to disseminate identifiers to county offices of education and school districts in spring 2008. Through CALTIDES, the teacher assignment data submitted to the state will be used for such purposes as automated teacher assignment monitoring. CALTIDES will determine whether a teacher has the appropriate credentials and authorizations to teach assigned courses based on common course codes. CDE is now working with districts to ensure that common course codes are used. 

Toward a more comprehensive system:

CALPADS and CALTIDES will yield important new information about student achievement – i.e. dropouts -- and teacher assignments – i.e. teachers teaching out of their subject area of expertise. But if our education system is to improve dramatically, that data system will need to tell us much more.

Below are just a few commonly identified priority questions facing policymakers and educators that California’s statewide data system as now designed will not be able to answer: 

1. What achievement levels in middle school indicate that a student is on track to succeed in rigorous courses in high school? 
2. What high school performance indicators are the best predictors of students' success in college or the workplace? 
3. What percentage of students graduating from a district/school is unemployed one year after graduation?
4. What percentage of high school graduates who go on to college take remedial courses?
5. Which teachers help their students achieve the strongest growth in a given year?
6. Which teacher preparation programs produce the graduates whose students have the strongest academic growth?
7. Which schools, using what strategies, are showing strong growth among groups of special concern (i.e. students in foster care, special education or alternative education programs)?

Other states have built comprehensive student and teacher information systems. Florida, to cite one example, collects the following information that California does not: attendance and absenteeism; class rank and GPA; disciplinary actions; juvenile justice placements; participation in vocational programs such as career academies, youth apprenticeships, or career tech; Title 1 services provided. Florida’s database also spans K-12 and higher education, enabling the state to judge how well K-12 schools prepare students for college. 

Getting Down to Facts was not the first report to call attention to California’s data shortcomings.

  • Education Week has identified California as being near the bottom of the pack in data collection. 
  • In March 2007, the US Chamber of Commerce gave California a “D” grade, ranking it 40th among states, in the efficacy of its education data system. In addition, California received an “F” for overall academic achievement and achievement of minorities – two areas in which improvement will depend heavily on the reliability and comprehensiveness of student-, school- and district-level data

Not just state, but local: The need for a comprehensive, statewide education data system should not overshadow a parallel need for better local data collection at the school and district level – the kind of timely, detailed information that teachers, site administrators and district officials can use to drive improvements in teaching and learning. According to the non-profit Springboard Schools, which works with school districts to improve their use of data for “continuous learning,” teachers and principals need “access to the kinds of summative data on progress that is provided by the state assessment system, but teachers also need formative or diagnostic data on what their students know and what they need… district and school leaders are especially interested in find-grained data on student performance and progress in targeted standards-based subject areas, topics, processes, and skills.” Policymakers should not lose sight of the importance of supporting district based data systems and expertise, and further exploration is needed to determine the scope of the ideal overlap and connections between state and local data systems.

 

Key Challenges:

  • Quality Control: According to the Data Quality Campaign, a national organization advocating for and providing technical assistance to states as they develop longitudinal education data systems, an assertive auditing function is a key element in any successful system. “Invalid or unreliable reporting by some schools and districts is a problem in a number of states, and this problem is likely to continue in the absence of checks on the accuracy and quality of the data submitted by schools and districts. Without a well-designed and well-implemented state data audit system, the public cannot have confidence in the quality of the information coming out of the state's public education system.” The DQC gives California good marks for its existing auditing efforts, but CDE acknowledges it will need to establish more regular and rigorous “quality checks” in order to ensure reliable data – especially for those data elements such as dropouts, which are used for accountability purposes – as CALPADS and CALTIDES come online.

  • Accessibility: For now, access to information contained in CALPADS and CALTIDES will be limited primarily to CDE and its contractors (although local districts will maintain access to their own data). There is a strong argument for opening up access to a broader set of researchers, to policy makers and school administrators, and to teachers so that all can be more strategic and efficient in their use of time and resources. The key will be to enable this broader access while safeguarding privacy protections that are guaranteed under the Family Educational Rights Privacy Act (FERPA).

The growing constituency for data: It has been said in recent years that efforts to build a comprehensive state education data system in California were stymied by the lack of an influential and vocal constituency for good data. Judging from recent public statements and efforts by a number of education interests, public advocacy organizations and prominent academics and think tanks, data may no longer be education’s forgotten stepchild. The following excerpts were gleaned from a series of white papers prepared for an October 2007 education policy convening organized by EdSource.

California Association of School Business Officials (CASBO):

“CASBO proposes that the first and highest priority for any major reform effort in public education be to invest in a comprehensive data system at both the local and state levels, to ensure local and state decision makers, as well as local educators and consumers, access to an integrated system that includes data on a broad range of information including student performance, local and state expenditures on educational programs and practices, and personnel practices that may impact student achievement. This needs to be accomplished prior to embarking on yet another round of educational program reforms. Without data, we are likely to continue… spending our limited resources on programs that my not produce optimal educational benefits to students.”

California Collaborative on District Reform, District Practitioner Working Group (comprised of 12 district superintendents, deputy superintendents and CFOs):

“Without an effective student tracking system, we as superintendents and administrators of California districts are significantly limited in our ability to determine the effectiveness of local policies, programs, and practices aimed at improving student learning and attainments… We believe it is critical for the state to fully fund and support a comprehensive longitudinal data system as well as support local efforts to collect, analyze, and use data to inform instruction.”

California County Superintendents Educational Services Association (CCSESA):

“The state must invest the resources needed to establish and sustain an education data system that is trusted, reliable, robust and accessible, with independent oversight by a group of key stakeholders to support a high quality data and information system… Investment needed: $50 million annually to develop and implement a comprehensive assessment and education data system; $20 million to support effective assistance to schools and districts in improving student achievement.”

California School Boards Association (CSBA): 

“The only barrier to implementing (a statewide data system) is political will on the part of state leadership. The state’s data system must be comprehensive enough to improve teaching and learning in the classroom, as well as local and state policy and resource allocations. It will be critical not only to capture the linkages between the programs provided to students and their impact on student achievement, but also to inform policymakers about the progress of students as they move beyond high school. Such linkages to higher education and employment can help K-12 revise and strengthen its support for students as a part of an ongoing commitment to continuous learning and improvement. A comprehensive data system should also provide direction to institutions of higher education about the quality of their teacher development programs.”  

Children Now:

“Only by developing the ability to ascertain what works and what does not will the state be able to make clear progress in regards to academic achievement. For California to regain its position at the top of our nation’s academic and economic indices, the state will need to develop a comprehensive, integrated, longitudinal data system that allows for continuous improvement on the part of students, teachers, administrators and policymakers alike.”

ConnectEd: The California Center for College and Career:

“To better assess the effectiveness of high school programs in preparing students for post-secondary education and employment, policymakers need appropriate indicators and data systems to collect the appropriate data. The indicators should be able to track students from high school to further education, training options, and employment.”

The Education Trust-West:

“A smart data system must house much more than data about K-12 education. When developed, CALTIDES and CALPADS need to link to data sets about higher education, including 2- and 4-year university data and postsecondary vocational participation data. The system should also link to university data and postsecondary vocational participation data. The system should also link to employment data; military service; incarceration; and health and human services to get a clear portrait of what happens to students as they journey from our schools into adulthood.”

Policy Analysis for California Education (PACE): 

“The Legislature should:

  • Move immediately to accelerate the implementation of CALPADS and CALTIDES, including sufficient funds for districts to support the training and infrastructure necessary to ensure that data collection is both timely and reliable.
  • Expand the scope of data collection… to focus on identifying policies and programs that enhance learning outcomes, and not simply on compliance with federal mandates. The marginal cost of collecting additional information while the system is under development is low; expanding the scope of data collection later could prove costly.”

The School Finance Exploration Partnership (a collaboration of the California School Boards Association, Children Now, the League of Women Voters and the California State PTA):

“Of interviewees that participated in this outreach process, nearly two thirds wanted better, more user-friendly academic and fiscal data. Most believed this information would help inform and engage the public, ensure accountability, and could lead to better state, local and site-based decision making.”