Can Early Teacher Evaluation Findings Help Change the Debate?

Over the past few years, states and school districts across the country have devoted significant resources to the design and roll-out of new teacher evaluation systems.  Driven at least in part by requirements attached to Race to the Top funding, the new systems have inspired heated debate over the efficacy of factoring student achievement data into a teacher’s performance assessment. The New York Times recently shared some initial findings from states that have launched new evaluation models including Michigan, Florida and Tennessee, reporting that the vast majority of teachers- upwards of 95 percent in all three- were rated as effective or highly effective. Although the analysis of these numbers has only just begun, the Times reports that some proponents of the new evaluation models admit that the early findings are “worrisome”.  And even though it is still early, we can reasonably anticipate that if the trend continues- and the findings from the new evaluation systems reveal no significant departure from more traditional methods of evaluation- we may start to have a lot more people looking at the complicated data analysis driving teacher evaluation systems linked to student achievement data and asking “what’s the point?”

It’s a good question, really, and one that probably hasn’t gotten enough thoughtful attention in the midst of the controversy surrounding them: What is the point of linking student achievement data to teacher evaluations?  Should we take it for granted that a primary goal- if not the primary goal- of these efforts is to identify and eliminate bad teachers?  If this is the case then these early findings should be a cause for concern, especially given the time and money being spent to collect and analyze the data.  If replacing bad teachers with good ones is the magic bullet for public education reform, it will take a pretty long time at this rate.

Of course, even opponents of the new evaluation systems would probably admit that the magic bullet theory is an oversimplification. Furthermore, it’s much too early to look at these numbers and extrapolate any meaningful conclusions about the actual number of ineffective teachers or even the accuracy of the evaluations themselves. Hopefully what these findings might do is allow us to finally begin to broaden the scope of our national conversation about how the linkages between teachers and students could actually drive education reform.  States and school districts implementing new evaluation systems have tried with varying degrees of success to communicate the message that linking student achievement data to teacher practice isn’t just about punitive measures- it also has important implications for improving professional development and teacher preparation programs by identifying shared practice linked to positive student achievement and replicating those practices in classrooms across the country. But that message is often overshadowed by the anxiety surrounding the punitive side of evaluation and underscored by public struggles with local teacher unions. If nothing else, these early findings might create an opening in the current debate for a more thoughtful discussion about the broader possibilities for linking teacher practice to student growth.

-Jacqueline Skapik

Across the Board: Who Else is Responsible for the Atlanta Cheating Scandal?

The media in recent weeks has focused a great deal of attention on the cheating scandal in Atlanta in which authorities have indicted 35 officials and teachers in the Atlanta Public Schools system for allegedly altering the results of students’ state standardized tests to reflect higher scores.

There have been similar allegations in other schools and districts around the country.  What is unique about Atlanta, however, is the scope of the alleged fraud.  Those who work in education know that it is virtually impossible to keep a secret about even the most trivial matters in a large urban school district.  One cannot help but ask, therefore, how a scheme of this magnitude (with so many teachers and administrators directly involved) could gain such momentum.

By all accounts, it appears that pervasive in the culture of the district was the “Machiavellian” notion that the ends of increasing student test scores outweighed the means, and that it was this culture that enabled such widespread cheating.  Indeed, many point to an array of incentives that the superintendent allegedly used to reward those who generated higher test scores, whatever the cost to students.

The superintendent, however, does not operate in a vacuum.  Even the superintendent is accountable to the school board.  The superintendent alone cannot establish policy.  Nor can she create financial incentives for employees without the approval of the school board. She takes her cues as the tone of her leadership and school district culture from the school board.  If not, the school board should remove her.  Why then has there been so little media attention on the role of the board in the Atlanta cheating scandal?

The school board, at least as much as the superintendent, is responsible for creating a culture of integrity, honor, and accountability within the district.  Effective school boards model these values both in the conduct of school board meetings and in their interactions with school communities.

Moreover, effective school boards recognize and constantly communicate to others the critical importance of accurate data in improving instruction and learning outcomes.  They establish policies to ensure the integrity of test results.[i]  They examine student data on a routine basis, and hold district and school administrators accountable for the effective use of the data.  In so doing, the board ought to catch extreme abnormalities in the data, ask probing questions, and err on the side of a full investigation any time there is any reason to suspect even the slightest impropriety.

Reading the media accounts of the Atlanta cheating scandal, one is reminded of the Enron scandal, in which high level executives engaged in fraudulent accounting practices that devastated a company, its shareholders and employees right under the noses of its board of directors.  In response to such abuses, the federal government passed the Sarbanes-Oxley Act, which imposes a higher standard of accountability on corporate executives and boards of directors.  Among other things, the Act requires top management to certify the accuracy of financial reports.  The Act also imposes heightened responsibilities on corporate boards, through standing audit committees, to oversee the actions taken by top management.

Testing data is to schools, students and parents as financial information is to corporations, shareholders, and employees.  Perhaps a set of rules similar to those imposed by the Sarbanes-Oxley Act is necessary to ensure the integrity of student data.  Perhaps top management (such as the Superintendent, and chief accountability officer) should be required to personally investigate and certify the accuracy of student test data.  Likewise, perhaps school boards should be required to have standing committees to review the process and ensure its integrity.

Many argue that the lesson learned in Atlanta and elsewhere is that “high-stakes” testing will inevitably lead to cheating.  This is a sad conclusion, indeed.  It reflects a lack of confidence in the ability of all students to improve, and excuses from responsibility those adults upon whom our students are relying to help them improve.  Worse yet, it excuses those adults from even behaving ethically, and endorses behavior that sets a terrible example for students.  Cheating is not symptomatic of an inherent flaw in high-stakes testing (though there may be flaws).  Instead, cheating reflects a lack of integrity, leadership and good governance that is essential to the success of our system of public education.

There are very few areas in which school districts require more rather than less regulation.  Sadly, it appears that this may be one such area.  With or without additional regulation, however, the critical role of the school board in preventing such abuses cannot be overstated or overlooked.

— Kim Clark

---

[i] For further guidance on best practices in testing security, see “Issues and Recommendations for Best Practices,” which is based on comments and ideas generated during a Testing Integrity Symposium that the U.S. Department of Education held in February 2012.  This publication can be found at http://nces.ed.gov/pubs2013/2013454.pdf.

How to Run a Computer Based Training Session: Three Indispensible Techniques

This week I’m really delighted to introduce Frank Nichols a talented consultant from our strategic partners at Strategic Urban Solutions. Strategic Urban Solutions will be guest posting for us from time to time, and this week will be sharing a training post with us.  

At Strategic Urban we tend to do a lot of work with large institutions: Cities, Non-Profits, Schools, etc. Typically, these institutions will need to move on from their old paper-based methods of doing business and adopt an organizational system. Let’s face it, this is usually long overdue and necessary.  When an organization’s staff need training on these new systems, it can be both rewarding and challenging to be in the position of the Trainer. I will be honest and say that I have not always been good at this. In fact, I wouldn’t be able to offer up any of this wisdom if I haven’t been thoroughly beaten up along the way. After many years and nearly 100 training sessions, I’d like to offer up three techniques that I have found indispensable.

1. Don’t Be a Policy Middleman

Many times when you are introducing a new system or process, it is due to big changes in an organization. It is inevitable that you, as a trainer, will be seen as the middleman between staff and management. In order to prepare staff for the new system, you might have to give them an overview of recent policy changes. Make sure they also understand your role and purpose: to help them adopt new technology. Don’t let your training session become a place for the airing of grievances. Negativity about an organization’s changes can carry over to negativity about the technology that you are introducing.

If you are consulting for an organization, and are not management yourself, you can position yourself as an advocate on their behalf. Show sympathy for the staff, while also maintaining positive representation of the management. One way to avoid becoming the policy middleman is to have the contact information of the policy expert(s) on hand. Inform the staff that they can direct specific questions to that contact so that you don’t get off track. Even better yet; if a policy expert is available to address the policy implications in person, during the introduction, you’ll be free to focus on technology for the rest of the session.

2. Positives Before Challenges

Showing staff a new system or process and then asking for questions can sometimes, understandably, lead to a wave of complaints. If one person comes up with a complaint the rest of the staff in the room might feel compelled to pile on. This is why it is important to take a few breaks throughout the session to discuss Positives and Challenges. I always start with Positives by asking “Now that you have seen some of the system features, what do you like most? Why is this system an improvement on what you have done in the past?” You’ll want to discuss Challenges as well…but hold those Challenges hostage. I won’t move on to Challenges until someone can offer up something positive about the system.

For Challenges, I like to ask “Do you anticipate any challenges in applying this system to your work?”. When you frame it this way, you’ll get thoughtful anecdotes from the Staff instead of complaints. They will help you to understand what they are dealing with when they go back to work, and you’ll be better prepared to use that context for the rest of the session.

3. Demo Before Practice

If you have a room full of staff with a computer in front of them, good luck getting their attention. I’ve been in the front of many training sessions, but I’ve also been in the back. A computer is not only an invitation to check email and social media, but also an invitation to explore the system ahead of the instruction. Getting ahead of the class in a focused computer training session sometimes means getting lost. Each section of the system comes with explanations, demonstrations, and discussions…all of which will be missed by someone who is staring at their computer and going on their own personal journey. How many times have you tried to get through an entire demonstration, only to be interrupted at various stages because someone is trying to click on this or that and it is not working? The solution is: clearly state when you are demonstrating and that the opportunity to practice coming up next. Demo before practice.

Before you introduce a part of the system, explain that you are going to first do a demonstration. More eyes will be on you (More, not all…I’m realistic, you can’t get everybody) and those staff will clearly see the current system component, they will hear your explanations and guidance, and will have an opportunity to ask questions. THEN, you can put them on a mission: “Now that you have seen how this component works, go ahead and complete this step on your own.” The beauty of this is that you can free yourself up to walk around and help people individually, before you command their attention on the next demonstration.

I hope you find these techniques valuable and that you experience the reward of a successful training session. Happy training!

–Frank Nichols is a guest blogger from our friends at Strategic Urban Solutions

Management Consulting for a Startup

Startups are a swirl of excitement, energy, and ideas.  But they are also often chaotic and chaos lowers staff performance, and leads to missed goals.  Even startups that eventually experience great success go through a phase of chaos as they figure themselves out.  So, what determines whether startups will mobilize all that energy and enthusiasm, or whether they will spin their wheels, wasting talent and resources?

The most important factor is whether they can establish the right kind of management systems.  Management systems determine whether ten brilliant people will spend all their time fighting over whose idea is the most brilliant or whether they will contribute ten useful parts to a dramatically effective whole.  And management systems are where we can be of tremendous service as consultants.

In particular, project management systems are crucial for startups.  Project management turns ideas (which startups have in spades) into output (which can be hard to come by early on).  Project management also greatly facilitates staff management; without effective work processes it can be unclear if poor performance is due to a failing of an employee, of their manager, or of the organization itself.  So what is project management?

People often think of project management as the process of setting deadlines and assigning tasks to team members.  While that is in fact part of the process, this characterization misses the point. The fundamental goal of project management is to help people understand how to work together.  You can set a deadline, but if your team is not well coordinated they will fail to meet it.   You can assign tasks to team members, but if you don’t have their buy-in or they don’t have clarity, they will not do the task to your standard of quality, within the timeline you need, or sometimes– not do them at all.

Instead, project management starts by building shared understanding.  At its core, this means getting everyone involved in a project on the same page about:

A) The goals of the project

B) The definition of success

C) The three fundamental constraints:

• Scope of the project
• Resources available (especially human resources)
• Time available to complete the project

With shared understanding around these critical factors, timelines and task assignments can be made based on the reality of what’s possible, rather than on unrealistic hopes.  When unexpected problems arise (as they will), and changes to the plan have to be made, these changes can be made based on understanding of the tradeoffs they imply.

As consultants, we can help a startup go from a twinkle in the founder’s eye to a highly functional team.  The success of this evolution hinges on thoughtful project management.

–SK

Leading Change in Education Reform Efforts

“A leader is best when people barely know he exists, when his work is done, his aim fulfilled, they will say: we did it ourselves.” —Lao Tzu

Leading people through a process of change is difficult, particularly in big, entrenched systems like traditional schools and districts. A school leader I know once told me that traditional school systems are like giant ocean tankers, you can make them change direction but it takes a lot of time and energy. Few school and district leaders are prepared to turn their ocean tankers around, particularly within the timelines and to the degree required for Race to the Top (RTT) to meet its ambitious goals.

As evidenced by the recent RTT year two reports released by US DOE, one of the greatest challenges to the success of the RTT reforms is not the content of the changes themselves, but simply that RTT entails significant change on the parts of individuals and systems, and change is hard.

Psychology research tells us that people don’t like change (status quo bias). Change takes effort, causes discomfort and sometimes can be downright painful. People fear the unknown. They wonder, “Will the work and the pain be worth the effort?”

Yet there is an urgent need for change in our education systems to ensure that all students are prepared for success in college, work and life. This need can be seen in student proficiency data from across the country. Rhode Island recently released the latest round of state assessment results, which were a grim reminder of how far we still need to go and how long it takes for systemic change to have an impact at the classroom level.

So how do we address the challenge of leading change?

In Rhode Island, one way we are supporting local leadership and spreading effective ideas to support RTT implementation is through the Collaborative Learning for Outcomes (CLO) process. Through the CLO process, Rhode Island district leadership teams meet regularly in facilitated sessions to share effective practices and learn from one another regarding RTT implementation strategies. The CLO process has provided a forum for district and school leaders from across the state to dig down into concrete strategies to support RTT implementation and to discuss mitigating the complex challenges they face on a daily basis with peers who struggle with the same issues.

In my work with the CLO teams in Rhode Island, the successful education leaders I have observed all share and act on the following beliefs about leadership:

(1) Communication must be a two-way street. To lead people into the unknown, you must listen, have honest dialogue, and be transparent about the work ahead. A number of district leaders who shared during CLO meetings that they created genuine opportunities for their teachers to express and receive answers to their concerns about the new RTT systems were the ones who were also most likely to report that everyone felt they were on the same team when it came time for implementation.

(2) Everyone must share ownership of the work. Distributing leadership responsibilities among those affected the most by changes builds internal champions and on-the-ground capacity, giving people responsibility leads to increased motivation to move the work forward, and getting implementer input on the “how” of the reform greatly increases the chances for success. Through the CLO process, I saw how school leaders who did not engage teacher-leaders in their buildings in developing implementation strategies were almost universally unable to move reform efforts forward with any reasonable speed. The opposite was true of those leaders who created real opportunities for teachers to hold responsibility for success.

(3) Leaders must support those on the front lines of change. Success depends on whether leaders can be creative about finding new resources and using existing resources, provide staff with needed training, and flexibly support staff to face the unknown. All educators in Rhode Island are working within the constraints of limited human and financial resources and the aspirational goals for RTT implementation. Many CLO discussions center on the challenge of stewarding resources wisely and creatively. While no one has found a magic bullet, those leaders who have acknowledged the insufficiency of the traditional structures for budgeting and using human capital are finding a variety of new ways to plan resource use so that they can provide as much support as possible for their staff.

By implementing innovative strategies, like the CLO process described here, we can help education leaders learn about and adopt practices that will increase the likelihood of success of RTT and other critical educational reform efforts.

–EFB

Securing Our Schools in the Wake of the Sandy Hook Elementary Tragedy — Pt I

The shooting last week at Sandy Hook Elementary School has prompted a great deal of debate across the country about gun control and access to mental health services.  The incident has also prompted increased scrutiny of school safety practices.  Of course, it is critical that schools review their lockdown procedures and other security measures on an ongoing basis, and ensure that staff members are well trained in those protocols.  School safety experts generally agree, however, that the security measures in place at Sandy Hook were appropriate and reasonable, and indeed saved lives.  Of course, all systems have limitations.  A criminal intent on breaking in at any cost will be difficult for any institution (other than a maximum security prison) to stop.  In fact, children are far safer in school than in other public places such as shopping malls, movie theaters, parks, playgrounds, etc.  And they are exponentially more likely to be killed in an auto accident than in an incident like the one that took place at Sandy Hook Elementary School.

Schools could increase police presence on campus.  Research indicates, however, that seeing armed police officers roaming the school can be scary for young children and undermine their feeling of safety and security. Moreover, those who criticize districts for spending too much on administrative as opposed to classroom expenses should be aware that school security, including on-campus police officers, is an administrative expense (which for many districts, is not insignificant).

Some, including Texas Governor Rick Perry, believe that allowing school personnel to bring guns to school is a valid solution.  They claim that a school employee with a gun who was properly trained could have stopped the perpetrator of the Sandy Hook shooting before he was able to kill so many people.   Statistically, however, it is far more likely that a legally purchased gun will be used not in defense of but against its owner or a member of his or her household (and by analogy the school it is intended to protect).

Even if this hypothetical gun-wielding employee turned out to be the James Bond in Governor Perry’s fantasy, i.e. capable of exercising good judgment and perfect accuracy under extreme pressure, allowing employees to bring guns into the workplace, and especially into schools, is a very bad idea.  The chance that most schools will ever experience anything like what happened at Sandy Hook is extremely slight.  Most schools, however, do experience some incidents of violence each year.  Add guns to this environment, regardless of who owns them, and the outcomes of those incidents are likely to be far worse.  It is a travesty that the perpetrator of the Sandy Hook shootings was able to gain access to legally purchased guns.  Locating more guns on-site and making them even more accessible will only escalate violence in our schools.  Moreover, if seeing police officers with guns on campus undermines children’s sense of safety and security, imagine what it would do to a child’s sense of security to receive a poor score on a homework assignment from a teacher packing heat.

If we want to invest in making our schools safer, we need to look at the areas of greatest risk to our students.  Fortunately or unfortunately, the greatest risk to students does not come from the outside.  The greatest risk comes from individuals students encounter on-campus with a colorable reason for being there.  That said, the one area in which many public school systems could be doing better is in conducting background checks of school employees, volunteers, contractors and others who come into contact with students on campus.  That subject, however, warrants a separate, more detailed discussion.  Accordingly, stay tuned for Part II, which will examine the ways in which some States’ and districts’ policies concerning background checks could be amended and/or supplemented to better protect students.  As for the adequacy of existing school security measures, and the suggestion that teachers be allowed to carry guns to school, please let me know you think.

The UPD blogger, Kim Clark is a senior consultant with UPD.  Prior to working with UPD, Kim served as the General Counsel for the Scottsdale Unified School District in Scottsdale, Arizona, as well as a labor and employment attorney at Steptoe & Johnson, LLP.

The Follower’s Manifesto

In my six years of teaching, I had plenty of colleagues who carried on non-stop private conversations through every faculty and department meeting they attended. The very educators who brought down the wrath of God on misbehaving or inattentive students became pouty, apathetic, or downright antagonistic when another adult had the gall to suggest that there was something these individuals needed to know or had yet to learn.

I know this mindset well, as I possessed it for a time:

“What does Vice Principal Smith know? He hasn’t been a teacher for 10 years…”

“I wish they’d let me get back to my classroom—I have so much to do and this is useless.”

“How could a consultant, who has never taught, possibly give me any advice about education?”

To be sure, some of this anger and indifference is well founded. I cannot count the number of faculty meetings I sat through where the principal read aloud (verbatim) from a schedule that affected 1/10^th of the school’s population. But to focus on this smaller point is to obscure a larger one: as much as we often hear that we lack good leaders in the education world, I believe the bigger problem is that we lack good followers.

Very few people have the privilege of holding a role in life in which they are consistently leaders, always laying out an agenda to be executed by those around them. Instead, most of us hold a more nebulous position—we are leaders of some and followers of others, and these roles change over time. Teachers are the perfect example of this—student achievement in the classroom requires great leadership on their part, but that leadership must be informed and supported through the following of administrative guidance, research-based standards of practice, community desires, and expert advice.  Yet while educational literature is rife with treatises on leadership (one of my primary introductory packets to Teach For America in 2004 was called Teaching as Leadership), there is little talk of following.

So what are the characteristics of a good follower, and how will they make a difference in education? With the help of the comparatively sparse followership literature[1], I’ve compiled this non-comprehensive list:

1. Good Followers are Open-minded. Too often in education, we assume that the best ideas for student achievement are contained in our own heads, or at the very least within our own dogma. We must be willing to adjust our approaches based on the advice, feedback, and new sources of information we receive.
2. Good Followers Disagree and Commit. Even good leaders will make decisions that their followers may not always agree with. This is perfectly reasonable, and followers should feel free to communicate that disagreement to leaders. However, once a decision has been finalized, followers must commit to act upon it as if it was their own. Refusal to act upon a decision prevents evaluation of the decision’s effects further down the road. This is the piece that I and my colleagues most often struggled with as teachers. It was easier to poo-poo a new administrative initiative about backwards planning for a million little reasons, than it was to buy into this initiative and change our ways.
3. Good Followers are Active Listeners and Collaborators. Listening to and participating in a conversation requires full attention and critical, collaborative thinking. The non-stop responsibilities of most jobs (especially teaching) can also function as excuses to mentally (or even physically) check out of one’s listening responsibilities. Grading takes precedence over listening to a department head, lesson planning replaces one-on-one time with a mentor. I know—I’ve been there. But I also know that listening and participating in collaborative opportunities is an important part of creating school culture and promoting practices that improve student achievement. It is through this collaboration that decisions are made and tested, and that leadership is held accountable.

With support from UPD’s Bob Pipik, Nick Goding, and (former employee) Dustin Odham, Highland Park High School in Topeka, KS has taken advantage of a federal grant to install a collaborative process of student and classroom data evaluation. Every progress report and grading period, teacher teams meet to examine trends in student attendance, grades, behavior, and test scores, both within their classroom and throughout the team. Students who are at risk are identified and intervened with as a team or individually using a “Student Tracker” created and molded through an iterative process of teacher and administrative feedback. This approach has led to a narrowing of the achievement gap between African American and White students, and has improved student test scores overall by almost 10 percentage points. And all of this has come as a direct result of attentive and excellent followership. It is true that school administration wrote the grant and initiated the data evaluation process (and for that they should be praised), but it was the school’s teachers who approached the process with an open mind, contributed to its functioning through collaboration with leadership, with outsiders (UPD), and among themselves during the teacher team meetings, and they have remained committed to its functioning for the past two and a half years.

It should be obvious that we can’t all be leaders all of the time, but that doesn’t mean we must resign ourselves to lives as desk jockeys, pushing paper for the man.  While my examples throughout this blog are based at the school level, the call for good followers is a universal one in the field of education (and beyond). Equity and excellence in public education will require that most of us make a commitment not just to lead, but to follow. From teachers to bureaucrats to consultants, we can shape and challenge our leaders, and the world around us, through our openness, our commitment, our action, our honesty. It’s time that “follower” stopped being a dirty word.

TM

---

[1] See Kellerman, Barbara. Followership: How Followers Are Creating and Changing Leaders. Harvard Business School Press, Cambridge, MA. February 18, 2008 as a prime example of the emerging field.

–Tim Marlowe

Stuck in the 80s

Although I have over 25 years of for-profit corporate America experience, I am not one to think that we should leverage everything from the business field to the education field.  However, I do think there are areas where the educational field could learn more from the business field.

One of those areas is student information systems, or SIS for short.  Simply stated, a good SIS manages student data and should cover areas like scheduling, grading, attendance, discipline records, etc.  I look at this as backbone stuff.  Every school has to do it and do it well.

I liken it to an accounting system in corporate America.  You have to do it well, but let’s face it, invoicing is invoicing.  Companies don’t get a competitive advantage from this activity.  (Of course they could be at a disadvantage if they don’t do it well).

When Madonna first started singing about being touched for the first time, corporate America had hundreds of accounting systems.  In fact, one company, depending on its size, could have several accounting systems.  Today, efficiency rules, and the likes of Oracle and SAP are generally used in large companies.  Large companies that have more than one accounting system are ridiculed as being behind the times.

Why then, are “a-ha” tunes still heard echoing in education halls?  I work in a small state, and there are at least ten SIS applications.  That means that data collection at a state level is complicated and knowledgeable resources in one SIS system cannot be leveraged across multiple LEAs.

The only reason to have variance is if it is bringing a significant value to the student experience.  If that advantage is not there, then one SIS should suffice for a region or state.  If you disagree, let me know and…”Hit me with your best shot.”

–AW

Improving the Ground Game in Education

We know political campaigns are driven by numbers. Since long before we started talking about data-driven decision-making as a key driver in education reform, political strategists had sophisticated models for crunching numbers and using them to determine where their money was spent, what their candidates said and where their rallies were held. But even though politics has always been a numbers game, in the aftermath of the 2012 presidential election, experts on both sides of the aisle are marveling about how the Obama campaign used innovative implementation strategies to take that game to the next level.

What does that “next level” look like? Broadly speaking, it is more precise, more dynamic and more individualized than its predecessors.  With a sophisticated strategy for using demographic data (including the 2010 census) to target resources and local campaign headquarters efficiently, Obama’s team built an unprecedented grass-roots level organization that generated support in key swing states.  They didn’t just crunch the numbers, they used them to develop a successful plan for turning them into votes.

As I read about the Obama campaign’s successful strategy, I was struck by the fact that several of the things they did very well are things that continue to serve as obstacles in implementing data-driven reforms at the state and local level in education. We know we should be using data to drive decision-making, but have we paid enough attention to our ground game?

Here are two areas where the Obama team’s strategy provides some insight into current challenges in education reform:

1. They connected their databases– When the Obama team realized that their fundraising database wasn’t connected to other key systems (such as their voter registration database), they immediately went to work, connecting all systems with critical voter information that previously didn’t talk to each other. This probably sounds eerily familiar to anyone who has worked in a school district or state-level education agency.  Separate databases are the rule, not the exception and connecting data sets to actually learn from the numbers is often a time-consuming task assigned to someone who is probably already working over-capacity. States are making progress with efforts to build longitudinal data warehouses, but progress is slow. In the meantime, people on the ground who need to use the numbers to drive decisions and resources don’t have the access they need. Instead, with the task of managing schools filled with thousands of students and teachers, leaders have to continue to build inefficient systems and work-around processes around disconnected data. We are moving to change this, but are we moving quickly enough?
2. They created an infrastructure in the field that supported their strategy– The Obama team developed sophisticated statistical models that allowed them to target groups of potential voters with extreme precision. With early warning systems and value-added growth data in student achievement, we are beginning to do the same in education. We are using data more precisely, and using it to plan instruction that targets specific students and specific skill sets. The critical difference is that the Obama team developed a new infrastructure in the field, designed to support that dynamic, individualized attention. They created small field offices in key counties, tight-knit teams of campaign staff trained specifically to provide the ground-level outreach to potential voters that their numbers told them was needed.  This is where education reform continues to lag behind. While we may be developing the systems and skills to help educators gather and analyze data, we have not made much progress in creating an infrastructure in public schools that allows us to use the data to drive action and resources quickly and creatively.

Education reform initiatives, like political campaigns, fail or succeed in their implementation. We should be looking for and learning from examples of successful implementation efforts wherever we can find them to ensure our ground game (like our president’s) is strong.

— JS

 

Will Data Quality Campaigns Stay in the Dark Ages?

Data about students that are high quality yield the best outcomes for students—this is the basis of nearly all education data quality campaigns. As education data systems mature across the nation due to investments from Department of Education Statewide Longitudinal Data System grants, states have not been able to find more innovate ways to ensure that the data feeding into these systems is high quality.

Assuming that high data quality is an important goal for grantee states, why then do they continue to rely upon tired professional development workshops, giant posters, and endless PowerPoint decks devoted to awareness campaigns to identify and solve data quality issues? If we’re going to rely on more modern data systems, we also need to modernize our thinking around diagnosing the root causes of data quality issues.

We think we’ve found a way to do just that.

These new systems generate mountains of rich data—data so fine-grained that we were able to develop a series of indicators to test various aspects of data quality, such as timeliness, uniqueness, and accuracy. But what we found truly exciting was what happened when we combined these indicators with survey data about district technology and capacity. We used these data to reveal stark contrasts between high-data quality and low-data quality districts. For example, we found that districts high on the data quality scale had an average of 4 full-time data staff and spent $20-$30k per year in fixed system costs, while districts on the low end of the data quality scale had an average of 2 full-time data staff and spent $10-$15k per year.

Using this approach, we can now provide districts with a real “pulse” on their data quality while also giving them greater information about the quality control levers at their disposal. If more districts adopted this approach, we would stand a much better chance of improving outcomes for students.

AK