inBloom, Train Wrecks, and Ed-Fi

As I sat down to write this entry, my day was interrupted most unusually.  Doug texted me the picture to the left.  The caption said simply, “Say hello to 26th street and the railroad track.”  In the picture I saw the same view I see every work morning from the “Big Table” here at UPD where many of us sit.  After more than 4 inches of rain over 36 hours, the ground right outside our office gave way taking more than a dozen cars and half the street with it.  If you watch the video (found below) of the ground as it collapses underneath the cars, you will see that it left the wall with nothing to hold, and fell under its own weight.  The stories on the news have since revealed that the neighborhood has know this was a problem for years, but their complaints and concerns met a deaf ear in the city and with the rail company.

 

It’s hard to see such a calamity and think not metaphorically about my originally intended subject: the collapse of inBloom. inBloom was, in lieu of a more boring technical description, a cloud based data integration technology, that would enable districts and states to connect their data to an “app store” of programs and dashboards that could sit on top.  The vision was a seamless and less expensive way for teachers and principals to gain easy access to data about their students.

 

inBloom was a very big deal.  Started in 2011, several big funders and education heavies devoted their credibility and more than $100 million to try to make it successful.  Their efforts succeeded in garnering several state and district partners.  But since its inception, consumer groups, parents, and privacy advocates have worried that placing their students data in the hands of a third party would not be safe.  Or worse, inBloom might “sell” their student’s data to the highest bidder.  Then came Edward Snowden, and what was a niche news story went prime time.

 

If you look at the technology within inBloom that transfers and stores data in the cloud, the critics did not have much of a leg to stand on. inBloom’s data protection technology is as good or better than just about any existing state or district.  If you look at inBloom’s license agreement, parents and privacy advocates had more explicit protections than they have now with many student data systems.  What caused inBloom to collapse as quickly as the wall outside my window was more fundamental: trust.  As citizens, we trust districts and states with our students’ data.  And for all of inBloom’s technical explanations on the security of the data, they never made the case that we could trust them as an organization.  With the withdrawal of Louisiana, New York, Colorado, and several districts, nothing could hold inBloom up.

 

Over the past year at UPD, we’ve done a lot of work with the Ed-Fi data integration and dashboard suite.  We successfully rolled out the system for the entire State of South Carolina in about nine months (public dashboards here) and are very excited to start work with the Cleveland Metropolitan Public Schools to implement Ed-fi there.  Ed-Fi is very different than inBloom, even though they both utilize the same underlying data model.  Based on extensive research on what teachers and principals say they need, Ed-Fi provides a set of powerful data integration and dashboard tools that a district or state can download for free.  Rather than shooting data up into the cloud, Ed-Fi lives where most people already trust, in the data centers of the district or states.  19 states and more than 7,000 districts have licensed Ed-Fi.

 

The tragedy of inBloom is that it was a great idea ahead of its time and stood to do a lot of good in education.  But the protectors of the status quo should see no victory in its collapse.  Teachers and principals are clamoring for better information to help their students.  Ed-Fi seems ready to pick up where inBloom left off, and do so with the trust this work requires.

———————————

This blog was written by Bryan Richardson. Bryan is a Partner at UPD Consulting and brings over thirteen years of experience in private and public sector management. Bryan holds national expertise in performance management, data systems, and complex project implementation. 

Asking the Right Questions—Data Privacy and Security

There are a lot of good signs to be seen in recent news about security and privacy in the education technology sector. Some of the key questions being asked by educators and administrators are “how well are student data protected from prying eyes and greedy corporations?” and “who has access and how are the data being used?” These are good questions, and they represent the vestiges of our struggles with adopting modern technology over the past 15 years. Conversations have matured from simple arguments around the value of computers in every classroom to philosophical debates about our organizations’ embrace of performance data as the ombudsman of quality education. Progress has clearly been made, but in our rush to catch-up with our corporate cousins we missed asking what turns out to be a pretty important question–who owns all this stuff?

 

That’s the question that ultimately sealed the fate of inBloom, a non-profit offering a cloud-based data warehouse designed to help districts and vendors share student information. Despite funding from big foundation names like Gates and Carnegie, inBloom collapsed under the weight of a five word question they were never able to answer well enough to satisfy concerned stakeholders. If data are stored on a machine that is not physically located in a building owned by the district, who really owns the data?

The data issue is really a matter of security and access, which isn’t so different from the days of paper records in filing cabinets–information was kept in a secure, locked location and only certain people had access. With data warehouses replacing filing cabinets, the difference is that the information is stored off-site and the keys are also in the hands of the data warehouse manager (in other words, the system or database administrator). inBloom failed to effectively communicate this subtle difference early on, and any answer they eventually provided came across as reactionary, slick, dishonest, and–my favorite new term–“hand-wavy.”

 

Schools and districts aren’t used to asking those questions, and the education technology sector isn’t used to answering them. This disconnect doomed the effort from the beginning. Had the question “who owns this stuff?” been asked early on, the answer would have at least brokered a conversation rather than distrust and eventual dismissal–not to mention a waste of about $100 million dollars in grant funding.

 

Ideally, that conversation would lead to a compromise where information storage and archiving solutions satisfy the security and access needs of all players–parents, teachers, administrators, and the general public. Perhaps the right solution keeps an element of the status quo: secure data such as individual names, contact information, and other personally identifiable information could be stored on-site with the keys in the hands of the same people, but the bulk of the data could be stored in the cloud. Hybrid solutions like this are possible with dashboard software like Ed-Fi where the software itself can be installed on-site along with the secure data and set up to pull the remainder of the data from the cloud.

 

In the consulting world at UPD, we see those disconnect problems all the time: Group A spends a ton of time and money solving a problem for Group B without ever truly engaging the members of Group B. inBloom undoubtedly engaged their stakeholders in the early stages, but not deep enough to where someone was able to ask “who owns all this stuff?” This is often the result of too much focus on delivering a solution and providing answers rather than asking questions and identifying the problem. It comes with the territory–we get so excited about the possibilities of new technology that we jump right into requirements gathering without stopping to think if we’re asking the right questions and solving the right problem. It might just be as simple as an issue of maturity; if we’re getting serious about our relationship with technology, it’s probably time we start asking about intentions.

———————————

This blog was written by Andrew Keller. Andrew is a Consultant at UPD Consulting and brings over 10 years of experience in education, policy, and data metrics.  

Literary Lots: Sponsorship Interview

Literary Lots, Kauser Razvi’s community revitalization project funded through Kickstarter, is a program that helps urban neighborhoods whip vacant lots into shape by transforming vacant lots into summer program spots for children. In Cleveland, several lots will take on literary themes from children’s books and spend the summer months as spaces for art and education. Working with Cleveland Public Libraries and LAND Studio, Literary Lots will transform 2 to 4 vacant lots adjacent to libraries into six-week summer program spots for children in inner-city Cleveland.

Between June and August 2013, local artists will use themes from specific children’s books to re-create places, concepts, or adventures from the book, creating a magical and educational space to engage local youth in art and culture.  The lots will be filled with books (naturally), and will feature reading and writing classes, in addition to providing interactive games for kids.  The hope is these spaces will bring neighborhoods, cultural institutions and artists together in creative collaboration to bring books to life… and keep books in our children’s lives.

UPD, a sponsor, was interviewed about our interest in the project. You can read Doug Austin’s, UPD President and CEO, interview here.

———————————

Kauser Razvi, an Account Executive at UPD, brings over 14 years of public sector management experience to the team. Her consulting engagements have included the organizing community members, funders, and businesses around the establishment of Global Cleveland, developing long term project plans for the Chicago Out of School Time project funded by the Wallace Foundation and the development of technical and organizational strategies around data systems to improve business functions and operations in Government and non-profit organizations. She holds a BA in Sociology and BS in Journalism from Boston University and a Masters in Urban Planning from the University of Michigan.

Reinventing The Wheel

For years, many cities have undertaken the task of developing a citywide plan, agenda, goals, etc. around children and youth development and success.  In most cases, this work is a collaboration between multiple organizations, including the school district, city agencies (parks and recreation, libraries), city funded agencies and community based nonprofits.  While the core values that these organizations have around youth success are common, bringing these organizations together to discuss and arrive at a common mission and set of goals, objectives, standards, and measures to work towards can take years to accomplish.  Examples of this type of work are the Nashville Children and Youth Master Plan, Milwaukee Succeeds, Grand Rapids Youth Master Plan, Minneapolis Youth Coordinating Board, Chicago Out-of-School Time Project, Ready by 21 Austin, among many others.  Even more examples are included here, on the National League of Cities site.

A sampling of some of these plans is included in a table below.  Even doing a quick scan of these initiatives reveals many common threads in the goals and objectives that were the result of the months/years of collaborative work: youth/children are prepared for school, succeed academically, are healthy, are supported by caring adults, and contribute to the community.

In a recent conversation I had in discussing how to start this type of work, the question was raised “why don’t we just use what has already been done?”  So why spend years redoing the work when it has already been done?

“Reinvent the wheel – to waste time trying to develop products or systems that you think are original when in fact they have already been done before” Cambridge Idioms Dictionary, 2nd ed. Copyright © Cambridge University Press 2006

The reason for spending the time, effort and resources is because the participation in this type of process is as important or more important than the output.  Bringing together leaders across the city who may or may not have worked well together in the past to discuss not only their own organizations, but also how as a city they can work towards a common set of goals and objectives is incredibly powerful.  Building these relationships and knowledge about each other’s work should increase the chances of success in work towards the common goals.

Even though there is a lot in common with the outputs (master plan, goals/objectives) from each of these efforts, they also each have a unique aspect to them.  Each of the efforts involved a unique set of people and organizations who have their own perspectives about priorities in their city and communities.  These citywide plans and goals are something that (hopefully) these organizations will be working together on for a long time to come, so it should be something that they each feel a connection with – something that they helped create.

Of course, this does not mean that efforts like this should happen in isolation, when there are clearly good examples of what worked well (and what didn’t work well) in the past.  So, these type of resources should be utilized to learn from, but not for the purpose of cutting out any of the important work in the development of the end product.

“At the same time “reinventing the wheel” is an important tool in the instruction of complex ideas. Rather than providing students simply with a list of known facts and techniques and expecting them to incorporate these ideas perfectly and rapidly, the instructor instead will build up the material anew, leaving the student to work out those key steps which embody the reasoning characteristic of the field.”

Questions like this continually come up in the work we do.  Why spend months developing a particular school district process with participation from unions, principals, teachers, parents, etc. when there are good examples that have already been developed using this same type of process in other districts?  Why hold another community meeting or  focus group session if you think you already know what people think about a particular topic?  Because the process of “inventing” is as important as the “invention.”

 

This blog was written by Cari Reddick. Cari is a Project Manager at UPD Consulting and has over 12 years of project management experience.

 

Samples of Citywide Youth Master Plans

Nashville	Milwaukee	Grand Rapids	Minneapolis
All children and youth will have a safe and stable home and a supportive, engaged family.	All children are prepared to enter school	Early childhood development, life-long learning & education	All Minneapolis children enter kindergarten ready to learn
All children and youth will have safe places in the community, where they are welcomed and supported by positive adult relationships	All children succeed academically and graduate prepared for meaningful work and/or college	Employment & financial independence	All Minneapolis children and youth succeed in school
All children and youth will develop valuable life skills, social competencies, positive values and become law abiding, productive citizens	All young people utilize post secondary education or training to advance their opportunities beyond high school and prepare for a successful career	Basic, physical & psychological needs	All Minneapolis young people have access to quality out-of-school opportunities
All children and youth will have confidence in themselves and in their future	Recognizing the difficult economic realities facing our families, all children and young people are healthy, supported socially and emotionally, and contribute responsibly to the success of the Milwaukee community	Mentoring, afterschool, cultural activities & strategic planning	All Minneapolis children and youth people have opportunities to prepare themselves for the responsibilities of an active civic live
All children and youth will have opportunities to have their voice heard and positively impact their community		Civic engagement, training & leadership
All children and youth will experience social equity regarding access to opportunities, resources and information that are critical to their success in the 21st century
All children and youth will experience a safe and caring school environment that supports social, emotional and academic development
All children and youth will achieve academically through high quality, engaging educational opportunities that address the strengths and needs of the individual
All children and youth will be physically healthy
All children and youth will learn and practice healthy habits and have access to the resources that support these habits
All children and youth will be mentally healthy and emotionally well
All children and youth will have access to and participate in quality programs during out-of school-time
All children and youth will have safe outdoor spaces in their neighborhood that provide opportunities for play and recreational activities
All children and youth will have safe transportation options that allow them to engage in activities, and access services and supports that the community has to offer

Can Early Teacher Evaluation Findings Help Change the Debate?

Over the past few years, states and school districts across the country have devoted significant resources to the design and roll-out of new teacher evaluation systems.  Driven at least in part by requirements attached to Race to the Top funding, the new systems have inspired heated debate over the efficacy of factoring student achievement data into a teacher’s performance assessment. The New York Times recently shared some initial findings from states that have launched new evaluation models including Michigan, Florida and Tennessee, reporting that the vast majority of teachers- upwards of 95 percent in all three- were rated as effective or highly effective. Although the analysis of these numbers has only just begun, the Times reports that some proponents of the new evaluation models admit that the early findings are “worrisome”.  And even though it is still early, we can reasonably anticipate that if the trend continues- and the findings from the new evaluation systems reveal no significant departure from more traditional methods of evaluation- we may start to have a lot more people looking at the complicated data analysis driving teacher evaluation systems linked to student achievement data and asking “what’s the point?”

It’s a good question, really, and one that probably hasn’t gotten enough thoughtful attention in the midst of the controversy surrounding them: What is the point of linking student achievement data to teacher evaluations?  Should we take it for granted that a primary goal- if not the primary goal- of these efforts is to identify and eliminate bad teachers?  If this is the case then these early findings should be a cause for concern, especially given the time and money being spent to collect and analyze the data.  If replacing bad teachers with good ones is the magic bullet for public education reform, it will take a pretty long time at this rate.

Of course, even opponents of the new evaluation systems would probably admit that the magic bullet theory is an oversimplification. Furthermore, it’s much too early to look at these numbers and extrapolate any meaningful conclusions about the actual number of ineffective teachers or even the accuracy of the evaluations themselves. Hopefully what these findings might do is allow us to finally begin to broaden the scope of our national conversation about how the linkages between teachers and students could actually drive education reform.  States and school districts implementing new evaluation systems have tried with varying degrees of success to communicate the message that linking student achievement data to teacher practice isn’t just about punitive measures- it also has important implications for improving professional development and teacher preparation programs by identifying shared practice linked to positive student achievement and replicating those practices in classrooms across the country. But that message is often overshadowed by the anxiety surrounding the punitive side of evaluation and underscored by public struggles with local teacher unions. If nothing else, these early findings might create an opening in the current debate for a more thoughtful discussion about the broader possibilities for linking teacher practice to student growth.

-Jacqueline Skapik

Across the Board: Who Else is Responsible for the Atlanta Cheating Scandal?

The media in recent weeks has focused a great deal of attention on the cheating scandal in Atlanta in which authorities have indicted 35 officials and teachers in the Atlanta Public Schools system for allegedly altering the results of students’ state standardized tests to reflect higher scores.

There have been similar allegations in other schools and districts around the country.  What is unique about Atlanta, however, is the scope of the alleged fraud.  Those who work in education know that it is virtually impossible to keep a secret about even the most trivial matters in a large urban school district.  One cannot help but ask, therefore, how a scheme of this magnitude (with so many teachers and administrators directly involved) could gain such momentum.

By all accounts, it appears that pervasive in the culture of the district was the “Machiavellian” notion that the ends of increasing student test scores outweighed the means, and that it was this culture that enabled such widespread cheating.  Indeed, many point to an array of incentives that the superintendent allegedly used to reward those who generated higher test scores, whatever the cost to students.

The superintendent, however, does not operate in a vacuum.  Even the superintendent is accountable to the school board.  The superintendent alone cannot establish policy.  Nor can she create financial incentives for employees without the approval of the school board. She takes her cues as the tone of her leadership and school district culture from the school board.  If not, the school board should remove her.  Why then has there been so little media attention on the role of the board in the Atlanta cheating scandal?

The school board, at least as much as the superintendent, is responsible for creating a culture of integrity, honor, and accountability within the district.  Effective school boards model these values both in the conduct of school board meetings and in their interactions with school communities.

Moreover, effective school boards recognize and constantly communicate to others the critical importance of accurate data in improving instruction and learning outcomes.  They establish policies to ensure the integrity of test results.[i]  They examine student data on a routine basis, and hold district and school administrators accountable for the effective use of the data.  In so doing, the board ought to catch extreme abnormalities in the data, ask probing questions, and err on the side of a full investigation any time there is any reason to suspect even the slightest impropriety.

Reading the media accounts of the Atlanta cheating scandal, one is reminded of the Enron scandal, in which high level executives engaged in fraudulent accounting practices that devastated a company, its shareholders and employees right under the noses of its board of directors.  In response to such abuses, the federal government passed the Sarbanes-Oxley Act, which imposes a higher standard of accountability on corporate executives and boards of directors.  Among other things, the Act requires top management to certify the accuracy of financial reports.  The Act also imposes heightened responsibilities on corporate boards, through standing audit committees, to oversee the actions taken by top management.

Testing data is to schools, students and parents as financial information is to corporations, shareholders, and employees.  Perhaps a set of rules similar to those imposed by the Sarbanes-Oxley Act is necessary to ensure the integrity of student data.  Perhaps top management (such as the Superintendent, and chief accountability officer) should be required to personally investigate and certify the accuracy of student test data.  Likewise, perhaps school boards should be required to have standing committees to review the process and ensure its integrity.

Many argue that the lesson learned in Atlanta and elsewhere is that “high-stakes” testing will inevitably lead to cheating.  This is a sad conclusion, indeed.  It reflects a lack of confidence in the ability of all students to improve, and excuses from responsibility those adults upon whom our students are relying to help them improve.  Worse yet, it excuses those adults from even behaving ethically, and endorses behavior that sets a terrible example for students.  Cheating is not symptomatic of an inherent flaw in high-stakes testing (though there may be flaws).  Instead, cheating reflects a lack of integrity, leadership and good governance that is essential to the success of our system of public education.

There are very few areas in which school districts require more rather than less regulation.  Sadly, it appears that this may be one such area.  With or without additional regulation, however, the critical role of the school board in preventing such abuses cannot be overstated or overlooked.

— Kim Clark

---

[i] For further guidance on best practices in testing security, see “Issues and Recommendations for Best Practices,” which is based on comments and ideas generated during a Testing Integrity Symposium that the U.S. Department of Education held in February 2012.  This publication can be found at http://nces.ed.gov/pubs2013/2013454.pdf.

Leading Change in Education Reform Efforts

“A leader is best when people barely know he exists, when his work is done, his aim fulfilled, they will say: we did it ourselves.” —Lao Tzu

Leading people through a process of change is difficult, particularly in big, entrenched systems like traditional schools and districts. A school leader I know once told me that traditional school systems are like giant ocean tankers, you can make them change direction but it takes a lot of time and energy. Few school and district leaders are prepared to turn their ocean tankers around, particularly within the timelines and to the degree required for Race to the Top (RTT) to meet its ambitious goals.

As evidenced by the recent RTT year two reports released by US DOE, one of the greatest challenges to the success of the RTT reforms is not the content of the changes themselves, but simply that RTT entails significant change on the parts of individuals and systems, and change is hard.

Psychology research tells us that people don’t like change (status quo bias). Change takes effort, causes discomfort and sometimes can be downright painful. People fear the unknown. They wonder, “Will the work and the pain be worth the effort?”

Yet there is an urgent need for change in our education systems to ensure that all students are prepared for success in college, work and life. This need can be seen in student proficiency data from across the country. Rhode Island recently released the latest round of state assessment results, which were a grim reminder of how far we still need to go and how long it takes for systemic change to have an impact at the classroom level.

So how do we address the challenge of leading change?

In Rhode Island, one way we are supporting local leadership and spreading effective ideas to support RTT implementation is through the Collaborative Learning for Outcomes (CLO) process. Through the CLO process, Rhode Island district leadership teams meet regularly in facilitated sessions to share effective practices and learn from one another regarding RTT implementation strategies. The CLO process has provided a forum for district and school leaders from across the state to dig down into concrete strategies to support RTT implementation and to discuss mitigating the complex challenges they face on a daily basis with peers who struggle with the same issues.

In my work with the CLO teams in Rhode Island, the successful education leaders I have observed all share and act on the following beliefs about leadership:

(1) Communication must be a two-way street. To lead people into the unknown, you must listen, have honest dialogue, and be transparent about the work ahead. A number of district leaders who shared during CLO meetings that they created genuine opportunities for their teachers to express and receive answers to their concerns about the new RTT systems were the ones who were also most likely to report that everyone felt they were on the same team when it came time for implementation.

(2) Everyone must share ownership of the work. Distributing leadership responsibilities among those affected the most by changes builds internal champions and on-the-ground capacity, giving people responsibility leads to increased motivation to move the work forward, and getting implementer input on the “how” of the reform greatly increases the chances for success. Through the CLO process, I saw how school leaders who did not engage teacher-leaders in their buildings in developing implementation strategies were almost universally unable to move reform efforts forward with any reasonable speed. The opposite was true of those leaders who created real opportunities for teachers to hold responsibility for success.

(3) Leaders must support those on the front lines of change. Success depends on whether leaders can be creative about finding new resources and using existing resources, provide staff with needed training, and flexibly support staff to face the unknown. All educators in Rhode Island are working within the constraints of limited human and financial resources and the aspirational goals for RTT implementation. Many CLO discussions center on the challenge of stewarding resources wisely and creatively. While no one has found a magic bullet, those leaders who have acknowledged the insufficiency of the traditional structures for budgeting and using human capital are finding a variety of new ways to plan resource use so that they can provide as much support as possible for their staff.

By implementing innovative strategies, like the CLO process described here, we can help education leaders learn about and adopt practices that will increase the likelihood of success of RTT and other critical educational reform efforts.

–EFB

Improving the Ground Game in Education

We know political campaigns are driven by numbers. Since long before we started talking about data-driven decision-making as a key driver in education reform, political strategists had sophisticated models for crunching numbers and using them to determine where their money was spent, what their candidates said and where their rallies were held. But even though politics has always been a numbers game, in the aftermath of the 2012 presidential election, experts on both sides of the aisle are marveling about how the Obama campaign used innovative implementation strategies to take that game to the next level.

What does that “next level” look like? Broadly speaking, it is more precise, more dynamic and more individualized than its predecessors.  With a sophisticated strategy for using demographic data (including the 2010 census) to target resources and local campaign headquarters efficiently, Obama’s team built an unprecedented grass-roots level organization that generated support in key swing states.  They didn’t just crunch the numbers, they used them to develop a successful plan for turning them into votes.

As I read about the Obama campaign’s successful strategy, I was struck by the fact that several of the things they did very well are things that continue to serve as obstacles in implementing data-driven reforms at the state and local level in education. We know we should be using data to drive decision-making, but have we paid enough attention to our ground game?

Here are two areas where the Obama team’s strategy provides some insight into current challenges in education reform:

1. They connected their databases– When the Obama team realized that their fundraising database wasn’t connected to other key systems (such as their voter registration database), they immediately went to work, connecting all systems with critical voter information that previously didn’t talk to each other. This probably sounds eerily familiar to anyone who has worked in a school district or state-level education agency.  Separate databases are the rule, not the exception and connecting data sets to actually learn from the numbers is often a time-consuming task assigned to someone who is probably already working over-capacity. States are making progress with efforts to build longitudinal data warehouses, but progress is slow. In the meantime, people on the ground who need to use the numbers to drive decisions and resources don’t have the access they need. Instead, with the task of managing schools filled with thousands of students and teachers, leaders have to continue to build inefficient systems and work-around processes around disconnected data. We are moving to change this, but are we moving quickly enough?
2. They created an infrastructure in the field that supported their strategy– The Obama team developed sophisticated statistical models that allowed them to target groups of potential voters with extreme precision. With early warning systems and value-added growth data in student achievement, we are beginning to do the same in education. We are using data more precisely, and using it to plan instruction that targets specific students and specific skill sets. The critical difference is that the Obama team developed a new infrastructure in the field, designed to support that dynamic, individualized attention. They created small field offices in key counties, tight-knit teams of campaign staff trained specifically to provide the ground-level outreach to potential voters that their numbers told them was needed.  This is where education reform continues to lag behind. While we may be developing the systems and skills to help educators gather and analyze data, we have not made much progress in creating an infrastructure in public schools that allows us to use the data to drive action and resources quickly and creatively.

Education reform initiatives, like political campaigns, fail or succeed in their implementation. We should be looking for and learning from examples of successful implementation efforts wherever we can find them to ensure our ground game (like our president’s) is strong.

— JS