Established in 2004, Boston After School & Beyond (BASB) emerged out of two previous organizations to serve as one centralized body responsible for strengthening and supporting the city’s out-of-school network and their collective efforts. Its challenge: align 57 agencies, operating in 136 sites and serving more than 10,000 youth, around specific shared and measurable goals.
But BASB lacked a standardized data collection platform that could integrate data from multiple inputs, and allow participating partners to compare their performance to aggregate citywide outcomes. “We didn’t know how many kids, exactly, were in our Boston programs,” says Chris Smith, BASB President and Executive Director. “Who was showing up? How often? And what were they doing?” In addition, as recognizing student skill gains was becoming more urgent, BASB needed an efficient way to evaluate progress and reward inpidual accomplishment.
Solution: Create a flexible platform that encourages participation
There was one more wrinkle: agency participation in BASB initiatives is entirely voluntary, not an obligation. To encourage participation, BASB’s data platform had to be both easy and rewarding to use. Working in close collaboration with BASB and its partners, Cityspan designed a solution, launched in 2014, that met urgent agency needs:
Alternative data entry methods
Participating agencies had conflicting data collection processes and priorities. The larger ones had already invested in data tools, such as Salesforce, and did not want to re-key data; the smaller ones were eager to acquire data tools they did not have. Cityspan designed a system that accepted inputs in two ways, from data uploads or online Web forms. Large agencies could upload their data without additional entry work; smaller agencies could use the online forms, giving them a data repository without demanding an IT investment on their own.
Integrated reports that align with national research organizations
Before Cityspan, report creation was a time-consuming affair based on the manual assembly of spreadsheets. In 2013 BASB designed a PRISM Report which integrates data from two of the most important national youth assessment organizations, the National Institute on Out-of-School Time (NIOST, at Wellesley College) and the Program in Education, Afterschool and Resiliency (PEAR, at Harvard University). Cityspan created an engine that produced the PRISM outcomes report automatically, pulling data seamlessly from multiple input sources across years. Integrating the PRISM Report into the system design vastly reduced report generation time and manual errors, allowing participating agencies to access reports that align to national research benchmarks, automatically.
STEM skill badges awarded by merit
Like other afterschool networks across the country, BASB wanted to foster greater accomplishment in “STEM”: science, technology, engineering and math. The awarding of badges can encourage excellence, but BASB wanted to track, measure, and recognize substantive improvement, not just attendance. Cityspan built a badging module that incorporates survey results from instructors that evaluate participants in five key competency areas – critical thinking, engagement, communication, perseverance, and teamwork – underpinning the badges with evidence that makes them legitimate credentials.
Results: Improved visibility and resource allocation
At the operations level, Cityspan gave BASB real-time visibility into program enrollment and daily attendance among its agency partners. The partners, through the PRISM reports designed by BASB and generated by the Cityspan system, got outcomes measurements that allowed them to assess their own performance, not just against national standards, but against the aggregate results of the BASB network.
“It gave us the ability to connect programs to a larger agenda, like the Mayor’s Summer Learning Program,” says Smith. He believes the biggest advantage lies within the evaluation data submitted by program staff: “We can look at the trends,” he says, “then direct resources to address emerging needs.”
The digital STEM Badge pilot proved successful, awarding 1,422 badges to 297 students in its first year of implementation. In the near future, BASB plans on connecting district school data to program data within the Cityspan system, an enrichment that will help agencies refine both their progress analyses and the nature of their programs.
“Our agencies love these improvements to the PRISM reports and are excited about the badging platform,” Smith concludes. “Our success depends on having a good product that engages partners year after year.
Cityspan benefits at a glance:
- Integrate data collection from 57 agencies operating from 136 sites
- Produce NIOST- and PEAR-compliant outcomes reports automatically
- In pilot, awarded +1,420 merit-based STEM badges
- Reduce cost, time-burden of repetitious operations tasks