Finding your place on the journey to data maturity, Part 1
Adapting to remote work as a result of COVID-19 may be quick and efficient for data-savvy nonprofits. For others, it may be chaotic and bewildering. While many offices remain partially or fully shut down, those offering frontline services have been forced to consider how to rapidly redesign their engagement and delivery approaches.
Information and communication are the lifeblood of any organization. Access to the right analysis and confidence in data security and quality translate to more informed and timely decisions on how to support vulnerable beneficiaries. As the global health crisis continues, harnessing the power of data is crucial to survival now more than ever.
For many nonprofits, the crisis has revealed weaknesses while also affording time to think about building their “core strength.”
Measuring how good nonprofits are with data
At Data Orchard, we’ve been researching how social sector organizations leverage their data since 2015. We’ve created and tested theories and models, developed assessment frameworks, and worked with hundreds of nonprofits to learn as much as we can about how the sector is progressing on the journey toward data maturity.
There are three key reasons we engaged in this research:
We wanted an evidence-based understanding of the problems and needs around data in the sector.
We needed a consistent, reliable way of assessing where organizations were with their data so we could determine the best way to support them.
We needed a repeatable way of measuring progress and outcomes from our support interventions.
Taking our operation online
We realized after our initial prototyping that if we carried on working with individual clients, we could only support 30-40 organizations in the UK each year—and mostly those who were accessible from the West Midlands, where we’re based. But if we made a tool available online on a self-assessment basis, we could support more data champions and partners globally and raise awareness and interest, while making a far greater impact in the process.
We set a goal with our digital partners Outlandish, to enable more nonprofits to understand their data capabilities and where they needed to improve. In doing so at scale we sought to evidence the state of the sector’s data maturity to mobilise action from funders, policy makers and support providers.
In October 2019 we launched a free online self assessment and benchmarking tool, with support from Digital Impact, the Esmee Fairbairn Foundation, and CAST. The tool scores organizations across seven key themes (uses, data, analysis, leadership, culture, tools, and skills) and gives a summary of strengths and priorities to enable leaders and data strategists to focus their energy and resources (see a sample summary page of the data maturity assessment and benchmarking report). The full version also gives benchmarking results and more detail on the next stages for each theme.
Accelerating data maturity: Two models
Overall, our evidence suggests reaching mastery with data is a long-term strategic process. (The most advanced organizations we spoke to still say they’re working at it after 8 or 10 years.) In order to find out what it takes to accelerate data maturity and how we could best support organizations, we launched our ‘Becoming Data Savvy’ pilot program, which offers nonprofits a mix of assessment, workshops, and planning.
We tested two support models – one light touch (10-day consultancy), the other more intensive and in-depth involving implementation of priority recommendations (70-day consultancy).
The employees and trustees of charities with similar income and staff size took a data maturity assessment at the beginning and end of a twelve-month project. While no progress was seen in the 10-day program, the 70-day program created a real shift toward data maturity for those who participated.
Data maturity before and after ‘light-touch’ 10-day support model
Data maturity before and after ‘intensive’ 70-day support model
Our analysis confirmed little if any change in most scores for the 10-day support intervention, with the exception of Skills, which increased considerably, and Uses, which decreased considerably. We attributed this to increased knowledge and a shift in understanding how data was (or was not) being used in the organization. Our 70-day support interventions, which provided a substantial injection of strategic-level capacity building, saw progress in all theme areas. The Skills theme again stood out as most improved.
Changes to data maturity themes before and after 10-day support programme, vs. 70-day support programme
Charity leaders identified a number of key benefits from the program:
The program provided an independent external analysis of the charity’s data maturity by experts in the field.
Staff, volunteers and trustees gained a better understanding of the different types of data for measuring impact and which ones are, and are not, being collected in the organization.
Impartial, product agnostic advice, guidance and information provided more options and empowered them to make their own decisions.
Experienced professionals raise necessary questions that are sometimes difficult to ask.
Connecting with other social sector leaders at different stages of data maturity in their organizations gave hope and inspiration (at Data Orchard’s first national Data4Good conference in Birmingham).
The ‘dos and don’ts’ of involving trustees
It is important to talk to trustees about data maturity and the value of data more broadly. This doesn’t mean they are good assessment candidates. In most cases, they can’t answer the questions because they often lack an intimate view of how their organizations operate. In the two cases where we involved trustees and staff in the ‘before and after’ assessments, trustees tended to be considerably under- or over-optimistic about the organization’s data maturity, as compared to staff.
A more in-depth onsite independent data maturity assessment concurred with the staff view in the online self-assessment. That said, we strongly encourage involving trustees in the process—by having them participate in workshops and respond to presentations of findings, e.g. reviewing impact and theory of change and data strategy.
Investing time and resources
A key point to note from the charity receiving 10 days of light-touch support, is that while some progress was made, there remained frustration around the lack of capacity (time or financial resources) to take forward the recommendations. But to accelerate data maturity, organizations have to invest.
Access to funding from grant-makers to build data capabilities is difficult. Moreover, the kind of costs involved in making the necessary strategic changes—particularly for technology—are nearly impossible to access. We estimate that a small- to medium-sized organization spends in the region of £50-£120k ($65-$150k) to upgrade and replace tools and systems, and to cover staff and consultancy costs.
In testing our 12-month “Becoming Data Savvy” support program, we found that a good quality diagnostic assessment can be completed sooner than expected i.e. 3 months vs. 12. It’s worth mentioning that Data Orchard’s online assessment tool is just part of what we offer. It would be fantastic to do more strategic long-term support interventions with organizations. However, given the current grant-making landscape, charities are rarely able to access the scale of investment needed to advance their data maturity. We welcome more programs from grant-makers that provide funding to support this among charities and social enterprises.
There are encouraging signs from some—SIB, Lloyds TSB Foundation, National Lottery Community Fund—which we hope will develop further. In the meantime, as part of our championing of Data4Good, we have published a list of data support providers and are looking into offering an advanced version of our data maturity assessment and benchmarking tool to others serving nonprofits.
Look out for part 2 of this blog, where we’ll share four lessons from our research.
For a detailed look at how charities work with us to achieve data maturity, visit our support page.
This blog first appeared on www.digitalimpact.io
Digital Impact Grants was a three-year initiative (2016-2018) supported by the Bill and Melinda Gates Foundation, to fund research and innovation to strengthen the safe, ethical, and effective use of digital resources in civil society.