“I don’t even know where to start with data!”
This is hands down the most common challenge I hear from nonprofits, big and small. And I get it. Do you need to hire new people, or train you existing staff? Who would you hire or what would you train in? Do you need a new CRM? Visualization software? Database management software? Should we even think about predictive analytics – and what does that mean anyway?!
So I’d like instead to give you ten actionable steps that you can take right now to up your analytics game. You’ll probably find that a few fit your organization and needs better than others, so start there. Or you can pick the easiest one for you to do. The idea is to take an action that can build momentum. You’ll probably find that many of these steps require collaboration or input from someone else, so it’s also a good chance forge alliances that will strengthen your analytic journey. This article will cover:
- Determining if you are measuring outputs or outcomes.
- Conducting a data inventory.
- Creating data dictionaries.
- Assigning strategic metrics.
- Assigning tactical metrics.
- Partnering with graduate programs for research.
- Running A/B tests.
- Buddying up on data.
- Appointing a “Devil’s Analyst.”
“Take the first step in faith. You don’t have to see the whole staircase, just take the first step.” ~ Martin Luther King, Jr.
#1: Determine if you are measuring outputs or outcomes
Every organization, no matter how small, is tracking something already. My first challenge to you is to find that thing (or perhaps things) and ask: “Are these outputs or outcomes?”
You may rightly wonder what the difference is. As nonprofits, we exist for a purpose. We have a burning need that we are addressing. That need or needs being solved is the outcome we care about. The solutions, programs, effort, and money we put into reaching the solution are the outputs we create.
Outputs are almost always easier to measure than outcomes, so you most likely will find that you’re counting outputs. For example, it’s far easier to track how many free lunches you deliver or yoga classes you lead than how much you’ve raised food security or eased working mothers’ stress.
If this is the case, can you think of at least one critical outcome that you could be tracking to ensure that you’re advancing your mission? Once you have that, you can use the next step to figure out if you can actually calculate that outcome metric.
#2: Take Stock of Your Data
Creating a data inventory is a great way to discover just how many resources you have available to your analytic cause. And you’d also be surprised how many of them were previously unknown either to you or others who would benefit from using them. By formally taking stock and documenting all data sources, analytics tools, and other resources, it becomes far easier to identify whether you can answer a specific question or tackle a specific analysis.
If such a data directory already exists in your organization (lucky you!), investigate how recently it’s been updated. This list should be a living document to stay current with organizational and programmatic changes. Once you have a working list, consider taking step 3 next.
#3: Create Data Dictionaries
For every major database or data source you have, you should also have a corresponding data dictionary. What’s a data dictionary? It’s the guidebook to each dataset that defines what’s contained. If someone hands you an Excel file full of numbers, you need to understand what each column, or field, represents, where it all came from, how it was calculated, and if there’s anything tricky you should know. A data dictionary should contain the following fields at a minimum:
| Field Name | Field Type | Description | Source | Notes |
|---|---|---|---|---|
|
The Field Name is simply whatever the column or field is called. It's important to know the exact name of fields from CRMs or other forms so you can match what you're seeing in the data table or export with the original source. |
Field Type is simply what kind of data are supposed to be in this field. Is it numbers? Text? Links to files? This helps users know if something might be wrong or invalid in the field. |
Even if the field name seems self-explanatory, it's helpful to describe exactly what the data represents. If it has been calculated, include the formula. |
If different fields in a database or dataset have different sources, it's helpful to list them out for each field. Otherwise, you can include a general note with the dictionary that states the original source for the whole dataset. |
This is where you can put any known issues or caveats to the field, or other tidbits that could be helpful to a new user of the data. |
|
E.g. "Graduation Year" |
Numeric, four digit year (e.g. 2020) |
Calendar year in which the student completed all requirements for and was awarded a degree. |
BlackBaud (student tracker) |
A student will have one graduation year, and thus one entry, for EACH degree earned. |
Data dictionaries may feel like a bunch of busy work, but just think about the next time someone asks you for a specific piece of information. If you have already pulled together a data inventory and data dictionaries, it would take you mere seconds to scan through and say, “Why yes, I can tell you how many students came from rural Montana for your grant” or “Yep, we track the age of every volunteer so I can make you a list of 18-24 year-olds to target for your new internship program.”
#4: Assign a KPI to Every Strategic Goal
You can do this for your organization’s strategic plan, your team’s, or your own personal one. The idea is to go through each strategic goal and create a measurable, meaningful metric or key performance indicator (KPI) that will serve as the compass for that goal. It needs to tell you if you are making progress, falling behind, or have successfully achieved it.
If you’ve done steps two and three, then once you have your list of metrics you should easily be able to determine if you have the information to calculate and track those metrics. If you find out you don’t, you can see how hard it would be to get the necessary data, or you can consider an alternative metric.
Not sure about how to create KPIs? You can get some ideas of common nonprofit KPIs here.
#5: Assign a KPI to your key tactics
Your strategic goals, and the metrics that track your movement towards success, are likely not enough to tell you what to do differently if you don’t like the direction you’re going. They will just tell you the finish line and how far you are from it. That’s why a good strategic plan should also lay out the specific tactics, or actions, that you and others are taking to get there.
But how do you really know if those are the right tactics? You’ll need a metric to show how successful each tactic is, so when a strategic metric comes up short you can determine which tactic isn’t pulling its weight. A good tactic KPI will capture not just the actual result, but also the literal or figurative cost to achieve it.
A common example of this is looking at funding sources. The strategic goal might be X dollars in fundraising. One of the tactics then could be targeting federal grantors. You would want to track not just how many dollars you’ve been awarded from federal grants, but also dollars per man-hour spent to get them. That way, you can see 1) are we getting enough in from this source to achieve our ultimate goal and 2) are we doing so in a sustainable or worthwhile way?
#6: Introduce Yourself to a Graduate Program
Did you know that every year there are thousands of grad students who have to find a decent question to research for their Masters or PhD thesis? Imagine how many of those students would love to know that their work will have real-world impacts rather than just gathering dust on the library shelf.
Look up your local higher education institutions, including community colleges, and see what programs they offer that might overlap with the work you do. Nonprofit administration, program evaluation, behavioral economics, organizational psychology, and others could all be relevant to your efforts generally. Or you could look at programs that align with the specific work you do, such as environmental studies for a non-profit focused on habitat conservation or urban planning for an organization combatting homelessness.
Once you have a few ideas, email the PI (principle investigator) or head of the department and introduce yourself. Explain what your organization does and that you are wondering if they would be open to collaborating on research projects or having you host a student’s project. You may already have a few research projects in mind that you can share, but you can start this conversation even if you don’t. They may have ideas of their own, too.
#7: Set up an A/B Test
An A/B test is a simple experiment to see whether approach A or approach B worked better to achieve goal Z. For example, you can test to see if sending emails in the morning or the evening gets better response rates, or if including a photo of animals vs people on a landing page increases donations. The idea is that you try each approach on a decently sized group of people, and then measure the output each approach gives you. Then you move forward with the winning approach on a larger scale.
If you run your email campaigns through MailChimp, Convert Kit, Constant Contact or any other major service, this ability is baked right into the service. If you’ve never done this before, you can simply Google “Run A/B test in Platform X” and you’ll immediately find how to carry it out in the email provider of your choice. Start by picking two possible subject headers for your next email, and send each to a subset of your list. Most of these platforms will then automatically use the header that got more clicks to send to the rest of your list.
Even if you don’t have this functionality built into your email provider, you can do it manually by creating your own “sample” lists of contacts from the full list of eventual recipients, sending two different emails to the groups, comparing the results, and sending on the winner to the rest of the recipients.
You may already be using A/B testing in your emails. If so, then I challenge you to find a new place you can use it. Perhaps in your next job posting? Google Ad campaign? Program format? Get creative!
#8: Find A Possible Nonprofit data-Sharing buddy
The power of nonprofits networks is well-known and undeniable. What’s less often considered is how nonprofits can collaborate to ease the burden of analytics and data, both on themselves and their constituents.
For example, I’ve been supporting one of my clients one such partnership. The American Indian College Fund is spearheading a collaboration among three other native scholarship organizations to create a shared student database. This collaboration will enable all organizations to access a bigger pool of student data, reduce overhead expenses for data maintenance, share analyst resources, and eliminate duplicative surveying of a population they often share.
You won’t build a collaboration like this in a day. But you could wander through some Google pages and find possible partners, or brainstorm ways that a collaboration could help you and a partner improve analytics. Perhaps you could even bounce some of those ideas off a trusted colleague or supervisor, and bring one or two vetted ideas to your leadership.
#9: Assign a Devil’s Analyst
Scientists don’t really think like “normal people.” While that may sometimes endanger their wellbeing, it comes in very handy when we want to break out of one of the most common mind traps we humans fall into: If we are asked, “Is XYZ true?” we inevitably will search for data and information to say that it is true.
The scientific approach is to do the exact opposite and look for the data and information that says it’s false. Finding even one thing that disproves a guess tells you it’s wrong. Finding one thing that supports it just tells you that it could be right – but you don’t know for sure. Yet we always want to look for things that confirm our first guess, rather than challenge it. Here’s a great video on this idea from Veritasium if you’re not quite sure what this means.
The point of this is that we all have this bias, and we aren’t going to get rid of it. Instead, we can create systems (like the scientific method) that help us look for different information. You can do that too by appointing a “devil’s analyst.”
This is a role I made up based on the idea of the devil’s advocate. I suppose you could also call it a “data ombudsman.” The idea is that this person (and you can rotate who fills the role) will ask of each major decision, idea or project, “What data or information would lead you to change your mind? What data would show this idea doesn’t work? What outcome would we see if the project isn’t successful?” By making sure someone is in charge of asking this question every meeting, every day, it will force us look for the most critical information we need to succeed.
Which will you do?
Hopefully you have at least one step you can take to move the needle on analytics in your organization! You probably don’t want to tackle everything in this list all at once… so pick your favorite and give it a shot. I love hearing success stories so be sure to drop me a line as you achieve new heights, or if you have any questions on how to step up your analytic game.
