Introduction
What organizational health is
Why we assess organizational health in our charity evaluations
Its positive correlation with organizational effectiveness
Its contribution to the success of the broader animal advocacy movement
Donors’ interest in organizational health
The organization’s policies and processes
The organization’s staff engagement and satisfaction
Additional context provided by the organization’s leadership
Some things that we do not assess
Why we focus on organizational health risks, rather than strengths
We are currently unable to fully investigate harassment and discrimination claims
Staff engagement surveys are inherently limited
Assessment results may be affected by the organization’s size
Our assessment may be subject to cultural biases
Our assessment takes time to complete
It is hard to find reliable data on specific elements of organizational health
Introduction
ACE’s charity evaluations aim to identify organizations that are the most likely to produce significant benefits for animals with additional donations. To do this, we assessed charities in 2023 based on four criteria: Impact Potential, Cost-Effectiveness, Room for More Funding, and Organizational Health.
In this blog post, we set out the rationale for assessing the last of those criteria: Organizational Health. We outline what we mean by organizational health, why and how we assess it, and some outstanding limitations and uncertainties with our assessments. We are focusing on this criterion because some have told us they find it less intuitive than our other evaluation criteria due to its less direct impact on animals.
Our assessment method is not set in stone. As we continue to expand our knowledge, engage with external experts, and learn from the charities that we evaluate—particularly those whose location, culture, and other circumstances we are less familiar with—we will continue to refine our methods to ensure they are as accurate and representative as possible.
What Is Organizational Health?
While definitions of “organizational health” vary, a healthy organization can be broadly defined as one where “culture, climate, and practices create an environment conducive to employee health and safety [and] organizational effectiveness.” Such an organization “improves employee performance and productivity, thereby positively promoting an organization’s image,” and “supports high employee performance, [with] employees [who] have an interest in the organization’s future success.”
For ACE’s organizational health assessments, we particularly seek to understand whether there are any aspects of an organization’s leadership or workplace culture that could pose a risk to the charity’s long-term effectiveness or stability, thereby reducing its potential to help animals. For example, an organization’s leadership team might be continually distracted by unproductive internal conflicts, or their staff might be unhappy and disengaged to the extent that the organization’s performance suffers. Such organizations are less likely to consistently deliver on their objectives and effectively use their human and financial resources, which ultimately translates into fewer animals being helped by their work. In other words, we focus on identifying whether a charity seems to have significant risks to its organizational health, rather than seeking to determine how good its organizational health is.
We recognize that interpretations of organizational health differ widely depending on the charity’s location, the demographics of its staff, and other unique circumstances (such as the organization’s age, size, or structure). Because of this, we seek to take a broad definition of organizational health rather than focusing too narrowly on any one element or proxy. A “healthy” organization, for us, is one that fulfills the following conditions:
It has adequate policies and processes in place to ensure the smooth running of its operations and the safety and wellbeing of its staff.
Its policies and processes, combined with the actions and behaviors of the organization’s leadership, translate into reasonable levels of staff satisfaction and engagement.
When potentially significant organizational risks are identified, the organization’s leadership team responds to those risks effectively.
Why We Assess Organizational Health in Our Charity Evaluations
Broadly speaking, understanding charities’ organizational health is valuable for our charity evaluations due to:
Its positive correlation with organizational effectiveness.
Its contribution to the success of the broader animal advocacy movement.
The level of interest in organizational health from our stakeholders.
The relative lack of focus on organizational health in the animal advocacy movement.
One factor missing from that list is the direct benefits for individual staff members at the organization. Healthy organizations are likely to be more pleasant places to work, and as animal advocates ourselves, we think that helping people enjoy their work is in itself a good thing. Positive staff wellbeing is also instrumentally valuable in that it is likely to result in better outcomes for nonhuman animals. However, our ultimate mission is to help nonhuman animals, and our assessment is not designed to promote staff wellbeing solely for its own sake.
Positive correlation with organizational effectiveness
The primary reason that we consider organizational health is its positive correlation with organizational effectiveness. This can help us to understand whether charities’ effectiveness might be constrained by organizational health risks within our two-year recommendation period, even if their work currently seems effective based on our other evaluation criteria.
There is evidence suggesting that strong organizational health is positively correlated with staff engagement, satisfaction, productivity, and retention. This means that staff at such organizations tend to carry out their work more thoughtfully and effectively and stay at the organization for longer, helping to maintain institutional knowledge.
For example:
Gorski et al. (2018) report that all of the activists they interviewed attributed their burnout in part to negative organizational and movement cultures, including a culture of martyrdom, exhaustion/overwork, the taboo of discussing burnout, and financial strain.
A meta-analysis by Harter et al. (2002) indicates that employee satisfaction and engagement are correlated with reduced employee turnover and accidents and increased customer satisfaction, productivity, and profit.
Schyns & Schilling (2013) report that poor leadership practices result in counterproductive employee behavior, stress, negative attitudes toward the entire company, lower job satisfaction, and higher intention to quit.
Waldman et al. (2012) report that effective leadership predicts lower turnover and reduced intention to quit.
Wang (2021) reports that organizational commitment among nonprofit employees is positively related to engaged leadership, community engagement effort, the degree of formalization in daily operations, and perceived intangible support for employees.
There are also some studies indicating that organizational health has a negligible impact on organizational performance. For example, a study by Xanthopolou et al. (2022) indicates that “strong cultures do not have a significant impact on organizational performance in the public sector”—though ultimately, the authors state that they are sufficiently unsure about the study’s findings that they can only confidently conclude that organizational culture and its effect on performance needs further investigation. While there are relatively few such studies, this may partly be due to publication bias; see the Limitations and Uncertainties section below.
There are some additional factors that we believe are reasonable assumptions but where we are not aware of empirical evidence supporting them, including:
When leaders encourage constructive feedback from their staff and instill a culture where staff feel able to give such feedback without any repercussions, we would expect a greater likelihood of identifying and addressing instances of ineffective leadership.
We would expect organizations with good organizational health to be less likely to have to divert resources toward dealing with severe HR issues and subsequent “damage control” should such issues become public. This means more resources are available to help organizations carry out their core mission of helping animals.
We would also expect such organizations to typically benefit from a better reputation within the animal advocacy movement, meaning they are more likely to attract funding and prospective staff.
Contribution to the success of the broader animal advocacy movement
Problems with leadership and workplace culture could negatively affect the reputation of the broader animal advocacy movement, as well as employees’ wellbeing and willingness to remain in the movement. Previous harassment cases within the movement, which received significant media coverage at the time, indicate that this is a valid concern.
We expect the strength of the broader animal advocacy movement to correlate positively with the health of individual organizations within it. A movement whose organizations’ leadership teams treat all of their staff well, encourage constructive feedback on their own performance, deal robustly with reports of poor behavior, and demonstrate genuine accountability is likely to be a more sustainable and attractive one than a movement whose leadership does not do those things.
This is particularly important for a movement like the animal advocacy movement, which some audiences perceive as radical and extreme. If we are to be taken seriously, welcome new members, and forge or maintain alliances with key stakeholders, we need to cultivate a positive external image. (This is easier said than done, and ACE has sought to be transparent about our own past mistakes in this area.) By assessing organizational health, ACE hopes to promote charities that are positively influencing the animal advocacy movement’s reputation and support leaders in the movement to consider how best they can put in place robust organizational practices.
Donors’ interest in organizational health
Many of our donors care about charities’ organizational health and want to ensure their donations go to charities with practices that contribute to an effective and sustainable animal advocacy movement. This is compounded by the fact that we are only aware of a small number of organizations working on improving the organizational health of the movement. Given the likely importance of this to the movement’s long-term effectiveness, many of our donors are keen that we help to fill that gap.
We are aware that some donors want us to go further in our organizational health assessments, acting as more of a watchdog organization. This is not currently possible as our team does not have the time or the expertise to conduct comprehensive, watchdog-type investigations into each of the charities that we evaluate. However, we see the value of such work and hope to see an effective, well-established organization filling this space in the future.
What Information We Collect
In line with the three components of organizational health that we set out above, we seek to assess whether there are any significant issues regarding charities’ organizational health by doing the following:
We inspect information on organizational policies and processes provided by top leadership staff.
We solicit feedback from charities’ staff and volunteers via our engagement survey, taking into account any whistleblower reports that we receive.
We discuss any potential concerns with the charities’ leadership to ensure we have the necessary context.
Below, we outline what this looked like in our 2023 evaluations. For further details, please take a look at the full reviews of our 2023 Recommended Charities.
The organization’s policies and processes
We provided charities with a list of policies that appear to be strong indicators of organizational health, based on a review of the relevant evidence, advice from external experts such as Scarlet Spark, the policies assessed by organizations such as Charity Watch and Charity Navigator, and our own learnings from charities in previous evaluations. We asked charities to indicate which of these they had implemented. These included policies and processes to ensure fair compensation, psychological safety, clear organizational design and communication, fair performance assessments, access to learning and development, and a fitting approach to representation, equity, and inclusion.
This list also included basic governance policies, including an anti-retaliation policy protecting whistleblowers and those who report grievances, a conflict of interest policy, a policy setting out procedures for the storage and destruction of documents, and a process for documenting minutes of board and board committee meetings.
We also considered the charity’s level of transparency with:
Their own staff, in terms of how they shared all of the above policies with their staff members.
The public, in terms of the documents they shared on their website (such as their financial records and the identities of their board members).
ACE ourselves, in terms of their willingness to share the information that we needed to conduct a thorough evaluation.
Lastly, we asked questions about the structure and membership of their board of directors to assess how they aligned with the best practices set out by BoardSource. Where relevant, we also asked about charities’ approach to international expansion, particularly how they navigated any potential power imbalances between offices in the Global North and the Global South.
We recognize that not all of these policies and processes will be common, or relevant, in all countries and contexts, which is why we followed up on any potential gaps with the charity’s leadership so that we could understand the full context.
The organization’s staff engagement and satisfaction
We solicited charities’ staff and volunteer perspectives via our engagement survey, based on the Gallup Q12 Employee Engagement Survey and supplemented by other models and frameworks such as the Maslach Burnout Inventory, Google’s Project Oxygen, and cross-cultural research by Culture Amp.
The engagement survey also contained a link to an anonymous Whistleblower Form, developed with support from legal experts at Animal Defense Partnership, for any staff or volunteers who wished to report issues of harassment and discrimination.
Please refer to our 2023 Engagement Survey for further details, including the full set of survey questions.
Additional context provided by the organization’s leadership
In some cases, our assessment indicated a potential issue or gap. For example, the charity might have been missing several important policies, or a significant proportion of staff might have reported low levels of engagement and satisfaction. In those cases, we asked the charity some follow-up questions, alongside any other questions regarding our assessment of the other evaluation criteria.
These discussions gave charities the opportunity to:
Provide additional context that mitigated our concerns, e.g., by explaining why their specific circumstances meant that certain policies weren’t necessary, or justifying why they had not been a priority to date.
Recognize that we had identified a potential issue, and set out a clear plan of action to address it.
With regard to our 2023 evaluations, these discussions sufficiently addressed any initial concerns that we had. This would not have been the case if, for example:
We received multiple reports of widespread harassment and discrimination at the organization.
The engagement survey indicated that a high proportion of the organization’s staff are unsatisfied and unmotivated.
A high proportion of staff reported that they did not feel safe voicing their opinions about the charity’s leadership and performance.
(If any of the above are true) The charity’s leadership did not take these complaints seriously or put in place any measures to address the underlying issues in the future.
How We Use Information
After gathering the information mentioned above, we held a meeting with the Evaluations Committee (formed of a subset of our Programs team) to discuss each charity’s organizational health assessment. This was to establish whether, as a team, we had any major concerns about the organizational health of a charity that would pose a significant risk to its effectiveness and stability, and should therefore prevent that charity from being recommended.
Once we completed our assessment of all four evaluation criteria, we held several team meetings to finalize our recommendation decisions. In 2023, our organizational health assessments did not yield any major concerns at any of the charities we evaluated, so performance on this criterion was not a deciding factor at this final stage of the process.
Factors we do not assess
Some factors that could be considered relevant to a charity’s organizational health are assessed through other criteria. For example, our Room for More Funding evaluation assesses charities’ budget and expenditures (both historic and projected), while our Cost-Effectiveness evaluation assesses how effectively charities have used their funding to achieve positive outcomes for animals.
We also do not ask charities for the demographics of their staff. While we think that it is important for charities to have robust procedures in place to prevent discrimination and promote healthy Representation, Equity, and Inclusion (REI) within the organization, we do not think that it is helpful to use demographic diversity as a proxy for this, particularly given that we assess charities in a range of countries and cultures where demographic diversity is likely to differ for a variety of irrelevant reasons.
Why we focus on risks, not strengths
In practice, gaining an accurate picture of charities’ organizational health is very difficult. For example, we recognize that our engagement surveys are subject to selection bias and may also not reflect employees’ true opinions, as they are aware that their responses could influence ACE’s evaluation of their employer.
In some of our past approaches to this criterion, we sought to give a more comprehensive assessment of charities’ strengths. However, we realized that we were not able to conduct a sufficiently thorough investigation to be confident in these assessments, some of which later turned out to be very different from the reality.
In 2020, we therefore changed our approach to this criterion. Since then, we have focused on determining whether there might be issues in the organization’s management that have a significant negative impact on staff productivity and wellbeing, and could be detrimental to the organization’s effectiveness and stability, with potential knock-on effects to the broader movement. While we do still note positive elements of charities’ culture and leadership, this is not the focus of our assessments.
Limitations and Uncertainties
While we strive to continually improve our assessment of charities’ organizational health, we recognize that several limitations remain. Below, we set out some of these limitations and some of the ways in which we have sought to address them.
We are currently unable to fully investigate claims
Limitation: We are currently unable to fully investigate harassment and discrimination claims due to a combination of time constraints, lack of expertise, and the often anonymous nature of the reports that we receive. This adds complexity and uncertainty to our assessments, and may also cause frustration among charities that we evaluate, especially when we are unable to share specific details about these claims for confidentiality reasons.
How we are addressing this: This year, we have sought to improve the channel for people to submit such reports, linking to the more comprehensive Whistleblower Form codeveloped with Animal Defense Partnership, rather than directly asking about harassment and discrimination in the engagement survey. We hope this helps to ensure that claimants understand the implications of providing such information, improve the comprehensiveness of any such information that we receive, better enable us to follow up with claimants, and better identify the level of detail we are able to share with the leadership of the charity in question. At the same time, we recognize that requiring claimants to fill out a separate, more comprehensive form may reduce the number of reports that we receive.
Staff engagement surveys are inherently limited
Limitation: Our engagement survey only provides a limited window into a charity’s workplace culture and may not fully represent the broad range of experiences within the organization. In particular, we recognize that surveying staff and volunteers can lead to inaccuracies due to selection bias and also may not reflect employees’ true opinions, as respondents are aware that their answers could influence ACE’s evaluation of their employer. We also recognize that our assessment represents a snapshot at a point in time and may not fully capture ongoing cultural shifts within an organization.
How we are addressing this: This year, we have included a wider range of questions in the survey and collaborated with the organizational consultants Scarlet Spark to help ensure these questions are likely to be effective predictors of organizational stability and effectiveness. As in previous years, we do not rely solely on the results of the engagement survey to make our assessment. Rather, we assess organizational health from multiple perspectives to arrive at the most appropriate decision within the time available based on all the information we have, including our follow-up conversations with the charity’s leadership.
Assessment results may be affected by organization size
Limitation: At larger organizations, there are more likely to be instances of harassment, discrimination, and other issues—not just because a greater number of staff members means a greater number of people who can commit or perceive misconduct, but because interpersonal and team dynamics inevitably become more complex at larger organizations. Conversely, smaller organizations may be less likely to have robust policies and processes in place, and their average engagement survey scores can be significantly affected by just one or two dissatisfied staff members.
How we are addressing this: For larger organizations, we do not expect a clean sweep of high scores across the board on their engagement survey. Likewise, if we were to receive a report of potential misconduct at a larger organization, our expectations of the charity’s leadership staff would differ depending on whether this appeared to be a widespread problem, or one restricted to one particular team within the organization. For smaller organizations, we account for the fact that they may simply lack the resources to introduce comprehensive governance processes. We also consider whether low engagement survey scores are due to consistently low scores from all staff, or due to a small number of anomalies.
Our assessment may be subject to cultural biases
Limitation: Our assessment may be biased toward certain Western workplace practices. As a U.S.-based organization with staff based predominantly in the U.S. and western Europe, our understanding of best-practice organizational health is inevitably skewed toward the cultures with which we are most familiar.
How we are addressing this: We seek to recognize this bias at all stages of the assessment and continually learn from the charities we evaluate, rather than imposing a “one-size-fits-all” approach onto each charity’s unique situation. For example, we recognize that not all of the policies and processes that we ask charity leadership about will be common or relevant in all countries and situations. Where there are indications that important policies and processes may be lacking, we follow up with the charity to gain a better understanding of the context. Particularly if the charity is based outside of the U.S., we are also eager to learn of additional policies they may have that they find to be important contributors to their effectiveness. In this way, we hope that this exercise can be mutually informative for ACE and for the charities that we evaluate.
This year, we also modified our engagement survey questions to reduce their focus on western cultures and piloted the questions with charities from different global regions to help ensure this was successful. More broadly, we continually seek to improve our understanding of different contexts, such as by engaging experts in different regions and attending conferences in those regions. We will continue to explore how best to improve the applicability of our assessment across all national contexts, using evidence from the countries where our evaluated charities are based.
Our assessment takes time to complete
Limitation: We are aware of the time demands that our organizational health assessment makes of charities, both leadership’s time in providing us with the information that we need, and staff time in completing the engagement survey.
How we are addressing this: After each year’s evaluations, we conduct a survey with the evaluated charities to gauge their satisfaction with our various assessments and understand how long the process took them. Based on these survey results and our own retrospection of this year’s evaluations, we will seek to strip out any elements of our organizational health assessment that do not appear to be relevant to our decision making.
It is hard to find reliable data on specific elements of organizational health
Limitation: As an evidence-driven organization, we seek to base all of our assessment methods on the best available data. However, reliable, larger-scale evidence on the impact of different aspects of organizations’ leadership and culture on organizations’ actual performance is fairly scarce. Such evidence specific to the nonprofit sector is even scarcer, and evidence specific to the animal advocacy movement is scarcer still. This is even more the case when looking for evidence that is specific to certain regions. There is also a chance that the available evidence is skewed by publication bias; that is, we expect that a study showing that inclusive leadership and workplace diversity typically leads to higher performance is more likely to get published than one showing that strong leadership and workplace diversity has a negligible impact on performance.
How we are addressing this: Despite the limitations mentioned above, based on existing evidence (such as the various studies cited in this blog post), our discussions with external experts, and on our own experience of evaluating charities, we are confident that organizational health is sufficiently relevant to a charity’s likely performance that it should be an important part of our evaluations. We will continue to review the available literature and consult with external experts to ensure our methods are based on the most up-to-date evidence.
Conclusion
We view our organizational health assessment as vital in ensuring that our recommended charities run transparent, sustainable, and compassionate organizations where staff are empowered to work as effectively and sustainably as possible. We are extremely proud to recommend charities that carry out their activities and conduct their operations in a way that is likely to help as many animals as possible over the long term. At the same time, we recognize that our approach has limitations, and we will continue to strive to address these, supported by guidance from external experts and feedback from the charities we evaluate.
To view all of the sources cited in this post, see the reference list.
The post Why We Assess Charities’ Organizational Health appeared first on the Animal Charity Evaluators blog.
Leave a comment
