Document(s) 9 of 13

Over the last 30 years, international development practitioners and researchers have identified the central role that organizations and institutions play in improving the use of development assistance (Eaton,1972; Uphoff, 1972; Savedoff, 1998; Picciotto and Weisner, 1998). As discussed in the last chapter, most if not all development projects have their origin in situations where a particular organization or group of organizations have not been efficiently or effectively carrying out their mandate ( discharging a function or providing a service), or want to improve their ability to discharge their mandate. In other words, they want to perform better.

It is therefore not surprising that most projects, besides providing direct programmatic assistance—hospitals receive funds to improve their “health mandate,” educational institutions their “educational mandate”—also include institutional strengthening components.¹ Presumably designed to modernize or strengthen the organization in question, these components typically involve training managers, purchasing new equipment, updating accounting and financial systems, and implementing structural reforms.

Those suggesting these institutional changes have what Drucker (1995) calls a “theory of the firm.” This is an implicit set of hypotheses or assumptions that characterize either how a firm is operating or how it should operate. For example, a review of 15 development projects for the Inter-American Development Bank dis-

__________

¹ In our review of development projects, support for both organizations and institutions is put under a category called “institutional strengthening.” The more subtle distinction between institution and organization made in this book does not usually occur in development projects.

cerned a pattern in which the accounting system of the agency executing the loan was always updated. The implicit assumption is that such systems will improve the financial controls, reporting and efficiency of both the executing agents and, in some instances, the organization. The assumptions and theories held by organizational members and development practitioners are operationalized through the process of organizational diagnosis, creation of projects or programs, and implementation of the project or program.

In this book, we have put forth a diagnostic framework used to articulate and make more transparent some of our ideas linked to a “theory” of how to improve the performance of organizations. From the perspective of our framework, every development investment is a test of a set of hypotheses about organizational change and performance improvement. Basically, our “theory of the firm” sets forth that organizational performance is a function of the environment within which the organization exists, its capacities, and its motivation. Any planned change to the environment, capacity or motivation of the organization occurs because of an implicit change theory.

If an organization or an investor (development agency) wants to change the performance of an organization—for example, increase the ability of the Ministry of Education to provide basic skills to students—then a diagnosis is undertaken and a series of hypotheses are developed and translated into some action. A project in this context is a deliberate act to improve performance. The ultimate purpose of a project undertaken by an organization is to improve organizational performance in the areas identified.

Over the past decade, there have been many new development innovations at the management, institutional and other levels aimed at improving the performance of development organizations. These include interventions such as total quality management, re-engineering, privatization, decentralization and performance management.

There are assumptions or hypotheses about how organizational change takes place, and in this context, the framework helps describe the rationale and potential logic for future donor investments. As aid continues to be questioned, and as we search for ways to communicate results, the gap between rich and poor countries grows, and global social and health problems spill over national borders. The search for better ways to organize and improve organizational performance becomes more pressing. To make our implicit assumptions and hypotheses more explicit, this chapter speculates about the use of the framework to create change and examines how we can learn to better intervene in organizations. The intent in this final chapter is not to summarize what we have already said, but to offer some observations about the use of the framework. This includes addressing the concerns cited below, all of which affect the diagnosis of an organization and how the diagnosis is used.

1) The concern about ownership is crucial to undertaking an organizational assessment. An important underlying hypothesis of the assessment is that the organization being evaluated is interested in using the results to improve itself. To do this requires paying attention to the issue of ownership. Who owns the results of the organizational assessment? Who is creating the hypotheses for change?

2) Related to ownership is the concern that organizational assessments can become “ceremonial” events to reinforce the status quo. This occurs when organizational members want to avoid the change orientation and transparency that an organizational assessment implies.

3) The concern about the use of “projects” as the primary vehicle to support and change organizations. Projects may distort the organization if they are not carried out within the context of an organizational performance framework—what we refer to as the project “trap.”

4) The concern about the timing of organizational assessments, particularly the need to consider the link between the organization and its “life cycle stage.” This involves the leadership, organizational and economic cycles, since they play key roles in the success of an assessment and in the meaning of the findings.

5) The concern about the link between logic models and organizational assessments. Here, we raise the issue of recognizing the need for dynamic use of the logical framework. Furthermore, we point to the need to recognize that a project logic and performance system might not be the same as one that helps improve organizational performance.

6) Finally, the concern about the application of existing diagnostic frameworks (such as the one presented in this book) and new organizational forms such as membership organizations and inter-organizational groups (networks, consortia, etc.) that may require different types of assessment. Many of these organizational forms have fuzzy boundaries, unclear ownership, and, in whole or in part, may be temporary structures. These characteristics can greatly alter the assessment in terms of the questions asked and the priority given to certain areas.

ORGANIZATIONAL ASSESSMENT AND OWNERSHIP

Organizational assessment is driven by both accountability and learning. From an accountability perspective, it may be required to demonstrate the performance of the organization to a donor, a licensing body, or a boss. This could either be to ensure continuity of a funding or licensing arrangement, or a new level of licensing. It may also be part of assessing a new phase of support. While such an assessment may result in organizational learning and change, that is not the main issue. The main issue is to determine the merit of the organization as part of a decision ( usually external to the organization) about some aspect of the organization’s funding or permission to operate.

Learning and knowledge also drive organizational assessment. Assessment provides a vehicle to better understand how an organization is functioning. While many of the questions in the organizational diagnosis remain the same as in accountability assessment, the intent is to internally use the information to move beyond a picture of the current state and to make operational decisions about how to improve the organization.

Our priority and concern is about the use of assessment for organizational improvement, and we situate ourselves in the “knowledge” more than the “ accountability” areas. We are concerned with issues such as how to make assessments more relevant to building knowledge that contributes to learning and improved organizational performance.

Our experience indicates that it is critical to look at who is defining and conducting an organizational assessment. If the assessment is carried out by a licensing body through a donor or a project implementation unit, it is generally unsuccessful in contributing to improved performance. This is consistent with findings over the last 40 years in development assistance. Those responsible must feel a sense of ownership—a commitment to success.

Ownership is associated with several factors that make up the organizational assessment. Staff members need to have the capacity to benefit from the work of the organization. They need to gain skills, change systems that inhibit successful work, and have an incentive system that supports processes of change. In sum, staff members need the commitment and ownership to stay with the change process. Ownership is important both at the leadership level as well as at the ground level where actions are carried out and decisions taken. One of the lessons learned from our previous work is that the data generated in the assessment needs to be seen as valid both at the top and at the bottom of the organization.

This often presents a paradox in development work. When the results of an external assessment deem that the existing organization is not capable of managing the loan and the related work, a project support or implementation unit is recommended as a mechanism to carry out the project. Project implementation units are established to avoid organizational and institutional shortcomings of the sponsoring organization that could, it is felt, result in delays, cost overruns or outright failure.

Thus a typical implementation unit has greater access to decision makers, such as the minister, and are exempt for normal procurement procedures. They are also able to attract qualified staff with better salary and benefit packages than the sponsoring organization can offer—all factors that speed up project implementation.

While the idea of an implementation unit is appealing from the project point of view, there is abundant literature and experience to support the notion that teams fail when they are not part of the organization in which they are attempting to produce change, or when the organization does not have full ownership of the change process. As stated in a report on project execution by the Inter-American Development Bank (2000, p. 21):

Project implementation units sometimes operate as enclaves in the overall system, and frequently do not assist, and in some cases, may even undermine the ability of executing agencies to subsequently manage project resources during the operational phase (which is generally when benefits materialize). Thus, a project financed by the Bank may have been implemented satisfactorily, but may not be sustainable in the long run, because underlying institutional problems have not been resolved. This is, and should be, a matter of concern to the Bank and to borrowers, since project completion is only a necessary—but not sufficient—condition for fulfilling overall development objectives.

This example suggests that relatively autonomous project implementation units can lack institutional and organizational commitment. They are therefore only suc-

cessful in project terms, not in terms of enhancing the performance of the organization that will have to carry on the work after the project is closed if it is to have a lasting impact. When the organizations that define and contribute to development in a society are circumvented because of their weaknesses, whether perceived or actual, a series of mechanisms are then created that further weaken the central organizations and reduce their ability to effectively participate in the governance process.

How to break this vicious cycle is critical in the ownership discussion. The inclusion of key interests (leadership staff, board, clients, partners) in appropriate ways is essential if these groups are to integrate the lessons from the project into their ongoing development work. In other words, how can the organizational assessment support ownership and commitment to a process of change?

We believe that an assessment aimed at improving performance has to be carried out by key organizational members who have some responsibility for the actions of the organization. When a diagnosis of an organization is conducted, it is important that the people involved in the day-to-day workings of the organization be directly involved in the process. They need to see the assessment of problems as their own diagnosis. Any ideas on “why things work or don’t work” should come from them. This way, they can create a hypothesis as to what is right or wrong in their own words, using their own thoughts and common sense (Weick, 1995).

If people do not own the organizational analysis, be it good or bad, they will not buy into any possible solution. Where some may say that the problem is always one of too little money, others who have taken the time to “own” the situation might be inclined to characterize the problem in a way that might lead to a solution. The problem then becomes theirs, as does the responsibility for helping to solve it.

Ceremonial Assessments

Assessing organizational performance often means that some individuals gain and others lose. It is therefore a sensitive and highly political process in which managers, in particular, but others as well, may open themselves up to criticism and punishment. Those incentives often lead to avoiding open assessment of organizational performance. This can result in a “ceremonial assessment”—the steps are undertaken, but in a very controlled manner, so that data is not released beyond the offices of a few individuals, and the report is carefully worded to keep all criticisms hidden.

An assessment of an organization should be a process of learning for all of the parties involved. An assessment should not be conducted just because someone,

somewhere, says that it should be done. An assessment is a large investment of time, money, resources and, most importantly, people. An organization must be ready both to do the assessment and to accept its results.

There are also instances when it is in the interest of the leader of the organization to keep the analysis of performance fuzzy. The leader controls the dialogue and discourse during the assessment. When queried on certain matters, the response is often “You do not understand ‘our organization’.”

But when organizations are transparent, the power relationship changes. Organizational assessments open up dialogue. They can bring new actors into the organizational power structure and bring about other positive changes. Sometimes organizational members or even project implementing units do not see such transparency as helpful. Change works against their interests. In such a situation, organizational readiness is in question. Those engaged in assessments must pay attention to this pernicious occurrence.

The issue is how to get those in power to participate in the assessment and use its results. The process and the findings have to be carried out in such a manner that there is positive benefit for both the individuals involved and for development of the organization as a whole.

As part of determining how to start the assessment and maintain its momentum, there needs to be careful consideration of the control of the resources necessary for the organization’s operation. This includes examining the resources controlled by management, external forces (donors, legislation, regulation), clients and staff, as well as considering how the assessment will affect each of these groups and how to generate a positive impact for all the interests involved. This does not mean that in every case everyone will be happy, or that there will be no organizational

changes that result in management and staff changes. Rather, it means that it is essential to consider all these factors in the decision to design and implement an assessment to avoid or manage any undermining of the process by those who feel threatened. In some situations, this could mean delaying the assessment.

Investing in Organizational Performance: The Project Trap

Is the project the best way to think about enhancing organizational performance? Are we not at risk of losing sight of the bigger questions: To what extent does the project support or limit the performance of that organization? Are conditions better? Are people more capable in decision making? In creating new societies? In building local development?

In a recent survey of evaluations in South Asia, the International Development Research Centre found a significant lack of emphasis on the organizational capacities of the partners, and a strong emphasis on the results of the projects themselves (Bajaj, 1997). The recipient organizations found this frustrating because the evaluations were primarily useful to talk about the success or failure of the project, rather than how the project supported the mission or performance of the organization. In fact, the project was the focal point of interest, not the learning needs of the implementing organizations.

Project support creates potential problems and paradoxes for organizations. On the one hand, they need funded projects to exist; on the other, the project too often becomes more important than the organization. When projects are the primary focus of action and performance measurement, development organizations lose sight of the more complex performance requirements of the implementing organization. In some cases, the organizations become fragmented and feel they have lost their sense of direction in responding to the requirements of a range of donor partners. They can become trapped by their own success and stand at risk of serious organizational decline.

Investment, then, is defined and measured in terms of individual projects. What our experience tells us, however, is that projects distort when they are carried out without due consideration of the organizational performance framework of the implementing organization. While projects are important to organizations, they must be seen as contributing to overall long-term organizational performance.

A critical future challenge is to find ways to address this issue while respecting

the needs of donors and international financial institutions in terms of accountability. This might include shifting the notion of accountability to include a stronger focus on the sustainability of efforts after the departure of the donor. While most donors already have this view of sustainability, the concept could be operationalized by focusing on investment in organizational performance.

A project is neither an organizational nor an investment model. As organizations struggle to find resources, the project should be seen as an intervention to aid the organization in its performance. Frequently, we have seen how projects upset an organization’s equilibrium. Leaders are drawn into power struggles to try to meet the stated objectives of the project, thus causing disharmony within the organization.

The inclusion of primary project objectives related to the performance of the implementing organization could help to shift the focus of work from the project alone to its impact on the capacity of the organization to perform effectively over the long term. If projects were to incorporate objectives related to organizational performance, then the evaluation of these projects would begin to take this into account as well.

The World Bank has reported that in the past, agencies have too often focused on how much money they disbursed and on narrow physical implementation measures of the “success” of the projects that they financed. It turns out that neither measure tells much about the effectiveness of assistance. The evaluation of development aid should focus instead on the extent to which financial resources have contributed to sound policy environments. It should focus on whether agencies have used their resources to stimulate the policy reforms and institutional changes that lead to better outcomes.

Organizational Life Cycles and Performance Change

Successful implementation of an organizational assessment requires a good understanding of the stage of development within which the organization finds itself. Organizations are quite diverse as social units: they come in many sizes, shapes and variations. Some organizations are old, others young. A young organization in a growth stage needs different types of support than a mature organization that is relatively stable. Similarly, organizations with an uncertain mandate are of a different nature than those whose mandates are clear. Organizational variations play a big role in understanding how to interpret the information from an organizational assessment.

Organizations also emerge in any number of ways and are strongly influenced by their leadership. There are new leaders with a mandate for change; departing leaders who want to influence the future; and even departing leaders who perhaps want to avoid having evidence of performance come to light.

As organizations are constantly evolving, there can be difficulties in creating ways to understand the mix of performance areas. Do young organizations pay more attention to their effectiveness and financial stability? Is this normal? Should this be encouraged in an organizational assessment? Do mature organizations pay more attention to efficiency concerns? Relevance? How does the organizational life cycle affect the organizational assessment process?

The only certainty, unfortunately, would appear to be that there is no certain answer to any of these questions. As Aldrich (1999, p. 1) has written:

I have been disappointed that most research on organizations focuses on structure and stability rather than emergence and change. By ignoring the question of origins, researchers have also avoided the question of why things persist. In contrast, the evolutionary approach to organizations treats origins and persistence as inseparable issues. In doing so, evolutionary models encompass many levels and units of analysis and thus typically take an interdisciplinary perspective.

Like many involved in the assessment process, Aldrich is intrigued by the complexity of organizations. Why do some organizations do well and others constantly fail? How does one identify the cluster of variables that can produce change? Why is it that some organizations resist change? We basically know that organizational

change or stability is inextricably linked to time-dependent historical processes. Since organizational assessments take place at a given moment, it is important to contextualize the assessment. Has leadership just changed or not? Is the economic or social environment in turmoil? Is the organization attempting to renew itself, or engaging in a new mission?

The point is that the hypotheses or assumptions about what affects organizational performance are often mitigated by the organizational life cycle. While life cycle analysis is included in our framework, it is often necessary to rethink the effects of life cycle changes when making conclusions or hypotheses about change, since employees would then have nothing else in their lives.

We know that many events occur simultaneously, rather than sequentially, in an organization’s life. Distinct capacities, motivation and environmental components may be separated out for analytical purposes, but in practice, they are linked in continuous feedback loops and cycles. So an assessment is really a snapshot of the organization at a given moment, using the analytical tools available. In this way, the assessment reflects the historical path of the organization’s accumulated actions to date.

Does this change the organizational assessment? Does it affect how issues are framed? Is it reasonable to assume that there is a process dip? How does this affect the implementation process? Do aid projects exacerbate the situation? These are questions that must constantly be posed when assessing an organization’s performance.

Logic Models and Organizational Assessments

Organizations are goal-oriented systems driven by the actions of many people. Their actions are not random events, but rather are driven by the assumptions held by these individuals. These mental models are known as managerial cognition (Schein, 1997), or perceived organizational culture. They are cultural patterns that translate a world that is often ambiguous and complex into a more understandable and familiar system that fits the needs and expectations of the organization, and in which the organization can take logical decisions.

The organizational assessment model presented in this text is a diagnostic tool aimed at helping development workers better understand the performance of an organization, and assess the various components that might affect that performance in the future. It is a framework that absorbs complexity and provides a way to organize the ambiguous, uncertain world of organizations. It is also a way to get people to learn and think. The diagnosis of an organization should lead to ways to change organizational performance.

Today, logic models as seen in logical frameworks are used to help development agencies and international financial institutions describe the project interventions they will make in organizations. Logic systems help to clarify the performance requirements and the resources needed to affect project performance. Of interest to our work is that many times, the organizational assessment must link its findings to a logic model or logical framework. Can it? After all, projects are short term, while change in organizational performance is long term.

Projects are driven by a logic that is relatively linear: inputs lead to activities, which leads to outputs, which leads to outcomes, which leads to impacts. While this logic is useful for more focused activities, rarely does organizational change occur in this linear pattern. Rather, change in organizational performance is better depicted as a set of interactive or clustered changes that are perceived by organizational members in different ways and in different time dimensions.

Can we link our organizational assessment work with that of the logic models? Our experience indicates both yes and no. On the positive or yes side of the equation, we found it useful to create logical linkages between areas of diagnosed change and our performance model. For example, there is an assumed link between training community health workers in clinics and improving the performance of those clinics. The logic is that improving individual capability affects organizational performance. However, we know that there are many other conditions that must be considered as well.

Most of the conditions for success in terms of capacity and the environment can be identified or subsumed within a logical framework system. However, we often do not have the tools to understand or depict the complexity of dealing with organizational and member motivation. How do we get ownership for change? What are the ways we need to change norms and values? Thus, while we have found the logic systems to be helpful in depicting some of the capacity and environmental aspects that lead to organizational change, we have been less successful in using the logic systems to help us understand organizational motivation and the dynamics of change. This is an area that requires work.

CHANGING ORGANIZATIONAL FORMS

This framework was developed with the standard organization in mind. It is focused on the performance of an organization with the standard attributes of a board of some sort, and a director who is responsible as the leader to take decisions such as hiring staff to carry out functions, etc. The organization has a defined functional purpose. For example, a Ministry of Health has a functional responsibility to make sure that clean water is available.

Increasingly, activities are not carried out by a single organization. More and more organizations are realizing that many of the tasks that need to be carried out require collaboration with other types of organizations, such as networks, consortia and pub-lic-private partnerships. These have characteristics that are different from those of single organizations. The newer organizations are an amalgam of different functional types. Decisions are not taken in one location, but rather are spread according to function, responsibility and need. Therefore, when we try to implement our framework, it may or may not make sense to the organization because of its structure.

While there are many different forms of collaboration, they share some common characteristics. In a single organization, there is a clear domain of operation for the organization. It provides a particular kind of service to a certain group of clients. A collaboration tries to meet a need that is not always clearly defined and that is changing over time. Therefore, it is not always precisely clear what the partnership is doing, nor who within the partnership is doing what. Partnerships are frequently not as structured as organizations; they may or may not have a legal existence. Frequently, they are built around shared interests and business relationships. But whether or not they have a legal existence, they are not clearly owned by one individual or one organization. Ownership is spread across the group that is participat-

ing, and the parts maintain their allegiance so long as they feel a sense of ownership and that the partnership is meeting a perceived need.

While many organizations are set up as permanent entities, partnerships are not always intended to last indefinitely. They can be set up to deal with a very specific problem, and once that problem is dealt with (or changes), the partnership dissolves and new partnerships emerge around new problems.

All of these factors have implications for diagnosis of the performance of the partnership. We are only beginning to explore the use of the framework with these types of organizations. What remains to be explored is whether the differences are primarily in definition of what performance means, or if there are some different elements that are fundamental to the framework to enhance its applicability to the assessment of partnerships.

In the organizations with which we have dealt we know who is a member and who is not. In newer organizations, there are part-timers, volunteers, temporary help and permanent-partial employees, all of whom see themselves as part of the organization. Because these members may have multiple loyalties and multiple boundaries, the boundaries themselves are somewhat fuzzy.

What experience to date tells us is that the factors noted above call for a very different consideration of the structure of performance and its assessment. Because the boundaries are fuzzy, performance assessment that is concerned with efficiency is problematic: so long as the boundaries are not clear, it is hard to determine whose efficiency to assess, and in what terms. The lack of clarity in ownership is combined with the central importance of ownership in sustaining effective partnerships. This means that relevance must be very carefully defined, both from the perspective of the problem (or problematic), and from the perspective of each of the partners.

CONCLUSIONS

Universalia has worked for almost six years with the Inter-American Development Bank and the International Development Research Centre to promote dialogue in order to improve organizational assessment. This book is the latest in that series. It is an update of our original book in 1995, with various portions reframed. The sections on performance and capacity have been rewritten based on our experience during this period, together with other agencies that have graciously shared their experiences and analysis with us.

Other organizations, such the World Conservation Union and the International Service for National Agricultural Research, are attempting to use the framework within their own organizational spheres. Many individuals and organizations have used the framework and have shared their experiences with us. The effort to help organizations improve their performance is ongoing. There are no cut-and-dried answers to the various problems encountered while conducting organizational assessments in less developed countries.

Document(s) 9 of 13