Alan Whaites^{[footnote 1]}
FCDO 2023

Summary

This guide is primarily intended for use by those conducting an institutional review of an organisation. It provides advice to reviewers on working with the counterpart organisation and is intended to be used in conjunction with other analytical tools, including country level institutional analysis and political economy analysis.

The contents will also be useful to those who commission institutional reviews, including development partners, it will inform the requirements, approaches and expectations for such work. It can be used to set out to review teams the methodology that the development partner wants them to use.

The approaches in this guide were developed by UK government staff over a number of years, primarily for use with public sector bodies (for example, government departments). Overall, the aim is for rigour with flexibility, the approach carefully builds layers of evidence and understanding. The approach also accepts that reviews will never go entirely to plan, each institutional review will be unique – the advice offered therefore provides a foundation of principles that can guide adaptation (rather than forming a rigid route-map).

Sections of this guide

Section	Contents
Section 1: Overview	Lays out the overarching questions that a review team should seek to answer, it explores the picture that the team will want to develop: their map of the organisation’s purpose and role.
Section 2: Conducting a review	Provides an overview of the five dimensions to institutional reviews. The description of each builds an understanding of how insights can be gained from a layered approach that allows conclusions to emerge over time.
Section 3: Criteria for analysis	Sets out some factors that can be used to interpret information and data, enabling the team to develop their analysis over the course of a review.
Section 4: Skills, tools and concepts	The make-up of a review team, and some of the concepts that may be used during discussions.
Annexes	Templates of questions to guide teams on the factual information and feedback that should be gathered, and detailed guidance on participatory tools to employ.

Section 1: Overview

1.1 What is an Institutional review and why do we do them?

This guide adopts the definition of institutions used by the cadre of governance experts within the UK’s FCDO:

Institutions are made up of organisations, norms and rules: they provide the systems, rules and processes (formal and informal) that enable or hinder human activity. Institutions are usually driven by actors with power, shaped and given direction by incentives. The impact of these drivers determines the degree to which institutions reflect inclusion, accountability and effectiveness.

This broad definition skirts some ideological divides around norms and organisations. It also recognises that ultimately institutions reflect the demands of those with the responsibility for setting policy agendas and determining priorities (`power’).

The approach below is focused on the organisational dimension of this understanding; however, no organisation can be understood in isolation from the rules (formal and informal) that shape its operation^{[footnote 2]}; Unpacking the complexity of organisations is therefore not as straight forward as determining how they are structured to deliver their roles, instead it requires juggling issues of politics, social dynamics, inclusion, tradition, and the vagaries of whether equipment and power supplies work. For these reasons while Institutional reviews may be bringing an organisation into focus it is the broader term that better fits the approach being used.

The starting point for reviews is often a desire by managers of a government department, or other public sector body (`the organisation’), to assess how it delivers expected functions, and whether/how this can be improved. Reviews may also be prompted by development partners as part of a programme of support. In either case, the primary owner of a review should always be the organisation involved,^{[footnote 3]} with a focus on their objectives.

An institutional review team will want to be clear why management feel that a review is important at this time – and what they hope that it will deliver. A good institutional review will deliver against these objectives, while challenging inaccurate assumptions about why the organisation is performing in a certain way, or what type of capacity development would be useful. It will work with the organisation to develop recommendations on positive change management.

To achieve these aims the review may map processes, systems and workflow, drawing out blockages or gaps. Organisations are, however, microcosms of society, with their own political and social dynamics, their performance is not a simple question of inputs and outputs. Reviews cannot be mechanistic; to properly understand an organisation they will address issues of trust, morale, leadership, and direction, this guide does this through five dimensions of inquiry.

Reviews also involve the need to balance objectivity with the natural subjectivity that can be at work. Institutional reviews are sensitive for those within the body concerned, staff and leaders are stakeholders in the process. There are also temptations for reviewers to see issues through ideas on reform (for example, New Public Management, `deliverology’ or e-governance). The approach below uses the gradual building up of layers of insights to help manage these objective-subjective tensions. Ultimately the recommendations should help an organisation deliver on its mandate or targets by supporting those involved to deliver their best.

1.2 Agreeing what a review will do – terms of reference

The starting point for a review is always to establish a shared understanding between the client (usually the organisation’s management) and the reviewer. Some types of development analysis can often involve a “subject” and “analyst” approach, in which the subject may not even know they are being analysed (for example, economic or political analysis). However institutional analysis only works well where the organisation is fully involved in the process, providing the reviewers with access and support, and developing shared conclusions. This can be easier where review teams include members of staff from the organisation, seconded into the team for the duration of the process.

The joint understanding of the aims and approach of an institutional review is normally captured in a terms of reference (ToRs) document. This defines the purpose, scope and methodology, and states who is responsible for follow-up on any recommendations. The review team might also encourage the partner to form a ‘focus group,’ composed of both staff and any external stakeholders. The feedback of this group can help to produce a review that is feasible and affordable. In all these steps the review team should take an inclusive approach (for example on gender, grade levels) that ensure it will hear voices from across the organisation.

The text of a terms of reference should set out the shared understanding of the purpose and objectives, clearly framing the questions that the organisation’s management want answered. They should then set out how the review will explore the layers of complexity involved. Detailing the dimensions to be considered:

1. Outside the institution – the country context, stakeholders and institutional networks, including related external organisations.

2. How the institution works – leadership, strategy, delivery, informal and formal norms and internal relationships. This includes the `effectiveness’ of delivery, using an understanding that goes beyond efficiency and cost to include a normative and qualitative lens. It also includes the resources available to the organisation.

3. Outputs, what the institution achieves and how, including impact, outcome, and inputs and the results that are experienced by the wider community.

4. Comparable experience (locally or internationally).

5. Analysis, action plans and recommendations.

When framing the ToRs it is important not to start with an inherently negative view, even where management want to focus on perceived problems (the review may find that given all the factors involved the organisation is already performing well). Preparatory discussions on the ToRs will help the review team to develop an initial `map’ of the partner, particularly by exploring a) what the units and staff want the review to achieve, and b) how those units and staff fit into the delivery and purpose of the organisation.

Terms of reference should also tease out how language is understood, Box 1 outlines how we understand the main technical terms. It is important to bear in mind that these same terms can have a specific meaning within an organisational culture. For example, the term `capacity’ may be a shorthand for staffing numbers. Reviewers need to be aware of the potential sensitivity involved in language and the need to avoid an overly dogmatic approach to terminology.

The review team will want to consider their `criteria for assessment’ at the beginning of the process. These issues are discussion in Section 3, including key concepts such as form following function, political drivers and human agency.

Discussions on the terms of reference should include risks. This includes risks to those conducting the work, and to wider stakeholders, these can be varied for example, reputational, financial or related to the personal security of those involved). There should be agreed approaches to mitigate and manage risks, with appropriate duty of care for those involved, and clarity on who is responsible for risk management. The terms of reference should also include clarity on appropriate standards for research ethics (see below).

Finally, while developing the terms of reference a duty falls on reviewers to encourage their counterparts to be `informed clients,’ helping them to consider the different approaches that might be taken. Clients should be aware of the types of issue that the review team is trying to understand and how these inform the eventual conclusions.

There are websites that offer useful guides for the `consumers’ of institutional reviews and the Learning Network on Capacity Development (LenCD) offers a quick overview of issues. The Governance and Social Development Resource Centre (GSDRC) discusses a range of aid based tools . Finally the United Nations University has also undertaken a review of tools aimed to analysis risks and resilience in FCAS (PDF, 411KB).^{[footnote 4]}

1.3 Institutional reviews are a people thing

Reviewers need to spend time with partners to understand their needs and expectations; ideally therefore an institutional review should not be a purely desk-based exercise.

The counterpart organisation will want to be sure that reviewers understand their perspective and concerns – for example on issues such as confidentiality and impartiality. A review team should discuss mutual expectations in depth with their counterparts (preferably with the whole leadership team).

This process will help the review team to identify the approaches for their work, recognising that any institutional assessment can be viewed with scepticism by stakeholders. This is particularly true if staff see the review as an assessment which the institution will pass or fail, or if they feel the review will bring change that may impact upon them or lead to a period of uncertainty.

Reviewers need to reach beyond any scepticism, building a collaborative approach with those inside the organisation. Institutional reviews therefore require good partnerships, sensitivity to the concerns of counterparts in the organisation, and a consciousness that reviewers are guests and any access they have within the institution places them in a very privileged position.^{[footnote 5]}

Given the inherent complexity and very personal dynamics it is particularly important that the review ensures the confidentiality of information shared by staff, and adopt appropriate research ethics in the handling of both information, and the context of interviewees (for examples see the Government Social Research Code – People and Products and GSR Ethical Assurance for Social and Behavioural Research).

Section 2: Conducting a review

The approach below owes a debt to the UK government methodology for capability reviews, and also a model developed in 2005 by EuropeAid. These have been adapted, updated and tested so that they can be used in any context, whether low, middle or high income, including states affected by conflict and fragility. The approach recognises that assumptions valid in one place may not be true for another (for example the reliability of utilities, financing and policy direction can vary). A process of discovery is used to tease out the layers of understanding that are needed to appreciate the unique universe of an individual organisation.

The five dimensions of discovery and analysis are not inherently linear (overlap is inevitable), they simply provide a structure to help build a picture of capacity, delivery and potential change. Students of institutional analysis will recognise that the approach shares a common heritage (along with many other institutional assessment tools) in the Burke-Litwin model developed originally in 1992. An important aim of that model has been to understand: the variables that need to be considered in any attempt to predict and explain the total behaviour output of an organisation, the most important interactions between these variables, and how they affect change.

The five dimensions will profile these variables and their interactions from different angles or perspectives – this helps to ensure `triangulation,’ the use of different sources to cross-check findings. Triangulation is an important tool in helping reviewers to overcome bias and subjectivity in analysis (whether their own or that of stakeholders).

The 5 dimensions of a review

The 5 dimensions below allows the review team to develop a set of initial `hypotheses’ concerning the impact of context, the quality of outputs and the nature of internal effectiveness. These can then be tested with counterparts before final conclusions are reached:

• focusing outside the institution – the country context, stakeholders and institutional networks
• understanding the institution – leadership, strategy, delivery, informal and formal norms and internal relationships. Including the resources available to the institution (for example, finance, staffing)
• focusing on outputs, what the institution achieves and how, including impact, outcome, and inputs
• considering comparable experience (locally or internationally)
• analysis, action plan and recommendations

2.1 Focusing outside the institution: the country context and institutional networks

The first dimension is to consider the context in which the counterpart organisation is operating, for example political economy analysis (PEA) may be available that sets out how tactics for managing power impact organisations. Such tactics could relate to the explicit or implicit functions of organisations; for example, providing employment for supporters of a ruling party, or offering a gesture on an otherwise neglected issue. PEA can show that organisations play numerous simultaneous roles for those who are managing power, potentially with a mix of positive and negative results.

PEA and other context analysis will identify both long and short-term trends which may impact organisational performance, including external pressures, social change, regional and gender dynamics and history. Many of these issues are viewed as `structural,’ which means that they are factors that have a broad time-horizon and can change slowly. External considerations will also include patterns of financial resourcing (whether revenue problems lead to frequent cash-rationing) and infrastructure issues (is electricity reliable? is there access to the internet and communication?).

This externally focused element of the review will help the team to develop a picture of the institutional universe of systems and relationships. For example, the institution will receive inputs and resources, or work tasks for which it is only one part of a wider process). These related external organisations will include the key bodies within the same part of the public sector, for example a public prosecutor’s office must working closely with the police, prison and court services. Mapping who these institutions are and speaking with them will ensure that the organisation is not considered in isolation. Relationships between organisations are likely to be complex, potentially with overlapping or unclear mandates, or functional gaps within the system (for example, the lack of an effective centre of government body to co-ordinate and guide).

The team’s picture of the institutional universe needs to include prevailing `cultures’ and norms. For example, there may be a tradition of rivalry and poor collaboration between Ministries. These broader norms will shape how staff behave and how formal rules/procedures are interpreted. The relationships and norms between organisations will also influence the messaging that is provided to the team, reinforcing the need for effective triangulation (see below).

2.2 Understanding the institution – leadership, strategy and delivery

The second dimension of review is to develop a more comprehensive picture of how the organisation operates, including its ability to change in both the short and long term. This part of the process will answer many of the basic questions about the organisation, such as how big it is, how is it funded and its governance/accountability systems.

The approach here is informed by the ‘Capability Review’ model developed by the UK Cabinet Office to assess ministries and government departments. The model provides a three-fold way of looking at organisations based around: a) leadership, management and team-working; b) strategy and c) resourcing. These three elements help to develop an understanding of how soft and hard issues interact with each other (for example, collaborative work-styles and budgets).

The table in Annex A outlines how an institutional review might explore these issues. The table offers questions on leadership/teamwork which will help the reviewers to establish how people relate to each other, how they are led and whether the organisation uses its capabilities effectively. The table in Annex A will also help the review team to gauge whether an organisation has clarity of purpose, and whether this is shared by all those involved. Finally, the team will need to explore issues of resourcing in detail – this includes the staffing of the organisation, financing and any other inputs that are required. How the organisation accounts for its resources, allocates them, and assesses budget requirements are all important issues to consider. This part of the process will further enable the team to develop a `map’ of the organisation – explaining its size, structures and operations.

The team will have the option to use analytical tools such as surveys. An important resource for understanding the potential of survey and other data in public sector bodies is the The Government Analytics Handbook: Leveraging Data to Strengthen Public Administration, by Daniel Rogger and Christian Schuster.

Combining structured questions with informal discussion will help the reviewers to tease out the difference between plans and formal processes/rules, and real everyday practice. Tools exist to help ask these questions and draw out the relevant issues (which are often sensitive); these tools normally involve a mix of participatory workshops and one-to-one interviews. SWOT analysis is a simple but effective way to prompt discussion with staff from partner organisations.

An essential element to consider is gender, including the subtle barriers that may exist to recruitment, promotion and effective working, some of which may hinge on how women are barred from effective working relationships (for example, opportunities to access information, or to share views). Considering gender issues means looking beyond numbers to issues of norms and power. It is also important to gauge how representative an organisation is of its wider society and particularly how it reflects exclusion, marginalisation and any underlying divisions within the wider context.

2.3 Focusing on what the institution achieves and how: inputs, outputs, outcomes and impact

The third element is to look at what the counterpart is expected to achieve and what it actually produces – what is the purpose of the organisation, and does it deliver? This part of the process represents an important technical challenge for the review team and can often raise some of the most sensitive issues for counterparts.

Identifying the most important outputs – Institutional reviews often take a ‘form to function’ approach (see section 3), and this entails firstly establishing what the customers of the partner want it to deliver. Every institution has `customers’ including, its citizens/public, the political leaders who set policy (for example, the Minister), and other government bodies. An organisation is expected to deliver impacts that meet customer expectations through outputs (for example, the operationalisation of a new policy to combat major social problems).

A review should always seek to find out how customers of the institution feel about its outputs. It should also determine which are the highest priority, this might be done by considering the mandate of the body (for example, does it have key roles set out in a constitution – such as provision of justice). In addition, the review team can ask leaders and other customers which of their products matter most. Public and political perceptions of priorities, and organisation performance may also be available from wider commentary (such as media coverage).

Assessing the quality of outputs – Once the highest priority outputs are identified the review team will need to evaluate their quality; is the organisation able to achieve what its customers want? This means identifying a quality standard for comparison. These points of comparison are `benchmarks,’ and should be based on reasonable expectations (measuring against the ideal is usually unhelpful).

In many cases the quality benchmarks will be self-evident from the organisation’s own targets, such as delivering universal literacy or a more skilled workforce. The organisation may already have in place feedback mechanisms for its customers, such as citizens report cards. However, sometimes quality is less clearcut, for example is a Prime Minister’s Office delivering good policy advice on the merits of regulatory reform? A review team may need to develop a practical quality standard, explaining what `good’ might look like for this output, and this outline will draw on practice for similar organisations and any wider learning.

For example, if the organisation’s main outputs are policies, then the team will outline criteria for gauging whether policies have been well crafted and are likely to have impact (such as, evidenced, costed, implementable, relevant to the strategy, clear, monitored ). This outline can then be used to test existing products to see if they meet the quality standard and to describe how close or far, they are away from it.

The review team’s judgement of quality then becomes a baseline – a future assessment will be able to judge whether the ability to deliver has improved, stalled or declined in comparison to the earlier review.

Identifying why outputs are like this – Delivering outputs is not the whole story, the review team will also want to gauge how successfully those outputs are delivered. This requires delivery that is both `cost efficient’ but also consistent with policy and with sustainable approaches that respect the rights of those involved. Ways of working are therefore important to understanding how successful an organisation has become, including its treatment of staff and internal norms (see section 3 below).

A key question is whether the assessed quality level is the result of the organisation’s ways of working or is it the victim/beneficiary of forces beyond its control? A helpful technique is `process mapping’ working with staff to visualise the chain of actions producing an output. Once the formal process is defined staff will then be asked to talk through issues around the inputs that they have received (for example, information from other actors) and also their experiences at the main decision/approval points. This approach will highlight the major challenges that occur and how problems are mitigated and managed.

2.4 Considering comparable experience

The fourth element of review is to understand wider experience in the organisation’s sector, including any specific case material. Reviewers should avoid the `best practice’ approach of dropping templates from elsewhere into what may be very different contexts. Nevertheless, the approach outlined here seeks to find locally-defined solutions that can draw on, not necessarily copy, wider experience.

Additionally, the comparative part of the review should work to reduce risks of predetermined assumptions about the nature of problems or solutions. Institutional change is complex, usually evolutionary, and often the product of internal debates and even contestation. As such, external ideas and models can best be seen as a spur to creativity (sometimes seen as ‘hybridisation’) rather than as a simple ‘fix’. This ‘best-fit’ rather than ‘best-practice’ approach is central to effective partnership, taking into account the findings of the five dimensions approach.

2.5 Action plans and recommendations

Once a team has consolidated their initial thinking this should be tested with stakeholders, including representatives of the organisation and any external sponsor. This testing calls on the review team to be open to challenge and encourages counterparts to offer further insights. For the review team it is important to remember that issues that look straightforward on paper can be complex in practice.

Once the analysis has been through a period of testing those involved in the review will want to set out potential solutions. Recommendations should always emerge from a collective understanding of the problem, and a sense that next steps are mutually developed, practical and achievable. A team should think of their final product not as a document, but rather as the level of ownership of the conclusions by the partner organisation. Advice on solutions should also be clear on what changes lie within the power of the organisation, and those that require wider co-operation or action (for example, legislative change).

Ideas on implementation can be centred on an `action plan,’ setting out broad approaches, rather than a detailed set of reforms. This leaves space for the organisation to work through resourcing and other implications. Developing recommendations will normally involve discussions that aim for:

• validation or improvement of the initial conclusions
• co-development of a draft action plan;
• leadership review of the action plan
• finalisation of the action plan

Section 3: Criteria for analysis

Section 2 has set out the detail of how to conduct a review, unpacking the different layers that gradually provide a rounded picture of an organisation, its functions, and where positive changes might be made. The approach recognises that organisations are shaped by factors that range from the physical (equipment), through to the norms and traditions of organisational culture. By considering five dimensions of inquiry reviewers can unearth a considerable volume of information, opinions and data. Turning this material into a report calls on their analytical skills, and this section sets out some guiding questions that should inform how the review’s findings can be assessed for the final report and recommendations.

3.1 How do people make sense of their organisation’s complexity?

The FCDO governance cadre definition of institutions recognises that for those within an organisation their `institutional’ universe combines seamlessly structures, norms, rules and the vagaries of human agency in a way that is intuitive and can change relatively quickly (a new manager can alter long-standing practice within days). For example, to an employee the failure of equipment may epitomise problems with organisational culture, rather than being seen simply as a structural or process issue.

As a result, we need to consider how counterparts see structures, norms and agency working collectively as a single dynamic (see Box 8), recognising that these factors are mutually influential and subject to change. This will include their understanding of organisational politics, and where power lies across the organisation and how it used. In our example of failed equipment, the employee may perceive a lack of enforcement of rules, budget problems, or an inability for bureaucratic processes to serve organisational needs. These failures may be consistent or contradictory to prevailing norms, and those norms will change (susceptible to influences such as demographic and educational change, wider cultural shifts and the potential for challenge by individuals unwilling to conform).

3.2 What is real the relationship between function, form and people?

Institutional reviews rests on an idea that should be remembered throughout the process. Form really should follow function – organisations are not best structured on the basis of who wants to do what job, or how many rooms are available. Rather organisations need clarity about what they are trying to achieve and what products would deliver those results.

This apparently simple chain of thinking is complex in practice; establishing systems to deliver on expectations can be a process of trial and error, guesswork or of incremental adjustments to traditional approaches designed for the needs of an entirely different age. An institutional review can provide a zero-sum view of whether mandates and expectations are being met by those systems, and whether they make best use of the most important resource involved – staff.

3.3 How does personal agency shape capability and impact?

In recent years behavioural economics has added to our understanding of human agency, including the fact that individuals with similar backgrounds, in similar circumstances, facing the same pressures and incentives, may make very different choices. This has led to a helpful increase in discussion of ‘actor based/focused change’, and in the attention given to behaviour within wider processes. Human agency operates both at a top level, through the preference and strategies of those with power, and at a core level, as the expression of the approach to work of all the stakeholders involved (including aspirations, beliefs and tactics). It will reflect their morale, career goals, biases, faults, strengths, and how these factors all come together collectively through ‘team spirit’.

Reviews must remember that changes of structures, systems and rules all boil down to changes in what people do, and how they do it. Those people experience institutions very differently depending on a number of factors, including grade, gender, personal background and relationships. The issues are also organisationally political, complex, and have competing influences, aspirations and concerns.

3.4 Politics and priorities

The approach set out above includes inquiry into how the political context shapes the ability of an organisation to operate. Within the same political environment institutions can vary greatly. In some cases, politics can actively undermine the performance of institutions, by using these as a tool for reciprocal politics (for example, using public sector employment to reward supporters), or it can promote them as essential to the aspirations of those with power. In many environments politics will not be hostile to institutions but neither will it be conducive, this may allow space for entrepreneurial organisational leaders, or lead to collective action problems. An institutional review

therefore needs to be politically informed, while also avoiding outright duplication of a political-economy analysis (see this guide to PEA).

Reviewers will also need to assess the degree to which institutions can shape the approach of politics in the context. Does space exist for the organisation to broaden the horizon of leaders on what can be done/achieved? or to influence their views? the relationship between political power holders and the institutional framework is not a one-way street. Reviewers should also be familiar with the OECD’s framework for drivers of trust in public governance), offering an overview of how organisations are perceived.

3.5 Is capability matched by success?

Graham Teskey provided a guide (Box 1) to how we understand capacity and capability and we can establish benchmarks for both. In addition, a review can gauge an organisation ability to change. A review will, however, need to avoid a simple assumption that the capability to do X or Y automatically equals success.

We can sometimes simplify `success’ to being completion of a set of tasks. Success should also have two dimensions of quality. The first being how well things have been done and secondly were they done in appropriate ways. For example, an organisation may deliver good quality products that match its mandate but still be riven by deep internal abuses, or might act in ways that are contrary to the welfare of citizens, including through gender discrimination.

A review should consider both dimensions of quality in order to determine success. An effective organisation is not simply one that `gets the job done,’ but also one that gets the job done in ways that achieve the bigger picture intended impacts, does so at a justifiable cost, sustainably and with due regard to appropriate ways of working (for example, inclusion, gender equity and transparency).

3.6 Does the organisation deliver value for money?

Value for money studies can be used within institutional reviews, drawing on methodologies from performance audits. However, these are sometimes critiqued as lacking appreciation of externalities, organisational culture, and the realities of how capability impacts delivery. An institutional review should deliver answers to many of the questions that prompt performance audits, while also going beyond these. Value for money principles are useful to this process, helping to frame questions that support the review as a whole. For VfM principles an important factor is that Value for Money is not simply looking for the lowest cost way to deliver x or y, but also a process of identifying how to deliver the best possible results from the resources that are being deployed. FCDO considers value for money to comprise 5 Es:

Economy: are we buying inputs of the appropriate quality at the right price?

Efficiency: how well are we converting inputs into outputs? (‘Spending well’)

Effectiveness: how well are the outputs of an intervention having the intended effect? (‘Spending wisely’)

Equity: how fairly are the benefits distributed, particularly among marginalised groups? (“spending fairly”)

Cost-Effectiveness: What is the intervention’s ultimate impact on objectives, relative to the inputs that we invest in it?

4.1 Engaging counterparts

A review team should bring together individuals with a breadth of skills and expertise and this may include previous knowledge of the context, expertise in Organisational Development (OD), change management, human resources management and public sector reform. The team may include members of staff of the organisation being reviewed, who are effectively seconded into the group for the period of the review. The team will need to commit substantive time to a conduct an in-depth review, spending several weeks working directly with the counterpart; if the team is working transnationally then this may be in two or three instalments.

The team will also need to assess the risks involved in undertaking the analysis (see section 1.2 above), for example could it create reputational risks for either the organisation or any supporting development partner? Are there physical or security risks for those involved? An open and frank discussion on risks should take place with stakeholders, with clarity on mitigation, management and responsibilities.

The team will need to use inquiry skills that depend on facilitating rather than leading discussion and an ability to triangulate information to gradually build up layers of understanding. Some of these skills have been captured in the literature on context analysis and a number of tools/approaches are described in FCDO’s guide to political economy analysis, (PDF, 671KB) and also the UK’s Beginner’s Guide to PEA (PDF, 708KB). In addition, Pablo Yanguas has outlined a `one hour’ conversational approach to inquiry and analysis that can be adapted for a range of purposes (see: ESID Briefing Paper No 12 (PDF,132KB)). An element of good inquiry through semi-structured interviews is consistency in the exploration of issues and themes – allowing for a picture based on multiple perspectives and insights to emerge.

Throughout the review the team will need to find an effective way of working together on `knowledge capture.’ Quickly writing up rough summaries of interviews will avoid losing important nuances while they are still fresh in the mind.

4.2 Useful concepts

The following are some key concepts that often surface during the conduct of a review:

Agency: the term used for the ability of individuals to impact the world around them, political agency is the capacity for actions that can impact the political sphere. The many variables that arise from an individual’s peculiar personality traits (knowledge, judgement and instincts), are all bound up in their agency.

Collective action problem: a group of individuals or organisations could all benefit from a reform, but the task of gaining alignment of the actors is too complex, particularly where some actors prefer to stand-by and let others press for change (potentially letting others rock the boat, or use up their political capital).

Confirmation bias: we tend to gravitate towards evidence and views that reinforce our own reading of a situation or problem, giving less credence to those elements that might challenge our reading. This helps to explain the importance of triangulation, and of questioning all the assumptions and perceptions that are offered.

Incentives: the factors that encourage and motivate people to act in a certain way go well beyond financial rewards, or even personal interests. For example, a strongly held belief might lead somebody to promote actions that harm their own financial position. Incentives can take multiple forms (as can disincentives) and where people have conflicting incentives, they will often `trade-off’ the gains/losses based on a prioritisation that may not be immediately obvious to others.

Norms/rules and values: people operate according to numerous sets of rules, written and unwritten, these include social norms (what constitutes acceptable behaviour). Traditionally, development workers talked about ‘informal’ and ‘formal’ rules, this distinction can be meaningless for counterparts for whom nearly all rules can be perceived as ‘formal’, because they are well understood and generally accepted. The greater issue is what rules are in operation, how are these understood, and what systems exists to mediate and navigate them.

Political institutionalism: those conducting a review should be aware of ideas on how politics can become more supportive of institutions. For example, Lant Pritchett has talked of a journey from ‘deals to rules’ in which early political deal making gradually solidifies understanding of how things should be done. Others have pointed to `transformational leaders’ with a vision to pursue a particular model of development. For a discussion of how leaders may view, and adjust, their approach to institutions see the World Bank’s 2017 World Development Report.

Reform/political space: a study by the World Bank of leaders who had led reform processes found that they need three ‘As’ in order to get things done (acceptance, authority, ability), but often they struggle to secure these factors in a consistent way. Others have reflected on the fact that leaders may not have a `reform horizon,’ they don’t know what reform is possible or how it has been delivered elsewhere. These problems and challenges are often cited as a lack of political or reform space – the room for change. Operating on the basis of the familiar (default tactics) may seem safer and more practical.

Rent-seeking: rents are the benefits that accrue from the control of resources, and rent seeking usually means pursuing those benefits by virtue of power (for example, corruption), rather than productivity. There are whole theories that are focused on rent-seeking. For reviews rent-seeking is both a useful concept in understanding power, and also a potential risk as a catch-all answer. Equally, rent-seeking can vary greatly across systems and groups, and generalisations are normally best avoided.

Social networks: the relationship between individuals shapes perceptions and decisions, however the nature of connections between people differ markedly based on cultural factors. Relational bonds can be forged (or broken) based not only on family or friendship ties, but also based on shared experience and interests. For example, in some contexts being in the same entry year for university or the military is important. These factors go beyond providing a form of loyalty between individuals, they can also make it easier to influence through insider knowledge of what might shape the `agency’ of key actors.

Triangulation: a finding should be confirmed by different sources (where possible from different perspective). Triangulation should ideally go beyond hearing the same thing from different people (urban myths can be triangulated if you ask people who use the same sources). If what is being said by interlocutors fits with the story from data, or from external perspectives, then triangulation is stronger. The team should avoid being influenced by one single powerful informant or document.

Conclusion

The process of conducting an institutional review will never go entirely to plan – for this reason the key principles and underlying approaches matter far more than a detailed route-map (which will never quite fit the context). However, it is important for review teams to aim for rigour in their process of discovery, gaining an informed picture of how the organisation works and offering evidence for their conclusions and recommendations.

Once a team has a clear sense of the five dimensions above, then it has the ingredients that support adaptation and improvisation. This process of adaption is easier when the relationships involved are strong; transparency and good communication are the common ingredients whenever reviews go well. Ultimately each review is for the counterpart – who will need to own the recommendations if any useful change is to emerge.

Annexes

Annex A: Questions for discovery

Some review models use a scoring system linked to their assessment criteria, for example, this is included within EuropeAid’s guide to Institutional Assessment and Capacity Development: What, Why and How? (EuropeAid, 2005, see also the resources available at their institutional assessment page). In addition, 3 tools are included within the follow-up booklet: `The Toolkit for Capacity Development’ (EuropeAId, 2010 copyright European Communities 2011). The 2010 toolkit introduces these tools by outlining a pared down 4 box model of factors that shape capability.

The 3 EU scoring tools are a helpful planning tool whether scoring is used or not. The use of recommended questions provides a guide to teasing out elements of a complex picture of relationships, motivations, organisational politics, systems, and structures, as a result using them as a simplistic tick-box approach could be counter-productive.

The table below is an adapted amalgam of questions based the UK capability review model, the EuropeAid tool and the experience of FCDO review teams. The issues are arranged based on the 5 dimensions of review. Similar checklists can be found in other methodologies (see the suggested reading below).

Questions/issues that inform the first four dimensions of discovery

Focusing outside the organisation – consider the country context, stakeholders and institutional networks

• are there comparable reviews of other organisations to provide a sense of context, including gender analysis?
• is there context analysis that speaks to issues of institutional norms, bargaining processes?
• are there perception studies of public attitudes to state bodies?
• predictable budgets, budget execution (Public Expenditure and Financial Accountability (PEFA) or similar assessments.
• PEA analysis (national, regional, sectoral, institutional), this might include short-term tools (actor or network mapping) or longer-term drivers of change/power analysis type tools (see FCDO’s guide to PEA (PDF, 132KB))
• employment audits, staffing numbers, appraisal and performance management systems within the wider civil service
• infrastructural issues (including access to utilities, communications and any issues with physical locations)
• is it possible to define a stakeholder map of those outside the organisation, including the nature and directness of their involvement?
• what is the basis for collaboration, communication and networking (for example, kinship or school groups)? Any distinct gender dynamics?
• legal frameworks, are there clear legal powers, constraints and mandates?
• legal oversight mechanisms (for example, Parliament and audit), how effective are these?

Focusing on outputs, what the organisation achieves and how, including inputs, outcomes and impacts.

• organisational mandate, legal or regulatory responsibilities
• formal stated objectives, targets, national development plans, public service commitments
• priorities outlined by policy leaders (for example, Ministers) and the outputs they believe will deliver on these
• how satisfied are key clients, particularly the policy leader? Is feedback on outputs shared?
• staff conceptualisation of outputs – what do teams believe that they are supposed to deliver? Are formal or informal priorities preferred? perceived realism and clarity of targets and trends in output and delivery
• priorities of users/clients of the organisation. Who uses the outputs, or has an interest in alternative outputs? Stakeholder view of outputs, delivery and performance, including a gender disaggregated view
• how are priorities communicated and what is the level of focus on these?
• is there a tension between government priorities and donor pressure (where relevant), is external involvement viewed as helping or hindering delivery?
• availability and quality of data on a) quantifiable outputs, b) consumer satisfaction and c) past and present trends
• degree of co-ordination, consultation and monitoring of outputs/delivery -is there a good flow of information?
• do clear `goal-owners’ exists for outputs/priorities/targets?

Understanding how the organisation works – leadership, strategy, delivery, informal and formal norms and internal relationships.

Planning and strategy:

• are organograms, departmental plans, budgets, available?
• how does the allocation of resources relate to priorities, outputs and any cost analysis of the resourcing necessary for each area?
• is tasking prioritised, clear and tracked? What is the quality of systems that prevent inequity in work distribution, overload or things falling through gaps?
• are there any gender variations in the allocation of resources?
• how predictable are resources, including levels of staffing; how have budgets changed over time, is there flexibility?
• supply side issues (availability of staff, raw materials, equipment, space)
• performance assessment mechanisms and monitoring for staff; including credible threat of sanctions for wrong-doing. How does feedback, quality assurance, and peer review happen?
• what formal and informal incentives exist and how are these organised (for example, around quality of outputs, or loyalty)
• do people receive the training that they need (including induction) – are skills developed to the level needed for high quality outputs?
• do people have the basic equipment that they need to operate, including IT?
• adaptability of processes, can systems and procedures flex in line with business needs, and delivery of outputs?
• can people access the inputs they need (for example, information, evidence )?

Leadership and teamwork:

• how does the policy owner, and political accountability mechanisms, view or characterise the organisation?
• effectiveness of practices in delegating responsibility (any gender distinctives?)
• are staff able to communicate with their stakeholders – can they chase inputs and consult over outputs? How are factors that restrict capacity dealt with?
• predictability of management systems and relative amounts of time/effort taken to secure approval
• checks and balance on the use of power within the organisation
• how do staff describe their own morale and that of the organisation as a whole, what makes them proud (or not) in working here?
• actual criteria for hiring, firing and promotion; including by grade, how is gender inclusion considered in this process? How do staff perceive these issues?
• how effective is internal communication, trust and collaboration across units? Do people know what tasks others are working on?
• do staff feel safe and free from harassment (including on the basis of gender)?
• on what basis do groups normally associate (including social, ethnic, religious, regional and gender distinctives)?
• what is the balance between formality and informality of leadership and management functions?

Strategy:

• does an internal change process exist (for example, pay and grade) and how is this perceived in terms of drivers, impacts and motives?
• what has the organisation learnt from previous reforms, what worked well?
• what are the internal assessments of strategic strengths and weakness?
• how are plans developed? What is the role of internal dialogue? What external reference points are used, how does the organisation determine solutions to problems and challenging tasks?
• how were procedures and systems developed and what relationship exists to the strategic planning process?
• if more than one strategy and/or set of goals exists what are the reference points for overall vision (for example, national plan, programme business case)?
• what accountability systems exist in relation to strategic direction (for example, Parliamentary committees)? And how do they play their role?

Considering comparable experience (locally or internationally):

• do plans/priorities for future capacity development and institutional reform already draw on any international models, peer experience or similar?
• do case studies exist of closely related/similar organisations?
• have the issues identified been the focus for any innovative, iterative or adaptive change processes elsewhere?

Annex B: Participatory tools

These tools/methods can be used to explore issues outlined in Annex A. In addition to participatory workshops a review team should also consider using complementary semi-structured interviews with a cross-section of stakeholders, an outline of an interview plan is included as Annex C. These tools should be used flexibly with workshop leaders adapting approaches to suit both context and the groups of participants involved. The questions and methods set out under each tool are based on covering the key areas of the five dimensions approach

Exploratory questions

These can be lists written on paper, surveys, or questions discussed in pairs, however they are often best used by picking 2 to 3 questions to be answered during the initial introductions at the start of a workshop.

How long have you been with the department?

How many different roles have you had?

How many people have left your team, and joined your team in the last year?

What are you most proud of doing at work in the last year?

What priorities have been set for your team?

SWOT analysis

Can be done in plenary or with participants breaking into groups of 2-4 to come up with lists to feed into the overall SWOT box.

SWOT analysis
Strengths	Weaknesses
Opportunities	Threats

Strengths: describe the positive capacities internal to the partner, what gives it its edge. These are things than can be done, and done well, within the partner organisation’s control.

Weaknesses: Issues within the partner that detract from its capacity to deliver, they reduce effectiveness.

Opportunities: Internal or external potential to achieve key goals, and to improve capacity and effectiveness.

Threats/constraints: Factors that call into question continued effectiveness or the realisation of opportunities, these can be internal or external and can be very big issues that are hard to change, or more micro issues (whether the wifi works!).

There are many good introductions to SWOT, a balanced view which looks at the tool through a business lens can be found at:

https://www.strategicmanagementinsight.com/tools/swot-analysis-how-to-do-it.html the UK Charity Commission has also done definitions of SWOT based on non-profit work: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/550689/Tool_2.pdf (PDF, 123KB)

SWOT questions (examples)

What are the strengths, weaknesses, opportunities and threats in relation to:

Outputs:

• the work we deliver each week
• our ability to deliver what our stakeholders want

How the organisation works:

a) Delivery

• the resources that we have to do our work with
• the incentive system
• how we evaluate and assess our work

b) Leadership and teamwork

• how we are guided and instructed
• the match between tasks/expectations and ways of working
• how we communicate and work together

c) Strategy

• how we develop plans and work out priorities
• what we inherited when we came to the department (legacy issues)
• the department as a whole

Process mapping

• work in small groups of 4 to 5 and ask each group to agree the most important/high demand output from their organisational unit
• give each group a sheet of flipchart paper and ask them to mark the final output on the right-hand side (left to right writing systems), the group should then draw a line backwards to the start of the process (when the work is commissioned), noting major decision or action points
• once the core process line is finished ask the group to add (using a different colour or post-it notes) the major challenges or blockages that occur
• the group should also mark in the points at which stakeholders are most involved (and who these are.
• finally add (a new colour) any approaches/tactics they have developed to ease the process
• ask the group to check whether the chart includes all the major areas of collaboration with stakeholders, including seeking inputs or consultation, if not then these should be added now
• check with the group whether they are happy that his represents how they deliver work – offer some prompts to double-check that issues have been considered, for example does it cover questions of: prioritisation (why is this work important?), resources, approval, quality assurance, impact and frustrations/problems

The pink wall

Provide each participant with a pink post-it note and set aside a white-board, window or space on the wall to be the `pink wall.’

Towards the end of the workshop explain that the session is formally finished for each participant when they have stuck their pink post-it on the pink wall.

The note can include any comment related to capability, and if possible, allow participants to write in any language. The note can be an observation, something that they would like the review to recommend, or any other comment.

Annex C: Semi-structured interview guide

Introduction

Semi-structured interviews should be conversational. The interviewer should adapt their questions and style to the individual interviewee; the interviewer is not completing a formal questionnaire and should avoid the impression that there is a rigid structure to the discussion. Questions should never push the interviewee towards narrow answers (yes/no), but instead be open-ended and offer space for the interviewer to ask follow-up questions.

Semi-structured interviews are not random, there is consistency across all the interviews in the themes addressed and even the specific issues that are raised. This allows the review team to `triangulate information,’ and to get several perspectives on the same questions and problems. One approach is also to reflect back (anonymously) emerging themes and comments when framing questions so that information and insights can be tested with counterparts (this may help to profile areas of contention).

Two useful introductions to semi-structured interviews, including guidance on good ethics and on planning an interview programme are the Evaluation Toolbox and tools4dev – practical tools for international development. Interviewers should also consult the FCDO’s Understanding Political Economy Analysis (PDF, 671KB) and the beginners guide to PEA (PDF, 708KB).

The questions below are based on the five dimensions approach to institutional reviews (and normally the interviews undertaken are complemented by participatory workshops and desk-based research. Questions should not be delivered as written, instead they can be combined and a good interviewer will adapt and reframe questions in line with the flow of the discussion, building naturally on the responses received.

Timing

A one-hour interview allows enough time for the exploration of no more than 3 of the questions outlined below.

The interview

Purpose

What is the overall purpose of this department/unit? Describe the overall context of this area of work?

What does your Department do?

What are your key products and outputs?

Who uses these outputs/products, do you have any feedback from them?

What do you think are the outputs that users would prioritise?

Targets

What are the targets that have been set for your work, are these clear and consistent?

How are targets and priorities communicated, and how is delivery checked?

Describe how work is spread across the whole area, who feels most pressured?

Delivery

What resources are you working with? Are they reliable/consistent (any areas where you think staff skills need to enhanced)?

Describe how you manage to ensure delivery and `get things done’?

What are the things that help motivate people (or demotivate them)….and can you describe how good work is recognised, or how people are offered critiques/feedback?

Leadership and teamwork

Can you describe the limits within which you work in terms of being empowered/delegated authority and need for approvals?

What happens when you need help, when there are difficult issues to work through?

What are your views on the fit between the capacity that your area has and the tasks that you are being given?

Strategy

Describe any plan for change in your area, what would you like to see happen to improve your working?

How are plans developed?

What have you learned from past changes?

Stakeholders

Tell us about the organisations and people that you engage with.

Who or what other offices or organisations, have key influence on your activities?

Who do you rely on to enable your work to happen, and who shows most interest in what you do?

Overall, what are the greatest challenges that you face?

What works well at the moment and contributes to effective delivery?

What have been the main advantages and disadvantages of the current structure/process?

What needs to be changed, removed or added?

Any other issues

If you were conducting an institutional review what issues would you want to focus on?

What do you hope the review will deliver, what changes would you want to see?

Background reading

• Matt Andrews, Jesse McConnell, Alison Wescott, Development as Leadership led Change: A Report for the Global Leadership Initiative, The World Bank, 2010

• Daniel Rogger and Christian Schuster, The Government Analytics Handbook: Leveraging Data to Strengthen Public Administration, 2023

• Asia Development Bank (ADB), Institutional Strengthening Framework A Guidance Note, 2014;
• Dustin Brown, Jitinder Kohli and Samantha Mignotte, Tools at the Centre of Government (PDF, 1.4MB), BSG 2021
• Burke & Litwin, ‘A Causal Model of Organisation Performance and Change’, Journal of Management, Vol 18, No 3 (1992), pp 523–545.
• Derick W. Brinkerhoff and Jennifer M. Brinkerhoff, Public sector management in developing countries: perspectives beyond NPM Orthodoxy; Public Admin. Dev. 35, 222–237 (2015)
• DFID – Promoting Institutional and Organisational Development: A Sourcebook of Tools and Techniques (PDF, 473KB), 2003
• European Bank for Reconstruction and Development, Capacity Assessment Toolkit: Manual and Guide Note, 2011
• Jean-Paul Faguet, Mahvish Shami, The Incoherence of Institutional Reform: Decentralization as a Structural Solution to Immediate Political Needs, Studies in Comparative International Development volume 57, pages85–112, 2022
• GSDRC, Public Sector Institutional Reform webpage
• Mona Lena Krook and Fiona Mackay – Gender, Politics and Institutions: Towards a Feminist Institutionalism, AIAA 2015
• Brian Levy, Working with the Grain: Integrating Governance, and Growth in Development Strategies. Oxford University Press. 2014
• Willy McCourt, Six Models of Public Service Reform (PDF, 544KB), 2013
• Peter Morgan, International Support for Capacity Development in Post-Conflict States (PDF, 4.3MB), 2011
• Jan-Hinrik Meyer-Sahling, Christian Schuster and Kim Sass Mikkelsen, Civil Service management in developing countries: what works? Report for DFID, Feb 2018
• OECD, Anticipatory Innovation Tools, 2022
• OECD, Inventory of Donor Approaches to Capacity Development: What we are learning, 2010
• UNDP, A Brief Review of 20 Tools to Assess Capacity, Capacity Development Group Bureau for Development Policy, August 2005
• UNDP, Supporting Capacity Development in Conflict and Fragile Contexts, 2012
• USAID, Human and Institutional Capacity Development Handbook, 2010
• The World Bank, Public Sector Employment and Compensation: An Assessment Framework (PDF, 9.5MB), 2021
• The World Bank, Climate Change Institutional Assessment, 2021
• The World Bank, The Capacity Development Results Framework: A Strategic and Results-Oriented Approach to Learning for Capacity Development, 2009