Eugene Bardach

QPAM: Problem Definition

Before a policy analysis can be performed, the underlying problem needs to be defined. Any problem definition is a function of what the author of the definition cares about and what they assume in terms of necessary relationships. Bardach suggested to be clear about what you care about and what you assume about the fact (Bardach, 2012). He claims that problems are caused by market failures, inequalities and non-efficient government solutions.

Value conflict resolution

An alternative approach to think about is a two-dimensional problem space that is defined by the involvement of money and social consensus (legislation could be passed and the law will remain long-term). The solution to a problem defined in this space would fall into the following categories

  • Private (companies): if no social consensus is there, but money can be made
  • Non-Profit (organisations): if no social consesus is there, but no money can be made
  • Public (government): if a social consensus is there, but no money can be made
  • Ambivalent (companies/government/organisations): if a social consensus is there and money can be made

References

Bardach, E. (2012). A Practical Guide for Policy Analysis (4th ed.). Thousand Oaks, California: Sage.

Sugar and Fat Policy in the US

A recent article in the New York Times illustrated the difficulties of creating policies. In particular, the article came to the conclusion, that the policy push against fatty food – a standard policy across the world – has been orchestrated by the the sugar industry.

The interesting part is, that they did not lie about the effects of fatty food. They merely paid scientists at Harvard in the 1960s to under-report the dangers of sugar for health. The headline of said research continues to dominate the way that health efforts by governments are understood today.

The scientific discourse was manipulated to remove alternatives (Bardach, 2012) that would later not show up in any policy analysis until the 2010s. The authors of those sugar industry papers continued in high positions in the US government and as head of university departments.

This article shows the limitations of efficiently transporting scientific findings into policies. A common strategy – also used in this case – seems to be to shift the goal post such that the focus moves to another problem, e.g. from sugar and fatty food to only fatty food.

The article went on to claim, that, potentially, today would be very different if sugar had remained a valid policy option removing many of today’s obese. This is probably a bit simplistic, but the question goes in the right direction. Industry-driven research will always be foremost about the industries interests and only secondary about public interest.

Good research can come out of an industry-driven context, but should always be contrast with research of non-interested /uninvolved groups. This is nonetheless a stretch as there seems to be little incentive for the uninvolved to invest enough into the research of the industry area without themselves becoming involved.

This particular situation of industry-driven research would probably require a change in the incentive structure such that the public can obtain alternatives that would be in their interest but otherwise be removed as they are opposed to the industries interest. This could only come by a policy change that puts industry under more pressure to also discuss unfavourable results.

This sounds at first contradictory, but moves in the pharmaceutical industry (“Spilling the beans,” 2015) have shown that there can be a construct that is beneficial to the industry as well as the public interest. However, it is likely to be necessary to devise a specific policy for each industry as the respective requirements should be wide.

References

Bardach, E. (2012). Step Three: Construct the Alternatives. In A Practical Guide for Policy Analysis (pp. 16–31). Thousand Oaks, California: Sage.

Bardach’s Eightfold Path through Policy Analysis III

In the last post we focussed on the gathering evidence and in this post we will discuss possible alternatives. Here are links to the articles of the different parts of the book:

Introduction

Step I: Define the problem

Step II: Assemble some evidence

Step Three: Construct the alternatives

The word alternative lamentably is used with varying meaning. In the exclusive sense an alternative replaces a policy and they cannot be enacted at the same time where as in the inclusive sense an alternative can be applied in conjunction with the policy and potentially enhance the results. It is recommended to clearly state what kind of alternative is talked about.

In the early stages of a policy analysis Bardach recommends to comprehensively look for alternatives and analyse as many as possible and over time narrow it down to the most promising alternatives.

Alternatives can be gathered by “window-shopping” at political, institutional or private outlets that propose their policies. Another way to obtain alternatives is by designing them, however, more often than note a creative brainstorm will not produce a new world-changing idea that has not been proposed yet.

Modelling

Casual models explaining the cause of problems can be useful to determine policies and intervention points but often a direct chain of causes cannot be constructed. The number of factor playing into a model is often obscure and can only be estimated making any model predictive at best. Nonetheless they offer great initial insights and therefore several common models are listed:

  • Market Models: Disaggregated suppliers exchange goods/services with disaggregated demanders and the model is thought to reach equilibrium through exchange.
  • Production Models: Identify parameters that make the production system (e.g. command-and-control regulation) vulnerable to breakdown, fraud, abuse, egregious diseconomies and the distortion of intended purpose. Another set of interesting parameters are those that induce the highest performance levels.
  • Evolutionary Models: Essentially they represent a genetic algorithm that represent a common process of change over time. In the policy view it consists of “variation among competitors”, “selection” and “retention”. The first consists of the kind of problems that come up, the second of how problems are assessed and the last of how problems are understood by the detector.

Conceptualize and Simplify

The list of alternatives should not be overwhelming, so only a small set of well-thought alternative should be presented. An alternative should be presentable by a simple sentence or even a phrase. However, it should not be reduced to its summary and an extensive version should be available. Variants of alternatives should be considered as one for this purpose and only be looked at once an alternative is chosen.

If the alternatives consist of a continuum, it is usually worth so segment the continuum into distinct units to all for a low number of alternatives.

Design policy alternatives

When pre-existing alternatives are insufficient – e.g. in the case of exploiting new opportunities – policies need to be designed. The process differs from merely comparing alternatives in that

  • it needs to be shown that the design fulfils the demands
  • it needs to be shown that the design is consistent
  • it might need to be adjusted to be realistic
  • it often requires trial-and-error

A good way to getting a hold of the loftiness of policy designing is to set targets (e.g. 20% increase in public transport) and budgets (e.g. improve public transport with $20 million).

Look for design pattern in previous policy designs (even in different fields) to find useful approaches to your problem. However, consider scale and environment before reusing design patterns as they may hinder applicability.

To get a stable design – especially under trial-and-error – Bardach suggests a few starting point:

  • The least flexible design element (e.g. budget)
  • The most powerful design element (e.g. desincentivise car use to favour public transport
  • The most robust design element (e.g. a subsystem of the design that could provided benefits without the whole policy being implemented)
  • The most transitory, least costly, design element (e.g. intermittent solutions to cover a switch in underlying designs)

As a result of the fluency of the policy development it is necessary to regularly check the assumptions. A helpful tool are logic models(“Logic Model Development Guide,” 2004).

Usually, the design process can be grouped into two distinct tasks. First the system needs to be designed in a steady running state and second a transition towards that state needs to be designed.

Stakeholders

It is useful to involve stakeholders as soon as possible to get feedback and find supporters and identify potential opponents and their reasons of opposition. However, if the design on the one hand is too rough potential stakeholders may not engage fearing early-mover vulnerability or may float their own ideas complicating the process.  If the design on the other hand is to polished stakeholders may oppose simply for not being consulted. Bardach calls the appropriate moment rough-but-not-to-rough.

References

Logic Model Development Guide. (2004). Retrieved September 17, 2016, from https://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide

Bardach’s Eightfold Path through Policy Analysis II

In the last post we focused on the problem definition and in this post we will discuss underlying evidence. Here are links to the articles of the different parts of the book:

Introduction

Step I: Define the problem

Step Two: Assemble some evidence

Bardach boils policy analysis down to two major activities: thinking and data gathering. Whereas the thinking clearly is the more important data gathering, the latter will consume most of the time. To make matters worse time pressure can be as bad or worse than political bias. Therefore data collection should be goal-oriented from the onset and focused on turning information into evidence (i.e. important information that affects existing believes).

Evidence is used for three main purposes:

  • Assess the nature and extend of the problem
  • Assess particular features of a concrete policy situation
  • Assess policies that have been thought to have been worked effectively in apparently similar situations

The range of assessments requires the evidence assembly to be performed several times indifferent stages.

Bardach recommends to actually focus on thinking through what kind of data to collect and how to use it rather than just starting to collect and look where you end up. This is due to the value of evidence. The cost of acquiring data must be compared to the strength of evidence it can achieve:

  • likelihood of the new evidence to cause a shift from a original decision to a substitute decision
  • likelihood of the substitute decision producing a better policy outcome than the original
  • magnitude of the difference between the original and substitute

To reduce costs “educated guesses” may be employed to rule out or require closer investigation. Guiding questions for this guesstimate should be:

  • If the data looks different than expected, what will be the implications?
  • Compared to the best expectation, how bad could the data be if it was really gathered?
  • How much does the evidence improve if I switch from guesstimate to real analysis?

Available Literature can be useful and is often readily available on the internet. However, it but must be used with care as it is often produced by biased advocacy groups that may favour publishing research favouring their cause.

Data can also be gathered by analogy where a problem (and its possible solution) distinct on the surface is queried to gain insides (how to disbar incompetent lawyers by analysing how incompetent physicians loose their license).

Many activities need long preparation times and/or need to be squeezed into busy schedules, so administrative steps should be taken as early as possible (e.g. request access to archives, schedule interviews, etc.).

Policy analysis are not performed in a vacuum and therefore the work should be exposed to critics in the process to mitigate complaints about the resulting policy. Biases can be countered by explicitly contacting critics of sources, sponsors or other directly interested parties.

A more potent feedback cycle can be used when a policy is implemented iteratively and takes in feedback from the participants. In that case the policy analyst becomes a partner/broker in the process.

 

 

Bardach’s Eightfold Path through Policy Analysis I

In the introductory post about Bardach’s work we discussed the overall context and content. In this post we will focus on the details of the first step of his guidelines. The other steps will be posted soon.

Step One: Define the problem

The most crucial step as it direct any further endeavours. Bardarch recommends to define problems quantitatively in terms of deficit and excess and specifically with a magnitude. Often magnitudes can only be estimated and it is worth providing point and range estimates (e.g. descriptive statistics with box plots). Magnitudes help to convey behavioral and concrete definitions. Since definitions of problems are by their own definition evaluative it is necessary to establish whether a problem is grave enough to concern the public. Bardach suggests that any form of market failure – i.e. a technical property of a good or a service malfunction – warrants an intervention but that on the contrary nearly no other kind of “problem” qualifies. Exceptions are

  • Breakdown of non-market systems (e.g. family relationships)
  • Low living standards for non-participants (voluntarily or involuntarily) of the market
  • Any form of discrimination (racial, minorities, etc.)
  • Government inability to deliver a service (public schooling, infrastructure, etc.)

Another issue with definitions is that they may also be diagnostic and in the worst case misleading if the diagnostic is mistaken. Causal claims of problem definitions need to be supported by evidence (see Step Two) or else they can easily fall victim to issue rhetoric.

Issue rhetoric

Bardach recommends to be wary of issue rhetoric that is usually loaded with connotations (partisan, ideological or personal) and recommends a more sober language to find a problem that is analytically manageable. Issue rhetoric also often enforces selective perception and allow parties (to the problem) to define their position as “correct”. At best issue rhetoric can used as the raw material for a first provisional problem definition.

Similarly, some words are connoted with a multitude of issues and therefore it may be necessary to provide a primary focus or exclude certain issues from the definition.

The odds

A special expression “The odds” should be used with care as it conveys vagueness and may under- or overstate an issue. It can be used to cover the uncertainty of the author about a specific event and it should be read this way in any policy analysis.

Latent opportunities

Policy analysis is mainly used to address existing grievances. However, there is a case to improve non-aggravated situations (against the modus operandi “if if ain’t broke, don’t fix it”). Bardach identifies five fields that can often yield improvements:

  • Operations research strategies allow to optimise the use of resources and improve productivity.
  • Cost-based pricing can reduce inequality by allowing for lower prices during off-peak hours and hand-off the price of peak-consumption to the actual users.
  • By-products of personal aspiration can be incentivised such that personal advantage coincides with social advantage (e.g. give a percentage of the cost reduction as a wage increase to the inventing employee).
  • Complementarity offers opportunities to join activities that have symbiotic effects.
  • Input Substitution can reduce costs by replacing over-qualified employees with well-qualified employees (e.g. civial clerk for police administration instead of trained officer).
  • Development support can provide opportunities to improve situation before they get out of hand (e.g. unemployment training before searching for a new job).
  • Exchange-oriented policies can facilitate the flow of services by mimicking markets (e.g. Obamacare and the online-market place for insurances),
  • Multiple Functions can be assumed by a single policy allowing to use synergies.
  • Nontraditional participants can give a new perspective on problems and allow for previously unknown changes for improvement.
  • Underutilized capacity can be put to use to reduce cost elsewhere (e.g. school buildings in the evening) but must be offered with caution in order to not endanger emergency capacities or baseline capacities.

The solution in the problem

A problem definition should not entail a solution and should be merely descriptive. Hidden solutions can be hard to see as exemplified by the problem definition “there are to little homeless shelters for families” compared to “too many families are homeless” where the first variation implicitly assumes that shelters are a solution the second options allows the exploration of stopping families from getting homeless in the first place.

As good definitions are hard to come by an iterative process to come up with a definitions is the recommended course of action according to Bardach. The iterative process should also increase the empirical and analytical understanding of the issue at hand.

Policy Analysis according to Bardach – Introduction

Bardach’s A Practical Guide for Policy Analysis (Fourth Edition, 2012) is an introductory work into policy analysis proposing an 8-step approach to policy analysis termed The Eightfold Path. It is directed at Master’s students with basic knowledge of economy.

Bardach focuses on the changes in policy analysis from a formal report to an interactive undertaking accompanying the process. The change is similar to changes in Software Engineering where the (static) Waterfall-Model has given way to the (dynamic/interactive) Agile Development paradigm.

The Eightfold Path – Overview

The Eightfold Path can be seen as a guideline to inexperienced policy analysts that struggle with balancing personal bias with organisational interests/biases. A quick overview (as listed in the book) is provided below:

  1. Define the Problem
  2. Assemble Some Evidence
  3. Construct the Alternatives
  4. Select the Criteria
  5. Project the Outcomes
  6. Confront the Trade-Offs
  7. Devide!
  8. Tell your story

The order as is not fixed as prescribed in the aforementioned list, but a rough guide. The problem-solving process has a circular logic to it meaning that several iterations may occur in a “trial and error” style similar to modern product/software development. The guidelines are tailored to practical situations but should be understood as conceptual and some parts of those guidelines may already be predefined by circumstances.

From the onset the book tries to foster efficiency as well as “correctness” (according the the defined/required standards of the problem). The details of the Eightfold Path will be discussed in future posts. In this overview (coinciding with the introduction of the book and summarising it) we will lastly look at the proposed resulting report of a policy analysis:

  • A coherent description of the problem
  • Several solutions
  • Projected outcomes for the application of any solution including path of reasoning
  • Trade-off analysis if no solutions can be clearly favoured
  • Recommendation of required

Extended Literature

As a practical guide the book should be read in conjunction with in-depth literature for policy analysis (listed in Bardach’s order):

[table id=3 /]