ONGOING ASSESSMENT OF PUBLIC HEALTH
INVOLVEMENT IN POLICY AND ENVIRONMENTAL CHANGE INITIATIVES
Prepared for:
The
Directors of Health Promotion and Education
Strategic
Health Concepts, Inc.
Ongoing Assessment of Public Health Involvement in Policy and
Environmental Change Initiatives
Introduction
During 2000 – 2001 and again in 2004 – 2005, the Directors of Health Promotion and Education (DHPE) contracted for an assessment of the involvement of public health agencies in policy and environmental change initiatives with a particular focus on interventions aimed at chronic disease prevention and control. Both of these assessments involved soliciting state government public health agencies for information on their activities and then analyzing the results.
The two studies had similar purposes in that both contain a detailed analysis of the findings and make recommendations regarding the state-of-the-practice of public health agency involvement in policy and environmental change. There are distinctions in the two studies that should be noted:
1. The 2000 – 2001 study was a first attempt to look at the state-of-the-practice and was responsive to a major reorientation of public health practice towards greater emphasis on policy and environmental change strategies for chronic disease prevention and control that occurred throughout the 1990’s. Many of the strategies implemented during that period were carried out by coalitions of various not-for-profit health organizations (e.g., the American Lung Association), government-based public health departments and, in some cases, for-profit organizations with an interest in health outcomes (e.g., hospitals, insurers). The two primary questions addressed in the initial study were:
a. What roles did public health agencies play in these broader strategies of policy and environmental change?
b. What was the capacity of public health agencies to fulfill these roles and what could be done to enhance that capacity?
Data for the initial study consisted of the responses to a number of basic, easy to answer and directed questions relative to public health agency involvement in and capacity for policy and environmental change strategies. DHPE and others have used the recommendations from that first study to develop a number of specific initiatives to enhance the capacity of state public health agencies to engage in policy and environmental change work.
2. The 2004 – 2005 built on the previous study results in that it provided a degree of comparable information on the level of involvement by public health agencies in policy and environmental change work. It differed from the earlier study in that it used a case study approach in which detailed information was obtained on the actual nature of the policy and environmental change, how the strategy was initiated and unfolded, who was involved and how the public health agency fit into the total picture, the results of the initiative, and lessons learned. The purpose of this study was to again describe the state-of-the-practice while gathering more in-depth information that would inform decision making about future DHPE and other initiatives to further develop public health agency capacity for this kind of work. Two case study versions were solicited:
a. Short case study – in which a structured set of questions was used to provide a 2-3 page summary of the initiative.
b. Long case study – in which a standard set of information was requested, with more options for a longer, personalized narrative description than was possible with the short version. These case studies were typically in the 8-12 page range in length.
Purpose
The purpose of this document is to address the question of what an ongoing assessment of public health agency involvement in policy and environmental change might look like. In taking into account the lessons learned from conducting the first two studies, it is clear that there are several broad options for structuring such an assessment and most of the remainder of this document addresses these. Before presenting those options, however, a brief review of lessons learned, underlying assumptions, and a general implementation protocol are provided.
Lessons Learned
There are a number of lessons learned from conducting these two studies that impact on the thinking about various options for an ongoing assessment. The most important of these include:
· Public health agencies are very interested in these types of studies and use the results to inform their work.
· There has been significant growth in interest, capacity and involvement in policy and environmental change between the time periods covered by the two studies.
· Finding the right person(s) at the state public health agencies to solicit a response was sometimes challenging.
· The time available for public health agencies to respond to requests for special studies such as these is very limited as increasingly these agencies seem to be asked to do more and more with less and less. The level of stress on the current public health system is extraordinary and must be factored into future plans for carrying out these studies.
· The level of effort and time required to solicit and collect information similar to those in these studies is extensive and in both cases was underestimated.
· The number of responses received was far less than the actual number of cases that exist even when every effort was made to make the process as simple and user friendly as possible. This means studies such as these funded at the level they are will almost always be illustrative snapshots of what is occurring rather than a representative picture. Nevertheless, the results are of great value given this limitation.
· The use of cash incentives ($1,000 per long case study) in the second study undoubtedly helped in soliciting the cases but did not change the time or level of effort involved in actually getting them in hand.
· Those who submit responses are very eager generally to share their stories and see the results.
Underlying Assumptions
There are some underlying assumptions that are inherent in the consideration of options for ongoing assessments such as the ones from the two studies carried out to date. These include:
· There will be continued change in the nature and level of public health agency involvement in policy and environmental change such that ongoing assessments will continue to be useful in guiding decision making about capacity development and supportive structures.
· The interest of public health agencies in the results of studies such as these will remain high.
· Public health agency staff will continue to be extraordinarily busy and soliciting their participation and follow up in future studies will require a commensurate level of proactive effort to be successful.
General Implementation Protocol
While the two previous studies differed somewhat in content, the general protocol for implementing them was similar and should be considered in implementing future studies. This protocol is generally chronological in order but some elements do overlap in time.
· Agree on study purposes and scope – for both studies DHPE designated an advisory group that was very helpful in outlining the study purposes and protocols. The previous studies defined the content of the study as focused primarily on chronic diseases. The issues of expanding the scope beyond chronic diseases came up to a limited degree during the initial study but was much more intensive during discussions of the second study. Arguments for expanding the scope in the future will probably continue to arise.
· Develop the data collection format and protocol – this often took several iterations before consensus was reached. One issue that came up in both studies was deciding who to actually submit the request for participation to. Possibilities included DHPE members and Chronic Disease Directors members, a few of whom belong to both organizations. Other considerations are directors of specific chronic disease programs/sections (e.g., tobacco control; obesity prevention). It has also been suggested that submitting requests through the state ASTHO member would be appropriate. This choice remains problematic because the degree to which these various people are aware of the full scope of policy and environmental change activities in the agency and even the degree to which they communicate well with each other is highly variable from state to state. Compounding this issue is the fact that turnover in any of these positions can be relatively high. Thus whatever address lists are used are always outdated and require personal follow-up to find at least a few of the current incumbents. In addition, there has been a desire to include information on what local public health agencies and territories are doing related to policy and environmental change work, but no solution for gathering this information efficiently has been identified to date. An effort was made to get at this need in the initial study but was not considered successful as it used the state agency as the key informant for local agencies. The second study attempted to gather information from territories but was not successful.
· Solicit and vet initial information – even with explicit information about the scope of the study, a sizeable number of ideas/cases are submitted that are out-of-scope. There remains some confusion in the field over what constitutes an intervention that is a policy or environmental change or one that is simply programmatic (e.g., a public awareness campaign). This confusion is understandable given that the same techniques might be used (e.g., a media campaign might be part of generating support for a policy initiative or it might be intended to solicit participation in some disease screening program). In the second study, potential submitters were asked to answer five 1-sentence questions to help identify those cases in- and out-of-scope. There was also a need to go back to the submitter in a number of cases for clarification before making this determination. This procedure is highly recommended for future studies.
· Solicit and review full information from those whose studies are identified as in-scope. Submitters are provided with the full data collection instrument/format. Submissions are reviewed for completeness and clarity and clarifying questions are asked. In both previous studies, clarifying information was required in a sizeable number of instances.
· Follow up with non-responders – a significant amount of time can be spent in contacting people who are interested in submitting but unable to meet a particular deadline. Great flexibility in deadline setting is needed; although at some point it is also necessary to decide to move on with what is in hand.
· Edit submissions and request approval of changes from the original submitter.
· Carry out the analysis of the combined responses – including comparisons with previous studies where appropriate.
· Draft recommendations and obtain input – an advisory body can be very helpful at this stage in the process.
· Draft a report of findings and obtain input.
· Finalize and submit the report – additional submissions of electronic versions of case studies may also be required.
· Prepare articles, presentations, and workshops and be prepared to respond to inquires after the report is issued.
Options for Ongoing Assessments
From the preceding information it is apparent that the major underlying issue in conducting ongoing assessments such as these is the level-of-effort required to carry them out. With that in mind, three basic options based on level-of-effort are presented below along with a discussion of the option including the pros and cons of each. It is recognized that elements of these options are not mutually exclusive and additional options cane be derived from them. Moreover, each of the options below requires some level-of-effort to review results, analyze the data and prepare reports.
Option 1 – A minimal level-of-effort option
In today’s electronic world it is relatively easy to set up websites and other electronic means of gathering information that can be used by potential submitters at their convenience. Potential submitters can be solicited through various means and directed to the website to respond. After initial set-up costs are incurred the efficiency of this form of data gathering can be quite high. The use of standard formats also reduces the amount of variability in what is submitted. This option would very well fit with a data collection protocol that is highly structured and for which closed-end responses could be well defined (something the first two studies’ purposes were not well-suited for). The downside of this approach is that it is dependent on the willingness of potential submitters to respond to solicitation requests. The experience of the first two studies suggests that hands-on follow up to encourage submission is essential. Also, with this type of ‘open-enrollment’ option, there will still exist a need to clarify open-ended responses should they be used.
Option 2 – A mid level-of-effort option
This option includes some designated level-of-effort on a periodic basis during the solicitation and submission period. A 100% FTE for whatever time period is specific during the protocol is recommended. The purpose of this person is to provide for the necessary one-on-one contacts to encourage submission, follow-up with non-submitters and to address needs for clarifying the information submitted and amending it if needed. This is somewhat analogous to what was done for the first two studies although in both a significant effort was put into defining the purposes and scope of the studies and in developing the study instruments and protocols. With the experience to date, it may be possible to develop a standard study protocol for future studies which could be periodically adjusted based on new conditions rather than doing a separate protocol for each future assessment.
Option 3 – A high level-of-effort option
This option suggests designating a full-time person(s) to act in the capacity of an ongoing surveyor of the public health field with regard to public health agency involvement and capacity for policy and environmental change work. It includes all the tasks identified above. Most importantly, this person would not only carry out future assessments, but keep their eye on the entire field of activity in such a way that when the assessments came up the person would know of important work to be included (and can solicit for it), have established relationships with people in the state public health agencies so that the right people to solicit can be readily identified and personally contacted, and speak on a continuing basis as a credible authority on public health involvement.
The issue of resource needs notwithstanding, it is suggested that some iteration of Option 3 above be considered as the next logical step in moving the public health capacity for policy and environmental change forward. There seems little doubt that policy and environmental change strategies are effective and the amount of effort devoted to them is certainly increasing, including at the local level. Given this and the strong, continuing interest in the field in this area of endeavor, it may be timely to move beyond periodic snapshots of what is happening to a more regular and persistent scanning of the field that leads to even more proactive capacity development and involvement in policy and environmental change in the future.