Success Measures for Community Stabilization

In response to the foreclosure crisis and its impact on neighborhoods and communities, Success Measures created a set of evaluation tools specifically designed to measure neighborhood change for those engaged in stabilization activities.  NeighborWorks® America is currently funding a pilot in which several nonprofit organizations engaged in stabilization are testing these tools.

Success Measures for Community Stabilization was designed with a general set of programmatic assumptions to guide the selection of the dimensions of change most important to measure as well as the focus of the specific tools.  Some of the key aspects of the programmatic context include:

Baseline and Change over Time: The framework is designed to examine change over time, using the initial data collected in three key dimensions as a baseline against which data can be compared over time. It is expected that change will be incremental, with the expectation that measurable change may not be evident for 2 or 3 years, depending on the depth of the challenges and scale of the efforts.

Three Dimensions of Community: The three key dimensions were selected because they represented three major facets of communities that are at risk in a place hard-hit by foreclosures: qualitative perceptions of the community (through the eyes of residents and key external players); housing market dynamics; and physical conditions of buildings and public space.  The assumption is that all three are directly or indirectly the focus of community stabilization work undertaken by community development organizations. Furthermore, from an evaluation design standpoint, they allow a robust mix of data collection methods and data types which enhance the power of the evaluation results.

Target Neighborhood: The framework is designed to be applied to a target neighborhood where the spatial, social and market dynamics intersect and where the scope of evaluation is reasonably consistent with the scale of the community stabilization work being done. While many community development organizations are doing community stabilization efforts across a larger geographic reach, the evaluation aspects of this framework are intended to be applied to a targeted geographic area where a concentration of community stabilization work will be focused over the timeframe of the evaluation.

Theory of Change:  The framework reflects a theory of change in which foreclosures in a neighborhood result in loss of confidence on the part of residents and stakeholders and a decline in the physical conditions of foreclosed/vacant properties and the buildings and space surrounding them.  This, in turn, negatively affects the housing market in terms of housing prices and the relative appeal of the neighborhood as a good place to live.

Through community stabilization efforts--including returning vacant properties to productive use, improving physical conditions of buildings and space, and engaging residents in improving their community--there can be improvement in the perception of the neighborhood which will in turn strengthen the appeal of the neighborhood to current and potential residents.  Over time, it is expected that this will be reflected in a relatively stronger housing market because of the appeal of the neighborhood as a good place to live. This framework is designed to evaluate the change over time in each facet of this “virtuous cycle” of improvements in perception of the neighborhood, physical conditions and housing market.

The Community Stabilization Indicator/Tool Framework

The Community Stabilization set is intended as a comprehensive set of data collection tools to capture change along several dimensions affected by community stabilization. They are:

• Set 1: Community Characteristics—A set of secondary data to frame the context of the community.
• Set 2: Community Image, Confidence & Management—A resident survey and key informant interview to understand internal and external perceptions of the community.
• Set 3: Community Physical Conditions—A set of observation tools at the block and parcel levels coordinated with some secondary data and an analysis framework so that an organization can measure change over time to the physical conditions of the community as well as determine which properties, if any, are reasonable to obtain and rehab.
• Set 4: Market Health—A set of secondary data to address the market health of a community.

 Set 1.  Community Characteristics

Information about Residents

Type of Information:  Secondary Data

How Collected: PolicyMap and Local Records

• Tenure in community
• Household structure and size
• Income distribution
• Demographics (e.g., race, age) of residents
• School performance data
• Employment and occupation
• Homeownership

Information about Geography

Type of Information:  Secondary Data

How Collected: PolicyMap and Local Records

• Definition of area boundaries
• Spatial relationship to larger geographic area
• Population size
• Number of households
• Concentration of subsidized housing

Set 2.  Community Image, Confidence & Management

Survey:  Resident Confidence in the Community

Type of Information:  Primary Data

Measures:                                                                          

• Level of satisfaction
• Perception of property values
• Perception of safety and security
• Perception of variety and convenience of amenities
• Perception of quality of public services
• Perceptions regarding community market health/quality of investment
• Perception of direction of community changes
• Perception of local government responsiveness to and involvement with community
• Confidence in the future
• Expected tenure in community
• Bonding and neighboring behaviors

Key Informant Interviews:  External Perception of the Community

Type of Information:  Primary Data

Measures:                                                                          

• Perception of community vitality
• Perception of direction of community changes
• Community comparison to other communities
• Attractiveness to people with choices
• Community identity
• History and experience with the community
• Past participation in economy of community
• Realtor behavior
• Investor behavior (e.g., purchases for rental or homeownership)

Set 3.  Community Physical Conditions

Observation and Records:  Block Conditions

Type of Information:  Secondary Data, Primary Data

Measures:

Part 1:  Available Records (Secondary Data)                        

• Community anchors
• Economic activity (e.g., commercial businesses)
• Availability of transportation

Part 2:  Block Conditions (Observation)                               

• Condition and use of space in the community
• Condition and use of commercial districts
• Existence and quality of Infrastructure
• Curb appeal
• Neighborhood-friendly interim-use strategies for properties

Observation and Records:  Residential Property -- Parcel Level

Type of Information:  Secondary Data, Primary Data

Measures

Part 1:  Available Records (Secondary Data)

• Mapping
• Rental, homeowner, owner-occupied with rental units
• Type of structure
• Code violations (optional)

Part 2:  Occupied and Vacant Residential Property (Observation)

• Condition of dwellings
• Condition of property (e.g. yard, fences)
• Standards of maintenance and upkeep
• Signage on home for sale
• Evidence of interim use strategies
• Neighborhood-friendly interim-use strategies for properties

Part 3:  Vacant Residential Property Only (Observation)

• Security treatment of vacant dwellings
• Health and safety hazards of vacant dwellings
• Interim maintenance of vacant dwellings
• Construction activity at vacant dwellings

Observation:  Non-Residential and Mixed Use Property -- Parcel Level

Type of Information:  Primary Data

Measures:

Part 1:  Non-Residential and Mixed Use Property (Observation)

• Condition of buildings
• Condition of property (e.g., parking lots)
• Standards of maintenance and upkeep
• Signage for sale
• Evidence of interim use strategies
• Neighborhood-friendly interim-use strategies for properties

Part 2:  Non-Residential and Mixed Use Property, Vacant Property Only (Observation)    

• Condition of buildings
• Treatment of structures
• Interim maintenance strategies

Observation:  Community Space and Vacant Land -- Parcel Level

Type of Information:  Primary Data

Measures:

Part 1:  Community Space and Vacant Land (Observation)  

• Condition of space
• Interim maintenance and use strategies

Part 2:  Community Space and Vacant Land, Vacant Land Only (Observation)       

• Condition of land
• Interim maintenance and use strategies

Records Form:  Vacant Residential Property Analysis 

Type of Information:  Staff Analysis

Measures:

• Requirements for acquisition
• “Obtainability” index (accessibility of owner, tax situation, lien status)
• Cost ands feasibility of rehab
• Community impact of vacant property

Records Form:  Vacant Non-Residential Property Analysis        

Type of Information:  Staff Analysis

Measures:

• Requirements for acquisition
• “Obtainability” index (accessibility of owner, tax situation, lien status)
• Community impact of vacant property
• Prior use (brownfields or other contamination)

Records Form:  Vacant Land Analysis

Type of Information:  Secondary Data, Staff Analysis

Measures:

Part 1:  Available Records

• Prior use (brownfields or other contamination)

Part 2:  Property Analysis

• Requirements for acquisition
• “Obtainability” index (accessibility of owner, tax situation, lien status)
• Community impact of vacant land

Set 4.  Market Health

Characteristics of the Housing Market                           

Type of Information:  Secondary Data

Note:  Most of this information will be collected and available on PolicyMap; other data will be collected by the organization and sent to PolicyMap.  

Measures:

• Sales volume
• Days on the market*
• Property value
• Vacancy
• Foreclosures (optional)*
• Homeowner investment in property

Collection and input by Policy Map

*Collection by organization

About Success Measures  

The Success Measures Data System (SMDS) is an outcome-based evaluation system for conducting full-scale participatory evaluations of programs to improve neighborhoods and communities. It provides everything needed, from start-to-finish, to do credible, useful evaluation, including:

• Step-by-step instructions on how to design and complete a participatory outcome-based evaluation
• Community development outcome indicators
• Data collection instruments for all indicators such as surveys, interview guides, and forms for tracking administrative and other records
• Directions and helpful tips on how to best use each data collection instrument
• A secure place for you to enter, manage and store all the data collected for an evaluation
• A reporting function that tabulates your data
• Guidance on how to integrate your learning into programs and advocacy

SMDS is a web-based application which does not require any software downloads onto your computer. It runs on the Web and is accessed through an Internet browser. SMDS updates are automatically available for all subscribers. For more information see www.successmeasures.org