California's Green Economy: Lessons Learned
This is the fifth in a series of blogs, which be posted on Fridays over the next few months on the Green Jobs Community of Practice. The series will focus on the definition of green jobs and will highlight green jobs reports from various states. This blog features California and its lessons learned.
Bonnie Graybill has been a manager in the California Labor Market Information Division since 1991, currently serving as Deputy Division Chief for Programs and Local Information. Beginning in Spring 2008, Bonnie and her staff have been studying the green economy. Bonnie’s career in State government began in 1973, in Employment Development Department’s (EDD) Personnel Office. In ensuing years, she managed various programs at the State Personnel Board and EDD, including serving as Planning Chief for EDD’s Director.
California’s LMI staff has been studying the green economy, and recently completed a survey of green jobs and green practices for the state, surveying over 50,000 businesses across the economy—all industries, all geographies and all size classes. Results from the can be viewed in a presentation entitled California's Green Economy.
California’s Lessons Learned from Conducting a Green Economy Survey of Employers
1. What are the top three or essential things to consider and resolve before you begin to develop your plan to conduct a survey of green jobs, occupations and/or industries?
- Define “green” with partners Work with stakeholders to explore possible definitions of green for you survey. It is critical to have definition that will allow employers to provide responses that unambiguously describe their situation. Consider definitions used by other organizations in your state and outside your state. Discuss how these potential definitions will address questions raised in your state. Building the definition with partners will contribute to the buy-in you will get once the survey is complete.
- Decide what questions you need to ask and design survey instrument Limit your survey to the absolute minimum number of questions needed to address the most important questions for your stakeholders. This may require a great deal of work to achieve agreement among your partners as to the most important items but will promote a higher response rate from employers. Be sure the questions are clear and will elicit the desired response. If you develop an online version of the questionnaire, make sure the questions are completed in the same manner as the paper version.
- Pretest the survey instrument Do an adequate amount of outreach with employers to pretest the survey questionnaire. Introduce the survey and the goals/purpose of the research to a subset of the sample (about 100 employers) and ask them to complete the questionnaire. Follow-up by interviewing individual employers about anything they thought was confusing, misleading, poorly worded, etc. Review employer responses and concerns with the survey team to identify potential clarifications and improvements to the questionnaire. Retest the edited survey with a subset of the pretest group to ensure that the edits corrected the issues they may have identified.
- Select your sample and verify addresses and phone numbers Determine how you are going to select your sample; all industry vs. particular industries, private industry only vs. including public firms, site selection vs. firm selection. Allow adequate time for address refinement after the sample is drawn. Look up/confirm telephone numbers at the same time you are refining addresses because there is a good chance you will be making follow-up calls to employers. Build in sufficient time to send introductory post cards to the sampled employers to announce the survey and help with address verification. If the post cards come back as undeliverable, you know you have address research to do. Tracking down correct addresses for employers, particularly in a declining economy, was troublesome for us. Even after three rounds of address refinement we had thousands of undeliverable surveys returned by the post office.
- Prepare for survey data collection by phone If you plan to collect survey data by phone, develop the training early and spend a sufficient amount of time training the phone staff. Two-to-four hours of training is not sufficient. Train the phone staff in how to overcome objections gracefully, tactfully, and with enthusiasm so employers are willing to stay on the phone long enough to complete the survey. Have the phone staff role play with each other until they are all comfortable with any potential scenarios. Train phone staff not to adlib on the questions and not to bail out halfway through the survey. Plan on supervising the phone staff at unannounced times (e.g. frequent physical “management by walking around” type monitoring.)
- Develop a survey transmittal letter The letter that accompanies the survey when sent to employers should be signed by someone who is respected in the field of research and not subject to political influence. California’s survey was sent under Governor Schwarzenegger’s signature at a time when trust/respect for the Governor was less than favorable If disenfranchised employers viewed the green survey as the Governor’s pet project they were less likely to respond to the survey except to voice their (strong) disapproval. In fact, our response rate doubled when we sent out a humble “last chance” letter signed by our division chief.
- Announce the survey in one or more press releases Prior to the first mailing of surveys, work with the media to post press releases regarding the coming survey and the importance of employers’ prompt, accurate response. Take a lesson from the Census media blitz that has been in the media for the past couple of months.
- Review completed surveys as they are received Review not only the questionnaire but compare industry and employment to QCEW. We did not review surveys completed online until we were into data analysis.
2. What was the most important lesson learned that will save time and grief?
- Build a database! Create the necessary technical infrastructure to house the complete sample and all related employer-specific data and all survey response data. Make sure you have the technical staff skills needed to process and analyze the data. It has been problematic for California to maintain all survey data in Excel spreadsheets because of the size of the spreadsheets, insufficient memory and/or processing power to work with such large files, and the limitation of only one person being able to work on a file at a time. Also, cleansing the response data; developing the weighting, non-response adjustment factors, benchmarks, and variances when working with five or six or more different files is extremely cumbersome and opens the door to potential errors. The process was laborious and time consuming, at a time when we wanted to move forward more quickly with our analysis.
What are some lessons that YOU learned from Green Jobs Surveys?
User Comments (0)
A Workforce3one.org account is required to submit comments. Please login now or sign up for an account.