Dear Readers:

In this issue of The EE Advocate we are pleased to bring you perspectives on the value and
process of evaluating programs. Evaluation to me is like the herb cilantro; when I was first exposed to it I didn�t care for it at all. Over time, I learned more about the use of cilantro to enhance recipes and I also grew to understand that evaluation is more than an imposed number exercise. It is rather an essential capacity building tool that guides planning, deepens learning, improves programs and ultimately enhances who we are as EE professionals and our ability to reach our goals. Now, I can�t imagine life without either cilantro or evaluation!

As Dr. Joe Heimlich states, "evaluation, at least for some of us, is fun when we forget it�s about �measurement� and discover it�s about doing what we do better". A short primer on evaluation by Dr. Heimlich can be found on page 3 as well as an article on page 1 about the use of evaluation in planning and improving Leadership Clinics. Keep an eye on the EE and Training Partnership (EETAP) as this group of national partners moves in the direction of more fully utilizing evaluation for activity planning and improvement. Results will be reported in future issues of the EE Advocate and available directly through EETAP�s website (page 8). Resources and Opportunities on page 11 offers many program evaluation resources to get you started or take you further in your evaluation pursuits.

Also in this issue the EETAP States and Demonstration Communities programs of EETAP are featured with updates of the progress state and local EE leaders are making towards achieving their EE capacity building goals. Also featured in this issue are model initiatives in North Carolina and Alaska and an update about capacity building sessions at the up and coming NAAEE conference.

In the next two years, NEEAP, with the assistance of EETAP�s outside evaluator Dr. Michaela Zint, will be evaluating state EE capacity building to glean greater wisdom about how we create EE programs, change education systems and communities and bring about environmental literacy and stewardship.

Thank you for your interest in the EE Advocate and your contributions to building capacity for EE,

Abby Ruskey, Co-Director, NEEAP

North Carolina�s Environmental Education Certification program establishes a standard for the
environmental education profession. This program is the first of its kind in the country,
although Missouri and other states are using North Carolina�s example as they establish their own environmental education certification programs. NC�s EE Certification program has proven to be a powerful and effective capacity building tool to strengthen the field of environmental education in our state.

An exciting part of this program is that it is open to everyone. Over 1,600 teachers, park rangers, non-formal educators and others are taking part in NC�s EE Certification training, a program that requires at least 200 hours of professional development. Participants develop the knowledge and skills to engage students of all ages in understanding how their individual decisions impact the environment.

Environmental education certification requires five major components:

A Certified Environmental Educator has opportunities for career advancement, and receives a resume-enhancing certificate and other benefits. On November 21, 2003, the Office of Environmental Education in the NC Department of Environment and Natural Resources will honor more than 350 of the newest Certified Environmental Educators at an exciting statewide ceremony.

For more information on North Carolina�s EE certification program, visit the NC Environmental Education Clearinghouse website at www.ee.enr.state.nc.us/ or email Libby.Wilcox@ncmail.net.

How many Project Learning Tree and Project Wild facilitators does it take to reach two teachers at a time who live hundreds of miles apart in a frozen arctic, road-less environment? "One," says Sandi Sturm, creator of the distance delivered Introduction to PLT/Wild course in Alaska.

The model takes teachers, who otherwise would not have easy access to environmental education materials, through a five-week, interactive workshop using Internet and telephone. Educators from the northernmost village of Barrow, where the nearest tree is 400 miles away, mingle and share cultural stories with educators from the rainforests of southeast Alaska, where totem poles stand majestically in village centers.

Plans are currently underway to develop a Train the Trainer program using the same model. "Every state has educators who would benefit from a distance delivered workshop," explains Sandi. "We all experience time and budget constraints when it comes to professional development opportunities." The new program would train facilitators and coordinators in other states how to deliver PLT/Wild workshops in their states.

At the Grassroots...Understanding What Evaluation Is Really About

Evaluation often brings shudders or moans of discontent from program personnel. In many cases, this is because our experience with evaluation is really not what those in the field of evaluation would consider "sound" evaluation practice. At its heart, program evaluation is about improving a program, an activity, an experience, an exhibit, or materials. The goal of program evaluation is continual reflection to ensure that a program is improving, continuing to perform at a desired level, and/or meeting its outcome goals. Evaluation is not just about how "good" a program is, but is about asking questions and knowing what will be done with the answers. And evaluation is not about an instrument or a series of instruments, evaluation is about understanding how and why a program works (or doesn�t) and how it can be improved or enhanced.

An evaluation is always unique to a program. The purpose of an evaluation is to determine how well a program is doing and how it can be improved or enhanced. This focus on use or utility is one of the major differences between basic research and evaluation.

An evaluation is not about gathering data, it is about answering questions.

Evaluation is asking questions�but we only ask questions for those things we really want to know the answers.

And we only ask questions for which the answers will lead us to doing something different (if we find out, for example, that 1/3 of our participants are lower income, are we really going to change what we do, or is it just information that we want to know?)

There are many ways to gather information from and about people. We don�t always have to ask the individual.

A major key to evaluation is systematic collection of information across time and audiences. We don�t ask one group, one time and make a decision about a program.

Often, evaluation is really finding out what we already know�but instead of trying to defend a program because we "know" it works ("just look at their faces"), we have evidence that shows how it works.

Evaluation is a way of thinking about how a program is organized, how the program is implemented, and how the program connects the desired audience with the desired information. Staff debriefing (formally) about a change in a program is evaluation if it is
systematic, rigorous, and leads to improvement in the program!

Evaluation, at least for some of us, is fun when we forget it�s about "measurement" and discover it�s about doing what we do better.

With the help of EETAP funding and leadership, we have engaged additional new leaders who reflect diverse facets of EE, rewritten our bylaws and framed a new governance structure that facilitates strategic board development. We held a successful conference in September 2001 and now have a listserv for electronic communication, a new website, an on-line database in progress, and new supporters throughout the state. We have begun to network more broadly and have built a palpable momentum. Arizona Association of EE (AAEE) has strong, determined leaders, working together as an effective team. They are committed to pooling resources, and continuing to strengthen and advance the capacity of the Arizona EE network.

"EETAP in Arizona has inspired us to work collaboratively on as many projects as we can. The Arizona EE community is working together, working smart and working hard: Project Power workshops, Arizona Studies Academy, joint workshops with FLP, WET, and PLT plus a conference in September.

"I believe the collaborative efforts that have produced AZ Studies Academy, Project POWER workshops, and other opportunities have been the greatest accomplishment to date. I believe also that collaboration has been important. It made all the program coordinators focus on just how interrelated these curricula are, just as earth systems are interrelated and interdependent.

"Some of the benefits from EETAP have been to strengthen our board, which has also strengthened the organization. The work we have done via leadership clinics and facilitated sessions have really helped to focus the team. Collaborations for workshops have happened in
various capacities. We have an impressive web site that is a reflection of the organization and what having funds can really accomplish. We are making strides to help EE organizations align with the standards (Correlation Project) and connect these to NAAEE standards."

The California team�s diversity initiative is our greatest accomplishment as an EETAP State. Through a variety of approaches, we have forged a process that will diversify environmental education leadership in the state and strengthen programs that reach diverse audiences. This long-term process began through involvement in the California Association for Bilingual Education and development of the EETAP Diversity Task Force. In creating the Diversity Task Force, the EETAP California team expanded upon the existing Golden State Environmental Education Consortium (GSEEC) community of leaders by recruiting statewide environmental
education leaders who serve diverse communities.

The Task Force efforts led to a Diversity Seminar focused on developing a road map to a diverse, effective and dynamic leadership in the field of EE in California. This successful seminar helped coalesce the strong and diverse team of leaders who attended the recent National Leadership Clinic. The efforts of the California team at the Clinic helped springboard new and innovative approaches to carrying our diversity initiative forward.

Illinois� experience during the EETAP States Project (CORE2 � Centers of Regional Environmental Education) can be summed up in the motto stated above. CORE2 is a networking project with the goal of coordinating environmental education trainings and opportunities making them available to everyone, no matter which region they live in. We�ve made great progress, but there�s still so much to do!

CORE2 began with an all-state strategy meeting on April 18th, 2002. Much time was spent on the importance of effective communication and what it might look like statewide. Participants discussed questions about the challenges to communication; brainstormed groups with whom they would need to communicate; and described best practices in communication. After a lively discussion, groups shared their responses and the notes that were generated were distributed within days of the meeting. The future seemed assured!

It�s now August of 2003 and the regions are just beginning to understand the benefits of communication. The partnerships that have developed, the money that has been saved, the personal opportunities for growth, and the organizational benefits are just now being realized. If it weren�t for EETAP, the EE providers within the state of Illinois would still be functioning in a vacuum. Fortunately, communication is improving between and within regions and that coordination cannot be far behind!

Following the recommendations of the 1994 Governor�s Task Force on EE, the EETAP States Project allowed Missouri to initiate projects to improve EE. One of our successes is our newly developed EE certification program that was launched in November of 2001; the following year we awarded our first certificate and will award 2 more in 2003. Currently 100 educators are enrolled. This equates to about 1/3 of the MEEA membership. The program has great potential growth as inquiries from new enrollees come in almost daily and new workshops are being approved and added to the program on a regular basis.

There are several things that make Missouri�s EE Certification Program successful. We looked at what Missourians said they needed, reviewed what was already in place in the state and
observed what other states were doing. Once we found a model, we adapted it to fit us. Missouri already had several partners providing quality EE programs. The certification program
provided a coordinated system that enabled participants to enroll in quality EE programs throughout the state. The certification
program not only requires enrollees to gather new knowledge and skills but also to put what they have learned into action in the classroom and their communities, further strengthening EE in MO.

Another piece of the EETAP program in Missouri that we are especially proud of is the Community Based Leadership Clinic held last April. Thirteen teams from across the state participated. Team memberships were diverse ranging from classroom
teachers and school administrators to community leaders and interested citizens. Some teams worked on action plans that would enhance their outdoor classrooms or their school curriculum while others worked on community betterment projects such as litter clean-up or improved community image. The amazing aspect was the networking that went on during the various sessions. The Clinic brought together people that the state EE association had not reached in the past, many of whom did not realize they were "environmental educators," and we all left with a better sense of community and partnership. Through follow-up efforts with these teams, we are looking forward to hearing success stories from across the state.

This past spring, the Environmental Education Council of Ohio (EECO) Executive Director and Ohio EETAP States Program project manager, Teresa Mourad, accepted the Operations Officer position with the North American Association for Environmental Education (NAAEE). Congratulations Teresa on your new position! Teresa made a real difference for EE in Ohio during her tenure there and will no doubt do the same with NAAEE. EECO is in the final stages of filling the Executive
Director position and looks forward to continuing their involvement with the EETAP States Program.

Oklahoma
Oklahoma Is More Than OK

It is hard to choose only one event that has stood out from our EETAP States Program Grant project. The process of building the OCEE Steering Committee from a diverse group of community
leaders has been a rewarding experience. The Steering Committee provided the foundation for the development of the Leadership Clinic held June 11-13, 2003 at the Oklahoma City Zoo. It took a year of planning to bring together community teams from across the state. Each team came with a particular community environmental education issue that needed to be addressed. Some teams were dealing with long-term concerns while others had recently formed to deal with current issues.

It was the participants� responses during the culminating activity that confirmed the value and impact of the clinic. The closing was patterned after the one we had experienced in Nebraska the summer of 2002. Each team member was asked what he or she was going to "walk away with" from the Leadership Clinic. Many expressed the feeling of support that they had received from each other, the
resource people, and the facilitators that strengthened their resolve. They were heartened to know that they were not alone and now had a network to call upon. Some felt renewed because they had learned new skills and strategies and now had a realistic action plan to implement when they returned to their community. Everyone was inspired by each others� stories and resolve to help their respective community and through their mutual effort to foster the development of an environmentally literate citizenry through their varied action plans.

Utah
Making EE Legitimate and Professional

While working with the EETAP project, a development team, comprised of environmental educators from around the state, designed a state-based EE Certification Program. The project evolved from a request by the Utah Society for Environmental Education Program Advisory Council (USEE PAC) to legitimize and professionalize EE in the state. The Utah EE Certification program consists of five themes in which environmental educators will become certified. Participants will go through the program working with a mentor to strengthen their knowledge and skills in: Foundations of EE; Environmental Literacy; Planning & Implementing; Professional Responsibilities; and Assessment and Evaluation. The program will be launched this fall, with several environmental educators excited to enroll.

The EETAP work in Washington State culminated in a three day institute in July of 2003 bringing together Deans of Schools of Education, Professors of Teacher Education, pre-service teachers, in-service teachers, superintendents, principals and non-formal environmental education providers. This group developed the work of the next 2 years, which is to form three major Hub�s connecting schools, non-formal EE providers and teacher preparation programs. At the core of each Hub is a
demonstration school district whose superintendent and principals have committed to making Washington�s environmental education benchmarks an outcome for learning at each grade level, K-12. They have also committed to using the standards based environmental education tests designed from and based upon the EE benchmarks. The Western Washington Hub will focus teacher preparation programs on the Tahoma School District, noted for its academic rigor and excellence in implementing education reform. The Central Washington Hub will focus local teacher preparation programs on the Cle-Elum-Rosslyn School District beginning its third year of Project CAT, integrating EE learning through the essential question, "Through a cougar�s (CAT�s) eyes, how is Cle-Elum-Rosslyn changing". The Eastern Washington Hub is now actively focusing on the West Valley School District, assigning pre-service teachers to be mentored in EE classrooms. The Environmental Education Consortium of non-formal providers (public and private) and education institutions (representing Deans, Superintendents and Principals) will provide professional development and support to schools undertaking this whole district integrated environmental
education approach. Our goal is to add more districts each year through this intentional and academically rigorous process.

As a result of recommendations made in the Year 3 evaluation of EETAP, the Year 4 evaluation will place a stronger emphasis on evaluation of individual activities. This is a departure from the evaluations of EETAP done in previous years. For example, the Year 1 evaluation focused on communication and Year 2 looked at capacity building. While the Year 3 evaluation looked at each EETAP activity, the review was done using existing documents such as progress and self-assessment reports.

By contrast, in Year 4 partners and EETAP States will incorporate evaluation as a part of implementing their activities. Funds were allocated to assist with the cost of evaluation and EETAP retained the technical assistance of Dr. Michaela Zint, Associate Professor of Environmental Education & Communication at the University of Michigan�s School of Natural Resources & Environment.

The evaluation process started with the design of activities and state scopes of work. Partners and EETAP States were asked to write meaningful and measurable outcomes and to include a plan for evaluating at least one outcome. The evaluation consultant will review all the outcomes and evaluation plans and provide suggestions for changes that will improve the outcomes and evaluation plans.

Once changes are agreed to and the outcomes and plans are finalized, partners and EETAP States will begin implementing their activities. Dr. Zint will assist Partners and States with the many facets of evaluation as needed.

Partners and EETAP States will then write evaluation reports as part of their end of year progress report. Dr. Zint will use these reports to make conclusions and suggest changes for Year 5.

The final step in the process is to use the results to improve EETAP�s activities in Year 5. Without this step the work up to this point is simply an exercise. According to EETAP Director, Dr. Richard Wilke, "the real value of evaluation is the information and insights gained that are used to improve the quality of our environmental education programs."

Chaparral Community Development Association (CCDA) has found a special niche that will protect the community�s groundwater and help bring in $40,000,000 for a waste water treatment plant. Through a series of meetings CCDA has reinvigorated the county�s efforts to fund, design, construct, and manage some very needed infrastructure. CCDA is coordinating, researching and providing the editorial services to create 5 bilingual inserts on the technical, environmental and health, economic, and public issues related to the construction and maintenance of the plant for its local paper. This information will be a critical piece to ensure the community�s "buy in" for some unique funding for disadvantaged communities along the border. Way to take action Chaparral!

The Oklahoma City Demonstration Community Team made big strides in planning and implementing activities this spring and summer. In May the team held a Project WET workshop attended by 30 teachers and school administrators. In June the team attended the Oklahoma State Leadership Clinic conducted by members of the Oklahoma EETAP States group. At this state clinic the community team developed detailed action plans which they continued working on at the 2003 EETAP Leadership Clinic in July.

In September the team held the First Annual East Side Environmental Education Expo featuring EE providers including Project WET, the Oklahoma City Zoo, Staples Recycling program, health and fitness educators and many more. Teachers who were trained in May brought their students who presented water education projects. In addition to school focused programs, the Oklahoma City team wrote four articles about water quality issues that were published in their local paper, and conducted a tour of superfund sites with civic, community, business and religious leaders. Connie Johnson, project leader, says "we are learning that community members are aware of the issues that impact the local environment, but don�t know what they can do to resolve them. This is the next step in our education effort."

From the inception of Orange County�s EETAP Demonstration Communities project it became clear that bringing together the Orange County Water District Groundwater Guardian Team and Orange County�s CREEC (California Regional Environmental Education Community) Network provided an excellent opportunity to build capacity for environmental education in Orange County. Our greatest accomplishment has been to successfully implement a collaborative model Stormwater Pollution Prevention and Education project for a school campus that: 1) utilizes the diverse technical expertise available within these two groups, 2) clearly addresses a community environmental issue, and 3) results in academic gain for students.

The Environmental Education and Training Partnership (EETAP) has redesigned it�s website (www.eetap.org) which offers educators information, resources, and links for promoting academic achievement and environmental literacy. A new feature of the site under resources is the on-line version of EETAP�s CD-ROM "Meeting Standards Naturally". Users can view the CD-ROM, download the on-line version, or request a hard copy from EETAP. The site now includes a link to, "Advancing Education through Environmental Literacy," a publication produced by the Association for Supervision and Curriculum Development in collaboration with EETAP. Users can read the publication on-line or order it from EETAP.

Education Program Evaluation (OUT8102) is a 4 day course offered by the U.S. Fish and Wildlife Service at the National Conservation Training Center in Shepherdstown, West Virginia January 27-30, 2004. The course provides an overview of conducting evaluations for education and outreach programs and a chance to practice skills in designing and using evaluation tools for surveys, interviews, and observations. Instructors are nationally known leaders and practitioners in the education evaluation field. Contact Georgia Jeppesen, Course Leader at eorgia_jeppesen@fws.gov or 304-876-7366 for an application form, additional information on cost, credit, and course objectives, etc. Information and registration is also on-line at http://training.fws.gov/courses.html

The EREC Program seeks proposals that offer unique approaches to evaluation practice in the generation of knowledge for the science, technology, engineering, and mathematics (STEM) education community and for broad policymaking within the research and education enterprise. The annual deadline is May 15th.

The ROLE program�s focus on research complements the EREC focus on evaluation. ROLE seeks to capitalize on important developments across a wide range of fields related to human learning and to STEM education. The annual deadlines are December 10th and June 1st. For more information on these grants, go to www.nsf.gov/pubs/2003/nsf03542/nsf03542.htm.

Information on other grants offered through the National Science Foundation can be found at www.nsf.gov/pubsys/ods/index.html by clicking on the "Program Announcements and Information" link.

The American Physiological Society offers the Planning An Effective Program Evaluation interactive short course either on-line or on CD. This course includes six lessons that lead the user through readings on evaluation basics, questions raised by program directors, and resources available both on and off line. A listing of print and on-line resources for planning and implementing a program evaluation can be found at www.the-aps.org/education/promote/links.htm and is updated regularly.

Drop everything, take this newsletter to the nearest computer and type in www.ucp-utica.org/uwlinks/outcomes.html! You�ll find a 7-page listing of Internet resources for nonprofits that deal with "Outcome Measurement and Program Evaluation". Slightly larger red titles and those labeled "Best Bet" are a great place to begin.

This site is courtesy of the United Cerebral Palsy and Handicapped Persons Association of the Utica Area.

Join us October 7-11 in Anchorage, Alaska for the 32nd annual NAAEE Conference. NEEAP is promoting some of the sessions which focus on EETAP-related capacity building activities.

Three community teams have worked with partners to increase local EE capacity through EETAP�s Demonstration Communities Project.  Team members from Oklahoma City, OK; Chaparral, NM; and Orange County, CA will present steps they�ve taken to build mutual trust, respect and plans with community members and partners from outside the community.  (Oct. 11th, 1:30-4:15pm)

State EE leaders convey their experiences with building capacity through the EETAP States Program. Learn what has helped and hindered their efforts to create more effective environmental
education. Participants will interact with state and national leaders to explore EE goals and implementation strategies applicable to their own state/province. (Oct. 9th, 9:15 am-Noon)

13 community-based teams came together in a participant-driven leadership clinic where they gained skills-training and resources needed to help develop EE action plans for their communities. Find out how the diverse makeup of teams and array of projects successfully extended the reach and benefits of EE in Missouri. (Oct. 11th, 2:30-3:15 pm)

EETAP is a consortium of ten organizations that are working together to build EE capacity at the national, state, and local level. Find out how this consortium functions and obtain an update of the project�s achievements in several key areas including the EETAP States Program, certification, NCATE and nonformal guidelines. (Oct. 11th, 1:30-2:15 pm)

EE providers of natural resource education in Arizona are partnering to �work smarter, not harder.� All "Projects" - WET, WILD, PLT, FLP, Out of the Rock, Energy - have combined to present one-day and one-week workshops for educators. Workshops are standards-based, integrated, and multi-curricular, leading to one Project POWER. (Oct. 9th, 9:15-10 am)

Are you a state or federal agency professional? Do you work at a zoo, museum or aquarium? Do you work in community settings or specialize in adult or family education? If you are an environmental educator in any of these situations, then join us to develop your EE competencies and share trials, tribulations and successes.  (Oct. 7th, 7-9 pm & Oct. 8th, 9am-5pm)

In 2002, NAAEE became a member of the National Council for the Accreditation of Teacher Education (NCATE), the body that provides national recognition of quality teacher education programs in the United States. This session will provide an update of the work of NAAEE over the past year in its involvement in the NCATE process. (Oct. 11th, 3:30-4:15pm)

Learn about EETAP�s on-line Fundamentals of EE course offered by UW-Stevens Point. You will learn about course content, results of student and instructor evaluations, and efforts to encourage other universities to offer the course to their students. Provide your ideas on how to expand the number of educators reached through this course. (Oct. 11th, 9:15-10am)