Pre Symposium Workshop

Evaluation Summary

There were 17 participants in the pre-symposium workshop, the majority of whom were working in Regional Tobacco and Healthy Lifestyle Teams. They commenced the day by writing a personal response to these two questions: What do you hope to gain from today? How will this make a difference to your program? Their responses addressed both questions. The main theme was learning more about different aspects of evaluation to apply in their programs, e.g. from planning through to implementation, analysis and reporting. However, another prominent theme was the value they hoped to gain through networking and sharing knowledge with participants from other Regional Teams about how they were implementing their programs. In a few instances, participants explained their intention to use their evaluation outcomes to be accountable for their work, and send clear messages to funders about the value of it. At the conclusion of the workshop, participants were invited to revisit their responses and assess how well their hope had been met in relation to both questions. They did this using a 5-point Likert scale with the following rating options: 1 = not at all, 2 = a little, 3 = somewhat, 4 = quite well, 5 = very well. Several people needed to leave the workshop a little earlier than planned, so the ratings represent 11 of the participants (65%). The results are shown in Figure 1. The evaluation process was run as a group rather than an individual activity, where everyone witnessed the ratings chosen and people were invited to explain the reasons why they chose their ratings to the whole group.

Personal Hopes

In relation to personal hopes about what they would gain, all but one participant indicated that the workshop met their hopes either ‘quite well’ or ‘very well’. Using the values applied to the different ratings, the average rating was 4.2. The main drawback that resulted in slightly lower ratings was having less opportunities to network with colleagues from other Regional Teams than initially hoped, which reflected the high regard and value participants placed on this. This was due to the majority of workshop activities requiring participants to work with colleagues in their team as they applied the planning and evaluation concepts to their local action plan. However, it was clear from participants’ comments about their choice of rating that all participants felt they had improved their evaluation knowledge and skills. The following quotes reflect comments that were commonly shared against the different ratings: “I learned a lot, just with the extent of the content I zoned out a bit, but I did get to break down the program I brought to the table…Maybe more group interactions that weren’t with our group.” (Rating = somewhat met) “I gained more understanding of effective measurement of our program, and got to know definitions of objectives and goals.” (Rating = quite well met) “The thinking on strategies and structural framework was different. It was like a benchmark way of reporting, good to learn that. I will go back with the team and do our work back in the community but knowing the differences in definitions.” (Rating = quite well met) “We have the opportunity to hear about what other people are doing because we are starting [our program] from scratch. We have the opportunity to put this in place from the beginning. It will be beneficial, a bit of an advantage for us.” (Rating = very well met) Figure 1: Participant ratings – how well were your hopes met? (n = 11)  

Will Your Learning Make A Difference To Your Program?

Participants expressed high confidence that their learning would make a difference to their program, as all chose either ‘quite well or very well’. Based on the values applied to the different ratings, the average rating was 4.5. They believed they had gained transferable skills that could be taken back to their communities for both their current or other programs. The concepts and tools presented during the workshop would enable them to set up more effective evaluations based on identifying specific indicators that set the criteria for real progress, and secure future funding. “This will help make our pilot program outcomes more presentable to the Board, and come up with more specific indicators. It will be easier to get funded.” (Rating = quite well) “Great to learn terminology for our team, help with accountability and equitable program and refunding purposes. It gave us a tool that I will use positively back in the region. We were really lucky to have had our entire team. We were enthused and had strengths in different areas across the day. It would be good to do it over two half days.” (Rating = very well) “With the cultural environment [we work in] I was able to realise the effects of colonisation are international and the way to address those effects is to use the mainstream tools provided to feed the information back in a way that they [funders] can digest, and fits their criteria, we can use their tools against them.” (Rating = very well)