On March 10, 2023, the Alberta Mentorship Program hosted an online Community Champions Mentorship Circle about monitoring and evaluating mentorship programs. Rosa Martinez, Mentoring Partnerships Coordinator at the Calgary Region Immigrant Employment Council (CRIEC), led the conversation.
Rosa is an experienced adult educator with a background in career advising, e-learning, workplace learning, and change management. With over ten years of experience in the non-profit sector, Rosa has worked with internationally trained professionals in different capacities, including as a facilitator, mentor coordinator, career advisor, employment coach, and program designer. Over the past few years, Rosa's professional interests have broadened into learning and development supporting mentoring programs. Rosa's approach to developing these initiatives uses systems thinking, collaborative leadership, and workplace learning.
On Linear Versus Non-Linear Thinking
“Sometimes you see mentoring or clients in the form of one linear line, from A to B,” Rosa explains. “They come and join mentoring; they have a mentor, and boom, they get a job."
The linear version considers each element in order, from beginning to end. This is orderly but can be limiting because it provides only one pathway through the mentorship program. In non-linear or systems thinking, everything is part of a bigger picture and there can be different pathways through the program including different starting and end points. This is more complicated but can provide more opportunities to meet mentor and mentee needs. This affects how an organization engages with all the elements within a program.
“In non-linear thinking, you can start anywhere. Sometimes mentorship may begin because of a need in the community, so one organization took the lead and got funding, community support, mentees and mentors. Or sometimes, it may start because funding becomes available, says Rosa.
A non-linear perspective allows CRIEC to find the interconnectedness among these elements and how they can work together within CRIEC and with other organizations. Ideally, this broader viewpoint will show how a mentee, mentor, employer, or funder experiences the program from different perspectives and how those experiences interact.
“Each perspective will give rise to a different journey," says Rosa. Understanding these differences will help organizations meet the needs of all their stakeholders.
“That linear approach is nice and clean and comfortable,” Doug Piquette, Executive Director of the Edmonton Region Immigrant Employment Council (ERIEC), adds. “But as you immerse yourself in this kind of work, you realize it is non-linear."
But this can be challenging to explain. Usually, funders prefer to see programs in clean linear pathways and boxes. It can even be uncomfortable for organizations to shift to this kind of thinking. But if they are willing to work through the discomfort, non-linear thinking will help organizations adjust to unexpected changes.
“It is messy!” Rosa says. “Sometimes, it goes in one direction and comes out another; these patterns are always in constant dynamic connection. Like last year, for example, all of a sudden, fewer clients! But it was not our fault; it was because of COVID restrictions. That’s not something you can predict on a linear model. Or, suddenly, there’s new funding that’s coming! How do we pivot to use that or demonstrate that we can use that funding? It is always very dynamic, and it is always very messy. Clients also tend to prefer a linear model. They want something fast and easy so they can start working. But nothing is fast and easy!”
Monitoring and Evaluation
So in this messy world, what information do you collect? How often? Who collects the data, and what do you use?
“With our funders, we have to count numbers, but that doesn’t reveal the whole story,” shares Doug. “When you’re immersed in this kind of work, you deal with the complexity of human personalities."
ERIEC ensures they collect the numbers and the data required for funders and other stakeholders. But they also collect stories and experiences through testimonials and anecdotes. Both forms of data are necessary to run an effective program.
"There are many other ways to ‘move the needle,’ such as the mentee or mentor's self-awareness, job preparedness, and intercultural dynamics. We see the changes and know what’s happening on the individual level," says Doug.
What is evaluation?
Evaluation is the process of determining the value or worth of a program. It is systematic, objective, and part of a planned, ongoing or completed intervention. They appraise data and information that will inform strategic decisions, hopefully improving the project or program in the future.
What is monitoring?
Monitoring is systematic and routine information collection to learn and improve practices and activities, provide internal accountability, make informed future decisions, and promote the empowerment of beneficiaries. Progress is checked against plans, and the data acquired is used in evaluation.
How does monitoring fit into the non-linear systems view of mentorship?
“Monitoring happens at all different points and stages along the mentorship journey,” Rosa says.
There are formal check-ins with mentees during mentorship to ensure the program can course correct and meet the mentees' needs during the program. After the mentorship ends, CRIEC completes three and six months follow-up check-ins to demonstrate accountability to funders.
CRIEC doesn't use formal check-ins with mentors. Instead, they ensure that there is regular informal communication so they can listen and respond to their needs constantly. Mentors are offered development opportunities and invited to a mentorship group that provides feedback to CRIEC on keeping their mentors engaged.
CRIEC collects Activity Output Targets, the quantitative data they provide funders. They set realistic, achievable targets or goals throughout the year and monitor and evaluate how they are doing against those targets.
“For the success of the mentoring, we must engage other parts of those messy systems, like mentors, mentees, funders, organizers, and more. Just like the non-linear model of thinking, monitoring is a continuous cycle with no beginning or end.
“We are always adapting, assessing, learning, and designing. All the time,” says Rosa. “It’s a moving system; when working in the field, we are responsive to needs as they emerge. From a systems perspective, whatever we communicate to management makes a difference to what we receive back.”
How do monitoring and evaluation come together?
Rosa recommends creating an M&E (Monitoring and Evaluation) Plan that will help you start and continue monitoring and evaluating your program. Using this plan, CRIEC can assess and collect information and self-correct at each stage of the program. This plan can consist of the following steps:
- Identify your evaluation audience (i.e., government funders, board, participants).
- Define your evaluation questions.
- Identify monitoring questions.
- Identify indicators and data sources (i.e., surveys, focus groups, interviews).
- Identify who will conduct the monitoring and evaluation.
- Identify the timeline (i.e., program start, end, year one).
Monitoring and evaluation become a continuous process at every level. They are used to assess if progress is being made toward targeted outcomes, to spot if implementation bottlenecks exist and how to correct them, and to highlight any unintended effects from the program or its activities – positive or negative. You cannot wait until the final report is made to assess these things. You must do it continually to achieve the best results.
The Alberta Mentorship program is sponsored by the Edmonton Region Immigrant Employment Council (ERIEC).
The Alberta Mentorship Program appreciates the funding from the Government of Alberta through Labour and Immigration Workforce Strategies. Our program is here to provide information and support to help organizations start mentorship programs.