Shadow assessment occurs when programs focus on what they think other people want to hear, as opposed to what is practical and meaningful for them. There are many reasons why programs engage in shadow assessment:
– Fear of assessment and evaluation.
– Lack of time, energy or motivation.
– Fear that assessment results will be used against someone, or to prove someone else wrong.
– Fear resulting from a dysfunctional or toxic work environment (see Where Assessment Works Best).
– Fear of having honest conversations about a program, usually leading to cynicism and skepticism.
With the exception of one item, the main theme running through shadow assessment is fear. This has almost everything to do with the organizational climate and culture of a program or institution. If a culture or climate is toxic, then the result is that most people are going to engage in shadow assessment, and just communicate what they think others want to hear. Because assessment is so dependent on genuine and civil conversations, I am not sure there is a lot one can do to foster a culture of genuine assessment.
In most programs and institutions, however, a culture of genuine assessment can be fostered. Genuine assessment is honest, transparent, and used for program self-reflection and improvement.
Note. Being genuine is not the same as being transparent. People get them mixed up all the time. Being transparent is about posting items to a website, message board, or sending an email. However, it’s a transparent conversation, not a genuine one. This is because it’s not a dialog – it’s a two-way monologue. Sharing assessment results on an online website may be helpful in terms of access to information, but it’s not the same as sitting down with someone or a group and engaging in a real conversation about program improvement. There is a significant difference between what the data are and what the data mean.
Institutional leaders and colleagues can do several things to foster a culture that values genuine assessment instead of shadow assessment.
- Reinforce the idea that assessment is about improvement first, and compliance second. In my experience, if a program is doing a good job with assessment anyway, they shouldn’t have to worry about compliance.
- To paraphrase George Patton, tell people what to do, not how to do it. They will surprise you with their creativity and ingenuity. Granted, there are best practices in assessment. And, there are a lot of great methods out there. Many programs are doing very creative things in regard to assessment. Some programs, particularly ones that are less mature in regard to expertise with assessment, will probably appreciate some kind of framework. Most programs, however, will naturally resist a process they feel like they have little control over.
- With responsibility, comes authority. If programs are going to be responsible for assessment, they need to feel like they have authority over the program. The worst situation arises when programs feel like an authority has forced an evaluation process on them, but the authority absolves themselves of all responsibility, which is pushed down to the program-level. In this case, programs should feel like they have support that will help share the responsibility.
- Resist commenting on the substance of a program. We cannot possibly be experts in every discipline. Expertise needs to be respected. Leaders and assessment professionals should remind programs they are only there to communicate institutional expectations and provide support, not judge the substance of a program. (Note. This may be different in program review, where budgeting and planning decisions need to be made. The point is to be clear about expectations and the purpose of the evaluation).
By focusing on support and professional development, as opposed to compliance and rewards, one can do a long way towards fostering a culture that discourages shadow assessment.