When Bad Assessment Happens to Good Student Affairs Assessment Professionals

Recently, I’ve conducted some bad assessment.

Two surveys on different occasions in which there were issues that any basic assessment professional should catch.

I’ve let a survey go out that has double barreled questions!

That same survey had political implications, asking controversial questions within a tradition laden population. It wasn’t for work or for formal research, I reasoned. So, I didn’t take the time to do the survey right and get people bought into asking the questions.

Another survey was pretty well done and had many eyes on it, but I realized the challenges of administering a home-grown survey using both paper and electronic modes. There were issues printing at natural question or page breaks. I printed eight versions before I got it perfect. Then I used print job seven for making copies and provided participants with an instrument in which three questions were hard to read because the words were cut in half – if only I had that extra .25 of an inch on the sheet.

Reality is that even I – someone who does assessment, and likes to think he does it well – can have those bad days. Even the best at developing instrumentation and protocol should involve others in the review and development. It is vital to examine political implications of assessment before “just putting an idea out there to see what people think”. No! Stop! Don’t do it. Don’t just put the survey out there just to see what they say. There are implications. Think about whether the implications are worth handling after the survey is sent out (even to a convenience sample of friends on Facebook).

It reminded me that we all have to be working toward maintaining and increasing our competence and confidence daily. We can backtrack. We can let things slide. Designing a poor survey likely won’t end up being the biggest tragedy around but there’s implications that must be addressed. I should have known better. People expect more from me.

What are the skills you need to improve/maintain that will make a difference in how others perceive you?

What are the mistakes you’ve made when aiming to do something you’re good at but ultimately you failed miserably?

2 thoughts on “When Bad Assessment Happens to Good Student Affairs Assessment Professionals

  1. Happens to the best of us, Dan. I’m an external evaluator on an NSF grant and just last week I had to tell the Principle Investigators that we couldn’t provide much data on one of their outcomes for a NSF-mandated report because I screwed up one of the response options on a post-workshop survey. It’s the first of several reports for a multi-year project and we have other relevant data so it’s not a fatal mistake but it’s embarrassing and another (unnecessary) reminder that mistakes happen.

    • Truth Kevin Guidry! Mistakes happen. In both cases I could have avoided the mistakes, but the crux of the post was that we have to be conscientious about our work. We don’t need to jeopardize the trust people have in us simply by running on autopilot or taking short cuts. Thanks for reading.

Leave a Reply

Your email address will not be published. Required fields are marked *

*
*