I have lived here in New Hampshire for the past 16 years when I came to be a hall director at the University of New Hampshire after finishing my master’s degree from Michigan State University in 1993. I am originally from Michigan and my parents still live there. My partner and I get back to visit them at least once a year. This past December was a “Henning Holiday” meaning that we would spend it with my family in Michigan. We were excited since my sister, brother-in-law, and nephew from Seattle were going to be at my parents’ house as well. 

As we began planning our trip, I began thinking about it as having an outcome – an outcome that impacts planning and assessment. As I have said in the past, assessment is not an activity. It is a state of mind – one that I always seem to be in. (Visit http://www.studentvoice.com/app/Blogs/Gavin/post/Assessment-is-a-State-of-Mind.aspx for that discussion.) Like developing outcomes and the strategies to promote them to help our students learn, thinking about our trip to Michigan helped us understand “where we were going, how will we get there, and how will we know when we arrive.” 

You may define some of the terms I use differently than I do, but I think you will get the gist of the analogy so that you can share this with others to educate them about outcomes.  (Visit http://wwwstudentvoice.com/app/Blogs/Gavin/post/Inputs2c-Outputs2c-OutcomesOh-My!-Nuances-of-Assessment-Language.aspx for a greater explanation of each of these terms.) 

So, in December, our goal was to drive to my parents’ house for a visit.  We had some inputs for the trip which were the raw materials to help us achieve our goal. The inputs included the car to get us there, the money to purchase the gas to fuel the car, snacks to keep us fueled, money for tolls, and the like. There were also outputs that served as quantitative metrics to document progression towards the outcome. These outputs included the total amount of time we took or the money that was spent on food, gas, and hotel during our trip (resources used rather than the resources available when we started), the average miles per gallon of gas used and the average miles per hour we traveled (efficiency of resources used), and the number of speeding tickets received (effectiveness of resources used). By the way, we didn’t get any speeding tickets this time. I learned my lesson from the Pennsylvania State Police on a previous trip. 

Now the outputs are helpful in understanding our trip. We can say how efficient and effective we used our resources, but we can’t use the outputs to help us determine if we met our goal or not (getting to my parents’ house). This is similar to how tracking the number of students that attend a program or aggregating their satisfaction with that program, while helpful, doesn’t tell us if they learned anything from the program (which is the ultimate goal).  

Our outcome is our desired result. For the trip to Michigan, the desired result was reaching my parents’ house. But this isn’t even good enough. Outcomes have to be more specific. As the intelligent folks at the University of Central Florida in their program assessment handbook say, outcomes should be SMART – Specific, Measureable, Aggressive, but attainable, Results-oriented, and Time-bound. So, our outcome could be more effectively written this way, “As a result of our trip to Michigan, Gavin and Terri will arrive at 12 Round Lake Road, Saginaw, MI at 10pm on Saturday, December 20^th.” The timing was important because we wanted to make sure we were there in time for my father's birthday on Sunday the 21^st. The specificity of the exact address of my parents’ house and the arrival time are important so that we could plan our trip and assess are accomplishment of that outcome.  

Once we had our outcome, inputs, and outputs, we needed strategies for reaching our outcome. Here is where specificity of the outcome was important, again. 

There are two major ways to get from where I live in Concord, NH to Saginaw, MI with some minor variations on these. For the sake of illustration, we will just focus on these two routes. Route 1, the most direct route, is west across New Hampshire, through New York state, then through Canada. Here is a map for folks like myself who are visual processors.              

Route 2 runs south through Massachusetts, Connecticut, and then west through Pennsylvania and Ohio and north into Michigan.                   

Route 1 takes about 15 hours from Point A to Point B while Route 2 takes about 17 hours. The natural inclination is to take Route 1 because it takes less time.. However, this is the moment where considering the strategies for reaching your outcomes are really important. While Route 1 is shorter, it actually takes more effort. Route 2 is almost entirely comprised of major interstate highways from New Hampshire to Michigan. Route 1, on the other hand, includes some winding, two lane highways, through New Hampshire and Vermont. While beautiful during the day, these country roads can be tricky at night and in the winter.  

There is also the issue of Buffalo, NY. I like Buffalo. Some of my good friends live in Buffalo. But, have you been to Buffalo in the winter? From my experience, you never know what you are going to get weather-wise. The rest of the state could be sunny and clear and at the same time there could be a nasty snow squall right over Buffalo. You just have to be prepared for that type of adventure – buckle-up, turn on the wipers, white-knuckle the steering wheel at 10 and 2 and let the fun begin! 

Another issue with Route 1 is that you have to go through Canada. Not that Canada is bad. It is a fantastic country and you should visit if you haven't been, but you just have to be more prepared since you will need a passport. Also, there used to be stretch of roadway going through Canada along this route where there was a vast expanse of nothingness. It was four hours without a place to stop for food, gas, or nature’s calling. With Route 2, you are guaranteed a nice service area almost every 45 minutes for all of your traveling needs.

Route 1 is faster, but you have to plan each step of the trip, or strategy, more carefully. 

Our choice was to be safe and take Route B. This choice had implications for our planning and implementation given the outcome we developed. In order to get to Saginaw by 10pm on Saturday, December 20^th and be as efficient and as effective with our resources (read: no speeding tickets which actually increase the cost and time of your trip) we would need to be on the road by 7am. This would allow for two stops for meals, gas, and bathroom breaks and three additional gas/bathroom breaks. We try to maximize our stops by accomplishing at least two of these tasks per stop. (Remember I said assessment and planning are a state of mind for me.) 

With this information, we can not only assess the achievement of our goal, but how effectively and efficiently we reached it which would allow us to plan better for our next trip to Michigan. Knowing where we are going (and when) can help us determine which road to take and allow us to determine if we have arrived (or met our goal). 

As you can see, developing a specific outcome impacts the process for selecting the strategies to support that. This is just what we do with our work with students. Once we determine a goal (e.g., develop leadership skills), we develop specific outcomes (e.g., as a result of the Emerging Leaders Course, students will be able to effectively facilitate a student organization meeting by the end of the course). We can then consider our inputs (e.g., staff and financial resources) and develop our strategies (e.g., identify three best practices for facilitating meetings) to foster that outcome. We can then measure our outputs (e.g., number of students in the class) and summative outcome (e.g., the extent to which students can effectively facilitate a student organization meeting) and make decisions for improvement.  Following this process keeps goals, inputs, outputs, outcomes, and strategies aligned increasing our efficiency and effectiveness and allowing for easier assessment. 

Apply this analogy to your next trip and let me know if assessment doesn’t start becoming a state of mind. 

References
University of Central Florida. (2004). Program assessment handbook. Orlando, FL. Author. 

Hello.

Thanks for taking the time to read my blog. It is my first. And to be honest I was a little nervous when I was asked to do this. I hope you find the content useful.

It might be helpful if you knew a little bit about me. Currently, I am the Director of Student Affairs Planning, Evaluation, and Research at Dartmouth College. I have been working in student affairs assessment full-time now for over eight years. Before that I spent seven years working in residential life but along the way I gained experience in judicial programs, diversity initiatives, leadership development and a few other student affairs areas. I love this kind of work because it satiates my curiosity as well as my need to do things the best way possible. The other exciting element of this job is I get to work with a variety of functional areas within student affairs so I am always learning something new. I also have the pleasure of serving as the Chair for ACPA’s Commission for Assessment and Evaluation.

Since I have been doing assessment work, one the biggest challenges is the time commitment needed. I often hear things such as “where I am going to find the time to do that. I don’t even have enough time to breath, let alone do assessment” Or, “my staff just doesn’t have the time. They are busy addressing student needs. Assessment takes them away from student contact.” Yes, assessment takes time. But, you have to look at it as an investment of time that will save you resources at the end. Just like saving for retirement, if you can invest a little bit now, the payoffs will be worth it. Also, assessment does help students directly since you and your programs and services are better able to foster student learning and meet their needs.

Here is an analogy that might be helpful for you and your staff as a way to overcome this time barrier. Have you ever been to one of those arcade pizza places like Chuck E. Cheese? They are the wonderful places with adults eating pizza while children run around the place jumping in piles of plastic balls, riding indoor merry-go-rounds, and spending tokens upon tokens on all sorts of arcade games. Well, one of the games many of these arcades have is the timed basketball game. You’ve seen it. It is the game with the basketball hoop about six feet away. There is netting or Plexiglas on the sides so the ball doesn’t bounce all over the restaurant after you miss the shot. You get three balls and the object is to get as many balls in the hoop as you can in the time allotted, generally one minute. Sometimes, the stakes are raised and there may be two hoops next to each other so you are not only competing against the clock but a real live person. The stakes become even higher when a 40-year old person like me has to compete against a 10-year old who is much more skilled and agile at these types of games. Depending on your score, you could even earn tickets that can exchanged for inexpensive rubber toys, stickers, and stuffed animals, that you just “have to have” before you leave.

Well, think about students affairs like this basketball game where getting a high score is equivalent to getting our jobs done well. We are trying to do a lot of things (read shoot many basketballs) with very little time (read a clock running down) and we hope that some of what we do hits the target (read fostering student development and learning).

The basketball game sets you up for ineffectiveness from the very beginning, the same way we are set up in student affairs – with a lot to do and not enough time. The clock on the game is large and the numbers are red. The game designers probably say red is easier to see, but think about the symbolism of red. It often means “stop” or “danger.” I think the red clock is a way to intimidate us. The next element of the game that set us up is that there are three balls waiting in the trough at the bottom of the game. Picture this, there is a large digital clock with red numbers staring you down and three balls that are screaming “shoot me.” The natural instinct is to start with one ball and toss them towards (not necessarily into) the basket as fast as possible so your arms look they are attached to the paddle wheel of a steamship, going around in a continuous motion until the game stops. Our hope is that the faster we shoot the basketballs, the more basketballs we shoot, and then the better chance we have of getting some through the net. We take the perspective that this is a game of chance and we shoot faster to increase our chances.

This is often how we operate in student affairs. There are too many things to do in very little time. When we try to implement strategies, which we aren’t sure will work, to foster student learning, it is like tossing those balls as fast as we can in hopes that some will go into the basket. We hope that if we try enough strategies some of them will work for some of the students. All the while we are looking at the semester clock wondering how much time we have left dreading that fateful buzzer called finals or perhaps graduation. This approach isn’t a very efficient or effective use of our resources.

There is another way. Assessment can help.

Imagine taking a deep breath before you even put the tokens into the machine. There is no pressure at this point. No money has been spent. During this breath you make some assessments: How far away is basket? Is the basket regulation size? Once you add your tokens, rather than simply shooting as fast as possible you take a couple of extra seconds and observe what is happening. Now that you can actually touch the basketball you assess it. Is it regulation size? How much air is in it? Once the ball hits the backboard and rim, what happens? Does the ball bounce far away from the rim or backboard because there is too much air in the ball, or the board or rim is “hard”? Is a “swish” the only the ball can get into the basket because anything that bounces off the rim or backboard doesn’t have a chance to go through the net? You can make these observations in a few quick seconds. Once you have made these observations, you can adjust your shooting to meet the needs of the environment. This is assessment. You have turned a game of chance into a game of skill. While you do need some skill shooting baskets, assessment can help you be more successful and maximize the skill and knowledge that you have.

While I haven’t measured this, I would bet a handful of Chuck E. Cheese tokens that if you took the breath before you began the game to assess the environment and took a few extra seconds with your first shots to assess how well you were doing, you would be more successful in the end being more effective with the resources that you had, regardless if you were playing against the clock or a 10-year old next to you.

This is what assessment is all about. A little time at the beginning, perhaps before students even arrive on campus, performing a needs assessment and taking a little time in the middle for formative assessment (measuring the process or progression towards goals) will help us be more effective. We can then take a look at the summative assessment (the number of baskets made or how effective we were in the end in reaching our goal) and make changes for the next time we play. After a couple of games we would become more affective. Fostering student learning and development takes assessment and practice.

If we take this little extra time for assessment, we will not only be more effective, but we might even be able to get enough tickets to get that Tweety bird stuffed animal we had been eyeing.

I hope this blog entry was useful. Please take a moment to share your thoughts.