Train the Trainer: Critical Thinking in the Wine Classroom

One of my rallying cries in wine education and ALL education for that matter, is to teach our student skills far, far beyond the basic memorization of facts.  I’ve spoken at many a conference on this matter, and have dedicated many of my posts on this blog to the subject.  It’s all about teaching to the “higher order thinking skills” known as (I’m not making this up) “HOTS” rather than just teaching to knowledge or memorization, known as “lower order thinking skills” or “LOTS”.

One of the best ways to teach to the “HOTS” is to guide your students to use the information rather than just remember it.  For instance, in my red wine class for beginners, I hope that my students will not just memorize the names of the red wines and the vocabulary words we discussed, but be able to use the information in some meaningful way, such as:

  • Describe what tannin is, and discuss how it adds to the overall character of a wine.
  • Discuss acidity in red wines and how it compares to acidity in white wine.
  • Describe a wine’s method of production and how it lead to some certain character of the wine…

What we are doing here is teaching critical thinking skills. Critical thinking can be traced in Western thought to the Socratic method of teaching used in Ancient Greece, and in the East, to the Buddhist Kalama Sutta. It is a part of the formal education process, increasingly significant as students progress through the higher grades, and should be the main concern in college and graduate-level teaching.  Obviously, critical thinking is an important part of just about every profession. 

According to Wikipedia, “There is debate among educators about its precise meaning and scope.”  Thank you, Wiki..very well said.  Crtical thinking is indeed one of those concepts, like “Politics” or “The Economy” that everyone understands but is hard pressed to actually define.  So here goes, my contribution to the debate…my definition of critical thinking:  The process of actively using new information by application, analysis, or evaluation. I tried to keep it simple.

Here is a list of my favorite techniques for adding critical thinking to the wine classroom:

  • Compare and contrast wines (or regions, or winemakers) by style.
  • Group wines into categories and explain why and how you chose the categories.
  • List wines in order (lightest to fullest, simplest to most complex).
  • Support an opinion with evidence. For instance, this wine is complex because…
  • Discuss whether or not a certain wine appeals to you, and tell us why/why not.
  • Describe a certain aspect of a wine (tannin, alcohol, residual sugar) and discuss how it relates to the other elements of flavor.
  • Predict how a wine will interact with a certain food.

And to make it easy, you can use what I call a “Bubbly Professor Brain Crank”.  I like to call this one “The Instant Critical Thinking Tool”: 

This wine is ____________  because ______________.

I like to use this sentence for class discussions, and it has a tendency to show up on my final exams.  I love it because it appears simple and un-intimidating, and yet you must use critical thinking to answer the question.  It’s also perfect for discussion because there are an infinite number of correct answers.  Students like that feature for test questions as well.

Here’s a sample…just how would you answer this question?

This Beringer Private Reserve Cabernet Sauvignon 1990 is ___________ because ________________.  

Made you think!  

 

The Bubbly Professor is “Miss Jane” Nickles of Austin, Texas…

Lessons in Teaching: Learned at TexSom

I am lucky enough to be attending the Texas Sommelier (“TexSom”) Conference this weekend at the lovely Four Seasons Resort in Las Colinas, Texas.  Being an educator and not a sommelier, I know that I attend conferences of this sort in a much different mind set than the typical attendee, but there is a plenty of valuable information and experience to be gleaned by sitting in the audience and watching someone else teach.  As a matter of fact, it’s my favorite type of vacation; all you fellow teachers out there will relate to the feeling of relief you get when someone else is on the teaching “hot seat” and you can lean back and watch.

So, it is with much gratitude for the conference leaders and presenters that I share with you a few great teaching lessons learned and reinforced by the seminars I’ve seen at TexSom this year!

Lesson in Teaching:  Tell a Story!

The Tomb of Eleanor of Aquitaine. Good wine story!

The first session presented on Sunday, led by Wayne Belding and Brett Zimmerman, both Master Sommeliers, was all about Bordeaux.  The presenters led us through the story of the history of Bordeaux, all the way back to the middle ages and the marriage of Henry Plantagenet to Eleanor of Aquitaine.  This marriage made the French region of Aquitaine an English terriority, which opened the wines of the Bordeaux region to the English market and eventually to the world stage.

The name Aquitaine itself means “the place of much water” and describes not only the main rivers of the area but then-marshly lands around the Medoc. Another fasciniating part of the story of the history of Bordeaux takes place in the seventeenth century, when English demand for the wines of Bordeaux increased to the point that Dutch traders began to drain the marshlands around the  Medoc and increase the land available to vineyards. This caused the vineyard area in Bordeaux to expand so much that no one could really tell what area their wine was from, which in turned led to the first delination of the vineyards of Bordeaux.  This classification was completed in 1725, with the collection of districts being known as the Vignoble de Bordeaux.  At this time, individual bottles began to be labeled with with both the region and the specific area from which they originated.  I was fascinated to learn this ancient history of Bordeaux and happy to add some more “tales of the vine” to my arsenal of teaching tools.

Why is storytelling a good lesson in teaching?  It is a proven fact that the use of storytelling – called “the narrative approach” in teaching lingo – is one of the most effective ways to add interest and engagement to a presentation.  Storytelling works in just about any educational platform from lecture to discussion to on-line, and can help you reach your audience on both an intellectual and emotional level.  A well-placed story can make theories or abstract ideas concrete and accessible, can spark interest in new material, and can help students memorize facts.

Lesson in Teaching:  Compare and Contrast 

Oregon Pinot: Did you prefer the Hippie or the Cowboy?

The last session I was able to attend on Sunday was all about Oregon Pinot Noir.  Presented by Fred Dame and Nate Ready, both Master Sommeliers, the session was subtitled “The Cowboy and The Hippie.”  Fred and Nate both presented some fabulously delicious Pinot Noirs, which would have made the session good enough, but what made the session even more interesting was the dichotomy in the way they both chose and presented their wines.  Fred represented the “Cowboy” approach and chose wines made in a “get ‘er done” style that produced a wine that was pure Pinot Noir – juicy, fruity, in some cases jammy – and showing lots of spice and floral aromas.  Nate represented the “Hippie” approach and chose wines that were done in a more “touch-feely” (some would say organic-biodynamic-light of the moon) style with lots of earthiness and subtlety.  The wines were presented two by two – with one wine is each set representing the hippie approach and one wine representing the cowboy style.  Presenting the wines this way enabled the audience to not just taste the wines but really think about them in terms of style; which by the way is the “language” most of our customers think in. 

Interestingly enough, my favorite wine of the flight was the 2010 Penner-Ash “Dussin Vineyard” – pure Cowboy.

Why is “compare and contrast” a good lesson in teaching? One of the ultimate goals for any teacher is to guide his or her students to not just “know” (or worse yet, “memorize”) new information but to use that information in a meaningful context.  In this way we are teaching critical thinking skills and such teaching is considered a “higher level” teaching skill, as opposed to just teaching a page full of facts. One excellent technique to involve the use of critical thinking skills in a class is to have students compare and contrast items or ideas.  By ending the Oregon Pinot Noir sesion by asking the audience “which wines did you prefer, the cowboy or the hippie” the speakers were forcing us (in the nicest possible way) to use critical thinking.  

This is a quick post from the hotel lobby…stay tuned for more “Lessons in Teaching:  Learned at TexSom” in the days to come!

The Bubbly Professor is “Miss Jane” Nickles of Austin, Texas.

My Advice to Aspiring Wine Educators

I get asked all the time, during classes, at wineries, in e-mails, and via just about every medium possible, “How do I get your job”? Well, I really want to answer by saying something like, “Study for twenty years, write a book, learn instructional design…”  But I’ve long since learned that’s not the answer people are looking for.

And it’s a good question, and since so many of you asked, here’s my real answer:

  • Know your stuff.  Enough said.
  • Spend just as much time studying and mastering presentation techniques, public speaking skills, and audience management as you do studying wine.  As a matter of fact, spend more time studying them.  Knowledge is everywhere in our society…every student in your audience has all the information you are going to spill forth for them readily available on the little e-gadget they have in their pocket.  They don’t need you to provide them with information…they need you to provide them with a way to understand, remember, and be engaged with the information.
  • Develop your own style of teaching and work at perfecting it.  Don’t beg or borrow other people’s teaching materials…look inward and produce your own.  I can’t tell you how many times I’ve had other educators e-mail me asking if they can have my power point presentation.  My only thought is “Why would you want to use my materials”? Can you really speak from the heart using someone else’s presentation?  Trust me, you will do a much better job at presenting using materials that are your own. 
  • Create opportunities to engage your class.  People who don’t undestand how to engage an audience think that it only has to do with energy or activity.  Points of engagement should be pre-planned, practiced, and executed at regular intervals in any class. 
  • Develop a skill or gimmick for remembering and using people’s names.  Draw yourself a room diagram with people’s names on it, write short descriptors (for your eyes only) of individuals on your class list or roster, or have people wear name tags.  The sound of one’s own name is the most engaging thing known to humankind.  If you have a large group, make sure you know and use the names of everyone in the front row.
  • Become an expert at A/V.  If you are going to be using projectors or microphones, be able to hook them up, break them down, turn them on and trouble shoot.  Nothing will throw you off your game faster than having to leave your area to go find a techie.
  • Arrive at your location an hour early.  No excuses.  Get set up, find your wines, figure out who is going to be in your audience, relax, and be ready to greet everyone at the door as they enter.  Introduce yourself to every attendee (or at least the first 25 if you have a big group).  Make small talk with your students in the time you have before class begins, and by the time you start you’ve already made friends.
  • If you are going to use Power Point, Keynote, Flip Charts or slides, study up on the latest research and advice concerning their use. One of the quickest ways I can detect an amateur is the use of wordy, detailed, or complicated visuals.  Same goes for handouts.
  • Don’t make excuses. If you are a good teacher, you can teach your subject with or without a projector, in a wine bar or a lecture hall, with the exact wines you ordered or with the wines they delivered by mistake. 
  • To get started; give one class for free at a wine bar, winery, or tasting society.  After that, you should be able to start to charge for classes.  If not, perhaps you need to practice some more.  Don’t go on too long giving it away…there are far too many “wanna be” wine educators out there who are willing to charge nothing for their services.  If you are providing a valuable service, you deserve to be paid. 
  • If attendees are answering questions or describing wines, find what’s right in every contribution.  No one should ever be embarassed or belittled for speaking up in class. Appreciate everyone’s attempts to answer your question or participate in your class even if they are technically “not correct”. 
  • Don’t get drunk in public.  Ever.  Especially not at your own events.  As a matter of fact, as long as you have the floor, the microphone or the name on the marquee, you should only take a sip or two of any wine that you are presenting; that’s all you need. (Ok, after the class is over go ahead and have a glass of wine to unwind while chatting up your students.  But JUST ONE.  End of rant.)
  • Don’t quit your day job.  Very few people make a living teaching wine classes. However, there’s nothing wrong with working the floor of a retail wine shop, pouring wine  samples in a grocery store, or waiting tables to to make ends meet.  While working at a retail store might not be your dream job, a job at a retail store (or restaurant or grocery store) just might lead to many opportunities to teach if you are willing to “make it happen”. 

And here’s my best advice:  Always keep growing.  Develop new teaching skills and new teaching materials constantly.  Appreciate the people, the surroundings, the wine, and the opportunity to share your passion.  Have fun, and if you can snag a full-time job with benefits that has anything to do with wine….take it.

The Bubbly Professor is “Miss Jane” of Austin, Texas

The Truth About Learning Styles (Again)

You’ve heard it before and you’ll hear it again…there are three basic learning styles, and everybody has one that they prefer.  Visual learners learn best by seeing information (graphs, maps, and pictures) , auditory learners learn best by hearing (speeches, lectures, recordings) and kinesthetic learners learn by doing (or touching, or manipulating materials).  You’ve probably taken a cute little test or quiz to determine your preferred learning style, and if you are a teacher you’ve been told to vary your lesson plans and teach to all three learning styles.

Before we try to credit or discredit this theory, let’s look at some known facts about cognition and memory.  First, people do differ in their visual, auditory, and kniesthetic memory abilities.  Certain people are better able to remember visual memory applications.  If ask you to visualize a pine tree, the Empire State Building, or the Mona Lisa; I am asking you to use your visual memory.  If I ask you to listen to three different recordings and  identify which one is Barak Obama’s voice, I am asking you to draw upon your powers of auditory memory. If I ask you to show me how to swing a baseball bat you would utilize your kinesthetic memory.  As I sure you realized at a very young age, certain people are just plain better at recalling visual images and certain people have excellent auditory acuity.  Of course, to the chagrin of many of us, some people are just great at remembering and re-enacting the perfect baseball bat swing, and some of us never seem to master it. 

Here’s one other thing we need to consider:  not all of our memories are mere sights, sounds, or movements.  Most of what we “know” is wrapped around meaning.  In other words, we remember things based on what they mean to us.  If a friend tells you a funny story about a co-worker who called in sick and three hours later was seen on TV, cheering on the home team at the noontime baseball game, you would remember the meaning of the story. 

I would remember the meaning of this story as “Somebody lied, somebody got caught, with technology these days you can’t get away with much, I’m so glad it wasn’t me, but it’s pretty funny that it was her!”  Most people would probably think the same thing about the story, or perhaps the “meaning” of the story would vary depending on whether or not you were friend or foe to the person involved.

I would also remember the visual elements of the story, such as the way my friend looked as she was telling it to me, or my visualization of what the TV broadcast would have looked like.  I would also remember the auditory elements such as what my friend’s voice sounded like as she told me the story.  I would also remember the kinesthetics of the moment, such as whether we were walking, sitting, or standing.  But the most important part of the story, the meaning, has a life of its own.

grapesThis part of the “learning styles” theory makes sense:  some people are better at certain types of memory than others. But the main, predictive element of the theory is that students will understand and retain information better (learn better) when the delivery of new material matches their cognitive style.  Let’s say we are teaching a class of 30 adults.  We’re giving a basic introduction to wine production.  We have a list of 30 vocabulary words we want the students to grasp. (Brix, Must, Crushing, Pressing,  Pomace, Racking, Fining, Acidification, Chaptalization, Malolactic Fermentation, Cap Management, Autovinification. Malic Acid, Lactic Acid, pH, Tannin, Sterile Filtration, Aging, Maturing, Blending, Carbonic Maceration….you get the picture!)

Let’s say we divided the vocabulary words into three lists of ten words each. For one list, we presented the information verbally…we read the vocabulary words and recite the definitions several times.  For another list, we present the information visually…we write the words and definitions on the board (or use power point slides).  For the third list, we somehow have our students act it out or use gestures to accompany definitions. If the predictive part of the “learning theory” holds true, the “visual learners” in the group would remember more of the words written on the board and the “auditory learners” would remember more of the vocabulary words that were recited.

Here’s where the theory takes a dive:  dozens of studies have been conducted along these general lines, and the theory is NOT supported.  Using a student’s preferred cognitive modality does NOT give that student an edge in learning new material.

How can that be?  Because what is being taught – the new material we want the students to grasp – is not necessarily visual, auditory, or kinesthetic information…it’s meaning.  If we wanted students to recall visual information, for instance, the way the grapes should look like when they are ripe, presenting that information visually is the only way to go.  Use an actual bunch of grapes or a good picture.  Just imaging trying to describe what something should look like using just your words. “Um…they’re purple, but sometimes kind of blue, and they are grouped together in a bunch with a stem, like a bunch of grapes at the grocery store, only smaller.  And tighter. And bluer. Sort of.”  It sounds funny to even try.

The same holds true for auditory information.  It must be transmitted via sound (I know it sounds a little goofy when stated like that – sorry)! If we wanted students to know how the batch of fermenting must should sound, an audible recording (or the real thing) is the only possible way to impart that information. Imagine trying to describe it using words.  “Oh you know, it should kind of sound like soda being poured into a glass, but then it stops everyone once in a while and goes sh-h-h-h-h-w-o-o-o-f.” Same with trying to teach a group of people what it feels like to crush grapes with your feet.  You just can’t tell them…you’d better go get a bus tub, some grapes, and tell everyone to wear slip-off shoes.

Remember what we are trying to teach here with our list of 30 vocabulary words…it’s not visual, auditory, or kinesthestic information only; it’s meaning. Presenting information via the visual channel has not been proven to give a “visual learner” any advantage in learning anything besides visual representations.  The same holds true for auditory and kinesthetic “learners”.  An auditory learner might very easily remember the sound of your voice while you were reciting the vocabulary words (and can probably imitate you quite well behind your back), but “the sound of the teacher’s voice” is most likely not one of your leraning objectives.

So if the “learning styles” theory is really quite wrong, why does it seem so right?  This theory is ultra- well entrenched in the literature and philosophy of education.   A quick google search of “learning styles” will deliver hundreds of pages on learning styles from the websites of well-respected universities.  A page with the VARK questionnaire on “What’s Your Learning Style?” tells me that over 25,000 people have taken their quiz this month.  I’ve been to several wine-and-food related educational conferences where a session on “learning styles” was offered. It’s almost one of those things that we just want to believe, or believe because everybody else believes, like “people believe 10% of what they read, 20% of what they hear….” which, by the way has also never been proven. And, of course, we have to concede that a small portion of the theory is true.  The fact that people vary in their visual memory ability is obvious.  Some of my students can memorize that wine map of Italy after just one look.  Others have to study for weeks.  They same holds true for the auditory and the kinesthetic.  But we can’t make the mistake of a false correlation between the natural ability of visual memory to an overall superiority of using the visual channel for learning the meaning of things, or for that matter, the learning of anything is isn’t visual.

Finally, a lot of the evidence surrounding learning styles is guilty of what is known as “confirmation bias.”  Once we believe something, we interpret ambiguous situations as being consistent with what we believe.  Suppose, for instance, that you discover that your students score better on tests when you provide them with a handout after your lecture.  You assume that while your lecture might be reaching the auditory learners, the handout reaches the visual learners so that class as a whole performs better.  You feel good about your class and your ability to teach to “the different learning styles.”  What actually happened, however, was more likely the fact that some of your students used the handouts to read the material and study after class, thus exposing themsleves to a repetition of the material.  Using a lecture and a handout is just a better way of teaching than relying on a lecture alone.

Even better, of course, is the student who actually listens to your lecture, reads the book, studies the handout and engages in some discussion or other use of the information. That’s a good way of teaching meaning, and if your classes motivate the students to do just that, you’ve done a good job teaching – to any learning ability and any learning style.

If you’d like more information on learning styles, you can see my previous post on learning styles:https://bubblyprofessor.com/2012/01/28/train-the-trainer-are-you-a-visual-or-a-verbal-learner/

I also highly recommend Daniel T. Willingham’s book, Why Don’t Students Like School?   (Jossey-Bass, San Francisco, 2009). It has a wonderful chapter called “How Should I Adjust My Teaching for Different Types of Learners?” that discusses  and debunks the three “learning styles” as well as many other theories of “multiple intelligences”. 

The Bubbly Professor is  “Miss Jane” Nickles of Austin, Texas.

Week One, Day One: The Introduction to Wine Class

Next week starts a brand new semester and among the classes I’ll be teaching this block is my sentimental favorite – The Introduction to Wine Class.  I offer Professional Wine Studies, Wines and the Culinary Arts, and Wine and Food Pairing as well as semester-long looks at both Old World Wines and New World Wines, but the introductory class remains my favorite.

It’s great to see wine newbies go from “What is Wine” to “The Legend of Sassicaia” in just over 12 weeks.  I always like to start Week One/Day One simply enough with “Wine, Defined.”  I am sure that every wine educator out there has their preferred version of the answer to the question, “what is wine”?   My is quite simply, “Wine is a beverage produced by the fermentation of fruit, mainly grapes”.  Of course this answer leads to many questions and further disucssions…what is fermentation, why grapes, and “can you make wine from Welch’s Grape Juice”?  Of course, the answer is yes…it just won’t taste very good!

And it never fails, within the first ten minutes of class someone will mention the following subjects:  Boone’s Farm, Four Loko, Sangria, Hellow Kitty Wines, Prison Wine, Mad Dog 20/20, Saké, Arbor Mist Blackberry Merlot, Thunderbird, Mimosas, Cristal, and Ace of Spades.  Fellow wine educators, I bet you have your own list, I would love to hear about what your students ask on day one!

And somehow, we get through it all.  I like to have a basic “learn how to taste” session on Week One/Day One as well, both to get the class off to an engaging start and also to lay the ground work for the more detailed, directed tastings we will have as the class progresses.

My introduction to sensory evaluation class is admittedly quite technical.  I tell the students what the wines are, but I ask them not to focus on that one particular wine but rather to use the wine at hand to learn about the sensory evaluation of  “every wine or any wine.” 

I use just three wines; an unoaked, crisp Chardonnay (A Macon-Villages is ideal), followed by a simple yet sweet white wine (I’ve been using Flat Creek Estate Muscato D’Arancia), and finish with Sterling Vineyards Napa Valley Cabernet Sauvignon. 

The basics of sensory evaluation class that I teach on Week One/Day One does not follow the basic steps of wine tasting.  We will get to all the expected steps (sight, swirl, sniff, snort, whatever…) in the course of the session, but not exactly in that order.  I tell the students that we are going to do approach the wines in the proper order (dry before sweet, white before red, light before heavy) and that we will let each wine “reveal” its secrets to us – in other words, each wine has something special to teach us. I choose my flight of three to include a wine that perfectly shows acidity, one that has sweetness, one with bitterness and tannin, and make sure that within the set of three, each of the major aroma families is there in an easy-to-recognize manner.  I want the class to be chock-full of “a-ha moments.”

Then I launch right in, teaching what I call “The Nine Elements of Wine Flavor.”  The nine elements are: Acidity, Sweetness, Bitterness, Tannin, Umami, Aroma, Body, Balance, and Alcohol.  I told you it was technical!  It does start off quite scientific, with discussions of pH, IBU’s, R.S. and ABV, but by the time we add aroma to the mix I make sure the step off the path of “paralysis by analysis” and let the students just relax and enjoy the flavor of the wine.  And somehow, it all comes together in the end.

If you’d like a copy of my handout about “The Nine Elements of Wine Flavor” just send me an email request to”  missjane@prodigy.net .

The Bubbly Professor is “Miss Jane” of Austin, Texas.

Finals Week in Wine Class

It’s Finals Week!

Final exam week in Miss Jane’s 12-week professional wine studies class has arrived!  To answer your question, NO…the wine final does not involve binge drinking, glasses clinking or happy hour. Like most college-level courses in hospitality management or culinary arts, we have both a practical and a written final exam.  Our written final exam is scheduled for this Friday; a 100 question multiple choice test followed by three essay questions.  

Please note that the title of this course is “Professional Wine Studies.”  While many of the wine classes taught around the world center on tasting, this course is centered on basic wine knowledge and how to use it within the context of a hospitality career.  I focus the class on learning about basic wine styles (white, red, sparkling, dessert…yes, that basic); how they are made, where they are made, and the world’s best known or most popular examples.  We learn how to taste wine so that we can talk about wine, and spend several class sessions role-playing the role of the server, sommelier, or salesperson.  We use my “mad libs for wine” to learn to write meaningful, concise wine descriptions. We learn about beverage costing as it applies to wines by the bottle and glass. We spend a good deal of time of food and wine pairing, which makes sense as most of my students see themselves as future chefs.  Finally, we spend a good deal of time discussing how to write a wine list and market wine in a restaurant or other setting.

So for my final practical exam this semester, I came up with the idea of an exercise in writing a wine list.  I started out by surfing the internet for nice, clear pictures of wine labels. This took a while as I wanted to use wine labels from wines we had studied and my students would be familiar with.  I also wanted a good mix of red, white, dessert wines and sparkling wines.  I came up with a word file full of about 30 wine labels that includes Bordeaux, Chianti, Rioja, Napa Meritage, Lodi Zin and Oregon Pinot for the reds.   For the whites I found Fumé Blanc, a nice German Riesling, an Aussie Chard…you get the idea.  I had six sparkling  wines including a few from California, a Cava, a Prosecco, and of course Champagne.  For the sweet wines I included Moscato d’Asti, Sauternes, Late Harvest Zin, Ruby Port and Muscat-Beaumes-de-Venise.  Remember, these are all wines that we had studied, and in most cases, tasted.

 I did a bit of cut and paste and gave every student a stack of 30 wine labels, and created a faux “wholesale price list”. Then, I gave the class two hours to write a wine list that was to include the following details:

  • Meaningful Categorization
  • Absolutely perfect listing of each wine to include producer, name of the wine, region of origin and vintage date (as applicable)
  • Progressive wine list format
  • A concise description of each wine (I like to use what I call a “five word description” such as “light, dry and crisp with fruity and floral flavors”.)
  • Two food pairing suggestions for each wine.
  • Pricing by the glass and bottle, as well as a spreadsheet detailing each item’s potential beverage cost and gross profit.

As they completed the project, I had every student bring their list up to me for a quick discussion and review.  Lots of learning can go in during that review period.  I had them describe how they chose to categorize their wines, how they arranged them in order and how the details of the list will be useful as a sales tool.

All in all, I have to say I think they all did a great job!  I was very impressed with the final projects, and think that it was a meaningful, active learning experience all around.  It was good exposure to the “nuts and bolts” of writing and designing a wine list.  Most importantly, we all had a great time and I feel it was a good example of active learning and a “flipped classroom”.

If you would like a copy of the materials I created for the class, click here: Bubbly Prof – Wine Labels for Wine List Project

The Bubbly Professor is “Miss Jane” Nickles of Austin, Texas – missjane@prodigy.net 

 

 

The ABC’s of MCQ’s

A few weeks ago, a friend of mine had me look over a 100-question multiple-choice test she had just written.  My friend works for a community college with a new culinary program, and she has been tasked with putting together a series of three professional wine studies classes that will ostensibly prepare students to take a certification test from one of the many groups in the U.S. that give wine people the opportunity to line the walls of their study with plaques proclaiming that the owner “knows a whole lot about wine” and lets them add a few letters following their name on a business card.  She has a lot of test writing in her future.

The first question on the final exam she was writing for the introductory class was, verbatim, as follows:

                1.  A wine region in South Africa is:
                                a.  Kirwan
                                b.  Tutuven
                                c.  Benguela
                                d. Robertson

 The answer is “D – Robertson” – but that’s about the only thing right about this question.

Much to my chagrin, the rest of her “test” was composed of 99 more questions, each very similar in style and content to the first. I knew we had a lot of work ahead of us and really needed to improve her questions, hopefully before they went out to any bright-eyed, hopeful wine students. After seriously considering how to word my comments to her, my response went something like this:

                1.  Which of the following items is wrong with the multiple choice question quoted above?
                                a.  The question is grammatically incorrect.
                                b.  The question tests single subject, knowledge-only learning.
                                c.  The question covers trivial information.
                                d.  All of the above. 
 
                The answer is “D – All of the above”!

 

It seems like the universe has deemed me something of an MCQ Maven, as I am tasked with reviewing tests such as these on a weekly basis.  Well, I guess that makes sense for an academic director. So, I thought I’d write down a few of the tips I was about to give my wine-teaching friend.  Perhaps they will help you as well!

 A.  When writing multiple choice questions, be grammatically correct! “A wine region in South Africa is Robertson” doesn’t sound correct even if I use my Yoda voice (which I do quite often).  To be grammatically correct about being grammatically correct, the question is confusing based on subject/predicate placement and uses the passive voice unnecessarily.  

Ideally, the stem of a MQC should be written as a complete sentence, and in the form of a question.  This is referred to in MCQ-lingo as the “direct question format” In the case of “question #1”, a proper direct question format would be:

                Which of the following is a wine region located in South Africa?
                Or…
                Which of the following wine regions is located in South Africa?

Using the incomplete statement format (also known as “fill in the blank”) is also considered appropriate, but most test writers still prefer the direct question format and use the incomplete statement format only when a direct question isn’t feasible. For the sake of illustration, a proper incomplete statement format for this question would be:

                 _____________ is a wine region in South Africa.

 It’s grammatically correct, but I still don’t like it. You’ll see why, below.

 B.  When writing multiple-choice questions, ensure the validity of your test by emphasizing higher-order thinking skills.  In other words, don’t write a test that can be passed by anyone who just locks themselves in a Holiday Inn for the weekend and memorizes the text, only to forget it all by Tuesday. 

Writing “knowledge-only” test questions is faster and easier compared to writing questions that test for understanding.  However, as in all things, you get what you give. Writing a test chock-full of single-subject knowledge-only questions is inexcusably lazy and might just get your test thrown out for being invalid and unreliable (the mortal sins of the testing world).  Another sin is to quote directly from the text in your questions, unless the entire purpose of the test is to rate photographic memory recall. 

Avoid these sins by writing at least half of your questions in a manner that tests understanding of the material by using memory plus application, cause-and-effect, analysis, synthesis, evaluation, or integration of subject matter. Here’s a better test question about Robertson:

                 Which of the following wines is mostly likely to be produced in the Robertson wine region?
                                a.  A sturdy, Shiraz-based red wine made from grapes grown in South Australia.
                                b.  A dry, Riesling-based wine made from grapes grown in Western  Australia.  
                                c.  A high-volume, fortified wine made from grapes grown in  South Africa.
                                d.  A  boutique-produced  Meritage blend made from grapes grown in South Africa.  

Is it obvious why this test question takes longer to write than our original question?  First of all, it  avoids testing for trivia in that each distractor and the correct answer involves not just one but two pieces of factual information (all of which must be properly vetted, no matter how smart the test writer thinks himself/herself).  It involves memory-plus application in that students need to know a bit more about the wine region rather than just where it is and integrates several types of information about the topic.

Incidentally, such a question is actually easier for a well-prepared test taker to answer correctly than a shorter question. At first glance, it would seem like a more complicated question, but in reality the test taker has more than just one chance at remembering the facts specified in the question, and can call upon a broader base of knowledge to ascertain the correct answer.  This is also a good example of how to build some “validity” and “reliability” into your tests.

For much more information on writing MCQ’s that test for higher-order thinking skills, see my previous blog post here:  https://bubblyprofessor.com/2012/02/26/train-the-trianer-taxonomy-of-a-wine-test/

 

 C.   When writing multiple choice questions, don’t test for trivia! If my friend was writing a 100-question test on the South African wine industry, a question about Robertson would have been appropriate.  The way it stood, however, had me humming “can you guess which thing is not like the other” for days. Keep in mind that her test was intended to be a final exam covering three months worth of classes and a 653-page textbook.  The kicker is that the only mention of Robertson in the massive text is a table on page 427 that lists over 75 South African regions, districts, and wards in 8-point type.  The items offered as distractors are just as obscure.  There is no way that any student can get that question correct unless they are just plain lucky.  They could be lucky in that they just happen to remember reading “Robertson” in the midst of that periodic chart of S.A. regions, or they just took a lucky guess.  Either way, this question is not a valid indicator of wine knowledge…but it would make a great “Trivial Pursuit – South African Wine Geek Edition” question.

When writing single-subject knowledge questions, ensure that the knowledge being tested is based on a student learning objective of the course, not just trivial information. You should only use your single-subject knowledge questions to test for important or significant information. When writing such questions, don’t be fooled into thinking that MCQ’s concerning broad information such as the definition of “malolactic fermentation” must by definition be “easy” questions. The ease or difficulty of such questions is totally under your command.  The more similar the distractors are to each other and the correct answer, the more difficult the question will be. If malolactic fermentation is a key learning objective of your class, feel free to make it a difficult question.

One good way to manage this process is to sit down and think of the ten or twenty most important  concepts you want a graduate of your program to understand.  These should, of course, be reflected in your course objectives. Then, write a fair question for each of them, varying the level of difficulty across the test.  For example, I tell my students on the first day of class that every day we will have a “word of the day” and it’s very important that they know them.  My “words of the day” include such bon mots as Carbonic Maceration and Botrytis. (Email me if you want a complete list.)  I also emphasize “Miss Jane’s Top Ten Terroirs” throughout the class and tell the class they really need to know these ten wine producing regions and what goes on there.  My top ten terroirs include Burgundy, Bordeaux, Chianti, Rioja…you get the idea.

Another good way to create knowledge-based questions is to write questions that cover more than one “tidbit” of information.  It is possible to write valid, knowledge-based MCQ’s if the question is integrated rather than single subject oriented.  Here are a few examples of integrated questions, all of which I suggested for use in place of my friend’s ill-advised question on Robertson:

                 Which of the following South African wine regions are located in the interior of the country?

                                a.            Worcester and Robertson
                                b.           Stellenbosch and Paarl
                                c.             Constantia and Elgin
                                d.            Elim and Klein Karoo

                (And the answer is… “A”!)

                 Franschhoek, Constantia, and Robertson are all well-known wine producing regions in which
                of the following countries? 
                                a.            New Zealand
                                b.            Australia
                                c.             Argentina
                                d.            South Africa 
               
                     (And the answer is… “D”!)
 
You’ll note that while these questions are knowledge-based (ok for about 50% of your test), they are not trivia-based in that they include more than one piece of information, both requiring and rewarding a broad base of knowledge and application as opposed to “I’ll take South Africa for 200, Alex”!
 

 Here’s a few more guidelines for writing MCQ’s:

  • Don’t quote from the text or test for information that is specific to just one source. In other words, don’t use your textbook’s description of Chenin Blanc (“apple, straw, melon”) and assume it is a universal definition.
  • Please keep in mind that there is a world of information beyond the text book that needs to be acknowledged.  I recently reviewed a test that asked “which of the following wines is exclusively red” and listed Châteanuf du Pape, Hermitage, Bordeaux and Beaujolais as possible answers.  You will note that none of these answers is exactly correct.   The supposed “correct” answer was Beaujolais, which I happen to know produces a white wine, albeit a somewhat obscure one.  My friend, who wrote the test, replied “but it’s not in the text.” Argh. By the way, mistakes such as this punish the student with broad knowledge and reward the last-minute text-memorizer.  Double Argh.
  • Properly vet each and every question and each and every correct answer and distractor.  That means solid research (well above and beyond just the text in use) on every question and question topic. This is to ensure that’s there is one (and only one) solid correct answer and yes, this this takes a lot of time. 

 

Good luck writing your multiple choice questions.  I am sure you will soon realize that while they do take a great deal of time to develop, a valid test is worth the effort. And don’t forget the payback…they are so quick to grade, as opposed to essay questions.  Stay tuned to this blog for future posts on writing and grading essay questions, by the way. 

 And please…help stamp out lazy test writers!