A vision in the mind of one man fifty years ago has led thousands of youth and adults to pursue a dream: that the skills of leadership can be defined and taught. So it might be a good time to reflect and ask, does the program work?
Evaluating Real Results
Since 1958, with programs currently in two locations, White Stag has put on 63 summer camps for an estimated 20,000 youth. Today, the children of participants from the 1960s and 1970s are taking part in the program.
As a non-profit, not held to the gun of showing a return for shareholders or adding to the bottom line, we are often satisfied to evaluate the success of our efforts in anecdotal terms. We see the enthusiastic response of the participants to the program, we witness their cheers and songs, we watch faces light up in recognition of new-found knowledge, and we even see youth apply their learning in new situations during camp. But does the program work?
Does the Program Work?
Our summer camp programs cost more than the typical summer camp. We advise all participants, "This isn't a fun-time camp." We give participants challenges and precious few direct answers. They go to bed well after dark, usually get up at dawn, and rarely get a break. They get intensive learning sessions two or three times a day. And every year, units repeatedly send youth to the programs' summer camps every year. We must be doing something right.
Yet the real measure of our success is whether the participants actually retain anything taught and are able to take it home and apply it. How do we know that we know?
How Do We Know That We Know?
The best measure of our success is writing effective, measurable objectives. Only in this way can we define our success.
Writing meaningful, clear, concise goals and objectives is one of the
most important activities we assume as managers-of-learning. If we cannot
clearly state the outcomes desired, we must ask ourselves if we really
understand the subject well enough for us to teach it to others.
Suppose you want to assess whether the participants have learned how to plan, so you give them a challenge to build a bridge. Your objective: "The candidates will use the five step planning process to build a foot bridge." They use all the steps of planning, and at the end of the day, you find that every lashing is tied correctly, but the bridge doesn't reach the other side of the river. Did they learn how to plan? By your objective, they were successful.
Writing Effective, Measurable Objectives
Meaningfully stated objectives:
- Succeed in communicating
to an unbiased third party your instructional intent.
- Define in concrete, measurable language the specific outcomes for the learning activity.
- Describe the behavior desired in measurable, specific criterion so an independant third party can judge whether the behavior is actually performed.
- Orient learners to new language
and systems of knowledge inherent in the objectives.
Use SMART Criteria
More objectively, you can apply the SMART criteria,
shown below. Ask yourself, is each objective:
| S - Significant |
Is the item significant? Is it important? Will it have
an impact on your unit's program? |
| M - Measurable |
Is the item measurable? How will you know when it's
done? Can the progress be tracked? |
| A - Appropriate |
Is this task within your job responsibilities? Does
it need to be done? Is this an opportunity that is available to you? |
| R - Realistic |
Can the item be accomplished? Can it be brought to
a successful conclusion? Do you have control over the task? |
| T - Timely |
Can it be done in a timely manner? Is it going to take
too much time to accomplish? Can it be done in a reasonable time? Place
a time limit on it, "A Dream becomes a Goal, with a deadline on it. |
Specifically, an objective identifies four things:
- Who is acting (the actor)
- What is being done (the action)
- How often it is performed (quantity)
- How well it is completed (quality)
Who is Acting
You can't evaluate a group very easily. A good objective names the actor who is being evaluated.
What is Being Done
The objective needs to define the task or behavior. To behave is defined as, "To act, react, function, or perform in a particular
way" Behavior is defined as, "The actions or reactions
of persons or things under specific circumstances."
You'll notice that very specific controls are required: a "particular
way;" "specific circumstances."
Behavior means taking action: you can see what the learner is
doing. It is a learner's direct, observable, overt action, not something you
infer or deduce from what they said or did. This calls for specific, measurable criterion.
How Often it is Performed
As you write an objective, you must describe the standard
or test that can be used to evaluate whether the objective is actually partially or completely achieved. If your learning outcomes are complex, you may need to write more objectives that allow participants to achieve partial
success. Shooting for the moon is great, but hitting the side of the barn is progress.
An objective refers to the behavior desired of a learner at the time
influence over him ends. The quantity described must be explicit, measurable and
verifiable (by other than whoever conceived it). It is a quantitative representation of the number of repetitions, steps, amount
of time, resources required or limited.
Example Objective
You have thirty minutes, two 6' spars, and two 12' lengths of rope. Use the five-step Manager of Learning competency to teach the other members of your patrol to tie a square lashing.
How Well it is Completed
The objective needs to describe the quality of the behavior desired. This can be quantified in both very specific physical, cognitive language and in emotional, affective terms as well.
Describing Physical or Cognitive Behaviors
Cognitive objectives describe the task in specific behavioral terms. It uses words that describe observable action:
to write, identify, recite, differentiate, construct, list, diagram,
draw, practice, compare, contrast, and so forth;
Cognitive objectives do not use nebulous, vague, and generic language like:
to know, enjoy, believe, grasp the significance of, be certain of,
realize, work, desire, like, be familiar with, remember, apply, understand,
have faith in, to really understand, to appreciate, to fully appreciate,
and so forth.
For a categorized list of verbs useful in a writing explicit objectives and a list of words and phrases to avoid, see Appendix
B - "Key Words in Instructional Objectives".
Example Objective
So
you would add to the previous example:
You have thirty minutes, three 6' spars, and six 12' lengths of rope. Use the five-step Manager of Learning competency to teach the other members of your patrol to tie a diagonal lashing. The lashings must be able to carry the weight of three patrol members standing on any cross-bar.
We want to write objectives that can be successfully met.
Example Objective
Sometimes the specific outcomes are not clear and need amplification. So we might further add:
You have thirty minutes, three 6' spars, and two 12' lengths of rope. Use the five-step Manager of Learning competency to teach the other members of your patrol to tie a square lashing. The lashings must be able to carry the weight of three patrol members standing on any cross-bar. Patrol members must be able to describe each of the five steps of Manager of Learning used during your session.
Describing Emotional or Affective Behaviors
One of the characteristics that differentiates White Stag from other leadership programs is its emotional impact on people. We want them to have a "True White
Stag Experience". We want to fill them with the spirit of cooperation
and friendship that current members know and enjoy. We want them to exhibit
qualities of leadership like compassion, empathy, and servant leadership. We aim to influence individuals' attitudes and values,
because they reflect an individual's deeper emotions, motivations and
reasoning.
Affective change can be evaluated by measuring the change in the rate of the behavior
targeted. Our challenge is to qualify and quantify the affective behaviors
desired. The attitudes and values we want to measure can be effectively described by qualifying the specific, verbal behaviors and physical actions sought.
Example Goal
The candidates will develop an appreciation of the White
Stag spirit and traditions, and feel enthusiastic about their experience
when they go home.
Example Objectives
The candidate will:
- Participate in a candidate neckerchief, legend, and phase neckerchief
ceremony.
- Paraphrase from memory at least three parts of the White Stag
Legend and describe how the principles evident in the Legend
are present in the program today.
- Write down two ways he can apply one of the principles of the
Legend.
- Have an opportunity to express in writing and verbally his feelings
about his week in camp.
On a practical level, program participants will only have an opportunity
to set goals and write objectives for themselves in relationship to their
personal growth. This is typically documented using Leadership Growth
Agreements (LGA). More complete information about LGAs and an example
LGA form in Resources for Leadership, Chapter
6 - "Evaluation Instruments".
Real Results: What Participants Say
Former candidate, youth and adult staff member Tim Madden wrote, "White Stag taught me several skills that I have added to my success. It has added to my communication skills, confidence, self reliance
and taught me what leadership really means. Most people in my generation don't know the
difference between management and leadership. Having a background in
leadership, I can both manage very well and lead as needed. This has helped me prove my value by providing leadership
in work groups while someone else who didn't know what it meant to be a
leader (but had the title) learned how. Learning how to be a follower
also helps as there are times when that is what is needed. And being able to
discern between the two is another skill that has been extremely helpful to me."
Excerpted from Resources for Leadership, available for download.