Grant Guru
-Gary Carnow, techLEARNING
Grant Guru Tip #29:
Determining Results
Breaking down your project into components will assist you in developing your evaluation plan. As you look at each component of your project you will be able to determine your results.
For example, let’s say you wish to develop a project for middle school students. These students have poor writing skills and you want to address their needs by developing a series of activities to get these students to write more and revise their work. Developing a needs statement is fairly straightforward. Read the following example.
The ability to write effectively is vital in today’s world. Writing is a survival skill in all aspects of K-12 and college instruction as well as an important social skill, enabling communication by email and letter. Writing is a source of voice and empowerment in a democratic society. Writing is a creative mode of self-expression. Writing is a life skill, needed to get a job, perform at a job, and advance. Writing is thinking. Students reflect, analyze and reconsider while they write. However, “in today’s schools, writing is a prisoner of time.” (The National Commission on Writing in America’s Schools and Colleges, 2003).
Writing is increasingly shortchanged in the language arts classroom. Students need frequent practice, opportunities to revise their work and authentic feedback to develop writing skills. The recent National Commission on Writing in America’s Schools and Colleges reports that students spend three hours a week or less on writing assignments; 15% of the time they spend watching television. The National Assessment of Educational Progress research reports that at grades four, eight, and twelve, about one student in five produces completely unsatisfactory prose, about half meet “basic” requirements and only one in five is “proficient.” Middle school teachers understand the need to increase writing frequency and provide writing instruction and assignments so that students develop proficiency; however, the daunting task of correcting 150 papers curtails the number of assignments in any given week.
If this sounds like a polished opening to a proposal, it is. This is the opening to a 2.2 million dollar Enhanced Education Through Technology grant that I was awarded and am currently implementing.
I use this as an example to illustrate the various components of a proposal that will assist in determining results. For this project, I needed to evaluate the results of the program (on students and staff) and also meet a series of goals. The participant groups includes students, their teachers, and a group of coach/mentors to assist in implementing the program. My student group consists of approximately 4,000 students in grades 6, 7 and 8 in thirteen K-8 elementary schools. The second group consists of approximately 90 language arts teachers at these schools. The third group, is a group of 19 coach-mentors, a subset of the 90 language arts teachers, who will assist in guiding the project.
In addition to the participating groups, there are also a series of goals. The first is to increase proficiency in writing. The second is to provide extensive access to computer resources to lower the student to computer ratio. The third goal is to increase communication and collaboration among home, school and community to support and improve student learning.
Getting students to write more in grades 6, 7 and 8 require teachers to teach more writing skills, assign more writing and grade more papers. For a teacher with five classes a day, if each student writes one paper a week, the teacher now has over 150 papers a week to read, comment and return. If each paper takes fifteen minutes to score, a teacher would need to find 2,250 minutes or 37.5 hours a week to complete this task. Is it any wonder that students don’t write more?
This particular project incorporates the student and teacher use of an on-line writing tool known as MyAccess by Vantage. The project provides the use of the prompt-driven, web-based writing environment (Vantage Learning’s online writing development tool). Each week (and often more), student writing responses are analyzed and scored with the assistance of Vantage’s Intellimetric artificial intelligence-based electronic essay scoring system. Vantage provides immediate instructional feedback allowing teachers to conduct early diagnosis and assign writing more frequently.
Each target group and project goal has its own set of intended results. Each project goal has an evaluation plan. The goal to increase writing proficiency is measured by standardized writing assessments, teacher evaluations and intellimetric scoring. To determine the extent that students have access to technology, I choose to look at milestone points in time to compare student to computer ratios. To measure the effectiveness of increased home and school collaboration, my evaluation plan includes opinion data of students, teachers, and parents.
As you can see, there is no one way or a right way to do conduct your evaluation and determine results. Grant programs are complex and require you to evaluate component parts. Next week, we will look at two evaluation models to assist you and add to your toolkit of evaluation strategies.
Grant Guru Tip #28:
Types of Evaluation
Planning an effective evaluation begins when you first develop your proposal. As you identify your needs and develop your program goals and objectives, be thinking of how you would answer the following questions:
- What questions will your evaluation answer?
- What are the time frames for each evaluation activity?
- What kind of data will you collect?
- When you will collect your data?
- What will you use to collect your data?
- Are there any specific instruments you will use in data collection?
The data collection methods that you choose will be summarized and briefly described in the beginning of your evaluation plan. Your evaluation plan will make sense if you find the right fit between your selected methods, the questions you want to answer, and the information needs of those that are providing funding.
There are several kinds of evaluations that could apply to your program. There are internal and external, as well as national evaluations. Internal evaluations are completed by your project personnel. External evaluations are conducted by someone that your organization has hired as a consultant. Occasionally, national evaluations may be conducted by the grant-awarding agency. National evaluations look at a variety of geographic locations where a particular project has been funded. This evaluation focuses on looking at results across similarly funded projects and organizations. Most evaluation plans you will design will combine elements of internal and external evaluation. Some RFPs will require you to hire an external evaluator (for example a college or university). In these cases, you will determine the cost and have line item budget detail to support your need. When not stated how to do so in the RFP, your organization will need to choose who and how your program is evaluated.
Some funders will mandate an evaluation type. In these cases, the cost and evaluator may have already been selected. You simply include the cost as a line item in your budget. This kind of evaluation is common among national projects.
When the evaluation type is not spelled out in the RFP, your organization would normally contract with a faculty member or another educational entity. External evaluations are most often completed by faculty from a nearby college or university. As part of your planning process you would negotiate an agreeable cost to include in your budget.
Very small grants, for example, under $5,000, usually have less exacting evaluation designs. A small classroom is often evaluated with a simple survey of project staff and students involved in the program. Review the guidelines in the RFP carefully to determine what is required.
Educational research is complex and there are many human factors involved in teaching and learning. As tempting as it may be to try comparing groups of students to a particular treatment, ultimately, you will find this to be very difficult. Many external factors make finding equal comparisons groups almost impossible. Additionally you will need to determine if a cause-and-effect relationship takes place. Overall, you may find it easier to collect baseline data and compare with data at future points in time. Think carefully about your baseline data. What kinds of numerical baselines will you use? Do you have a pretest or posttest in mind? Will you be looking for quarterly progress or a one-year follow-up? How will you determine that you implemented your program as planned? Will your evaluator create written reports? Who will the evaluator share the gathered reports with?
Your evaluation plan will be strong if you are able to answer questions important to both the grantor and the grantee. What do you and your organization hope to learn, and what does the granting agent hope to learn? Blending the needs of both will create a strong project.
Grant Guru Tip #27:
Designing an Evaluation Plan
Evaluation plans tell you and your funder how you will know that your program is working and if not, why not. Good evaluation plans are tightly integrated into the proposal and links the evaluation to your project goals and objectives. When responding to an RFP, the evaluation plan usually is one of the last parts of your narrative.
As you plan your program, you need to ask yourself what do you want to learn? What information will you need to measure your effectiveness? What purpose will the evaluation serve? Who is the audience for the evaluation information? From what sources should you collect the information? In what format will you collect your data? What resources will you need to collect the data?
I was really surprised to find that there are over five million hits on Google for “designing an evaluation plan.” I didn’t realize this was a hot topic. Is it due to this age of educational accountability?
Just as with goals, objectives, and activities, the evaluation section of your proposal should continue to follow the same parallel structure that you have already started. For each major objective you should be able to measure the effectives of your activities/implementation. I like to keep the numbering of the evaluation in alignment with the rest of my proposal. For example, my objective 1a, will have an integrated activity 2a with an evaluation statement 3a. As always, refer to the RFP and follow the numbering and heading scheme provided in the grant directions.
When you develop your need statements, you usually have reasons why you have identified this as a need. Likewise, as you begin to think about the activities that kids and teachers will do, you plan how you will measure and report your project success. Your outcomes do not always provide you with positive results and so you need to build benchmarks into your proposal to keep your activities on track. Doing what you said you would do is why you were funded. Even if your outcomes do not meet your expectations, this result may be important to those that try to adopt or adapt your program methodology into the future.
Evaluation experts have their own language that the beginner grantwriter may find overwhelming. You don’t need to be an expert in evaluation but it helps to understand the language. Let’s take a look at some of this terminology. Evaluators use the term assessment to mean a measurement tool. If the measurement occurs over a specified time, it is known as a longitudinal study. Evaluators speak of qualitative and quantitative data. The main difference is that qualitative data usually involves the collecting of opinion surveys and anecdotal stories from participants. Quantitative data relies on numbers. Qualitative collects “soft” data while quantitative collects “hard” data. As you carry out your evaluation process, you operationalize your project to meet measurable outcomes. Your level of success is often measured by indicators. As you measure changes due to the result of your program, you are using benchmarks to match points in time. As your project progresses, you use the collected data and indicators to provide continuous improvement.
Grant Guru Tip #26:
Grant Activities
Grant activities are what you do to reach project goals. Well-designed projects have specific objectives that grantees interpret into the who, what, when, where, why and how of the proposal. Request for Proposals (RFPs) usually ask for a detailed plan of action or action plan. The action plan provides the grantor a blueprint of what you, the grantee will do, and when you will do it. We have already discussed how objectives are the steps toward achieving the goals of the project. Now, your job as grantwriter is to figure out how you will achieve each objective through your grant activities. These plans need to be detailed enough to help the reader understand that you have thought through the steps you will follow. On the other hand, these activities need to be concise enough so that you don’t go over your narrative’s page limit as specified in the RFP or promise more than you can deliver.
Each part of your plan, each activity, is best stated along with information about who is responsible for caring out the task. These action steps often add additional tasks for the people you work with. Before expanding other people’s daily responsibilities, it is good practice to bring in high-level administration early on in your project design. In some cases, grant funding may pay for additional help or extra duty pay for current employees. Seek guidance on how to proceed so that you build ownership and community for your project. The culture and readiness of your particular situation will help you determine if this kind of potential change is even possible. Having the perspective of others up-front will save you grief down the road.
Murphy’s laws will prevail. Nearly everything you plan to do will take longer than you think. Nothing will be as easy as it looks. And every solution will breed new problems. Although Murphy’s laws may be credited to events at Edwards Air Force Base in 1949, perhaps, or attributed to one of many other theories, the meaning is not lost here. When you write grants, you have a tendency to throw in as much as possible to build an exemplary solution set for the grantor. In doing so, you must keep asking yourself, will we have time to do this? How long will this activity take? What steps do I take if my actions create unplanned for reactions?
The RFP will guide you as to whether your action plan, persons responsible and timeframe go in separate sections or if you can combine them. When allowed, I like to create a table or a grid. This is an efficient way to communicate a great deal of information in a short amount of space. Grids or tables work well because it helps break-up the narrative for the reader. If the grid is particularly complex, turning the page into landscape-mode works well (along with the entire grant), as long as the RFP allows this. Be sure to read the RFP carefully, some RFPs are very prescriptive, even delineating the size of fonts you are allowed to use in a table. Some RFPs may forbid color graphs and charts. Remember that you have a variety of ways to display information in black-and-white, gray-scale shading, and fonts. Make sure that your grantor can easily photocopy your proposal easily. Particularly when using shading, keep it to a light shade so that it is presentable after photocopied.
Some processes you may describe just don’t work well in a grid. Steps to show continuous planning and improvement cycles are often times better served by diagram. Diagrams lend themselves to circular processes. Whether you choose to use grids or diagrams, do so when they condense your text and make it easier for the reader to see what you are trying to accomplish.
The activities you select are best described in terms of students, teachers, and other project participants. As you write your activities, if space permits, give the reader a word picture of what kids will do. This is particularly important for an educational grant. Keep your activities aligned and presented in the order that you have presented your goals and objectives. Keeping this parallel structure throughout your proposal will help the reader better understand your ideas. Next week, we will look at your proposal’s evaluation plan. It will be no surprise to you that we continue the same parallel structure.
Grant Guru Tip #25:
Goals, Objectives, and Outcomes
If you can write a lesson plan, I believe you can write a grant. Teachers write lesson plans every day. Grants are a lot like lesson plans. A lesson plan states objectives, has objectives, itemizes materials, describes steps, incorporates assessment, and is based on the needs of students. Grants do too. Today’s tip will look at the heart of a grant, the part that is most like a lesson plan.
At this point in your proposal you have introduced your project idea and have defended the need. Now it’s time to spell out what you are going to do. Most Requests for Proposal (RFP) guidelines ask for you to describe your plan in broad statements that we call goals. Goals are the targets we aspire to. Your goals are not necessarily measurable. When I write a grant for a technology-infused program I like to create at least three goals, a student goal, a teacher goal and an over-all program goal. Each goal will have its own set of objectives. Whereas a goal is a broad statement, the objective will describe the program in measurable terms. Objectives describe the program in terms of students and staff and tell the reader how the program will benefit the participants. The outcomes you select will become the basis for your program’s evaluation. What will change because of your project? The answers to this question are your outcomes. Now, how will you measure the change? The answer to this question will become your evaluation plan.
I like to also include a timeline for meeting the objectives. These points on a timeline are my benchmarks. Each objective should relate to an obtainable and measurable outcome. The art of grant proposal development is to find a balance on what you promise to do to meet the expectations of your funders and at the same time, not over-promise what you can actually pull off. I have noticed that in recent RFPs the term “measurable outcomes” has taken the place of “goals and objectives.” Although is not unusual for an RFP to ask for all three, goals, objectives, and measurable outcomes. Depending upon the RFP, goals, objectives, outcomes and timelines may lend themselves to charts within your narrative. The actual activities (for example, what students will do) are covered in the activities that follow your chart or goal/objective narrative.
Objectives are what you plan to do and your activities are the steps you take to complete your objectives. Your measurable outcomes are the changes that take place because of your activities. At different points in time, you should plan to collect indicators about how well you are doing in meeting your objectives and it is good practice to mark these benchmarks as milestones. You may wish to do this periodically; at the beginning and the end of the grant period; or at a pre-determined time, for example, quarterly. Your overall timeline will reflect your milestone events. This timeline also becomes your grant schedule, your to-do list, which will guide your program implementation. It will help keep you on track.
Knowing what will change and determining how you know the change will take place will lead to an outcome. Let me share with you a part of a grant that I wrote in the mid 1990s. These were the days of bilingual instruction in California schools and this particular school K-8 elementary school was in need of assistance in developing ways for limited English proficient students to access core social studies curriculum. My needs statements for this grant were divided into three parts, for students, for staff and for parents. I listed my needs and followed in parenthesis how I knew this was a need. For example my need for staff was as follows: staff need strategies and skills in social studies instruction with sheltered English and primary language instruction, as appropriate (limited number of language development certified teachers, bilingual program mandates).
I created a parallel staff objective to match the need. It read as follows: staff objective (and expected outcome, based on staff need): Staff is trained in sheltered English language techniques to increase students acquisition of the social studies curriculum. Staff commits to integrate technology into units of study to enhance student knowledge and provide anchors for limited English students. Each grade level selects social studies units of study and each teacher commits to enhance the unit with the creation of a technology-infused student project (a HyperStudio or PowerPoint presentation, a video production, or a social studies newspaper, for example). This objective was placed in a chart that went on to list the activities, the person responsible for each activity and a timeline for each activity over the course of the grant project. The evaluation section of the proposal paralleled the structure set-up previously in the need, objective, and activity sections of the proposal. The grant was easy-to-read and score. We were funded for over $20,000 and went on to create wonderful technology projects with our students.