The Living Schoolbook

I spoke this morning with John Gosney from IUPUI about their plans to move the Goal Management project forward. One of the functional areas that seems to need a little attention was the need to have multiple rubrics. A while ago I emailed the OSP portfolio list this bit:

Rubric needs GM-122

During our time in Ann Arbor we talked about the need for managed rubrics.

I don't know if anyone has any experience with other rubric management or creation tools out there...I know rubristar (http:// rubistar.4teachers.org) because of another project I was involved in a few years back. After revisiting the rubristar site and talking with Joe Shedd about his expectations, I thought I would bounce some ideas off the community about the requirements for GM-122 (http:// jira.sakaiproject.org/jira/browse/GM-122).

A rubric is usually represented by a matrix of goals/outcomes (usually shown on the left side) and a rating scale (shown across the top) and in each cell is some statement about performance indicators that would inform an evaluator how to choose a score for each outcome. I think that a common use of the OSP matrix has been to set up a structure where the goals are the "Criteria" and the "Scales" and statements are embedded in the evaluation forms in each cell of the matrix. The horizontal axis of the matrix is often thought of as a "time" or "stage" dimension.

When I think of this in terms of Goal Management, I see the linking of Goals with a Rating Scale as setting up a rubric structure. The current implementation provides a blank text area where the author of the assignment/datapoint/wizard page/matrix cell can write in plain text how they plan to score student work. This works ok if there is no programatic effort to organize how the rating scales are used, but the next step is obviously to support such an effort. I believe that there are a few such efforts that we are hearing that we would like OSP to support. Note, from here on out, when I say "rubric" I am using it to describe the descriptive text that would go in each cell of just one row of a "real" matrix (performance indicators as they relate to just one Goal).

Here are some of the rubric organization scenarios I have heard about in my conversations:

1. Managed assessments: Some rubrics are rather specific to (and must be tied to) a particular assessment item and must be approved by an "assessment coordinator" for educational QA purposes as part of a larger assessment system strategy. Changing the assessment/rubric in this case involves more than just the teacher. (Saginaw, Syracuse, RINET)
2. Generally reusable (but unchangeable) rubrics: Some rubrics may be general purpose rubrics that are NOT tied to an assessment, but the dissemination of these approved rubrics may be a strategy of an institution to push forward an agenda of best practice for assessment by providing a handy reference library of general purpose writing, mathematics and science rubrics (for example). While the choice whether or not to use one of these "off the shelf" rubrics (and which one) is left to the teacher, providing some information to the teacher about the schools expectations of its students at different stages (and perhaps suggesting an appropriate rubric for this grade level/stage of development) would make this service more valuable. (Indiana?)
3. Reusable rubric templates: Similar to the above, but the library of "off the shelf" rubrics are merely starting points. There is not a priority to ensure that everyone is doing assessment the exact same way. When a teacher uses one of these rubrics, they can easily edit the performance indicators to suit their needs and create a new rubric, just for their new assignment. (Rubristar approach)
4. Sharing of rubrics: This is a bottom up approach to establishing "best practice". As the teachers create their own rubrics against goals, they have the opportunity to publish them as part of the "reusable" library so other teachers can use/edit/republish them. (Someone?)

Do these make sense? Which efforts do you think we should place a priority on? Should they all be supported eventually?