Friday, December 16, 2011

Foundations Series: What is Assessed Curriculum?

I love Winter. Almost as much as I love Fall, which is my favorite session. And, almost as much as I enjoy talking about curriculum alignment. And assessment. Which is why you are in luck stopping by my blog today, because I am going to be talking about both curriculum alignment and assessment today. I am continuing my Foundations Series right here, right now! Specifically, I am going to dig into the question "What is Assessed Curriculum?"

But before I tackle Assessed Curriculum, I hope you don't mind if I spend just a little more time on why I love Winter so much. I promise, it will greatly enhance your understanding and appreciation for curriculum alignment and assessment. Or, at least, it will be loosely related.

So...Winter, why do I love you so? It's not that I really enjoy the bitingly-cold winds of a mostly-flat Iowa. But here are some things that I do enjoy about Winter: falling snow, playing in big snow piles with my dog Buddy, being warm and cozy inside watching TV with my wife Suzy and once again my dog Buddy, drinking a nice hot cup of coffee, and seeing my breath when I walk outside. For some strange reason, I just feel 7% cooler when I can see my breath outside. And believe me, I can use all of the cool points as I can get. Though I'm pretty sure that my wife would say believing this actually makes me not 7% more cool but 27% less cool.

You want to know what else I really love about Winter? I love wearing hats and gloves. I really only wear the same four pairs of shoes. But I do have a fairly extensive glove collection. My favorite is a pair of wool convertible glove-mittens. Ah yes, here they are:


Now these are cool gloves. Or should I say, really warm gloves. And so versatile, too. On cold, but not too-cold days, I can let my fingers breath in the crisp cold winter air. If it is just bitterly freezing, or if I'm going to be outside for a long time, I just flip the tops of the gloves over my frigid fingers and rock them mitten-style. Anyway, I feel like these gloves can do it all! Maybe you have a favorite pair of gloves that you love. Or maybe it's something else like blanket, a screwdriver, a dress, or a remote control. Whatever it is, we all have our favorites things. Now hold that thought, I'll come back to it in a little bit.

Let's look back a month, shall we? Last month, I spent some time digging into the question "What is Enacted Curriculum?" I get excited about the Enacted Curriculum for a number of reasons. First, its role in impacting student learning is huge, according to a review of research literature. But it also really interests and excites me because it's an area of education that is really, really hard to capture accurately. Think about it for a second, as a classroom teacher: do you have time to record everything that you actually taught each student? Most teachers I know do not, which is why survey methods are both interesting and promising.

Those are the sorts of things I talked about last month. This month, I am digging into the area of Assessed Curriculum. Recall that I use a multi-dimensional framework based on the work of Andy Porter that includes the intended, enacted, assessed, and learned curricula:


In this framework, the assessed curriculum is:

the knowledge and skills (i.e., the content) that are measured to determine student achievement.

In other words, the assessed curriculum is the "what" gets measured when we are trying to figure out where student learning is at. I know not many folks like using formulas to understand the world, but I do, so hang in there with me. I think of it like this:

content = stuff + what students do with the stuff

Simple enough, right? We are trying to dig into what students are learning when we assess. I'm guessing that most of you can get behind that idea. What is sometimes harder for folks to wrap their head around is that assessment is a type of curriculum. And I'm not just talking about things like the tests that come at the end of chapters in textbooks. I'll get into that a little later. 

The definition I used for the term curriculum in my blog "What is Curriculum?" is based on Andy Porter's work and goes like this:

Curriculum is what students are supposed to learn, what they get the opportunity to learn, what gets assessed, and what is actually learned.

So, as strange as it may seem, from this perspective assessment is part of curriculum Another perspective to take is a curriculum alignment perspective. We want the assessment practices we use to align with what we teach, right? Curriculum alignment is about the "what" or the "stuff" of curriculum. Assessments have stuff. What teachers teach has stuff. If this logic is getting distracting, don't get bogged down in it. Just take the ideas themselves from this blog and don't worry about it.

Types of assessment

Remember when I said to not let perspectives get in the way of the big ideas in this blog? Like, one sentence ago? Yeah, this is on of those times. Folks spend a lot of ink and hot air...er...normally-temperatured breath, arguing about what to call different types of assessment processes and tools. I don't really want to go down the debate path on the vocabulary here, though I'm happy to have that discussion with anyone who is so inclined. For now, I'd just like to briefly present an assessment framework that I find handy for understanding the different types of assessment decisions that can/should be made.

I provided a general definition of assessed curriculum above. I'd like to get a little more specific now. Check this description out:

Assessment is a system of processes and tools that are used to determine the extent to which students are acquiring or have acquired the knowledge and skills listed in the curriculum and delivered via instruction (Niebling, et al., 2008) In general, there are four types of assessment decisions:
  • Summative: Summative assessment tends to be comprehensive in nature, provides accountability, and is used to check the level of learning at the end of a unit of study. (RtI Action Network)
  • Formative: Formative assessment is a collection of evidence about student learning that is used to inform instructional decisions in an ongoing manner. Progress Monitoring, a type of formative assessment used in RtI systems, is a scientifically-based practice used to assess students’ academic performance and evaluate the effectiveness of instruction. It is the process used to monitor implementation of specific interventions. (RtI Action Network)
  • Screening: Screening assessment is a quick check of all students’ current levels of performance in a content or skill area. (RtI Action Network) The purpose is to help identify potential academic and/or behavioral concerns in need of additional assessment. (Midwest Instructional Leadership Council)
  • Diagnostic: Diagnostic assessment is used to confirm screening results and to inform intervention by determining a student’s particular academic needs. (RtI Action Network)
Hey, didn't you tell us to hold onto our thought about our favorite things?

Yes, yes I did. Thanks for reminding me. I want you to think about some of your favorite things again. And, if you have enough brain space left (it is the holiday season after all), think about what you know about assessment as well. I just laid out different purposes of collecting assessment information. Now let's see, what can my gloves teach us about making assessment decisions? After all, these are really a sweet pair of gloves. They can do all sorts of things. For example, they can train my dog, Buddy.


There's my guy. Pretty handsome, huh? I'm not quite sure why he's so spazzed about having fake antlers on his head. But clearly, he needs some training! Well guess what, my gloves can train Buddy!


Remarkable! As you can see, my gloves can give Buddy a command and sure enough, Buddy follows the command. Well, I'm feeling quite empowered now. I wonder what else my gloves can do?

I don't know about you, but I find driving during the holidays to be really stressful. And I've got so much else to do. Maybe my gloves can drive a car! If they can drive a car, then maybe they could go run errands for me. Let's see...


Ok, the gloves appear a little distracted, but they are still on the road. So far, so good...


Whoa! My gloves almost hit a strangely-calm dog and a very handsome man who were trying to cross the street.


Well, that didn't end very well, but it could have ended much worse. Maybe I expected too much too soon from my gloves. I mean, not only did they almost run over that dog and handsome man, they seem to be developing an attitude problem. Maybe I should scale it back a little bit.

I know! I don't really enjoy cooking dinner at night. I'd rather be reading about curriculum alignment and assessment. Perhaps my gloves could quick cook some dinner for me from time to time...


Now this seems to be working much better for my gloves. This glove has found the penne pasta and Alfredo sauce I set out. And it looks like my glove even got out some paper and something to write with, perhaps to write down a tasty recipe from a cookbook. I bet my other glove has started boiling some water to put the penne pasta into...


Yep, sure enough, there is my other glove, boiling a pot of water. I'm getting hungry just seeing these pictures. And I'm stoked, because it seems like I've found another amazing thing that my gloves can do! I'll check back in a few minutes to see how my penne pasta is doing........ok I'm back. Let's see how it's going...



Well, that didn't end too well, either. Maybe I've expected too much from my gloves. Maybe, after all of that, they didn't really train my dog Buddy at all. Maybe there was someone out of frame helping the gloves, making the gloves seem like they could do something they weren't really made to do.

If you are still reading this blog, congratulations, and thank you for putting up with my strange sense of humor. Let me get to my point. Like my gloves, like many of your favorite things, we may love them. We may think they can do a lot. But in the end, my gloves and you favorite things became our favorites because they were really good at one or two things, not everything.

Assessment practices and tools are the same way. When we try to make 1 minute oral reading fluency probes into diagnostic tools, they can let us down if that's all we use to try and get diagnostic data. When we try to turn measures that take 30-45 minutes per student into screening tools, we are likely to waste a lot of time and resources when we could have used a reliable and valid curriculum based measure (CBM), for example.

Furthermore, it's my opinion that when we demonize different assessment processes and tools, we are failing to take responsibility for our own actions or the actions of other educators who are misusing the processes/tools. Are you mad because state accountability assessments get misused and abused? Fine. But don't be mad at the measures. Be mad at the people misusing them. And help them learn more appropriate ways to use them. Because you know what, large-scale standardized assessments can provide helpful screening and/or summative data. They aren't useless, at least not in the hands of those who are properly trained to use them appropriately.

That's a bit of a soapbox, but I find it necessary to share my thoughts on this topic. Let's not just yell and scream about professional malpractice around assessment (yeah, I said it). Let's work together to promote and improve appropriate assessment practices.

What the research says about alignment and assessed curriculum

created using text from
http://en.wikipedia.org/wiki/Research
I'll tell you what research says about alignment and the assessed curriculum. After 20+ years, and millions of dollars invested, I can tell you with a very high degree of confidence the following:

Students do better on assessments when they've been taught the stuff that's in the assessments.

Genius, right! I know, I know. That is really common sense. But, having research data to back that up is handy. Here's what's really important though. This phenomenon holds for students who come from low socio-economic backgrounds, have low prior achievement, have disabilities, or belong to minority ethnic group. In other words, giving all students equity in opportunity to learn what they are to be assessed on can really help level to playing field. Check out the articles by Cohen (1987) and Gamoran and his colleagues (1997) to see some of this research.

Now, there are a few nuances to these findings. First, this doesn't mean all students can accomplish this learning by being taught the exact same way. Some kids will need more time so the content can be taught more slowly, or they may need to content taught differently. But we still need to give them an opportunity to learn. The second main nuance here is that it seems that when examining this opportunity to learn (i.e., alignment between the enacted and assessed curriculum), we have to look at both the topical/conceptual knowledge as well as the cognitive complexity of the content. These are topics for future blog posts, but I wanted to highlight them here.

Practical implications of research findings

Last month, I really hit on the importance of opportunity to learn for all students, which I reiterated with the summary of some research above. Instead of continuing to repeat myself, I'd like to take a slightly different angle here. Let me start with some questions (in no particular order, just numbered to keep track of the conversation):

  1. Have you ever heard a phrase that goes something like "this test is aligned to the standards"?
  2. Do you know how well aligned the the tests you acquire, are given, or create are with what you teach and/or state standards? (sorry, that's a long question)
  3. Do you know how well aligned the state tests are with state standards?
I'd be neglecting my professional duties if I let a question like #1 above go unchallenged. Really, "aligned"? Alignment is not a black and white issue. It's a matter of degree. There isn't, nor should there be, any test that covers all state standards (i.e., intended curriculum) or everything a teacher teaches (i.e,. enacted curriculum). Now, all of the items on a single test may all hit at least on or more state standards, or they may each be something that the teacher taught. But do you see how this question depends on which direction you looking at alignment? And we haven't even gotten into the multi-dimensional nature of alignment itself. Soon. 

picture source - Winners Dehli News
Regardless of shiny packaging or how large the promises, there has to be some evidence of where things are aligned and misaligned. And, for that matter, how the creator of said shiny packaging made those alignment determinations should be clear as well. Hey, those are two very Tweetable statements. Give me some Twitter love!

As an educator then, how would you respond to questions #2 and #3 above? Can you? Should you be able to? If the test isn't really measuring what is being taught, or at least what is planned to be taught (e.g., screening, pre-testing), is that really fair to the students or teachers? I'll let you decide.

If you hear absolute statements about curriculum alignment, please raise the red warning flag, or your baloney meter, because you're probably about to hear/read something that isn't entirely true. Even if you hear a more conservative statement, like "strong alignment," does the maker of that statement explain how they came up with that conclusion? If not, they should. You deserve better than that, and so do our students. 


Yet another revisit of textbooks and related materials

Let's dig into the matter of textbooks and related materials, shall we? It wouldn't be a very good blog post about curriculum and alignment matters if we didn't methinks. To date, I've made the claim that textbooks and related materials can be considered to be both intended and/or enacted curriculum, depending on how and when they are used. Furthermore, I've argued that textbooks and related materials shouldn't hold the honorable distinction of being exclusively considered "the curriculum." But can textbooks and related materials actually be considered assessed curriculum? You know I like scenarios, so here's one for your consideration:

picture source -
http://school.discoveryeducation.com/
At the end of the school year, a district's Director of School Improvement wants to know the extent to which the Common Core State Standards were assessed across the district. She started by asking teachers where they got their tests from that they used in the classroom (i.e., tests other than those directed by the state or the district). In this district, teachers overwhelmingly responded that they used the tests at the end of the chapters and units in their textbooks. Therefore, the Director decided to start her examination with the chapter/unit tests. What sort of alignment comparison would they be making?

I would say they are making an assessed-to-intended curriculum alignment comparison. Here is my thinking...

First, the desired examination is looking at things that were already implemented, similar to last month's scenario. And like last month, textbooks and related materials seem to be at the center of the desired examination. The difference in this scenario, however, is that a specific aspect of the textbooks and related materials is being examined: the chapter/unit tests. These are assessments. Second, and like the scenario last month, the desired alignment examination is with the Core Content Standards and Benchmarks (i.e., the intended curriculum). That's why I think that in this case, textbooks and related materials can be used in an assessed-to-intended curriculum alignment comparison. What do you think? Leave a comment here or tweet me.

Check back here next month when I discuss the next topic in the Foundations Series: "What is Learning Curriculum?" and how is it different that the assessed curriculum. Thanks again for taking the time to read my thoughts. I hope all of you have/had a great holiday season. See you here next month!

Resources Used

Cohen, S.A.(1987). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16, 16-20.


Gamoran, A., Porter, A.C., Smithson, J.L., & White, P.A. (1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19, 325-338.

Niebling, B. C., Roach, A. T., & Rahn-Blakeslee, A. (2008). Best practices in curriculum, instruction, and assessment alignment. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology, (4)5, 1059-1072. Bethesda, MD: National Association of School Psychologists.

Porter, A.C. (2006). Curriculum assessment. In J.L. Green, G. Camilli, & P.B. Elmore (Eds.), Complementary methods for research in education (3rd edition). Washington, DC: American Educational Research Association.

RtI Action Network. Glossary of Terms. Retrieved from http://www.rtinetwork.org/glossary.

Wednesday, November 23, 2011

Foundations Series: What is Enacted Curriculum?

Welcome to another fascinating installment of Alignment Foundations! Last month, I dug into the question "What is Intended Curriculum?" Last month, I defined intended curriculum as what students are supposed to learn. I also took issue with textbooks and related materials exclusively (or at least almost exclusively) defining the intended curriculum. However, I do acknowledge that textbooks and related materials do constitute at least part of the intended curriculum, before they are implemented, or enacted.

Hey, look at that! What a great segue into this month's topic in the Foundations series, "What is Enacted Curriculum?". The enacted curriculum is another part of the multi-dimensional framework developed by Andy Porter that also includes the intended, assessed, and learned curricula:


In this framework, the enacted curriculum is:

the content actually delivered during instruction (i.e., instructional content), as well as how it is taught (i.e., instructional practices). Typically, the content targets are based on the intended curriculum.

In other words, the enacted curriculum is what students get the chance to learn, as well as how teachers "deliver" the content. Now, I realize that most folks would rather spend more time talking about instructional practices, or the "how" of the enacted curriculum. It's understandable. The "how" of instruction is what teachers spend a good portion of their day thinking about and doing. However, curriculum alignment is fundamentally about the "what" of instruction, and this blog is about curriculum alignment.

OK, that's really not a good enough reason to focus on the "what" of enacted curriculum to any degree, much less an entire blog. I get excited about this work because of the potential positive impact for students, plain and simple. Let me share some research on alignment and the enacted curriculum.

What the research says about alignment and enacted curriculum

Don't worry, I'm not going to turn this into a thesis on curriculum alignment. As a matter of fact, I'm just going to talk, very briefly, about two studies. If you want a more extensive treatment of alignment research on student learning, check out this link. Otherwise, just stay here and keep reading :).

The first resource I'd like to share is actually a summary of several studies that was done by S. Alan Cohen in 1987. Now, I know what many of you are already thinking: 1987, seriously?!?! Is that the most recent research you can come up with? Old research is no good!

First, no, it's not the most recent research I can come up with, as I will share in a little bit. And second, the age of the research is really irrelevant if it hasn't been refuted through more recent research with equal or great methodology. In this case, I haven't found anything that meets those criteria. So, in my mind, this research is still applicable. Let's see what Cohen has to say, shall we?

          Cohen Studies. Briefly, what Cohen found across all of the studies he summarized in this article is that the alignment between what was taught (the enacted curriculum) and what was assessed (the assessed curriculum) had a significant and large impact on student learning (the learned curriculum). The findings were significant in that groups of students in higher-alignment situations performed better than students in lower-alignment situations. The findings were large based on the effect sizes calculated in the studies. Cohen found effect sizes to generally be between 1.0 and 3.0. If you want some information on how to interpret these numbers, check out this link. Here is my interpretation...THAT'S FREAKING HUGE!

          Gamoran Study. Adam Gamoran and his colleagues, in 1997, found in their study what many of us would consider common sense: as opportunity to learn what was assessed increased, so to did student outcomes. In other words, students did better on assessments when they had a chance to learn what was on those assessments. This is not earth-shattering news to most educators. What was particularly interesting in this study is that the alignment between the enacted and assessed curricula COMPLETELY WIPED OUT the impact of students' prior achievement levels, socio-economic status, and ethnicity, all factors that are typically considered to have negative impacts on student learning.

Practical implications of research findings

I believe that major point to walk away with from findings like these is that opportunity to learn/alignment has the potential to have both a significant and meaningful impact on student learning. So much so that at times, alignment can largely negate the impact on factors like poverty on student outcomes. Now, what I'm not saying is that poverty is not important. That would be foolish. What I am saying is that when we provide all students with an opportunity to learn, it can help level the playing field and students will generally learn.

If these findings and ideas are indeed the case, implications extend beyond credit requirements and textbook adoption cycles. This is about what students actually get a chance to learn. What happens day in and day out in classrooms and other learning environments crafted and facilitated by teachers. In my mind, to ensure all students have the opportunity to learn what they are supposed to learn (the intended curriculum) and what they are assessed on (the assessed curriculum), several questions are worth posing and pursuing. For example:

  • What processes and policies are in place to facilitate all students being able access these learning opportunities?
  • Are teachers comfortable teaching perhaps a broader range of students?
  • Are teachers comfortable enough in their content knowledge and ability to learn new knowledge to meet these demands?
  • What supports does the system have in place to ensure teachers can accomplish these goals?
These are not small questions. Fundamentally, it calls for the system to work as a just that, a system. Teachers need to not only have access to information and materials, but each other for conversations and collaboration. As a matter of fact, to accomplish these goals, I believe teachers should have access to what other teachers are teaching. As in, observation and self-report data. And by self-report data, I'm not talking about consensus curriculum maps or "model" lesson plans. I mean reflective data, i.e., what got enacted. Furthermore, I believe these sorts of data should not be used for accountability purposes. My reasoning is simple: attach high stakes to self-report and observation data, we can surely expect those data to decrease in reliability and validity. And without that, we have nothing of use.


Let's revisit textbooks and related materials, shall we?

Last month, I explored the issue of where textbooks and related materials fit into this curriculum framework, since I'm pushing so hard against the idea of just calling them our curriculum and calling it good. I like using scenarios, so here is another one:

picture source -
school.discoveryeducation.com
At the end of a school year, a district's curriculum director wants to know the extent to which the curriculum materials acquired over the summer were implemented across the district, and then how that implementation aligned with the Common Core State Standards. What sort of alignment comparison would they be making?

I would say they are making an enacted-to-intended curriculum alignment comparison. Here is my thinking...

First, the examination is looking towards what was implemented. In this case, I believe we can use the terms implemented and enacted to mean the same thing. Second, the examination is looking at implementation (i.e., what was enacted) relative to something else, (Common Core State Standards). I have previously used the Common Core as an example of intended curriculum. In this example, it's all about the timing. Since the examination of the curriculum materials is occurring after they are implemented (or not, as the case may be), the materials change from intended to enacted. Make sense? What do you think? I'd love to hear your perspectives.

Check back here next month when I discuss the next topic in the Foundations Series: What is Assessed Curriculum. Thanks again for taking the time to read my thoughts. See you here next month!

Resources Used

Cohen, S. A. (1987). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16, 16-20.

Ellis, P. D. (2010). What are some conventions for interpreting different effect sizes? Retrieved from http://effectsizefaq.com/2010/05/30/what-are-some-conventions-for-interpreting-different-effect-sizes/

Gamoran, A., Porter, A. C., Smithson, J. L., & White, P. A. (1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19, 325-338.

Porter, A.C. (2006). Curriculum assessment. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Complementary methods for research in education (3rd edition). Washington, DC: American Educational Research Association.

Wednesday, October 26, 2011

Foundations Series: What is Intended Curriculum?

Last month, I continued my Foundations Series on Curriculum Alignment by exploring the question "What is Curriculum?" Recall that I took issue with the idea that things like textbooks and related materials hold the exclusive right to be called "the curriculum," no matter how practical it may be to do so. I of course left this as the big cliffhanger to get you to check back in this month. ;)

Let's take a step back first, and set the stage for this discussion. This month, I am going to dig a little deeper into one aspect of curriculum, the intended curriculum. The intended curriculum is part of a multi-dimensional framework for working with curriculum developed by Andy Porter, with the other three dimensions being the enacted, assessed, and learned curricula (Porter, 2006). Visually, this is how I picture it:


In this framework, the intended curriculum is:

the content target for the enacted and assessed curriculum, statements of what every student is to know and be able to do. It is often captured in content standards or other similar documents that define specific points in time when students are to learn the knowledge and skills.

In other words, the intended curriculum is what students are supposed to learn. This may be a reasonable description of what the intended curriculum is (I'll let you decide that), but it does not address who should define the intended curriculum and how it should, or at least could, be defined.

Who and How Should the Intended Curriculum Be Determined?

This is a very sticky issue, one full of emotion for most educators. From my experience working with teachers and administrators, I'd say there is a strong feeling out there among many (I won't say most, because I don't know that) educators that the intended curriculum should be determined locally, not by politicians, test and curriculum developers, and other talking heads.

At the same time, I have also worked with many (again, I won't say most) educators that say that with all of the growing stressors and challenges of being a teacher, they would just like to be told what to teach, and be left alone as to how they should teach it. I'll get into the latter part of that statement when I blog about the enacted curriculum. For now, I'll stick to the what part of the conversation.

I'm guessing that many of you can appreciate this perspective (i.e., tell me what but not how). For me, I think it's completely understandable, and maybe even a partial explanation as to why many teachers rely so heavily on textbooks and related materials for their daily instructional decisions.

Currently, the United States continues more and more towards content being defined centrally/nationally, so to speak, not at the local level. No Child Left Behind created perhaps the most centrally-directed defining of intended curriculum at the time, requiring states to codify their own state content standards. The ante was significantly upped with the creation of the Common Core State Standards (CCSS) in English/Language Arts and Mathematics, which was led by the National Governor's Association (NGA) and the Council of Chief State School Officers (CCSSO). And it appears that there is now movement to create similar  documents in the areas of Science and Social Studies. You can see how these standards were developed here, at least for the English/Language Arts and Mathematics.

Even if a state adopts/adopted the CCSS, there is still room there for states (as in, state departments of education/instruction) to define an "additional 15%" of content (i.e., intended curriculum) that students are to learn. I put 15% in "" because there really isn't any guidance as to how you would determine how much is 15%. Still, it's latitude for states to define intended curriculum.

Now, I'm no math major (I was a psychology major). But reading what I just wrote, it looks like the CCSS and state's additions add up to no local/district decisions about content. So now I'm really shifting from the "who" to the "how" the intended curriculum should be determined. I do believe that their is an important role for folks at the national/federal, and state levels to determine the intended curriculum. Honestly, if the work is done defensibly well (a big if for many of you I'm guessing), I'm perfectly ok with the foundation of the intended curriculum being developed at the national/federal and state levels.

Why on earth would I say such a thing  you ask? I'll tell you why...the process these groups use is usually pretty rock-solid. Some really smart, experienced educators (i.e., folks from universities, national organizations, administrators, and teachers) get together and hash out their ideas to come up with what they think is best. Their ideas get nationally vetted by virtually every stakeholder group in education. They have professional editors polish up their work. As a result, the finished product is usually pretty darn good.

Now, that's not to say that it's perfect. There are often holes in this work. It will be biased to some extent 100% of the time. It will be missing some things and have things that many think shouldn't be there. Human beings write these documents. The voices of dissension can help shape and improve this work over time. In addition, decisions made at the district, building, and teacher level can help balance out many imperfections in the national/federal/state work. This compensation, if you will, comes in many forms. Discussions in professional learning networks, materials adoption and development, and lesson planning are just a few examples of work that can be done to bring both clarity, local context, and specificity to the intended curriculum.

At the same time, hopefully the national/federal/state work can provide compensation for shortcomings in the district/school/teacher level of decision making and implementation around the intended curriculum. In my mind, their are two big areas in which having a national intended curriculum can provide a tremendous benefit for teachers and students. First, having a common instructional target can help, if followed by teachers, eliminate discrepancies in opportunity to learn for all students. Second, a national intended curriculum can provide local educators freedom to use their dwindling resources focused on other things that repeatedly trying to write standards without the experience or resources necessary to do it well.

So...where do textbooks and related materials fit into this approach to defining curriculum?

You didn't think I was going to forget to respond to my very own question, did you? :) Let me try to address this question with a hypothetical example. Consider this scenario:

A district is planning to adopt and acquire new curricular materials for K-3 reading. They want to know the alignment between the content of materials from multiple companies and the Common Core State Standards. What sort of alignment comparison would they been making?

I would say they are making an intended-to-intended curriculum alignment comparison. The Common Core defines what students are supposed to learn in this example. The materials can't be enacted or assessed yet, because they haven't been adopted or acquired. The materials can't be learned, because students are the focus on what is learned, not materials. Whatever materials the district in this scenario selects will still be intended before they are implemented by teachers and students.

Ok, there it is. I have made my declaration: textbooks and related materials are intended curriculum. You know what though? I also think textbooks and related materials can be enacted and/or assessed curricula as well. "What" you say? Didn't I just say textbooks and related materials couldn't be enacted or assessed curricula? Well, guess what? You're just going to have to check back in next month when I explain myself with my next installation in the Foundations Series: What is Enacted Curriculum?

Thanks for taking the time to read. See you here next month!


Resources Used

Porter, A. C. (2006). Curriculum assessment. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Complementary methods for research in education (3rd edition). Washington, DC: American Educational Research Association.

Thursday, September 29, 2011

Foundations Series: What is Curriculum?

Welcome to my blog! If you got here from the monthly newsletter of the Midwest Instructional Leadership Council (miLc), welcome! If you found your way here from my Twitter feed, welcome! If you just stumbled here by accident, welcome! I hope you stay and check it out.

Last month, I "officially" kicked off my Foundations Series on Curriculum Alignment, although I technically started the month before by broadly defining curriculum alignment. To review:

Curriculum alignment is "the extent to which and how well all policy elements work together to guide instruction and, ultimately, facilitate and enhance student learning. (Webb, 1997)

The topic for this months' blog in the Foundations Series is exploring the question "What is Curriculum?" This is actually, in my opinion, going to be the most challenging blog post to write and discuss in the Foundations Series. Why? Because this concept/term gets tossed around and used by virtually everyone connected to education and it evokes strong emotion. Curriculum is therefore incredibly difficult to define to any degree of consensus. This EdWeek blog does a nice job of capturing these issues, in the context of the Common Core State Standards.

What is an alignment guy to do?


Are Textbooks the Curriculum?

Well, let me start off with what is likely to be viewed by at least some as at least ignorant, if not inflammatory: I think we need to stop calling textbooks and related materials the "curriculum." I am by no means anti-textbook. I understand that for many educators, their textbooks almost completely drive their instructional decisions from day-to-day. It is my opinion, however, that defining textbooks as the curriculum is incredibly narrow, preventing educators from having a comprehensive perspective on what curricular factors influence student learning. This is especially true for curriculum alignment, which requires a broader view of curriculum to really be worthwhile work for educators.

A Multi-Dimensional Approach

So, if textbooks and related materials don't hold the exclusive right to be called "curriculum" in my approach, then what is curriculum? In my curriculum alignment efforts, I've decided to start with the work of people who have been exploring this issue for much long than myself. In particular, I have found Andy Porter's multi-dimensional approach to be incredibly helpful in trying to define and understand the concept of curriculum. 

Perhaps the best resource I've found for defining and exploring curriculum by Porter is a chapter he wrote in the Handbook of Complementary Methods in Education Research called Curriculum Assessment, which you can download here from Porter's website (he has many other great resources there as well, I highly recommend checking it out). In this chapter, which is based on his larger body of work, he breaks curriculum down into four parts: (a) intended, (b) enacted, (c) assessed, and (d) learned. To quote:

"Curriculum can be divided into the intended, enacted, assessed, and learned curricula. For K-12 education, the intended curriculum is captured most explicitly in...content standards - statements of what every student must know and be able to do by some specified point in time. The enacted curriculum refers to instruction (e.g., what happens in classrooms). The assessed curriculum refers to student achievement tests. States, districts, and the U.S. government test various subjects at various grade levels. Teachers use their own tests to monitor student performance." (Porter, 2006) 

For me, this description of curriculum turns into the following picture:


This figure is likely familiar to most, at least in terms of its structure. Many have used a triangle to paint a picture of curriculum, though different terms are put at the corners of the triangle in different frameworks. I am of the opinion that this version of the triangle, when properly used, can incorporate any and all facets of any other approach to understanding curriculum, and then some. It is remarkably comprehensive yet simple, flexible yet consistent.

So...What is Curriculum?

Each of these terms will be explored in more depth in future blogs in the Foundations Series for Curriculum Alignment. For this blog, I use the following phrase when answering the question posed in the title of this blog, "What is Curriculum?":

Curriculum is what students are supposed to learn, what they get the opportunity to learn, what gets assessed, and what is actually learned.

Where do textbooks and related materials fit within this approach? My answer to this question, as it is to many questions, is...it depends. :) You'll have to check in to my Foundation Series blog next month, when I address the question "What is Intended Curriculum?" Or, hit me up on Twitter and ask. Thanks for reading my blog!

Resources Used

Porter, A. C. (2006). Curriculum assessment. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Complementary methods for research in education (3rd edition). Washington, DC: American Educational Research Association.

Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 8). Madison, WI: National Institute for Science Education, University of Wisconsin–Madison.

Monday, August 8, 2011

Introduction to Curriculum Alignment Foundations Series

Welcome back blog readers. As I've started on this new professional journey with the Midwest Instructional Leadership Council (aka, miLc), blogging has thankfully been something I can attend to more frequently. At least, for now it is. In the past, my blogging has typically been about something that has come across my radar that I felt compelled to comment on. For those that enjoyed reading about those mini-"inspirations" I plan to continue doing that here in this blog.

But I have also had many ideas lodged away deep in my brain to blog about when it comes to curriculum alignment, but lacked the vision and time to do the work. Well, as I said, that has changed. So, let me unveil the first of these ideas...

I just started a new line of blogs called the "Foundations Series." More descriptively, I plan to regularly blog about foundational curriculum alignment topics and issues. These blogs will occur once a month, and will coincide with the dissemination of the miLc monthly newsletter. You can subscribe to our newsletter by clicking here and fill out the form. Of course, you'll be able to read these blog posts just by coming here directly. But our newsletter will have plenty of other goodies in it, so you should check that out anyway :).

I actually posted my first blog in the Foundations Series last month when we sent out our first newsletter. You can read that blog post here. The Foundations Series will consist of the following topics, in order:

1. What is Curriculum Alignment?
2. What is Curriculum?
3. What is Intended Curriculum?
4. What is Enacted Curriculum?
5. What is Assessed Curriculum?
6. What is Learned Curriculum?
7. What is Alignment?
8. What is Directionality?
9. What is Dimensions?
a. What is Topical/Conceptual Knowledge?
b. What is Cognitive Complexity/Demand?
c. What is Emphasis?
10. What is Level of Analysis?
11. Pulling the Curriculum and Alignment Models Back Together

So, for you math fanatics out there, this adds up to about a year's worth of blog posts, once per month. I'll also have my topical posts and a series of other series I plan to launch soon as well. So you'll have plenty of alignment blogging goodness to enjoy. I'll also always tweet a link to my latest blog, so you might want to follow me on Twitter as well.

That's all for now. Stop back often!

Wednesday, July 20, 2011

Foundations Series: What is Curriculum Alignment?

The alignment of intended, enacted, and assessed curriculum, is an under-examined and misunderstood aspect of the educational process and often taken for granted. Despite this relative lack of attention, the term alignment can often be seen in the documents we read and the conversations we have with our colleagues. Alignment is a term that means different things to different people. So, how do we define curriculum alignment, and why does it matter?

In this blog posting, I'll begin to address the first of these two questions: what is curriculum alignment? If you'd like to read a "dictionary" definition, check this out. It's pretty good, actually. A bit wordy though. I like definitions that we have a shot at conjuring up ourselves if asked. I'm a fan of Norman Webb's definition. It goes something like this:

Curriculum alignment is "the extent to which and how well all policy elements work together to guide instruction and, ultimately, facilitate and enhance student learning."

That definition is a little less wordy, but could mean just about anything, right? I'm afraid that to really get at the heart of alignment, we're going to need to break this down a little more. And that, my blogosphere friends, will take some time. Lucky for me, this gives me plenty of things to blog about for awhile. For now, let me leave you with three thoughts:

1. Process vs Event. Curriculum Alignment, done well, is a process, not an event. Furthermore, the focus needs to be on student learning. If you can't draw a one or two step connection between the "alignment" work you are doing and how it will directly impact students, it may not be the best use of your time.

2. Defining Curriculum is Complex.

The term curriculum alone conjures up a wide range of responses and emotions. In a future blog, I'll dig into some different definitions, including the one I use in my work. For now, here's a picture of the framework I use to define and understand curriculum.


3. Defining Alignment is Complex. As I alluded to at the beginning of this blog posting, defining alignment is not an easy task. One blog entry can't do it justice. So, like I just did for the term curriculum, I present here for your viewing pleasure a picture of the framework I use to define and understand alignment.


Check back soon to see how I dig into curriculum alignment some more.

Resources Used

Porter, A. C. (2006). Curriculum assessment. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Complementary methods for research in education (3rd edition). Washington, DC: American Educational Research Association.

Porter, A C (2002) Measuring the content of instruction: Uses in research and practice Educational Researcher, 31, 3-14.

Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 8). Madison, WI: National Institute for Science Education, University of Wisconsin–Madison.


Thursday, June 30, 2011

My International Alignment Meeting

I had the distinct privilege of attending an international meeting for the Surveys of Enacted Curriculum (SEC) on June 27th and 28th, 2011. It was at Loyola University in Chicago. They were excellent hosts, and the facilities were outstanding. The conference was "international" because there was a gentleman there from Qatar. Qatar! I couldn't believe it. I had no idea how widespread SEC use really was. Apparently pretty widespread. I was very honored that John Smithson, one of the original creators of the SEC, invited me to attend.

Let me back up. The SEC is a set of processes and tools that can be used to examine the alignment between the intended, enacted, assessed, and learned curriculum. In other words, it can be used to explore the connections between what is supposed to be taught, what is actually taught, what is assessed, and what is learned. In my opinion, the SEC is the gold standard in measuring curriculum alignment. I highly recommend checking it out. Here are two links where you can learn some more about the SEC here and here.

Here are a few of the points I took with me from this meeting:
  1. Socially desirable response problems to the teacher survey don't really seem to be a major issue. In other words, it seems that most teachers respond honestly. This is important, otherwise the information would be worthless.
  2. SEC data should not be used to evaluate teachers. Furthermore, if the experience is going to be useful, teacher trust is a must.
  3. There could be a future for the SEC using teacher-developed assessments instead of just large-scale, standardized assessments for alignment work.
  4. The SEC still performs well, but there is a desire to collect more reliability and validity data for these tools.
  5. Student engagement becomes increasingly clear as a critical variable to explore when looking at alignment.
  6. The Common Core State Standards for English/Language Arts and Mathematics has been coded using the SEC language framework, and is now in their database. That means for anyone wishing to do alignment work with the Common Core, it is sitting there for comparison in the SEC database. Teachers just have to complete their survey, and then comparisons can be made.
I learned a lot more, but it's pretty nerdy, and I'd hate to bore you :). Rest assured, I got my alignment nerd on in a major way. I left the two-day meeting feeling very excited about the future of alignment in our country. There are some great minds doing great work. I believe it is our imperative then to both benefit from and contribute to this cause.

Monday, June 27, 2011

New Beginnings

Well, if you've made it to this blog posting, you know that I no longer work for Heartland AEA 11. Or, at least, you just figured that out because I just wrote it right there. Either way, here I am, separated from Heartland for the first time in eight years. It's sort of crazy to realize that for me even now, over two weeks into my vacation.

If you want to read my letter to Heartland's HR department explaining my decision not to sign a contract for the 2011-12 school year, click here. There's nothing very juicy in it. But it's my official statement on the matter. I will say this on the matter here in this blog, and just leave it that way: I am deeply committed to the work I do to help teachers and administrators make sense of data, and how to use those data to make the educational experience better for all students. I do this mostly with curriculum alignment data, though I do a fair amount with response to intervention (RtI) as well. That means I will always pursue employment opportunities that will allow me to pursue my passions.

So, what are my work plans now? I have joined a non-profit that I helped start a few years ago called the Midwest Instructional Leadership Council (miLc). Counting me, we have 2.5 employees, an Executive Director, and an amazing Board of Directors. I love working with each and every person associated with miLc. And I am stoked to be working for miLc. I feel refreshed and excited to continue pursuing my passions with this amazing group of world-changers.

We don't sell promises or products that will solve every problem. No silver bullets. Instead, we will work shoulder-to-shoulder with whomever wants to partner with us to bring evidence-based practices to the classroom. We do this with passion, determination, and enthusiasm. We work with teachers, schools, districts, education agencies, and anyone else that thinks we can help.

I sincerely hope that I can continue working with Iowans (and beyond) to achieve these goals. But even if I don't work directly with you, know that it is my sincere hope that all is going well for you personally and professionally. Take care and I hope to see you down the road.