Foundations Series: What is Dimensions? - Cognitive Complexity/Demand

Well, it's the end of June. Most students are out and about enjoying the summer break, as are many teachers. As well they should be. You all have earned a break after another long and challenging year, but hopefully a rewarding one. But knowing educators as I do, you're never really completely done or unplugged. So, I am here for you, to help you keep those neurons firing. And what better topic than curriculum alignment! Here we go.

Last month, I began to unpack the alignment dimension of topical/conceptual knowledge. I defined topical/conceptual knowledge as "the subjects, information, and ideas that students are supposed to learn," (Niebling, Roach, Rahn-Blakeslee, 2008) otherwise know as stuff kids need to learn. I know, not exactly fancy. But I think it gets the job done. This month, I am going to unpack the second dimension in the alignment framework I use, known as cognitive complexity/demand. I define cognitive complexity/demand as

what students are expected to do with the topical/conceptual knowledge. (Niebling, Roach, Rahn-Blakeslee, 2008)

The idea is this: during the schooling process, we don't just try to plug topics and concepts into students' heads to live there unused. We want our students to do something mentally (i.e., cognitively) with those topics and concepts. Oftentimes, we generically describe the "do something mentally" process as understanding. Now, understanding is a loaded concept; it could mean just about anything. My goal isn't to unpack this concept, but rather to use it as an example as to how we can be as objective as possible in dealing with cognitive complexity.

In the field of alignment, it is common practice to use a cognitive complexity/demand framework as a lens to examine the intended, enacted, assessed, and/or learned curriculum. One of the end products of such an examination is to determine or explain the type of mental activities called for in the curriculum. Or, to take it a step further, cognitive complexity/demand frameworks can help us examine the degree of alignment along this dimension between two different curricular elements (e.g., enacted to intended).

What are some examples of cognitive complexity/demand frameworks?

Great question! I thought you'd never ask. I use three different cognitive complexity/demand frameworks in my curriculum alignment work: (a) Bloom's Revised Cognitive Taxonomy, (b) Webb's Depth of Knowledge Framework, and (c) the Surveys of Enacted Curriculum Cognitive Demand framework. I'll explain each of these very briefly. The titles for each bullet point below are also links to more information about that taxonomy/framework.

Bloom's Revised Cognitive Taxonomy (RCT): Almost every educator has heard of "Bloom's." It comes up in almost every teacher preparation program. It's now called the Revised Taxonomy because, well, they revised it. There is a knowledge taxonomy as well as a cognitive taxonomy. The cognitive taxonomy is content neutral, and includes the following levels: Remember, Understand, Apply, Analyze, Evaluate, Create. It is organized, for the most part, from less to more sophisticated thinking, though there is overlap between the different levels. Furthermore, it is not assumed that a student must master skills at the lower levels to be able to engage in the higher-leveled thinking skills. Finally, when examining any curricular element, multiple levels may be assigned (e.g,. to a single standard or test item).

Surveys of Enacted Curriculum (SEC) Cognitive Demand Framework: The SEC cognitive demand framework is similar to Bloom's RCT, in that is generally organized from less to more complex. Furthermore, multiple levels can be assigned to a curricular element  (e.g,. to a single standard or test item). Unlike Bloom's RCT, the SEC is content-specific, with frameworks for English/Language Arts, Mathematics, Science, and Social Studies. For example, the framework for Mathematics is as follows: Memorize Facts, Definitions, Formulas; Perform Procedures; Demonstrate Understanding of Mathematical Ideas; Conjecture, Analyze, Generalize, Prove; and Solve Non-routine Problems/Make Connections.

Webb's Depth of Knowledge (DOK) Framework: Webb's DOK framework shares one primary similarity with the SEC: it is content specific. There are DOK frameworks for Reading and Writing, Mathematics, Science, and Social Studies. The generic labels for DOK levels are Recall, Skills/Concepts, Strategic Thinking, and Extended Thinking. Unlike both Bloom's RCT and the SEC, typical application of the DOK framework involves assigning one DOK level for each component of a curricular element  (e.g,. to a single standard or test item). There are exceptions to this practice. For example, in many cases, multiple DOK levels have been assigned to the Common Core State Standards in English/Language Arts and Mathematics.

Ok, that was a lot, and I really only scratched the surface. Cognitive complexity/demand is, well, complex. This post is already getting long, and I haven't even provided examples of using each of these approaches to cognitive complexity. I'm afraid I'll have to save that for another time. Hopefully the links can provide you with some additional helpful information. Let's dig into some research, shall we?

What does research say about cognitive complexity/demand?

To understand what research has to say about cognitive complexity/demand, it's important to know that this concept is also studied in the context of "rigor" and "higher-order thinking skills." This matters, because research in these related areas can provide us information on the role of cognitive complexity/demand when it comes to student learning. Some of what we know about cognitive complexity/demand and alignment is fairly broad, and certainly correlational. For example, international studies such as the TIMSS (Trends in International Mathematics and Science Study) have pointed to practices in countries where students outperform U.S. students that could ultimately help improve student learning in the United States. Included in these high-performing countries are a focus on fewer topics/concepts, and working with students to support deeper (i.e., more cognitively complex) thinking.

A more direct approach to examining the role of cognitive complexity/demand and alignment was taken by Adam Gamoran and his colleagues back in 1997. In this study they examined the enacted to assessed curriculum relationship to see how that impacted growth in student achievement. What they found was that as enacted to assessed curriculum alignment increased, so to did student achievement growth. Interestingly, this relationship was found only when cognitive complexity/demand was included in the alignment examination, while looking at just topical/conceptual knowledge alignment did not. Although correlational and not causal, this study was well designed and executed, and the results were compelling. In this study, alignment accounted for over 40% of student score variance at the classroom level. That means, out of all the things they studied plus the error that happens in every study, over 40% of what explained student scores was alignment. That's a pretty big amount. 

Unfortunately, the profound results of this study have not been replicated to the same extent since then when it comes to cognitive complexity/demand, though I'm hopeful that the work being done by Alexander Kurz and Steve Elliott with the My instructional Learning Objectives Guidance System (MyiLOGS) and my work with the Iowa Curriculum Alignment Toolkit (I-CAT), as well as continued work with the SEC will yield similar results in the future.

What are the practical implications?

In my mind, there are several very practical implications of cognitive complexity/demand. But what is practical for me isn't necessarily practical for most everyone else. With that said, there are a few biggies worth mentioning here:

1. Leaning about cognitive complexity/demand: You should learn about cognitive complexity/demand frameworks, as well as how to use them. It's not magic, and the work can be challenging and at times frustrating. But if you start to get it, my guess is that you will start to think about your instruction and assessment in a different way, and hopefully help you implement not only a more rigorous enacted curriculum, but a tighter-aligned enacted curriculum as well.
2. Designing and delivering instruction: Whether you use a published set of textbooks/materials or develop your own, it's not good enough to look at the standards or state test and say that some part of your instructional materials "matches" them. That's not even good enough for topical/conceptual knowledge. Typically, in my experience, cognitive complexity/demand doesn't even enter this situation, but it needs to. Use a cognitive complexity/demand framework in addition to your standards to help you select or design instructional materials. How you ultimately deliver those materials is your enacted curriculum. What sort of cognitive processes did your students engage in?
3. Designing and delivering assessment: I'll make this one easy. Everything I just said in #1 applies to designing and delivering assessment.

Final thoughts

If nothing else, I hope you remember this: it's all about student thinking, and frameworks help us capture the type of thinking found in the different curricular elements. We've got a pretty compelling research foundation for the importance of cognitive complexity/demand, though that research is primarily correlational. I also think that cognitive complexity/demand is really the bridge between what we teach and how we teach it.

That's it for this month gang. Check back next month when I dig into the next alignment dimension, emphasis. Until then, follow me on Twitter. Thanks for reading!

References

Gamoran, A., Porter, A. C., Smithson, J. L., & White, P. A. (1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19, 325-338.

Niebling, B.C., Roach, A.T., Rahn-Blakeslee, A. (2008). Best practices in curriculum, instruction, and enacted curriculum. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology 5 (Vol. 4), 1059-1072. Bethesda, MD: National Association of School Psychologists.

Foundations Series: What is Dimensions? - Topical/Conceptual Knowledge

Greetings alignment fans! I hope that all of you had a fantastic Memorial Day Weekend, and are feeling rejuvenated. I know I am. I spent the weekend with my wife and dog in Fargo, ND visiting some dear friends. Believe it or not, I didn't even think about alignment. Ok, I hardly thought about alignment. Sometimes it just jumps right in ol' noggin. So, last month, I more or less took off from blogging. Nevertheless, I'm here to dig into another foundational alignment concept. Before I get too far down that path, let's briefly recall the working definition for the term alignment:

"the extent to and how well all curricular categories and the elements within them (e.g., content standards, instructional content, and assessment practices) work together to guide instruction and, ultimately, facilitate and facilitate student learning." (e.g., Webb, 1997)

The point of this definition isn't just that alignment means that curriculum is coordinated/similar. The bigger point is that this coordination is done for the purpose of supporting student learning. If that's not our clear purpose, then our work is misguided. Ok, moving along...

The month before my brief hiatus was focused on the question "What is Directionality?" You can see the overall alignment picture for the framework I use, as well as where Directionality fits, under the green oval on the left.

This month, I will focus on the question "What is Dimensions?" You can find that in the figure above under the blue oval in the center. That question, I realize, is a bit clunky the way it is phrased. Think of it this way: the term Dimensions is a categorical label. Anyway, Dimensions is such a broad and important concept, I'm actually going to be breaking it down into three sub-categories, each of which I will explore in single blog posts. I will start with the alignment dimension of Topical/Conceptual Knowledge.

What is topical/conceptual knowledge? 

Briefly, topical/conceptual knowledge is:

the subjects, information, and ideas that students are supposed to learn. (Niebling, Roach, Rahn-Blakeslee, 2008)

A less elegant way to state that definition is this: stuff kids need to learn. Through this lens, we aren't concerned about what students are supposed to do with the "stuff" they are learning, what sort of cognitive activity we hope to evoke in their minds. No, this is really just about the facts, topics, ideas, and concepts they are supposed to learn. Here are some examples:

• math facts
• the Civil War
• photosynthesis
• author's voice

That's pretty straightforward, right? As stated, the above examples are just topics or ideas. Let's ratchet this up a notch, shall we? Let's look at some Common Core standards and see if we can pick out the topical/conceptual knowledge in them. I'll provide on English/Language Arts example and on Mathematics example. Here we go...

English/Language Arts

Standard RI.5.3. Explain the relationships or interactions between two or more individuals, events, ideas, or concepts in a historical, scientific, or technical text based on specific information in the text. 

What is/are the topical/conceptual knowledge of this standard? Think for a minute. Then scroll down and see what I think. :)

Got your answer? Here's what I thought. I believe the topical/conceptual knowledge in this standard can be found in this portion of the standard: "relationships or interactions between two or more individuals, events, ideas, or concepts in a historical, scientific, or technical text...". The standard calls on students to learn about relationships or interactions that are detailed or described in different kinds of texts. Do you agree or disagree?

Mathematics

Now, let's try a math standard:

Standard 3.MD.4. Generate measurement data by measuring lengths using rulers marked with halves and fourths of an inch. Show the data by making a line plot, where the horizontal scale is marked off in appropriate units—whole numbers, halves, or quarters.

Ok, same as before. What is/are the topical/conceptual knowledge of this standard? Think for a minute. Then scroll down and see what I think.

Got your answer? Here's what I thought. Here are the parts of the standard statement that I thought contained the topical/conceptual knowledge of the standard: "measurement data...halves and fourths of an inch...line plot...horizontal scale...appropriate units—whole numbers, halves, or quarters." In my opinion, this one wasn't quite as neat and clean as the ELA example. But I believe you can see several concepts described in the standard. There are a variety of types of measurement data (e.g., temperature, length, weight, etc.), each of which can be broken down into different units. Data can also be displayed in a variety of ways (e.g., line plot, bar graph, pie chart, etc.), again broken down into different units.

In each of these examples, we've isolated the topical/conceptual knowledge called for by the standard, but we haven't gotten into what students are supposed to be able to do with that topical/conceptual knowledge. That is critically important, but I will save that issue for next month :).

What does research say about topical/conceptual knowledge?

Sadly, we know little from research about topical/conceptual knowledge. Candidly, most research on opportunity to learn and alignment does not break down Dimensions into different categories. There really isn't any acknowledgement that Dimensions is a "thing." In my opinion, we can look to two general areas of research that can shed a little light on the subject. One area is descriptive, the other predictive.

Descriptive research

When it comes to descriptive research, we can look to work done to describe the degree of alignment between two curricular elements (e.g., intended, enacted, assessed, learned curricula). Norman Webb's (e.g., 1997) work on alignment does examine what I would consider to be a form of topical/conceptual knowledge. I would say his version of this concept is what he calls categorical concurrence. Basically, there is a high degree of categorical concurrence across standards and assessments if the same or consistent categories of content appear in both the assessment and the standards.

Andy Porter and his colleagues (e.g., Porter, 2002) have created a multi-dimensional alignment framework known as the Surveys of Enacted Curriculum (SEC). Part of the SEC framework includes a set of topical descriptors that relates quite closely to the concept of topical/conceptual knowledge. Examples of these topical descriptors includes "Linear equations" and "Main idea(s), key concepts."

Predictive research

Unfortunately, we know even less about the predictive nature of topical/conceptual knowledge than we do about it's descriptive potential. Put another way, just looking at topical/conceptual alignment hasn't been specifically explored much in research. Perhaps the most telling piece of research comes from Gamoran and his colleagues (1997) who found that just looking at topical/conceptual knowledge alignment didn't predict how well students would perform on assessments. Only when cognitive complexity was added to the analysis could those types of predictions be accurately made. We'll revisit the cognitive complexity issue next month.

What are the practical implications?

In my opinion, the practical implications boil down to a few simple ideas.

1. It can be helpful to just think about topical/conceptual knowledge alignment first before getting into cognitive complexity. I can't provide you with empirical support for that opinion. It's based on my conversations and work with teachers and administrators who consistently tell me that thinking about cognitive complexity is harder. 
2. Most work done under the title of "alignment" really only looks at topical/conceptual knowledge, and typically at a coarse-grained level (more to come in future blogs on this concept). Which is a decent-enough place to start, but limited. 
3. The work gets more interesting (and challenging) when we start looking at cognitive complexity in addition to topical/conceptual knowledge.

Final Thoughts

In my experience, most folks can wrap their heads around the idea of topical/conceptual knowledge as being a "thing." For the most part, folks are also relatively comfortable looking at topical/conceptual knowledge alignment. Perhaps you fit into this description. Where there rubber meets the road, however, is with cognitive complexity. So, that is where I shall end this blog, and it is where I will pick the next alignment foundation series blog. Until then, hit me up on Twitter, and happy aligning!

References

Gamoran, A., Porter, A. C., Smithson, J. L., & White, P. A. (1997). Upgrading high school mathematics instruction: Improving learning opportunities for low-achieving, low-income youth. Educational Evaluation and Policy Analysis, 19, 325-338.

Niebling, B.C., Roach, A.T., Rahn-Blakeslee, A. (2008). Best practices in curriculum, instruction, and enacted curriculum. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology, 4(5), 1059-1072. Bethesda, MD: National Association of School Psychologists.

Porter, A C (2002) Measuring the content of instruction: Uses in research and practice Educational Researcher, 31, 3-14.

Webb, N.L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 8). Madison, WI: National Institute for Science Education, University of Wisconsin-Madison.

Foundations Series: What is Curriculum?

Welcome to my blog! If you got here from the monthly newsletter of the Midwest Instructional Leadership Council (miLc), welcome! If you found your way here from my Twitter feed, welcome! If you just stumbled here by accident, welcome! I hope you stay and check it out.

Last month, I "officially" kicked off my Foundations Series on Curriculum Alignment, although I technically started the month before by broadly defining curriculum alignment. To review:

Curriculum alignment is "the extent to which and how well all policy elements work together to guide instruction and, ultimately, facilitate and enhance student learning. (Webb, 1997)

The topic for this months' blog in the Foundations Series is exploring the question "What is Curriculum?" This is actually, in my opinion, going to be the most challenging blog post to write and discuss in the Foundations Series. Why? Because this concept/term gets tossed around and used by virtually everyone connected to education and it evokes strong emotion. Curriculum is therefore incredibly difficult to define to any degree of consensus. This EdWeek blog does a nice job of capturing these issues, in the context of the Common Core State Standards.

What is an alignment guy to do?

Are Textbooks the Curriculum?

Well, let me start off with what is likely to be viewed by at least some as at least ignorant, if not inflammatory: I think we need to stop calling textbooks and related materials the "curriculum." I am by no means anti-textbook. I understand that for many educators, their textbooks almost completely drive their instructional decisions from day-to-day. It is my opinion, however, that defining textbooks as the curriculum is incredibly narrow, preventing educators from having a comprehensive perspective on what curricular factors influence student learning. This is especially true for curriculum alignment, which requires a broader view of curriculum to really be worthwhile work for educators.

A Multi-Dimensional Approach

So, if textbooks and related materials don't hold the exclusive right to be called "curriculum" in my approach, then what is curriculum? In my curriculum alignment efforts, I've decided to start with the work of people who have been exploring this issue for much long than myself. In particular, I have found Andy Porter's multi-dimensional approach to be incredibly helpful in trying to define and understand the concept of curriculum. 

Perhaps the best resource I've found for defining and exploring curriculum by Porter is a chapter he wrote in the Handbook of Complementary Methods in Education Research called Curriculum Assessment, which you can download here from Porter's website (he has many other great resources there as well, I highly recommend checking it out). In this chapter, which is based on his larger body of work, he breaks curriculum down into four parts: (a) intended, (b) enacted, (c) assessed, and (d) learned. To quote:

"Curriculum can be divided into the intended, enacted, assessed, and learned curricula. For K-12 education, the intended curriculum is captured most explicitly in...content standards - statements of what every student must know and be able to do by some specified point in time. The enacted curriculum refers to instruction (e.g., what happens in classrooms). The assessed curriculum refers to student achievement tests. States, districts, and the U.S. government test various subjects at various grade levels. Teachers use their own tests to monitor student performance." (Porter, 2006) 

For me, this description of curriculum turns into the following picture:

This figure is likely familiar to most, at least in terms of its structure. Many have used a triangle to paint a picture of curriculum, though different terms are put at the corners of the triangle in different frameworks. I am of the opinion that this version of the triangle, when properly used, can incorporate any and all facets of any other approach to understanding curriculum, and then some. It is remarkably comprehensive yet simple, flexible yet consistent.

So...What is Curriculum?

Each of these terms will be explored in more depth in future blogs in the Foundations Series for Curriculum Alignment. For this blog, I use the following phrase when answering the question posed in the title of this blog, "What is Curriculum?":

Curriculum is what students are supposed to learn, what they get the opportunity to learn, what gets assessed, and what is actually learned.

Where do textbooks and related materials fit within this approach? My answer to this question, as it is to many questions, is...it depends. :) You'll have to check in to my Foundation Series blog next month, when I address the question "What is Intended Curriculum?" Or, hit me up on Twitter and ask. Thanks for reading my blog!

Resources Used

Porter, A. C. (2006). Curriculum assessment. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Complementary methods for research in education (3rd edition). Washington, DC: American Educational Research Association.

Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 8). Madison, WI: National Institute for Science Education, University of Wisconsin–Madison.

Introduction to Curriculum Alignment Foundations Series

Welcome back blog readers. As I've started on this new professional journey with the Midwest Instructional Leadership Council (aka, miLc), blogging has thankfully been something I can attend to more frequently. At least, for now it is. In the past, my blogging has typically been about something that has come across my radar that I felt compelled to comment on. For those that enjoyed reading about those mini-"inspirations" I plan to continue doing that here in this blog.

But I have also had many ideas lodged away deep in my brain to blog about when it comes to curriculum alignment, but lacked the vision and time to do the work. Well, as I said, that has changed. So, let me unveil the first of these ideas...

I just started a new line of blogs called the "Foundations Series." More descriptively, I plan to regularly blog about foundational curriculum alignment topics and issues. These blogs will occur once a month, and will coincide with the dissemination of the miLc monthly newsletter. You can subscribe to our newsletter by clicking here and fill out the form. Of course, you'll be able to read these blog posts just by coming here directly. But our newsletter will have plenty of other goodies in it, so you should check that out anyway :).

I actually posted my first blog in the Foundations Series last month when we sent out our first newsletter. You can read that blog post here. The Foundations Series will consist of the following topics, in order:

1. What is Curriculum Alignment?
2. What is Curriculum?
3. What is Intended Curriculum?
4. What is Enacted Curriculum?
5. What is Assessed Curriculum?
6. What is Learned Curriculum?
7. What is Alignment?
8. What is Directionality?
9. What is Dimensions?
a. What is Topical/Conceptual Knowledge?
b. What is Cognitive Complexity/Demand?
c. What is Emphasis?
10. What is Level of Analysis?
11. Pulling the Curriculum and Alignment Models Back Together

So, for you math fanatics out there, this adds up to about a year's worth of blog posts, once per month. I'll also have my topical posts and a series of other series I plan to launch soon as well. So you'll have plenty of alignment blogging goodness to enjoy. I'll also always tweet a link to my latest blog, so you might want to follow me on Twitter as well.

That's all for now. Stop back often!

Foundations Series: What is Curriculum Alignment?

The alignment of intended, enacted, and assessed curriculum, is an under-examined and misunderstood aspect of the educational process and often taken for granted. Despite this relative lack of attention, the term alignment can often be seen in the documents we read and the conversations we have with our colleagues. Alignment is a term that means different things to different people. So, how do we define curriculum alignment, and why does it matter?

In this blog posting, I'll begin to address the first of these two questions: what is curriculum alignment? If you'd like to read a "dictionary" definition, check this out. It's pretty good, actually. A bit wordy though. I like definitions that we have a shot at conjuring up ourselves if asked. I'm a fan of Norman Webb's definition. It goes something like this:

Curriculum alignment is "the extent to which and how well all policy elements work together to guide instruction and, ultimately, facilitate and enhance student learning."

That definition is a little less wordy, but could mean just about anything, right? I'm afraid that to really get at the heart of alignment, we're going to need to break this down a little more. And that, my blogosphere friends, will take some time. Lucky for me, this gives me plenty of things to blog about for awhile. For now, let me leave you with three thoughts:

1. Process vs Event. Curriculum Alignment, done well, is a process, not an event. Furthermore, the focus needs to be on student learning. If you can't draw a one or two step connection between the "alignment" work you are doing and how it will directly impact students, it may not be the best use of your time.

2. Defining Curriculum is Complex.

The term curriculum alone conjures up a wide range of responses and emotions. In a future blog, I'll dig into some different definitions, including the one I use in my work. For now, here's a picture of the framework I use to define and understand curriculum.

3. Defining Alignment is Complex. As I alluded to at the beginning of this blog posting, defining alignment is not an easy task. One blog entry can't do it justice. So, like I just did for the term curriculum, I present here for your viewing pleasure a picture of the framework I use to define and understand alignment.

Check back soon to see how I dig into curriculum alignment some more.

Resources Used

Porter, A. C. (2006). Curriculum assessment. In J. L. Green, G. Camilli, & P. B. Elmore (Eds.), Complementary methods for research in education (3rd edition). Washington, DC: American Educational Research Association.

Porter, A C (2002) Measuring the content of instruction: Uses in research and practice Educational Researcher, 31, 3-14.

Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education (Research Monograph No. 8). Madison, WI: National Institute for Science Education, University of Wisconsin–Madison.