Over 100 Years of American Film History in 1 Spreadsheet

Click here for the spreadsheet

The American Film Institute's theater outside Washington, DC -Photo by Kate Mereand

Twenty years ago on June 16th 1998, I was on vacation in the Outer Banks, and I couldn't tear myself from a show on CBS. Even as a relatively young person, I already had the priorities of a octogenarian. While the people I was vacationing with were busy doing things like partying in the sand, relaxing by the pool, or playing cards and games in adjacent rooms, I was glued to the television.  Needless to say, I was not the coolest guy on the trip. I even grabbed my artist sketch pad I had brought to do drawings, so I could keep notes on the proceedings. My priorities may not have been yours, but, dammit I was committed to them.

The American Film Institute had put together a TV special based on counting down the one hundred greatest American films of the 20th Century. It was a broadcast so specifically designed to appeal to all of my key interests that to this day I worry I may have been used as a guinea pig in some scientific experiment that designed it. I loved movies. I loved ranking things. I have an inherent subservient tendency that love when authority figures tall me what is valuable in art. That's why I love when the Oscars declare with autocratic confidence what was best in film in a given year.  Well this special, titled AFI's 100 Year... 100 Movies, felt like an Oscar ceremony for a whole century of movies!

A love affair began between me and the American Film Institute that continues to this day. The next day I all I wanted to do was talk about the list with people. I want to debate it, and analyze it, and break it down to the nth degree. Instead of wasting the wall space of my bedroom on photos of supermodels or rock stars I clipped a newspaper article with the AFI's list and taped that up.  Years later I would rejoice at the opportunity to move close to the AFI Silver theater in Maryland. I was so excited to take part in their many events celebrating the past and present of filmmaking. Then, like any good creep, I would then try to form a relationship with the good folks there over the internet.

In the years following my initial introduction to their work AFI created more rankings shared across more specials. Like Waylon Smithers with Mr. Burns, I would dutifully knowledge each one, copy down everything they said, and nod in agreement. Over the years the AFI ranked the 100 greatest comedies, the 100 greatest romances, the 100 most thrilling movies, the 100 most inspiring movies, the 50 greatest heroes and 50 greatest villains, the Top Ten of ten different genres of film, and several more lists denoting cinematic greatness. I recorded all of these carefully in notebooks before moving on to spreadsheets on a computer. While the AFI released their final list ten years ago, I have gone back to those lists over and over again for my own purposes.

Eventually, I began to consider the idea of compiling all of those lists into a single resource. In a move that my therapist called "an obsessive's dream" I took it upon myself to create one comprehensive List of Lists ("Because YOU demanded it!"). Thankfully AFI was generous enough to share tons of very useful information on their website. There I could find the lists from their 13 special events and the background information of all the lists of nominated films they used to build their rankings on their website. From those raw materials I crafted a single spreadsheet to summarize everything that was great from over 100 years of American film history. 

If you think my work with AFI lists is obsessive, you should see my Oscars research

Now here comes the part of the piece where I describe my methodology in excruciating detail that almost no one will care about. However I have spent too much time in the company of academics to skip this step. So if you want to insight into my obsessive mind, read on! Everyone else can skip to the next picture. After that I will talk about fun things you can do with the List of Lists.

The first step in making a List of Lists is determining all the lists you have to list. Over 11 years the AFI produced 13 events announcing a set of rankings. They were, in chronological order:

• June 16, 1998- AFI's 100 Years...100 Movies, ranking the 100 best American movies to that point all-inclusive (all other lists will also be limited to films produced in the United States or with major involvement from Americans) 
• June 16, 1999- AFI's 100 Years...100 Stars, ranking the 25 greatest male and 25 greatest female movie stars who had made their film debut before 1950 or who had died prior to the creation of the list (so more like the 50 most popular stars you'll find on TCM)
• June 13, 2000- AFI's 100 Years...100 Laughs, ranking the 100 greatest comedies
• June 12, 2001- AFI's 100 Years...100 Thrills, ranking the 100 most thrilling, or "pulse-pounding" in the words of the telecast, movie; there was not a strict genre restriction on this list like there were on others
• June 11, 2002- AFI's 100 Years...100 Passions, ranking the 100 greatest love stories in films both dramatic and comedic (but disappointing some, none that were pornographic)
• June 4, 2003- AFI's 100 Years...100 Heroes & Villains, ranking the 50 greatest heroes and 50 greatest villains portrayed in films
• June 22, 2004- AFI's 100 Years...100 Songs, ranking the 100 greatest musical moments, not limited to those in movie musicals or even songs written for films, rather any instance of music being used in the body of a movie
• June 21, 2005- AFI's 100 Years...100 Movie Quotes, this one should be self-explanatory
• September 23, 2005- AFI's 100 Years of Film Scores, ranking the 25 greats film scores, this was presented in a special one-night only event at the Hollywood Bowl instead of a broadcast
• June 14, 2006- AFI's 100 Years...100 Cheers, ranking the 100 most inspiring films; again this list was not confined to any specific genre
• September 3, 2006- AFI's Greatest Movie Musicals, ranking the 25 greatest musicals, this was also presented at a live performance rather than over television
• June 20, 2007- AFI's 100 Years...100 Movies – 10th Anniversary Edition, revising the original list of the 100 greatest movies, both to incorporate films since released and changing opinion regarding certain films (so goodbye to 1915's The Birth of a Nation)
• June 17, 2008-AFI's 10 Top 10, ranking the 10 greatest films of 10 genres: animated films, court room dramas, epics, fantasy, gangster movies, mysteries, romantic comedies, science-fiction, sports movies, and westerns.  

My main purpose for this spreadsheet is to create a database of movies judged to be of holistically high quality. That means that some of these lists would not be conducive to my goal. 

The list of 50 stars was the most difficult to deal with as I could not figure how to account for it. The AFI surely did not mean it as an endorsement of every film in the combined body of work of over 50 actors. I determined that The Plastic Age did not merit inclusion in the spreadsheet by virtue of an uncredited appearance by Carole Lombard. Had the AFI based inclusion on that list for specific films starring the actors as they did with the heroes and villains then I could use it. Instead I did not include that list at all.

Also I divided the 10 Top 10 into ten separate lists. Since each top ten was based a specific list of nominees that occasionally overlapped, it seemed appropriate to treat them as distinct entities.


The list of heroes and villains had some unusual features. Heroes and villains were both given the same number of ranked spots, but were not equally represented among the nominees. Further the nominees were not categorized as heroes or villains. Thus I treated it as one list of characters with split rankings.

This left me with a total of 21 lists to consider. The lists represented a combined 1,949 films ranging chronologically from 1912 to 2006. Some where represented in multiple rankings, while a majority of them were only nominated on a single list. Simply listing all the nominated films would leave me with a useless catalog that would not communicate the various ranking the AFI went to so much trouble to create. To try to capture the relative merits of each movie according to the lists, I would need a system to account for the frequency and placement of a film among the combined rankings.

First I decided to divide the rankings into tiers. I did not want to simply rank films by their ordinal number for a few reasons. There was no clear way to deal with the hundreds of nominees which had no ordinal ranking. Since I cannot be privy to the specific mechanations that determined the rankings, I couldn't be sure how much value to assign the difference between the 53rd and 52nd film on a list. However tiering could assign proper weight to the kind of intuitive ranges that the juries creating this lists likely had in the back of their minds. In a list of 100 films distinguishing the top 50 from the top 25  or top 10 makes sense. Tiers also made it easier to compare scores across lists.

From there I would assign each film a score for appearing on a list in any position weighted relative to it that position. I awarded a movie a number of points equal to the number of films nominated for that list divided by how many other films fell in the same tier (or higher) as that movie ( #Nominees/# in Tier=Points). If that quotient didn't result in an even number I rounded to the nearest integer. Thus each film that was nominated and not ranked earned 1 point (#Nominees/#Nominees=1). Then the higher up a list a movie ranked the more points it would earn. Lists of 100 would be have tiers for their top 50 and top 25. All lists had a tier for the top ten. Then the top film on each list was awarded as many points as there were nominees in the list ( by the logic that it was in a tier of 1 and #Nominees/1= #Nominees). Isn't this all sounding more interesting than anything you could do on a subtropical beach?

Ranked films that were drawn from larger pools of nominees (like the 500 nominees considered for the greatest comedies) earned more points than those from smaller pools (the 10 Top 10 lists each came from only 50 nominated films). That gave appropriate weight for films that stood out among crowded fields versus those selected within narrow genre restrictions.

The heroes and villains list again posed some challenges. Since I had decided to treat it as one list of 100 characters, I chose to split the points evenly between the two sub-categories. So I awarded the top 25 heroes and top 25 villains the number of points they would have earned for being in the top 50 of another list. Then I created a top 20 tier instead of a top 25 to include an even number of both goodies and baddies while also observing a number that seems like an intuitive division within such a list. The top ten and number one of both lists were treated as ties and shared the respective points (i.e. #Nominees/2).

Because the 10th Anniversary was openly an attempt to improve upon the original list by altering the rankings of some of the films, I also included an adjustment based on movement from one list to the other. The formula was designed to reflect both upward and downward changes by subtracting or adding a number of points equal to the difference between the movie's position on the 1998 list and its average position on both lists.

In keeping with my goal to create scores that were reflective of a film's totality, I grouped the lists into different two groups. Most lists ranked films as a whole or for a major component (e.g. the hero or villain of the story). I added all of those together for one score. In the List of Lists I kept those on the sheet labeled "List Set 1". For the lists that only ranked relatively minor aspects of a movie (the score, musical moments, and quotes), I included those points in a secondary score, the sheet labeled "List Set 2". You may choose to rely on either score as you see fit.

After that I could begin using the List of Lists to explore any number of research areas. I tried to treat the points like Bill James treats Win Shares. I keep their limits in mind, but I use them as a tool when answering certain questions. If creating the list was an obsessive's dream, then this was the sort of extended fantasy sequence for an obsessive that Hollywood might portray in a lavish musical number.

Unlike a beach vacation the fun never ends with a database like this!

The first and most obvious thing to do with a List of Lists is to combine the scores from all lists. This gives you an impression of which films succeeded in multiple area. While the movies that were number one on a list will dominate, they will be separated by scoring well across multiple genres, or by have attributes like be both thrilling and inspiring.

For example, Casablanca, was ranked on eight lists and was nominated for another. The Wizard of Oz reached the top 10 in six different lists including topping two of them. Perhaps most impressive is Schindler's List which was not the top movie on any list yet had the 11th highest primary score and 14th highest secondary score, topping many movies that had claimed a number one ranking! (I included that exclamation point so you can understand how much more fun I find this than whatever fun and games other people were having in the Outer Banks all those years ago.)

Another use for the list is to group films together in various ways. I included the year each film was released in my data and combined this with their cumulative scores to compare what films stood out most from each year. You can compare this to contemporary attempts to adjudicate the best movie each year, say for example the winners of the Academy Award for Best Picture. Those comparisons help us understand some of what goes into helping a film stand the test of time.

Given the amount of shade that gets tossed at Oscar and all his friends at AMPAS, you might expect hindsight to reveal the Best Picture winners to frequently underperform. Yet the AFI and the Academy actually agree on the best picture quite often (but of course they don't always agree). In the first year the Oscars were awarded based on the calendar year a film was released, 1934, both picked It Happened One Night as the top film. The last year of movies that were released before the majority of AFI lists were determined was 2002. In that year we again have a consensus pick in Chicago. In between those two extremes the Best Picture and highest score film line up may other times, e.g. Titanic, Unforgiven, Out of Africa, The Deer Hunter, The Godfather, The Sound of Music, On the Waterfront, Casablanca, etc.

However I also found plenty of disagreements that were pretty revealing. The Greatest Show on Earth is frequently cited as one of the weakest picks to win Best Picture. The AFI certainly supports that judgement. Across all 21 list it only scored 1 point. The highest scoring film of that year, 1952, was Singin' in the Rain has one of the ten highest point totals for any of the nearly 2000 films listed. And it wasn't even nominated by the Academy! Perhaps the happy and poppy musical seemed light weight compared to the big scale production by Cecil B. DeMille. However hindsight has allowed critics to recognize that beneath the song and dance numbers is an interesting story about the end of the silent movie era with a nuanced message about artistry and authenticity.

Two of the also-rans from 1987 accumulated more than ten times the points of that years Best Picture winner. While The Last Emperor is the sort of historic epic that people tend to associate with the Academy Awards, both Moonstruck and Fatal Attraction are better remembered today. Maybe the Academy overlooked them because they are stories primarily driven by women. Remember that #OscarsSoWhite campaign? This helps demonstrated that #OscarsSoMale too.

Conversely this method can identify some instances where the Oscars must have been very competitive. In some years multiple movies will have very high point totals. The race in 1977 may have come down to the Star Wars movie that started it all and eventual winner Annie Hall. They both racked up lots of points with the AFI with the winner depending entirely on which list set you favor. The Sound of Music had to face Doctor Zhivago in 1965, and according to my primary set of scores they are exactly tied! The year before may have been even harder though. Three of the nominees for Best Picture -Mary Poppins, My Fair Lady, and Dr. Strangelove: Or How I Taught My Eyes to Skim Past the Inordinately Long Subtitle- all rank among the 100 highest scoring films of all time.

I even tool breaking things down by year one step further. I assess each year based on how many films from that year appear in the List of Lists, how many total point were earned by the films from that year, and the average number of points earned by the listed films of that year.  You can find that data in the sheet labeled "Year by Year", including a set of charts to help visualize the numbers.

Measuring years as a unit tells an interesting story of cinema history. The number of listed films each year is low through the early years of the 20th century reflective of an art form still establishing itself. Things pick up in the late 1920's as the silent film era peaks, then really take off with the introduction of the talkies. After that noteworthy films appear each year at a fairly steady pace with the exception of a brief drop in the mid-1960's marking the gap between the "Old" and "New" Hollywood eras. The numbers only tail off in the 21st century because a decreasing number of the AFI lists cover those years. This suggests that the film industry has basically been in good health since the introduction of sound.

The point totals and averages provide some more nuance to the story. The total points show some distinct peaks in filmmaking. The end of the 1930's and start of the 1940's stand out far above all other years. 1939 has frequently been described as the greatest year in American movies, and this data certainly bears that out. It has the most total points by far and the best average to a less extreme extent. This marks the peak of the studio system in Hollywood when movie making enjoyed a level of artistic and financial success it will likely never see again. Unfortunately the effects of World War II damaged the film industry taking away talent and resources for a higher purpose. The movies regained their footing, but they wouldn't hit another height like that until the period from 1959 to 1962.

The graph showing the average points has a shape very similar to a wave pattern. This suggests that the art form rises and falls as every few years new styles, approaches, and technologies revitalize it. While it appears as though the wave is ebbing with the peaks decreasing, I believe this is a natural effect of human retrospection. Those movies that have had longer to influence others and being studied and appreciated will of course have higher standing. However as they fade from living memory and more recent films have their time in the sun the shape of the wave will shift. Doubtlessly some folks in the mid-20th century would have argued that the silent films they remembered were far superior to the musicals and special effect laden blockbusters that followed. I conclude from this that the art of movie making is as alive and vibrant as ever with new peaks still to come.

You can do so much more with this data. I have already tried a few different experiments that I may share in later posts. Hopefully you will be inspired by the 20th anniversary of the AFI 100 Years... series and pursue your own ideas. If you do please tell me about them in the comments. Until then thank you for joining me in celebrating the wonder of the movies and strangeness of a guy who would rather watch a TV show about movies than party on the beach.

Does Gravity Have a Schedule? Or: When Will Reality Hit Trump?

Gravity is undefeated

The cleverest thing I heard during the 2016 U.S. Presidential campaign came from Republican commentator Rick Wilson. He was describing the success of Donald Trump to that point by saying, "Donald Trump is like a man who jumped off a hundred story building, has fallen to about the fiftieth floor, and now thinks he can fly." (I can't find an exact transcript of his remark, so forgive me if I paraphrased any of this incorrectly.) Basically he was alleging that gravity still applied to Mr. Trump even if he hadn't felt its full effects yet.

Now obviously Mr. Wilson was operating under the assumption that then-candidate Trump would soon hit the ground in the form of losing to Hillary Clinton by an embarrassing margin. However, since we live on a planet God is punishing for some reason, the former reality-TV star actually won that election. (Oh, in case any of my previous material didn't make it clear, I'm a super-liberal Democrat. The results of 2016 have pushed me to be more open and vocal about it on my microblogging/mass-texting service.) To this day now-president Trump still has the appearance of a man in midair. While for a time we all began to question whether he would ever touch the ground, increasingly it seems as though the pull of gravity will force him to Earth soon. The most obvious downward pressure on the President may be the investigations which many expect to produce evidence of criminal wrongdoing. You also have to consider the political opposition that has risen up like bitchy yeast in the aftermath of the Republicans' 2016 victories. By this time next year the odds are very good President Trump will have to deal with Democrats controlling at least one house of Congress. However I would like to highlight a different force that may cause our POTUS to plummet and on a predictable schedule too: betrayal by Republicans in Congress.

My theory stems from an interview one unnamed Republican member of Congress gave to notable conservative media figure, Erick Erickson. This Congressman (while concealing their exact identity Erickson apparently felt it safe to let us know what kind of genitals this MoC has) contacted his buddy Erick to vent about the frustrations of being a member of Donald Trump's party when he is constantly causing political problems. Some people have taken part in a parlor game to guess this mysterious public servant's identity, but I would rather focus on the content of some of what he said and try to extrapolate from it.

Mr. Erickson begins his story by quoting the unidentified lawmaker as saying, "If we're going to lose because of him, we might as well impeach the motherf**ker".  That certainly raised my eyebrows. The brows reached new heights when I then read that this secretive legislator has gone on television multiple times to defend said motherf**ker. Thus he confirms the suspicion some have that other Republicans in DC hate the man they nominated and elected, even if they won't come right out and say it. That made me wonder if there are a set of conditions in which the members of the President's party on Capitol Hill would ultimately brings the President crashing down.

Thankfully we get an indication of what that might look like later on in Erickson's piece where he provides for the full context for the opening line.

Judiciary is stacked with a bunch of people who can win re-election so long as they don't piss off Trump voters in the primary. But if we get to summer and most of the primaries are over, they just might pull the trigger if the President fires Mueller. The sh*t will hit the fan if that happens and I'd vote to impeach him myself. Most of us would, I think. Hell, all the Democrats would and you only need a majority in the House. If we're going to lose because of him, we might as well impeach the motherf**ker. Take him out with us and let Mike [Pence] take over. At least then we could sleep well at night.

He also details the internal strife Congressional Republicans are experiencing, the ramifications of the Democrats expected reclaiming of House of Representatives in the next election, and all the ways President Trump has infuriated him. In all it paints a picture of a President who is much closer to hitting pavement than he might realize. It's really just a question of when, not if, the gravity of the situation become more than he can overcome.

Crucially the concealed Congressman hinges all of his predictions of impeachment on the condition that President Trump fire the special counsel currently investigating him. That is an important caveat to keep in mind as all of my following predictions cannot come to pass with that inciting event. However since it is already something many people are already speculating about, some reasonable parties are preparing for, and that almost happened once already, I think it's fair to treat the attempted termination of the investigation as read. 

With that out of the way, let's focus on the meat of this obscured politician's veritable smorgasbord of revelatory comments. He focuses on the House Judiciary Committee as the decisive factor in all of this as they are the folks in Congress who would first consider any articles of impeachment against a sitting president. He predicts they could do this once they no longer need fear reprisal from voters that support Trump. While he vaguely refers to "the summer", we can actually be much more specific than that by working out the exact date when enough committee members will cross that threshold.

We can begin determining the date by defining the threshold. The House Judiciary Committee has 39 member which means for any articles of impeachment to be successfully voted to the full House would require at least 20 votes. Our anonymous official is likely correct that all Democrats (or close enough to make no difference) would vote to impeach President Trump, and they represent 17 votes in Judiciary. That should mean we only need 3 Republican votes, except I doubt it's as simple as that. In the modern House nothing moves with support primarily from the minority party, especially when Republicans control it. Former Republican Speaker of the House Dennis Hastert, established the "Hastert Rule", which stated simply that a majority of the majority party must support something for it to be voted on. Republicans have typically followed that rule, which would mean at least 11 and preferably 12 of the Republicans on Judiciary would have to be willing to vote for impeachment before we'd see any actual action. So when will 12 members of the committee from the GOP be free from the so-called "Trump base"?

Our next step will be to identify any Members of Congress in this pool of 22 who cannot afford to lose the support of the maybe 25% of voters who are deeply devoted to President Trump. We cannot judge this purely on those who speak in favor of the President publicly as our unnamed elected office holder clearly demonstrates that is no proof of true support. However a few people on this committee have demonstrated a loyalty to Trump, his voters, or his agenda with more than just lip service.

I believe 5 of the 22 are "Trump loyalists" who have gone above and beyond the call of party loyalty. First Rep. Steve King has repeatedly represented himself as a member of the "Trump Base" and certainly shares the President's views on some key issues. Next we can note that Rep. Jim Jordan has not only actively sought to undermine the special counsel, but that he has made some pretty indefensible claims supporting Trump as well. Then we should acknowledge that both Reps. Louie Gohmert and Andy Biggs have called for removing Robert Mueller from the investigation. So they are pretty unlikely to support any impeachment vote resulting from the President doing exactly that. Finally, I would not expect Rep. Matt Gaetz to turn on Donald Trump when they already share so much, including political mentor Roger Stone.

Another 4 committee members may need all the Republican votes they can get for a general election. Two of them, Reps. Karen Handel and Steve Chabot, both face potentially tough races for reelection, according to Cook Political Report. Two other current members are leaving the House as they each run for governor of their respective states. In Idaho Rep. Raul Labrador is leading in the polls over other Republican primary candidates, and Rep. Ron DeSantis is currently enjoying a similar position in Florida. Even if neither of them wins their primary they have clearly shown an interest in higher office suggesting that neither of them would want to upset a large segment of Republican voters.

That leaves 13 of our original 22 a possible votes for impeachment once they are free to vote their conscious. We can list 7 of them as "free" right this moment. Reps. Lamar Smith, Darrell Issa, Ted Poe, and Trey Gowdy are all retiring from Congress and seem to be leaving the worries of pleasing Republican primary voters behind them. More importantly the chair of the committee Rep. Bob Goodlatte is retiring as well. If the chair wasn't willing to hold a vote in committee there would be no chance of impeachment. On top of that we can add Rep. John Ratcliffe who is already past his primary and Doug Collins who has no opponent in his.

Then it's just a matter of looking through the calendar and finding when each of the remaining members is finished with their primary. Whether these incumbents win or lose, they will be past the concern that a faction of Republican voters can end their careers. The 11th member will become "free" on June 27th, before the Sicario sequel hits theaters. If you want to be extra sure the majority of Republicans would be free to vote for impeachment, you could wait until August 15th when Rep. Jim Sensenbrenner who has already voiced his support for the special counsel investigation.

So depending on how events pass we may not have to wait for November, 2020 to see the Trump Presidency come undone. We may not even have to wait for Labor Day. Keep that in mind if you ever find yourself stressed about political news.  President Trump may still act like he can fly, but the rest of us know it's only a matter of time before gravity wins.


 

A Mathematical Proof 2500 Years Too Late

As a teacher I learned that one of the most important steps in educating someone is understanding how they think through a problem. To best understand how you take in and retain information, you must "unpack" your own thought process. You have to acknowledge every step your mind takes when it studies, analyzes, and solves a problem. It can be difficult to slowly and deliberately review each step of your thinking process like this, but you will be rewarded with improved problem solving skills. It's just like taking apart a machine to understand how it works so you can then make better use of it and take better care of it.

I recently went through this process myself when I got a particular mathematical ant in my pants. I am currently reading a novel that made a mention of the Pythagorean Theorem in passing. The character in the novel had such natural mathematical talents that the Theorem made immediate intuitive sense to her. Reflecting on this savant-like brilliance, I tested my own understanding of the Theorem. Almost everyone who has studied geometry remembers Pythagoras's most famous equation,  "A squared plus B squared equals C squared." If you were a good enough math student, you also remember that this equation describes a natural relationship between the three-sides of any triangle that has one corner at an angle of 90-degrees a.k.a. a right triangle. My problem was I couldn't remember how to prove the Pythagorean Theorem. This seems especially silly to me because Pythagoras originally demonstrated his proof about 2500 years ago.

In mathematics it is never enough to simply trust that an equation or rule works, you must be able to prove it. You can't just demonstrate it inductively either. To prove Pythagoras it would not be acceptable to show that 3^2 + 4^2 = 5^2 (feel free to pause for a second and check that arithmetic if you want) and a triangle with sides measuring 3, 4 and 5 units would have a right angle. Mathematicians don't take a rule that applies to a handful of examples and say, "I assume it works like that all the time."  An effective proof takes established rules and applies them to a generic example so as to create a universally applicable statement.

Lacking the ability to leave questions unanswered, I took my ignorance as an opportunity to test my mathematical thinking. I also decided to write down my process so that I could "unpack" the logical process necessary to prove something in math. Because I have blogged about education topics in the past, I though some segment of my readers might take an interest in unraveling the bundle of thoughts involved. By making my geometric contemplations explicitly clear and simple, I hope to help fellow educators find ways to help their students think through math problems.

An illustrative triangle

To prove the Pythagorean Theorem I needed to use deductive reasoning. So I began by writing down all the facts I could think of that related to triangles. From those points I could begin building my reasons to demonstrate the relationship between the sides of a right triangle.

I began by noting that for any triangle the angles of the corners must add up to exactly 180 degrees. This makes intuitive sense, because you can imagine a triangle as a path that takes three turns to point you in the opposite direction. As we all know, a reversal of direction is 180 degrees -or pi radians if you prefer.

Next I recalled the equation for calculating the area of a triangle, Area= 1/2*base*height. You can see how this works with a simple thought experiment. If you take any triangle and add a conjoined copy like a reflection across one of the sides, you have quadrilateral (i.e. a 4 sided shape) with an area equal to the triangle's base time's the triangle's height. Naturally the area of the original triangle is half the area of this new quadrilateral or 1/2*base*height.

The last fragment of knowledge I drew on was a little more advanced. I knew that there was a relationship between the size of an angle at the corner of a triangle the length of the side opposite to it. This is the basis for those trigonometric functions that probably traumatized most people in geometry class, sine, cosine, and tangent. This makes intuitive sense when dealing with a right triangle. The largest side, called the hypotenuse, is always opposite of the 90 degree angle (which must be, by definition, the largest angle in the triangle).

An example of a 45-45-90 triangle

This is useful because it establishes that in a right triangle with two equal angles of 45 degrees each, a 45-45-90 triangle, the two sides other than the hypotenuse must be equal because they are opposite equal angles.

The reason I focused on the example of the 45-45-90 triangle is because it has some special properties. In the same way we could imagine a quadrilateral made out of two identical triangles, we can create one square out of four identical 45-45-90 triangles. Just picture the hypotenuse as one side of the square with the four 90 degree angles joined in the middles.

One square which contains four 45-45-90 triangles

We can analyze this square to help prove the Pythagorean Theorem (in case you forgot during my babbling about triangles facts, that was the point of this exercise). First, we establish the total area of the square. Since all sides of the square are the length of the hypotenuse,or side C, we can easily calculate its area. The area of a square is the same as any quadrilateral the length multiplied by the height. In a square those lengths are both the same as the length of one of the sides. So area = side * side = side^2. In this square the length of the side is called C so the area is C^2.

We also know that the total area of the square is the same as the combined area of the the four triangles. As discussed above the area of a triangle is given by the equation area = 1/2*base*height. You multiply the area of one of those triangles by 4 and you get this:

4 * area = 4 * (1/2*base*height) = 2*base*height = total area of square

Since the base and the height, which we can call side A and side B, are both the same length we can simplify the above equation for the square's area.

2*base*height = 2*side*side = 2 (side^2) = side^2 + side^2

We can substitute side A or side B in for either side length in this equation. That allows us to rewrite the area of the square as Area = A^2 + B^2 = C^2. This proves the Pythagorean Theorem for all 45-45-90 triangles. 

I started with proving the Theorem for 45-45-90 triangles, because I knew they had special properties that made them easier to work with. Unfortunately you can't create a simple square out of every right triangle. Still, knowing that I had worked the proof for one class of triangles, gave me an insight for working a broader proof. In order to prove Pythagoras correct for all triangles, I had to find a more sophisticated method that I could apply to any type of right triangle. I was able to work from the foundation of my earlier exercise by searching for ways to create squares using the sides of triangles.

My breakthrough came when I realized that any right triangle can be used to define a square area. (This was important because, as I would learn later, earlier generations had used this as the basis for other proofs of Pythagoras.) You can form a square by arranging four identical copies of so that their right angles form the corners of the square.

The square defined by the combines sides of  four copies of a triangle

In order to keep the sides equal length we'll combine all the triangle legs of length A against all the legs of length B. This gives us a square where the length of each side is A+B. Since we get the the area of a square by multiplying the length of a side by itself, we can calculate the are of this square in the following way:

Area = (A+B) * (A+B) = A^2 + 2AB + B^2

When you arrange the triangles in this manner, the hypotenuse of each triangle forms the side of another smaller square with the square discussed above. Since all the sides of this square have the length of the hypotenuse, which we call C, you can define the area of this square as C^2. The area of the square with sides of A+B can be redefined as the area of the hypotenuse square plus the area of the four triangles. Each of the triangles has an area of 1/2*A*B.

Area = C^2 + 4(1/2*A*B) = C^2 + 2AB

We know from our earlier definition of the larger square's area that the area is equal to A^2 + 2AB + B^2. Therefore

Area = A^2 + 2AB + B^2 = C^2 + 2AB

Because the two quantities are already equal, we can perform any mathematical operation to both sides and they will still be equal. In this case we can subtract the quantity 2AB from both sides because in exists in both expression. This leaves us with A^2 + B^2 = C^2. This proves the Pythagorean Theorem for all right triangles.

Just for clarity, I do not recommend any math teacher tries to explain the Pythagorean Theorem using my meandering thought process. Instead I hope teachers of all kinds appreciate that though my method may not seem perfect it achieved the desired result. The purpose of unpacking is often to find the twists and kinks in a person's reasoning, but it can also demonstrate that differing methods of thinking can be acceptable. Students need to know that there is no one right way to go about thinking. Every teacher should encourage students to unpack their thinking, so they can see that -however they go about reasoning- they can always find ways to sharpen their thinking. You can always improver your problem solving  even if it means solving a problem 2500 years too late.

Number Crunching the Oscars

There are many things a person can do in anticipation of the annual Academy Awards telecast. Plenty of people make plans to hold or attend an event filled with fellow cinephiles and much merriment. Some will immerse themselves into the prediction game, reading magazines and blogs about all the behind the scenes politicking and buzz mongering. Hopefully, quite a few people will use the attention the Oscars shine on the nominees as an inspiration to take in some of the talked-about films. Then there is that tiny and bizarre minority -no, minority is too generous, splinter faction is probably more accurate- that decides the best way to celebrate the year in movies is to do a bunch of math.

In my defense, I blame the Academy of Motion Picture Arts & Sciences for my recent rush of calculations. They recently adopted a few changes in the Best Picture category that have greatly complicated matters. First they decided to double the number of movies nominated for the top prize from five to ten. Okay easy enough, just print bigger ballots. Then they noticed that a movie could garner as little as 10.01% of the vote and win the most coveted award in Hollywood. Put it in other words, given the Academy has about 6,000 members, 5,400 of them could hate one of the ten nominated movies, but if 601 people vote for it it would be recorded as the best film of the year.

Rather than allow a highly fractured vote lead to an embarrassing result, someone proposed a new kind of ballot called "preferential voting" or "instant runoff". This basically means the Academy members rank the ten nominated film from their number one choice down. (On a personal note, I recently participated in an instant runoff election, and I can say from experience it was fun, easy and seemed like it could work in more elections.) Once the ballots are collected, the fine folks at PriceWaterhouse Coopers -a barrel making firm if I understand the definition of cooper correctly- then check to see if any film got a majority of the first place votes. Unless a film is an instant winner (unlikely to happen unless one film has an overwhelming level of support), the tabulators start weeding out the nominees that received few first place votes and redistributing their votes among the remaining films.

Since this is all a little confusing, I think it might be best to walk through a simulation of how the vote might break down among this year's Best Picture nominees. The inimitable Nate Silver -the only other person on Internet I could find nerdy enough to mix math with the Oscars- performed a similar exercise already. However he crunched his numbers before the nominees were announced, though he did accurately predicted nine of the ten. Also he used critics rankings, because he could find reliable data for them, instead of attempting to gauge the opinion of the AMPAS voters, a much trickier proposition. Fortunately for you, I don't have Silver's commitment to accuracy to worry about or his stellar reputation for predictions to preserve, so I can make a foolhardy attempt to predict how members of the Academy might rank the films. I will combine my knowledge of past Oscar races, my understanding of the AMPAS membership, and a watchful eye to the current dynamics of the awards season to estimate the level of support each nominee can expect.

We can begin breaking down the vote distribution by breaking down the membership of the Academy. Between the 6,000 AMPAS members and the 15 branches within the Academy, I can identify 4 major voting blocs, each based on different definitions you can use to determine the Best Picture of the year.

First you have those voters who believe that the Best Picture award should go the grandest film of the year. They like seeing movies you could describe as "epic" winning the top prize, films like Gladiator and The Lord of the Rings. This group probably contains mostly producers and directors, with some cinematographers, writers and a few actors mixed in. I estimate these voters account for about 35% of the total number of Best Picture votes.

Balancing out the first group you have those who believe that "best picture" means "most moving". They want to reward the movies that have the strongest emotional impact. Recently these voters likely supported Best Picture winners like Million Dollar Baby and Slumdog Millionaire. I would say a lot of the actors in the Academy vote with this block along with some of the writers and directors, probably a fair number of the artistic members who do art direction, makeup, and costuming too. These voters seem to represent around 35% of the vote as well.

The third major group comes from the less famous branches of AMPAS, the ones who are nominated in the so called "artistic and technical categories". Since every member can vote in the Best Picture category this undervalued lot can have a powerful impact on the race. They tend to hold that the Oscar brand should fall to those films they deem the most well made or best crafted. The last two Best Pictures, The Hurt Locker and No Country For Old Men, seem to have had strong support from this bloc. All told they likely comprise 25% or so of the Academy.

Finally, you have a small group that can play a big role if there is ever a close race. These are producers, publicists, and studio executives who simply -some might say cynically- contend that the "best" movies are the one's that make the most money. So their rankings of the years films pretty much mirror the final box office earnings. Their support helped put Titanic over the top to earn a record number of Oscars. Last year they probably supported Avatar, but the other factions trumped them. They probably add up to no more than 5% of the votes available for Best Picture.

So if you divide the Academy into these four groups, you likely can guess which groups would vote for which nominees, and therefore how much support each nominee has. It is important to remember that each group will likely vote for more than one film, and many nominees will receive votes from more than one group. Also, I will assume that each group has a selection of movies that they all rank above the others. So you when a movie that had a share of the "most touching" vote is eliminated it's votes will probably go the other emotionally powerful movies before they go anywhere else. With all of that explained, I think we can begin breaking down the actual votes.

Grandest Picture Bloc- 35% of the total 1st place votes
- The King's Speech: 35% of the bloc vote = 12.25% of the total vote
- The Social Network: 30% bloc = 10.5% total
- Inception: 20% bloc = 7% total
- True Grit: 10% bloc = 3.5% total
- Winter's Bone: 5% bloc = 1.75% total
These are all the options from among the nominees which seem to portray a grand vision, or tell an ambitious story. In some cases this is because of the amount of work involved (for period pieces like True Grit or The King's Speech). Sometimes it its because of the level of technical sophistication that the filmmakers employed (such as in manking The Social Network or Inception). Also a film could just have overachieved and accomplished a lot with relatively meager means (as I believe some will argue Winter's Bone managed to do).


Most Powerful Movie Bloc- 35% of the total 1st place votes
- The King's Speech: 40% of the bloc vote = 14% of the total vote
- The Fighter: 20% bloc = 7% total
- Black Swan: 15% bloc = 5.25% total
- The Kids Are All Right: 10% bloc = 3.5% total
- Winter's Bone: 5% bloc = 1.75% total
- Toy Story 3: 5% bloc = 1.75% total
- 127 Hours: 5% bloc = 1.75% total
Among this group you can find the nominees that have been most praised for the emotional power the convey. Here you will notice some overlap with the first group, most notably The King's Speech. While other films from the first group have been left out. I am yet to hear anyone call either Inception or The Social Network "the most touching movie of the year".

Best Made Film Bloc- 25% of the total 1st place votes
- The Social Network: 30% of the bloc vote = 7.5% of the total vote
- The King's Speech: 20% bloc = 5% total
- The Fighter: 15% bloc = 3.75% total
- True Grit: 15% bloc = 3.75% total
- Inception: 10% bloc = 2.5% total
- Winter's Bone: 5% bloc = 1.25% total
- 127 Hours: 5% bloc = 1.25% total
Now all of the nominees have to have been well crafter movies to even be considered for an Oscar. However, I believe this list contains those pictures who derive the biggest chunk of their merit from their technical values. Each of these nominees has impressive filmmaking elements that made it stand out from the other movies released in 2010.

Biggest Hit Bloc- 5% of the total 1st place votes
- Inception: 60% of the bloc vote = 3% of the total vote
- Toy Story 3: 40% bloc = 2% total
Finally this group has only two movies with big box office earnings to vote for. In most years I would imagine the vote might be pretty monolithic -whichever nominee earned the most gets the most votes. However this year I think the voters will split between the highest grossing movie of the year, Toy Story 3, and the biggest money-maker among the live action films. I say this because some members of the Academy still believe that animated films should not be compared to live action films. They involve different kinds of filmmaking and they generally aim to accomplish different goals. Therefore any method of comparison, even total grosses, can't be easily evaluated between animated and live action films. Do you reward Toy Story 3 for selling so many tickets, especially when you earn less off a children's ticket? Do you support Inception for bringing droves of adults to the cinema, something Hollywood has struggled to do for years? There's no easy answer, so I predict a split vote here.

Let's put all that information into one chart, so you can see how much of the vote each nominee received.

Total Vote- Round 1
- The King's Speech: 31.25%
- The Social Network: 18%
- Inception: 12.5%
- The Fighter: 10.75%
- True Grit: 7.25%
- Black Swan: 5.25%
- Winter's Bone: 4.75%
- Toy Story 3: 3.75%
- The Kids Are All Right: 3.5%
- 127 Hours: 3%

None of the nominees have a majority, though The King's Speech has a huge lead over all the others. That means there is no instant winner and the voting no goes to multiple rounds. In each round the film with the lowest share of the 1st place vote is eliminated it's votes are then redistributed to those nominess that remain. The votes are spread out by going down the ballot to the highest ranked film still in contention and giving that film an additional 1st place vote.

The first film to go is 127 Hours. I know it feels like losing an arm to lose one of these nominees (rimshot!), but that is how the process works. We start by identifying where the votes for 127 Hours came from then determine which of the survivors profit from the departure of this film.

Some of the Most Powerful Movie vote, 5%, went to 127 Hours, so we can split that among some of the other touching films in contention. This is how I expect those votes to break down:

1st Place- 127 Hours
- Black Swan: 30%
- The Fighter: 30%
- The King's Speech: 20%
- Winter's Bone: 20%

To clarify what these numbers represent, I am dividing up just 1.75% of the total 1st place votes. When I say The King's Speech gets 20% of the votes from people who judged 127 Hours to be the emotionally touching movie of the year, I am say one-fifth of 1.75% of the Academy's 6,000 members, or about 21 people, would now have their 1st place vote go to The King's Speech instead of their original pick.

Of cour 127 Hours got some votes for being the best made film of the year as well. Here I have a different pool of films among which to divide the 1st place votes. I would predict the votes would spread out like:

1st Place- 127 Hours
- The Social Network: 30%
- The Fighter: 30%
- Winter's Bone: 20%
- The King's Speech: 10%
- Inception: 10%

After I calculate the effect these changes have on the total 1st place vote distribution, I can make an updated chart of the standings.

Total Vote- Round 2
- The King's Speech: 31.725%
- The Social Network: 18.375%
- Inception: 12.625%
- The Fighter: 11.65%
- True Grit: 7.25%
- Black Swan: 5.775%
- Winter's Bone: 5.35%
- Toy Story 3: 3.75%
- The Kids Are All Right: 3.5%
- 127 Hours

Since we still lack any nominee with over 50% of the total vote the process continues. Next on the chopping block we have the critically acclaimed dramedy, The Kids Are All Right. Since this fillm got all of it's support from one bloc of voters, the Most Powerful Movie crowd, it is pretty easy to divvy up the votes appropriately. I imagine that the supports of The Kids Are All Right, a largely comedic film with an indy vibe, will have ranked other films that are humorous or independent highly. Thus:

1st Place- The Kids Are All Right
- The King's Speech: 50%
- Toy Story 3: 30%
- Winter's Bone: 20%

That brings our updated standings to:

Total Vote- Round 3
- The King's Speech: 33.475%
- The Social Network: 18.375%
- Inception: 12.625%
- The Fighter: 11.65%
- True Grit: 7.25%
- Black Swan: 5.775%
- Winter's Bone: 6.05%
- Toy Story 3: 4.8%
- The Kids Are All Right
- 127 Hours

Now we move on to Toy Story 3 and it's supporters. The folks who voted for it because it was the highest earning movie of the year are easy to account for. All of their 1st place votes would just go to the next big money maker, Inception. The people who held it as the most touching of the nominees will require a more involved breakdown. This block no longer represents just the 1.75% of the Academy that gave it a 1st place vote originally. Now we also have to spread out the votes it received as a result of other films getting dropped. That means we now have 2.8% of the vote to work with. (I know that's a staggering amount of change to deal with. You might want to take a second and catch your breath before you move on.) Here is how I would imagine the votes are divided among this group:

1st Place- Toy Story 3
- The King's Speech: 70%
- The Fighter: 10%
- Black Swab: 10%
- Winter's Bone: 10%

Factoring in those adjustments, and the earlier bump to Inception's standings, we have these updated standings:

Total Vote- Round 4
- The King's Speech: 35.435%
- The Social Network: 18.375%
- Inception: 14.625%
- The Fighter: 11.93%
- True Grit: 7.25%
- Black Swan: 6.055%
- Winter's Bone: 6.33%
- Toy Story 3
- The Kids Are All Right
- 127 Hours

In an interesting twist, Black Swan, which started with more of the 1st place vote share than Winter's Bone, is actually the next movie eliminated. Please note that all of this has been my speculation. I cannot actually know if more members of the Academy ranked Winter's Bone high on their ballot than did for Black Swan. I do however believe that this sort of change in the standings is possible, and even likely, once you start eliminating nominees and redistributing the vote. Having said that and remembering that only films still active can receive new 1st place votes, this is how I break down the vote:

1st Place- Black Swan
- The King's Speech: 60%
- The Fighter: 20%
- Winter's Bone: 20%

Those adjustments mean these are the new standings:

Total Vote- Round 5
- The King's Speech: 39.068%
- The Social Network: 18.375%
- Inception: 14.625%
- The Fighter: 13.141%
- True Grit: 7.25%
- Black Swan
- Winter's Bone: 7.541%
- Toy Story 3
- The Kids Are All Right
- 127 Hours

The changing share of the votes bring another intriguing change. Winter's Bone has now lept True Grit as well. This is mostly because True Grit has not gained any new votes from the eliminated movies. Part of this is because a lot of the movies we've dumped drew their support from different sources than True Grit. Another factor is that some films may be widely liked, but not widely loved. This year True Grit seems to be a good example of that. Without many people outside of it's ardent supporters ranking it highly on their ballot, it doesn't stand much of a chance of winning Best Picture.

The voters who considered True Grit the biggest and most ambitious movie of the year probably split their other top votes like this:

1st Place- True Grit
- The King's Speech: 40%
- Inception: 30%
- The Social Network: 20%
- Winter's Bone: 10%

While those who rated True Grit as the best crafted film likely have their support spread like this:

1st Place- True Grit
- The King's Speech: 30%
- Inception: 30%
- The Social Network: 20%
- The Fighter: 10%
- Winter's Bone: 10%

I would like to note that in this round I assumed Inception would benefit strongly from the elimination of the only other film with strong action/adventure content. That means the new standings look like this:

Total Vote- Round 6
- The King's Speech: 41.593%
- The Social Network: 19.825%
- Inception: 16.8%
- The Fighter: 13.516%
- True Grit
- Black Swan
- Winter's Bone: 8.266%
- Toy Story 3
- The Kids Are All Right
- 127 Hours

Now we have to finally cast aside Winter's Bone. Since it is being removed that means we have a few points to consider when spreading out the ballots it had gained. First we can see that the Grandest Picture bloc has begun to shift. Because the people who rated Winter's Bone high likely are comfortable voting with a smaller skill picture that has a big vision, I predict The King's Speech will benefit more than Inception or The Social Network. Also I expect that the voters in the other corner will favor The King's Speech over The Fighter. I say this because I can look to the trends in the earlier vote distributions. Anytime group of voters that gave a boost to Winter's Bone tended to favor The King's Speech pretty strongly. That translates to a big boost The King's Speech compared to modest gains for the others.

The Grandest Picture vote would divide like this:
1st Place- Winter's Bone
- The King's Speech: 60%
- The Social Network: 30%
- Inception: 10%

The Most Powerful Movie vote would divide thusly:

1st Place- Winter's Bone
- The King's Speech: 70%
- The Fighter: 30%

This is what would happen with the Best Made Film vote:

1st Place- Winter's Bone
- The King's Speech: 40%
- The Social Network: 20%
- The Fighter: 20%
- Inception: 20%

Which means our updated standings would now be this:

Total Vote- Round 7
- The King's Speech: 46.6067%
- The Social Network: 20.83%
- Inception: 17.385%
- The Fighter: 15.1783%
- True Grit
- Black Swan
- Winter's Bone
- Toy Story 3
- The Kids Are All Right
- 127 Hours

Finally we have reached a point where a movie is within easy striking distance of the 50% mark. That means this next round should be enough to give The King's Speech the golden man-shaped trophy. This is especially true when you acknowledge that The King's Speech is the only film left that had the support of the Most Powerful Movie Bloc, so it now has 35% of the 1st place votes all to itself.

Breaking down the votes from the people who named The Fighter the best crafter movie of the year, I estimate this is how things would stand:

1st Place- The Fighter
- The King's Speech: 40%
- The Social Network: 40%
- Inception: 20%

Add it all up and you finally have a clear winner.

Total Vote- Round 8
- The King's Speech: 58.86%
- The Social Network: 27.78%
- Inception: 18.36%
- The Fighter
- True Grit
- Black Swan
- Winter's Bone
- Toy Story 3
- The Kids Are All Right
- 127 Hours

What can we learn from any of this? (Other that I am a very odd duck to devote so much arithmetic to such a trivial matter) The first thing that strikes me is that asking one of ten nominees to get a majority of the vote means it is unlikely any movie will ear the Oscar after only a few rounds of voting. Further I find it interesting that the three films left standing, The King's Speech, The Social Network, and Inception, can each point to an area where they had a strong body of support, the Most Powerful vote, the Best Made vote and the Biggest Hit vote respectively. It would seem those campaigning for any of the nominees would be well served by targeting a demographic that can give you a solid block of votes, then hoping that as films get eliminated you will rise up. Also, I like the fact that in this exercise we saw how Winter's Bone was able to climb the standings by being broadly supported if not having the level of high profile support the others did. This suggest to me that any high quality film could wind up winning Best Picture even if it doesn't have the box office numbers or publicity of other movies.

So does any of this mean anything? Not really, it was mostly an exercise I came up with to amuse myself and hopefully my audience. The only way to know if I am correct in my estimations, and that The King's Speech will walk away as the big winner is to tun in and watch the Oscars on February 27th. I know I'll be watching, and I hope you do to.