{"id":141,"date":"2014-11-29T04:33:07","date_gmt":"2014-11-29T04:33:07","guid":{"rendered":"https:\/\/noahs-blog.net\/?p=141"},"modified":"2014-11-29T04:48:18","modified_gmt":"2014-11-29T04:48:18","slug":"the-method-of-least-squares","status":"publish","type":"post","link":"https:\/\/noahs-blog.net\/?p=141","title":{"rendered":"The Method of Least Squares"},"content":{"rendered":"<p>Recently I got into an on-line &#8220;debate&#8221; (yes I know, I should have known better, but let&#8217;s make this a learning experience for all; shall we?) about the world population. This got me thinking about how useful it would be to show people about &#8220;the method of least squares&#8221;.<\/p>\n<p>The method of least squares is a mathematical tool used for analyzing scientific data. Here&#8217;s how it works: First we take our scientific data. This data could have come from any number of sources. Perhaps it came from some sensor that is used to measure something in a scientific experiment, or perhaps it is just some statistical data about the population; like this chart I made from data I got from <a href=\"http:\/\/esa.un.org\/unpd\/wpp\/Excel-Data\/EXCEL_FILES\/1_Population\/WPP2012_POP_F01_1_TOTAL_POPULATION_BOTH_SEXES.XLS\">here<\/a>:<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-159 size-full\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1.png\" alt=\"population-chart\" width=\"1366\" height=\"689\" srcset=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1.png 1366w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1-300x151.png 300w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1-1024x516.png 1024w\" sizes=\"auto, (max-width: 1366px) 100vw, 1366px\" \/><\/a><br \/>\nNow it has been claimed before that the world population is going up exponentially, and who are we to argue with that? Besides it looks kind of like the beginning of exponential growth.<br \/>\nThis means that the formula for the world population at any given time will look something kind of like this:<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/our-formula.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-147\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/our-formula.png\" alt=\"our-formula\" width=\"191\" height=\"34\" \/><\/a><\/p>\n<p>Where y is the world population, x is the year, and a, b, and c are some kind of number that we don&#8217;t know the values of. Now what if we wanted to predict what the world population will be in the future?<br \/>\nObviously we need a formula, and obviously we need to find out what a, b, and c are.<br \/>\nLet&#8217;s look at that graph again:<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-159 size-full\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1.png\" alt=\"population-chart\" width=\"1366\" height=\"689\" srcset=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1.png 1366w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1-300x151.png 300w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-chart1-1024x516.png 1024w\" sizes=\"auto, (max-width: 1366px) 100vw, 1366px\" \/><\/a><br \/>\nDid you notice how it&#8217;s all bumpy and not a nice smooth curve like this one?<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/nice-curve1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-158 size-full\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/nice-curve1.png\" alt=\"nice-curve\" width=\"804\" height=\"609\" srcset=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/nice-curve1.png 804w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/nice-curve1-300x227.png 300w\" sizes=\"auto, (max-width: 804px) 100vw, 804px\" \/><\/a><br \/>\nIt seems reality doesn&#8217;t like making things nice and simple. The people who were finding this world population data probably made a few mistakes. Maybe some people lied on their tax forms (which is probably how they got much of this data) at certain points when their economies were bad, or maybe some people were accidentally counted twice.<\/p>\n<p>In other words these results have some random error in them. This is unfortunately a problem that you face whenever you measure anything: you&#8217;re going to get inaccurate results. Whether you&#8217;re measuring world population, the charge on fundamental particles, or just distances with a ruler; the result will always be slightly off.<\/p>\n<p>As technology improves measurement tools get more accurate, and don&#8217;t introduce as much error, but scientists still have to find some way to deal with this.<\/p>\n<p>So how does one deal with this?<br \/>\nFirst what we do is take a formula that makes a graph that we think looks like our data. In this case it&#8217;s a second degree polynomial; AKA this thing:<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/our-formula.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-147\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/our-formula.png\" alt=\"our-formula\" width=\"191\" height=\"34\" \/><\/a><\/p>\n<p>Now let&#8217;s replace a, b, and c with the Greek letter beta (because this will separate the true math nerds from the fakers). So it&#8217;ll look like this:<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/beta-substitute.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-148\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/beta-substitute.png\" alt=\"beta-substitute\" width=\"246\" height=\"65\" \/><\/a><\/p>\n<p>This way we can maybe we can use this for other crazy formulas in the future like:<\/p>\n<p><a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/crazy-function.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-149\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/crazy-function-300x63.png\" alt=\"crazy-function\" width=\"300\" height=\"63\" srcset=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/crazy-function-300x63.png 300w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/crazy-function.png 302w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>What we want to do is find the values of the betas this formula,<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/beta-substitute.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-148\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/beta-substitute.png\" alt=\"beta-substitute\" width=\"246\" height=\"65\" \/><\/a><\/p>\n<p>fit the data as closely as possible. So what we could do this for every data point:<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/least-squares.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-150\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/least-squares.png\" alt=\"least-squares\" width=\"150\" height=\"41\" \/><\/a><\/p>\n<p>In other words, every point in that graph is a pair of x and y values. For each one we take the y value, and subtract this from it<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/the-function.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-151\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/the-function.png\" alt=\"the-function\" width=\"179\" height=\"38\" \/><\/a><br \/>\nWhere x is the x value for that particular pair. This way we&#8217;ll find the difference from the value that our theoretical formula gives us, and the actual value from the data. The trick here is to find the beta values that minimize the difference between the theoretical value and the actual value.<\/p>\n<p>If we do this subtraction and squaring for every data point (or more realistically: have a computer do it for us), and add them all up we might end up with something that looks like this horrible mess:<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/horrible-mess.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-152 size-full\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/horrible-mess.png\" alt=\"horrible-mess\" width=\"880\" height=\"82\" srcset=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/horrible-mess.png 880w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/horrible-mess-300x27.png 300w\" sizes=\"auto, (max-width: 880px) 100vw, 880px\" \/><\/a><br \/>\nNot only does this give us what will be the error in our formula, but it&#8217;s also the phone number of Satan.<\/p>\n<p>Now we can use calculus and algebra to find out what the betas should be. First we need<br \/>\nto take derivatives with respect to each of the beta variables (<a href=\"https:\/\/www.khanacademy.org\/math\/differential-calculus\/taking-derivatives\">here<\/a>&#8216;s a bunch of Khan Academy videos on how to take derivatives, and what derivatives are. Be sure to look for the videos talking about some rule (like the &#8220;exponent rule&#8221;, or the &#8220;chain rule&#8221;)).<\/p>\n<p>Next we set all those derivatives equal to zero.<br \/>\nIn this case it&#8217;ll look like this crime against nature:<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/where-did-my-life-go-so-wrong1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-167 size-full\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/where-did-my-life-go-so-wrong1.png\" alt=\"where-did-my-life-go-so-wrong\" width=\"837\" height=\"149\" srcset=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/where-did-my-life-go-so-wrong1.png 837w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/where-did-my-life-go-so-wrong1-300x53.png 300w\" sizes=\"auto, (max-width: 837px) 100vw, 837px\" \/><\/a><br \/>\nAnd yes this will be on the final exam (of your existence).<\/p>\n<p>Next we can use <a href=\"https:\/\/www.khanacademy.org\/math\/cc-eighth-grade-math\/cc-8th-systems-topic\/cc-8th-systems-with-substitution\/v\/the-substitution-method\">back substitution<\/a> to find out what the values are. Then that&#8217;s it! We&#8217;re done! And to think I&#8217;ve probably only lost two thirds of my audience when I showed the first picture!<br \/>\n<a href=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-formula.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-154\" src=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-formula-300x35.png\" alt=\"population-formula\" width=\"300\" height=\"35\" srcset=\"https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-formula-300x35.png 300w, https:\/\/noahs-blog.net\/wp-content\/uploads\/2014\/11\/population-formula.png 322w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><br \/>\nThis formula will tell us the World population (in billions) at any given time &#8216;x&#8217;. Where &#8216;x&#8217; is the year.<br \/>\nWe can then use <a href=\"https:\/\/www.khanacademy.org\/math\/algebra\/quadratics\/quadratic-formula\/v\/using-the-quadratic-formula\">the quadratic formula<\/a> to find out when the we will go extinct from overpopulation!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently I got into an on-line &#8220;debate&#8221; (yes I know, I should have known better, but let&#8217;s make this a learning experience for all; shall we?) about the world population. This got me thinking about how useful it would be to show people about &#8220;the method of least squares&#8221;. The method of least squares is &hellip; <a href=\"https:\/\/noahs-blog.net\/?p=141\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">The Method of Least Squares<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-141","post","type-post","status-publish","format-standard","hentry","category-math"],"_links":{"self":[{"href":"https:\/\/noahs-blog.net\/index.php?rest_route=\/wp\/v2\/posts\/141","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/noahs-blog.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/noahs-blog.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/noahs-blog.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/noahs-blog.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=141"}],"version-history":[{"count":10,"href":"https:\/\/noahs-blog.net\/index.php?rest_route=\/wp\/v2\/posts\/141\/revisions"}],"predecessor-version":[{"id":168,"href":"https:\/\/noahs-blog.net\/index.php?rest_route=\/wp\/v2\/posts\/141\/revisions\/168"}],"wp:attachment":[{"href":"https:\/\/noahs-blog.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=141"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/noahs-blog.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=141"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/noahs-blog.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=141"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}