Graphing Real Data With Excel: Fill & Download for Free

Young’s modulus, E, is a function of the shape of the curve of potential energy with interatomic distance. Since the shape of this curve is governed by fundamental considerations (Felix Chen's answer to Why does Young's modulus of material increases by hammering?), it’s not affected by things like defects or second phases. But since temperature distorts the equilibrium bond distance and thus alters the curvature of the interatomic potential curve, Young’s modulus is changed by temperature: It decreases as temperature increases, by how much exactly we’ll see right now.The quantitative dependence of E with temperature T has been proposed as[1][math]\text{(1)}\;\;\;\;\displaystyle E=E_0\Big[1-a\Big(\frac{T}{T_m}\Big)\Big][/math]where [math]E_0[/math] is elastic modulus at 0 K, [math]T_m[/math] is absolute melting temperature, and a is given as about 0.5. Using real data for E with temperature for different materials[2], we can check the accuracy of this equation. We’ll start with aluminum.The above equation for aluminum is plotted below (if anyone’s wondering, I used Mathcad to create these):Now let’s check real data versus the prediction in the above graph:500 K: real datum is 64.2 GPa; graph gives 64.4 GPa400 K: real datum is 68.6 GPa; graph gives 69.1 GPa200 K: real datum is 78.2 GPa; graph gives 78.6 GPaIt can be seen the correspondence between experimentally measured values and predicted values is excellent.We’ll try out one more metal, annealed 304 stainless steel. Its graph per Eq. (1) is presented below:Comparing experimental data with the predictions of the above plot shows500 K: real datum is 178 GPa; graph gives 184.1 GPa400 K: real datum is 186 GPa; graph gives 190.5 GPa200 K: real datum is 200 GPa; graph gives 203.2 GPaAgain, as with aluminum, Eq. (1) fits the empirical numbers very well for 304 stainless steel. Thus, Eq. (1) characterizes the temperature dependence of elastic modulus adequately, although of course a won’t always be 0.5 for all materials. Last, one final observation is that per Eq. (1) E near the melting point drops by about half.Footnotes[1] Mechanical Behavior of Materials[2] Design for Thermal Stresses

tl;dr: Do a project you care about. Make it good and share it.There’s a lot of interest in becoming a data scientist, and for good reasons: high impact, high job satisfaction, high salaries, high demand. A quick search yields a plethora of possible resources that could help -- MOOCs, blogs, Quora answers to this exact question, books, Master’s programs, bootcamps, self-directed curricula, articles, forums and podcasts. Their quality is highly variable; some are excellent resources and programs, some are click-bait laundry lists. Since this is a relatively new role and there’s no universal agreement on what a data scientist does, it’s difficult for a beginner to know where to start, and it’s easy to get overwhelmed.Many of these resources follow a common pattern: 1) here are the skills you need and 2) here is where you learn each of these. Learn Python from this link, R from this one; take a machine learning class and “brush up” on your linear algebra. Download the iris data set and train a classifier (“learn by doing!”). Install Spark and Hadoop. Don’t forget about deep learning -- work your way through the TensorFlow tutorial (the one for ML beginners, so you can feel even worse about not understanding it). Buy that old orange Pattern Classification book to display on your desk after you gave up two chapters in.This makes sense; our educational institutions trained us to think that’s how you learn things. It might eventually work, too -- but it’s a unnecessarily inefficient process. Some programs have capstone projects (often using curated, clean data sets with a clear purpose, which sounds good but it’s not). Many recognize there’s no substitute for ‘learning on the job’ -- but how do you get that data science job in the first place?Instead, I recommend building up a public portfolio of simple, but interesting projects. You will learn everything you need in the process, perhaps even using all the resources above. However, you will be highly motivated to do so and will retain most of that knowledge, instead of passively glossing over complex formulas and forgetting everything in a month. If getting a job as a data scientist is a priority, this portfolio will open many doors, and if your topic, findings or product are interesting to a broader audience, you’ll have more incoming recruiting calls than you can handle.Here are the steps I recommend. They are optimized for maximizing your learning and chances to get a data job.1. Pick a topic you’re passionate or curious about.Cats, fitness, startups, politics, bees, education, human rights, heirloom tomatoes, labor markets. Research what datasets are available out there, or datasets you could create or obtain with minimal effort and expense. Perhaps you already work at a company that has unique data, or perhaps you can volunteer at a nonprofit that does. The goal is to answer interesting questions or build something cool in a week (it will take longer, but this will steer you towards something manageable).Did you find enough to start digging in? Are you excited about the questions you could ask and curious about the answers? Could you combine this data with other datasets to produce original insights that others have not explored yet? Census data, zip-code or state level demographic data, weather and climate are popular choices. Are you giddy about getting started? If your answer is ‘meh’ or this feels like a chore already, start over with a different topic.2. Write the tweet first.(A 21st century, probabilistic take on the scientific method, inspired by Amazon’s “write the press release first” practice and, more broadly, the Lean Startup philosophy)You’ll probably never actually tweet this, and you probably think tweets are a frivolous avenue to disseminate scientific findings. But it’s essential that you write 1-2 sentences about your (hypothetical) findings *before* you start. Be realistic (especially about being able to do this in a week) and optimistic (about actually having any findings, or them being interesting). Think of a likely scenario; it won’t be accurate (you can make things up at this point), but you’ll know if this is even worth pursuing.Here are a few examples, with a conversational hook thrown in:“I used LinkedIn data to find out what makes entrepreneurs different -- it turns out they’re older than you think, and they tend to major in physics but not in nursing or theology. I guess it’s hard to get VC funding to start your own religion.”“I used Jawbone data to see how weather affects activity levels -- it turns out people in NY are less sensitive to weather variations than Californians. Do you think New Yorkers are tougher or just work out indoors?”“I combined BBC obituary data with Wikipedia entries to see if 2016 was as bad as we thought for celebrities.”If your goal is to learn particular technologies or get a job, add them in.From Shelby Sturgis: “I built a web application to help teachers and administrators improve the quality of student education by providing analytics on school rank, progress on test scores over time, and performance in different subject areas. I used MySQL, Python, Javascript, Highcharts.js, and D3.js to store, analyze, and visualize California STAR testing data.”“I’ve used TensorFlow to automatically colorize and restore black and white photos. Made this giant collage for Grandma -- best Christmas ever!”Imagine yourself repeating this over and over at meetups and job interviews. Imagine this in USA Today or story or Wall Street Journal (without the exact technologies; a vague “algorithm” or “AI” will do). Are you boring yourself and having trouble explaining it, or do you feel proud and smart? If the answer is “meh”, repeat step 2 (and possibly 1) until you have 2-3 compelling ideas. Get feedback from others -- does this sound interesting? Would you interview somebody who built this for a data job?Remember, at this point you have not written any code or done any of the data work yet, beyond researching datasets and superficially understanding which technologies and tools are in demand and what they do, broadly speaking. It’s much easier to iterate at this stage. It sounds obvious, but people are eager to jump into a random tutorial or class to feel productive and soon sink months into a project that is going nowhere.3. Do the work.Explore the data. Clean it. Graph it. Repeat. Look at the top 10 most frequent values for each column. Study the outliers. Check the distributions. Group similar values if it’s too fragmented. Look for correlations and missing data. Try various clustering and classification algorithms. Debug. Learn why they worked or didn’t on your data. Build data pipelines on AWS if your data is big. Try various NLP libraries on your unstructured text data. Yes, you might learn Spark, numpy, pandas, nltk, matrix factorization and TensorFlow - not to check a box next to a laundry list, but because you *need* it to accomplish something you care about. Be a detective. Come up with new questions and unexpected directions. See if things make sense. Did you find a giant issue with how the data was collected? What if you bring in another data set? Ride the data wave. This should feel exciting and fun, with the occasional roadblock. Get help and feedback online, from Kaggle, from mentors if you have access to them, or from a buddy doing the same thing. If this does not feel like fun, go back to step 1. If the thought of that makes you hate life, reconsider being a data scientist: this is as fun as it gets, and you won’t be able to sustain the hard work and the 80% drudgery of a real data job if you don’t find this part energizing.)4. CommunicateWrite up your findings in simple language, with clean, compelling visualizations that are easy to grasp in seconds. You’ll learn several data viz tools in the process, which I highly recommend (it’s an underrated investment in your skills). Have a clean, interesting demo or video if you built a prototype. Technical details and code should be a link away. Send it around and get feedback. This being public will hold yourself to a higher standard and will result in good quality code, writing and visualizations.Now, do it all again. Congratulations, you’ve learned a lot about the latest technologies and you now have a portfolio of compelling projects. Send a link to the hiring manager on your dream data science team. When you get the job, send me a Sterling Truffle Bar.

It is an interesting question, and one that is rarely addressed in any great depth in introductory stats courses. After that, the fact of the normal distribution becomes so “normal” that it rarely is addressed in more advanced courses either. I will try to keep my reasoning on a fairly intuitive level, so a general audience can get some benefit from it (I hope).First off, the average person might well ask “what is a normal distribution”? It is that mathematical creature that you might have also heard of as a “bell curve” or perhaps a Gaussian distribution. The former term relates to its visual appearance (as seen below), while the latter relates to one of its most famous discoverers, the brilliant mathematician Gauss.This picture represents something called the standard normal distribution. The x-axis represents different values of some real data distribution, and the y-values represent the rate of change in the probability of a variable actually taking on that value. All data that is normally distributed can be converted to this standard normal distribution, and therefore can be analysed by one set of principles, that apply to this picture.In the middle is the mean value, what people generally refer to as the average. To the left and right are values that deviate from this average, expressed in a form called standard deviations. As the picture shows, in a normally distributed set of data, about two thirds of all values will fall within one standard deviation of the mean, with 95% falling within two standard deviations, and 99% falling within three standard deviations. So, normally distributed datasets have some very predictable properties, which is nice. They also have the nice property of being symmetrical around the mean, as the picture shows. These two properties are invaluable to statisticians and data scientists, in terms of using mathematics and algorithms to predict many features of the real world.The other important thing about the normal distribution, is that many, many situations in the real world can be modelled by a normal distribution, or at least come very close to a normal distribution. In fact, it tends to be the “go-to” distribution, for most purposes. Some examples are the heights of a random population of people, an IQ distribution or the pattern of misses that a shooter makes around a bullseye.Getting back to the original question, why is it that so many real-world data distribution take this form? The usual explanation is given by another name for the normal distribution, which is the “error distribution”. The idea is that errors are generally random, so that they are as likely to go in one direction as in the other. For example, the marksman is as likely to shoot a bit to the left, as a bit to the right, or a bit high as a bit low. Thus, a graph of how far the shots are from the bullseye will reflect this random tendency, and be symmetrical around the mean. Similarly with height and intelligence – many genes (perhaps thousands) contribute to these outcomes, as do a great number of environmental factors, such as nutrition, illnesses, low income and so forth.As for the “bell shape” of the curve, that seems to relate to some other facts about probability, the Bernoulli process and the Central Limit Theorem. A Bernoulli process is a process that has a set probability of success or failure, like tossing a coin. The Central Limit says that if you take a large number of samples from any distribution, and analyse some statistic from that group of samples, you will eventually get a normal distribution for that distribution. I put those two facts together, in the experiment below.In this experiment, I tossed a coin sixteen times, and counted the number of heads. As I increased the number of trials, that distribution became closer and closer to a normal distribution. I simulated this in an Excel spreadsheet, with the results shown below:You can see how the graph becomes more and more like the classic “bell shaped curve” as the number of simulated trials goes from 40 to 4000. Just how many trials are needed to get “close enough” to a normal distribution is somewhat debatable, but for many statistical purposes, it’s probably “normal enough” at about 100 trials, as many statistical and/or data science methods are fairly robust, in this regard.Here’s a quote from a book I own called “The Pleasures of Probability”, by Richard Isaac:“The Central Limit Theorem is sometimes used to give a theoretical explanation for the frequency with which normal or approximately normal distributions describe natural phenomena. It is said that the height of an adult, for example, is due to a multitude of causes: genetic makeup, diet, environmental factors, etc.. These factors often combine in an approximately additive way, so that the result is, by the Central Limit Theorem, close to normally distributed. It is true that all these factors contributing to an individual’s height do not in general have the same distribution, nor are they always independent, so the version of the Central Limit Theorem discussed here may not apply. There are, however, generalizations of the Central Limit Theorem valid when there are departures from the identically distributed assumption, and even from the independence assumption. Such results could offer a reasonable explanation of why many phenomena are approximately normally distributed.” (page 138)It is worth noting that there are many other statistical distributions that show up in real data. One of the most important of these is the power law, which describes many natural (e.g. the size distribution of craters on the moon) and social (e.g. book or movie sales) data distributions.It’s important to recognize when normal distribution assumptions are valid. The author of the popular economics book “The Black Swan” goes into this in some detail, but that’s another story (basically, unexpected things happen a lot more often than we expect from our assumptions of normality, and when they do, they can have very drastic consequences, like stock market crashes).It is also important to know the difference between a normal distribution and a ghost: