Many people have a biased view of what econometrics really is and how it can be applied to business problems. Econometrics has evolved largely from the inability to randomly double your taxes while keeping your neighbors taxes constant to test how you react. Similar problems face decision makers who turn to data for insight: Randomizing prices can be really bad pr and econometrics can help you solve this problem among many others. In the following posts I will introduce some methods from applied econometrics, structural modelling and panel data/time series tools and show how they can be used to solve problems where standard statistical and machine learning tools do not work.
What econometrics is?
People often have misconceptions about what econometrics really is. This is largely due to the fact that macroeconomic and public policy decisions such as interest rate cuts and tax increases get most of the news coverage. However, econometrics has a lot more to offer than politically motivated macroeconomic forecasts. In the following posts I hope to give you a better idea of what econometrics is, what kind of problems it tries to solve and how it can be applied to produce useful insights.
Probably the most famous definition of econometrics comes from Ragnar Frisch, the father of econometrics and the first recipient of Nobel prize in economics (or Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel if you want to distinguish economics from “real science” badly enough to say/type the monster):
“Intermediate between mathematics, statistics, and economics, we find a new discipline which for lack of a better name, may be called econometrics. Econometrics has as its aim to subject abstract laws of theoretical political economy or ‘pure’ economics to experimental and numerical verification, and thus to turn pure economics, as far as possible, into a science in the strict sense of the word”
To put it simply, econometrics is a set of tools that has been developed to confront economic theory with the real world data.
Why not economic statistics?
I will not try to cover the differences between econometrics and statistics in detail, but will direct a more interested reader to start from here or here. I will make a few key points though.
As mentioned above, Frisch had a vision of making economics as scientific as possible with “experimental and numerical verification”. Even though there has lately been a serious effort to conduct field experiments, economists have not been able to study the big social questions with standard experimental approach. This is for a good reason since allowing economists to test how you respond to random policies and hyperinflation would probably not end well. The inability to experiment has probably been the biggest reason why econometrics has produced a unique set of tools that are, in my opinion, underutilized in business analytics.
Another reason is simply a different type of problems. Econometric analysis is usually conducted with "model driven" approach as opposed"data driven" approach in statistics. This is why econometricians face very unusual problems from a statisticians point of view.
And perhaps most importantly, what is it good for?
Questions that economists study are usually causal and sometimes predictive in nature. Politicians might want to know what kind of effects a proposed policy would have, how would a merger of two competitors affect the market structure and hence competition and welfare or how much will the GDP grow next year. These questions are great examples of the types of problems economists are dealing with. In the coming posts I will be digging a little deeper how the tools developed for this kind questions can be applied in business analytics and will content myself with a short prelude.
When A/B testing can't used
Especially in the web analytics, the gold standard for mining causal effects is A/B testing. Showing different ads randomly to similar people is a great way to determine which ad (directly) increases traffic on your website. In some cases, just like in economics, random experiments are impossible or just not feasible. The politician considering a tax reform wouldn’t be able to randomly assign citizens to a new tax scheme while taxing others like before. In some cases, randomization could be too costly. A good example would be Amazon’s random price tests that lead Jeffrey Bezos to publicly state that Amazon would never do random price tests based on customer demographics again. Because of the inability to conduct experiments, econometricians have been forced to take advantage of experiment-like occurrences. My next post will cover such techniques and discuss how they could be applied to gain business insight.
When there is "no data"
My next example was the question how a merger of two large companies would change market structure. It is a shame that we can't create parallel worlds and do a merger in some of them to get the experimental data. On top, this question also has another problem. When the market structure changes, the statistical process that generates the data also changes. So we don’t have any data on how consumers and firms behave under the new market structure and hence the techniques that rely on observational data do not work.
A common way for economists to tackle such problems is to use economic theory and explicitly model how the market works, estimate the key parameters from the data and test how the model behaves if you change the strucutre. A good example of a business related problem in this realm would be a change in the business model. There would be observational data of customer behavior within the old model but no data within the new model. Third post of this series will cover structural models.
When you have panel or timeseries data
Lot of the data economists deal with is structured panel data (such as administrational data) or time-series data (financial data, macroeconomic time series) and the amount of research done about the panel and time series methods is staggering. In my fourth post I will cover some basic methods as well as some newer techniques such as now casting developed by Google’s chief economist Hal Varian.