By Perceptive Analytics
Imagine that you were to build a model to predict the mileage of a car from different attributes of the car. How will you do it?
The simplest approach will be to take one parameter or attribute (which one attribute to choose can be a question for endless debate) that affects the mileage most and build out a regression model to predict the mileage. But do you think this is the right approach? No, because the mileage of a car depends on multiple factors and not just a single factor. So, let’s go a step further and expand our model to make it more robust and include other attributes of car.
In the second approach, we will identify various attributes of a car such as horsepower, capacity, engine type, engine variant, cylinders, etc. All these will form predictor variables (also known as independent variables) for our model and the mileage will be response variable (also known as dependent variable).
What’s the difference between the first and second model?
In the second model, we have multiple factors or variables contributing to the final output variable. Intuitively, the accuracy of this model should be higher. Right?
The first model is called a simple linear regression; while, the second model is called a multiple linear regression model. In this case, assumption is that there are multiple independent variables which all affect the output variable. But, what if one of the independent variables is a dependent variable on other independent variables? For example, mileage is a dependent variable on horsepower, capacity, engine type, engine variant and cylinders; but what if, horsepower is a dependent variable on capacity, engine type and cylinders?
In such a scenario, the model becomes complex and path analysis comes handy in such situations. Path analysis is an extension of multiple regression. It allows for the analysis of more complicated models. It is helpful in examining situations where there are multiple intermediate dependent variables and in situations where Z is dependent on variable Y, which in turn is dependent on variable X. It can compare different models to determine which one best fits the data.
Path analysis was earlier also known as ‘causal modeling’; however, after strong criticism people refrain from using the term because it’s not possible to establish causal relationships using statistical techniques. Causal relationships can only be established through experimental designs. Path analysis can be used to disprove a model that suggests a causal relationship among variables; however, it cannot be used to prove that a causal relation exist among variables.
Let’s understand the terminology used in the path analysis. We don’t variables as independent or dependent here; rather, we call them exogenous or endogenous variables. Exogenous variables (independent variables in the world of regression) are variables which have arrows starting from them but none pointing towards them. Endogenous variables have at least one variable pointing towards them. The reason for such a nomenclature is that the factors that cause or influence exogenous variables exist outside the system while, the factors that cause endogenous variables exist within the system. In the above image, X is an exogenous variable; while, Y and Z are endogenous variables. A typical path diagram is as shown below.
In the above figure, A, B, C, D and E are exogenous variables; while, I and O are endogenous variables. ‘d’ is a disturbance term which is analogous to residuals in regression.
Now, let’s go through the assumptions that we need to consider before we use path analysis. Since, path analysis is an extension of multiple regression, most of assumptions of multiple regression hold true for path analysis as well.
- All the variables should have linear relations among each other.
- Endogenous variable should be continuous. In case of ordinal data, minimum number of categories should be five.
- There should be no interaction among variables. In case of any interaction, a separate term or variable can be added that reflects the interaction between the two variables.
- Disturbance terms are uncorrelated or covariance among the disturbance terms is zero.
Now, let’s move a step ahead and understand the implementation of path analysis in R. We will first try out with a toy example and then take a standard dataset available in R.
Now, let’s create our own dataset and try out path analysis. Please note that the rationale for doing this exercise is to develop intuition to understand path analysis.
Now, we have created this dataset. Let’s see the correlation matrix for these variables. This will tell us how strongly and which all variables are correlated to each other.
The above chart shows us that Y is very strongly correlate with X2; while, Z is strongly correlated with X2 and Y. The impact of X1 on Y is not as strong as that of X2.
The above plot shows us that Z is strongly dependent on Y and weakly dependent on X3 and X1. Y is strongly dependent on X2 and weakly dependent on X1. This is the same intuition that we have built earlier in this article. This is the beauty of path analysis and this is how analysis can be used.
The values between the lines are path coefficients. Path coefficients are standardized regression coefficients, similar to beta coefficients of multiple regression. These path coefficients should be statistically significant, which can be checked from the summary output (we will see this in the next example).
Let’s move to our second example. In this example, we will use standard dataset ‘mtcars’ available in R.
In the above summary output, we can see that wt is a significant variable for mpg at 5 percent level; while, dsp and crb are significant variables for hp. ‘Hp’ itself is not a significant variable for mpg. We will examine this model using a path diagram using semPlot package.
The above plot shows that mpg is strongly dependent on wt; while, hp is strongly dependent on dsp and crb. There is a weak relation between hp and mpg. Same inference was derived from the above output.
semPaths function can be used to create above chart in multiple ways. You can go through the documentation for semPaths and explore different options.
There are few considerations that you should keep in mind while doing path analysis. Path analysis is very sensitive to omission or addition of variables in the model. Any omission of relevant variable or addition of extra variable in the model may significantly impact the results. Also, path analysis is a technique for testing out models and not building them. If you were to use path analysis in building models, then you may end with endless combination of different models and choosing the right model may not be possible. So, path analysis can be used to test a specific model or compare multiple models to choose the best possible.
There are numerous other ways you can use path analysis. We would love to hear your experiences of using path analysis in different contexts. Please share your examples and experiences in the comments section below.
This article was contributed by Perceptive Analytics. Prudhvi Potluri, Chaitanya Sagar and Saneesh Veetil contributed to this article.
Perceptive Analytics provides data Analytics, business intelligence and reporting services to e-commerce, retail, healthcare and pharmaceutical industries. Its client roster includes Fortune 500 and NYSE listed companies in the USA and India.
- Optimization Using R
- Next Generation data Manipulation with R and dplyr
- Choropleth Maps in R