Linear Regression vs Logistic Regression

Theoretical Background:

• Objective: Linear regression aims to find a linear relationship between the independent and dependent variables, minimizing the mean squared error. In contrast, logistic regression aims to predict the probability that a given input belongs to a certain class, using a logistic curve.

• Assumptions: Linear regression assumes a linear relationship between the input variables and the output, independence of errors, homoscedasticity, and normally distributed errors. Logistic regression assumes a linear relationship between the logit (log odds) of the outcome and the input variables.

• Mathematical Formulation:

  • Linear Regression: $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \ldots + \beta_n x_n + \epsilon$
  • Logistic Regression: $P(Y=1|X) = \frac{1}{1 + e^{-(\beta_0 + \beta_1 x_1 + \ldots + \beta_n x_n)}}$

Practical Applications:

• Linear Regression: Used in scenarios where the outcome is continuous, such as predicting house prices, stock prices, or any measurable quantities.

• Logistic Regression: Used for binary classification problems, such as spam detection, medical diagnosis (disease vs. no disease), and churn prediction.

Code Examples:

from sklearn.linear_model import LinearRegression, LogisticRegression

# Linear Regression
lin_reg = LinearRegression()
lin_reg.fit(X_train, y_train)
predictions_lin = lin_reg.predict(X_test)

# Logistic Regression
log_reg = LogisticRegression()
log_reg.fit(X_train, y_train)
predictions_log = log_reg.predict(X_test)

Diagrams:

graph LR
  A[Linear Regression] --> B((Continuous Output))
  C[Logistic Regression] --> D((Binary Output))

For more detailed explanations, you can refer to these resources:

• Linear Regression - Wikipedia
• Logistic Regression - Wikipedia