PYTHON / MACHINE LEARNING WITH PYTHON
The scikit-learn workflow
Use scikit-learn's estimator contract: construct with hyperparameters, fit on training data, read underscore attributes, predict, and wrap it in a Pipeline.
What you will learn
- Pass hyperparameters to the constructor and data only to fit; the constructor computes nothing
- Read anything learned from data off trailing-underscore attributes: coef_, mean_, classes_
- Call fit/fit_transform on training data, transform/predict on every other array
- Wrap preprocessing plus a model in a Pipeline so the pair behaves as one estimator
Understanding The scikit-learn workflow
Every object in scikit-learn follows one contract. The constructor stores your hyperparameters verbatim and does no work at all, fit(X, y) does the learning and returns the object itself, and whatever was computed from the data is stored as attributes whose names end in an underscore. On top of that base, transformers add transform (plus the shortcut fit_transform) and predictors add predict, usually with score and sometimes predict_proba. Because roughly every class in the library obeys this, swapping Ridge for RandomForestRegressor is a one-line change and nothing downstream needs editing.
The mental model to hold is who owns which state. Names without an underscore are yours: alpha, n_neighbors, max_iter, and you can read them back with get_params() or change them with set_params() before fitting. Names with a trailing underscore belong to the data: coef_, mean_, scale_, classes_, n_iter_, and they simply do not exist until fit has run, which is why hasattr(model, "coef_") is the honest test for "is this fitted". fit is destructive rather than cumulative: calling it a second time discards the previous solution, and clone(model) hands you an unfitted copy with identical configuration, which is precisely how cross-validation and grid search get a clean model per fold.
As soon as the workflow has more than one step, stop carrying intermediate arrays by hand and use Pipeline, which is itself an estimator. Its fit calls fit_transform on every step except the last and fit on the last; its predict calls transform on those earlier steps and predict on the last. That structure makes "fit the scaler on training data only" a property of the code rather than something you must remember, and nested configuration is reachable as step__param, so a pipeline drops into any place a plain estimator is expected.
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
X = np.array([[0.0, 0.0], [0.1, 0.2], [0.2, 0.1],
[1.0, 1.0], [0.9, 1.1], [1.1, 0.9]])
y = np.array([0, 0, 0, 1, 1, 1])
pipe = Pipeline([
("scale", StandardScaler()),
("clf", LogisticRegression(max_iter=1000)),
])
print("clf has coef_ before fit:", hasattr(pipe.named_steps["clf"], "coef_"))
pipe.fit(X, y)
print("clf has coef_ after fit :", hasattr(pipe.named_steps["clf"], "coef_"))
print("scaler mean_ :", pipe.named_steps["scale"].mean_.round(2))
print("clf coef_ shape :", pipe.named_steps["clf"].coef_.shape)
print("predictions :", pipe.predict([[0.05, 0.05], [1.2, 1.0]]))
print("training accuracy :", pipe.score(X, y))In scikit-learn, hyperparameters go into the constructor, data goes into fit, and everything learned from data comes back out on attributes ending in an underscore.
Worked examples
fit learns state, transform reuses it
Shows that a transformer has no usable state before fit and that fit_transform is nothing more than fit followed by transform.
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.exceptions import NotFittedError
X_train = np.array([[1.0], [3.0], [5.0]])
scaler = StandardScaler()
try:
scaler.transform(X_train)
except NotFittedError:
print("transform before fit ->", "NotFittedError")
a = scaler.fit_transform(X_train)
b = StandardScaler().fit(X_train).transform(X_train)
print("fit_transform == fit then transform ->", np.array_equal(a, b))
print("learned state ->", scaler.mean_, scaler.scale_)
print("new row reuses it ->", scaler.transform([[7.0]]))Example explained
Line 1transform raises NotFittedError because it checks for mean_ and scale_, which only appear after fit.
Line 2fit_transform(X) is defined as fit(X).transform(X), so a and b are element-for-element identical.
Line 3mean_ = 3.0 and scale_ = 1.633 are the learned state; the trailing underscore marks them as data-derived.
Line 4transform([[7.0]]) reuses the stored 3.0 and 1.633 instead of re-estimating, which is why unseen rows go through transform, never fit_transform.
Hyperparameters versus learned attributes
Demonstrates get_params, set_params and clone, and that changing a hyperparameter does not change an already-fitted result.
from sklearn.base import clone
from sklearn.linear_model import Ridge
X = [[1.0], [2.0], [3.0], [4.0]]
y = [2.0, 4.0, 6.0, 8.0]
model = Ridge(alpha=1.0)
print("alpha (set by you) :", model.get_params()["alpha"])
print("fitted? :", hasattr(model, "coef_"))
model.fit(X, y)
print("fitted? :", hasattr(model, "coef_"))
print("coef_ (learned) :", model.coef_.round(3))
twin = clone(model)
print("clone alpha/fitted :", twin.get_params()["alpha"], hasattr(twin, "coef_"))
model.set_params(alpha=100.0)
print("coef_ before refit :", model.coef_.round(3))
model.fit(X, y)
print("coef_ after refit :", model.coef_.round(3))Example explained
Line 1get_params() returns configuration only, so alpha is readable before any data has been seen.
Line 2hasattr(model, 'coef_') flips from False to True at fit time; that transition is the whole definition of 'fitted'.
Line 3clone copies the constructor arguments and nothing else, which is how cross-validation gets an untouched model per fold.
Line 4set_params(alpha=100.0) edits the recipe, not the result: coef_ stays 1.667 until fit runs again and gives 0.095.
A Pipeline is an estimator too
Shows nested step__param configuration and how the pipeline routes fit to fit_transform and predict to transform.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge
X = [[1.0], [2.0], [3.0], [4.0]]
y = [2.0, 4.0, 6.0, 8.0]
pipe = Pipeline([("scale", StandardScaler()), ("reg", Ridge(alpha=1.0))])
print("reg__alpha" in pipe.get_params(), "scale__with_mean" in pipe.get_params())
pipe.set_params(reg__alpha=10.0)
print("nested alpha :", pipe.named_steps["reg"].alpha)
pipe.fit(X, y)
print("scaler mean_ :", pipe.named_steps["scale"].mean_)
print("predict(2.5) :", pipe.predict([[2.5]]))
print("steps :", [name for name, _ in pipe.steps])Example explained
Line 1get_params() on a pipeline exposes each step's parameters under the step__param naming scheme.
Line 2set_params(reg__alpha=10.0) reaches inside the pipeline and rewrites the Ridge object's alpha.
Line 3pipe.fit runs scale.fit_transform then reg.fit; pipe.predict runs scale.transform then reg.predict.
Line 4mean_ is 2.5, the training mean, and predict never touches it, so no test row can influence the scaling.
Important notes
fit returns self, not a new object, so model.fit(X, y).predict(X_new) works but the original model was mutated in place; there is no 'unfitted copy' unless you ask for clone(model).
A few estimators support warm_start=True, in which case a second fit continues from the previous solution; that is the documented exception to the rule that fit forgets everything.
Common mistakes
Calling scaler.fit_transform(X_test): it overwrites mean_ and scale_ with test statistics, so the test features no longer match the scale the coefficients were learned on and the scores become meaningless.
Passing data to the constructor, as in LogisticRegression(X_train, y_train): the first positional argument is a hyperparameter, so you get a TypeError or a silently misconfigured model instead of a fitted one.
Setting model.alpha = 5 (or set_params) after fitting and then reading coef_ or predict: the stored solution is unchanged, so you report results for the old hyperparameter until you call fit again.
Try it yourself
Change, predict, then run
Build Pipeline([('scale', StandardScaler()), ('clf', LogisticRegression())]) on a hand-written 6-row, 2-feature dataset, print hasattr(pipe.named_steps['clf'], 'coef_') before and after fit, then use pipe.set_params(clf__C=0.01), refit, and print coef_ both times to see the hyperparameter shrink the learned weights.
Open the Python workspaceCheck your understanding
You fit a StandardScaler and a LogisticRegression on the training set, then run X_test_s = scaler.fit_transform(X_test) before calling model.predict(X_test_s). Why are the predictions untrustworthy?
- predict only accepts raw, untransformed features, so scaling the test set at all is a mistake.
- fit_transform re-estimates the mean and standard deviation from the test rows and overwrites the training ones, so the test features no longer live on the scale the coefficients were learned for.
- Refitting the scaler resets the model's coef_ attribute to its unfitted state.
- StandardScaler cannot be reused after fit, so the second call silently returns zeros.
Show answer
fit_transform always refits: it replaces mean_ and scale_ with statistics computed from whatever array you hand it, so the numbers reaching predict are centred and scaled differently from the ones that produced coef_. Option 3 is tempting because it sounds like corruption, but the scaler and the model are independent objects; coef_ is untouched, and the damage is entirely in the feature values you fed to a correctly fitted model.