How to train a neural network in Python without leakage

A scaler fitted before the split lets test rows leak into preprocessing. The fix is mechanical: split first, learn scaling from training rows only, and keep the test set out of every tuning decision. This PyTorch salary-regression walkthrough gives you the exact blocks to prepare data, train a neural network, and inspect predictions back in salary units.

ProjectPro's salary-prediction example supplies the dataset layout and network architecture. Netguru's development guide draws the evaluation boundary: preprocessing must not learn from held-out data, and the test set stays outside the tuning loop.

Define the salary target before choosing layers

This is regression: input is position level, output is salary. Not a chatbot, not a pre-trained language model — just a small network fit to the supplied numeric examples.

Set an acceptable salary error before adding layers. ProjectPro names mean absolute error and mean squared error as regression metrics; this walkthrough uses squared error during training and reports absolute error in salary units at the end.

That target needs interpretation. Netguru warns that historical records encode past business decisions as labels. Predicting recorded salary is not the same as determining what someone should be paid. Write that distinction next to the acceptable error before starting any experiment.

You need a local Position_Salaries.csv matching the source layout: position level in column two, salary in column three, both numeric. The tutorial doesn't bundle the data or provide a missing-value strategy. Check those inputs before running the notebook rather than treating conversion as cleaning.

Prepare Python to train a neural network

Run the blocks below in order, same notebook or session. Keep the CSV in the working directory so the relative path resolves.

1. Install the example's dependencies

   Install Pandas, PyTorch, Matplotlib and scikit-learn in the environment that will run the example.

   pip install pandas torch matplotlib scikit-learn

   Pandas handles the CSV, scikit-learn creates the splits, PyTorch trains the network, Matplotlib plots the result. No model download is part of this workflow.

2. Load the numeric input and target

   This block reads the CSV, converts values to numeric where possible, and creates a feature tensor and a target tensor.

   import pandas as pd
   import torch
   import matplotlib.pyplot as plt
   from torch import nn
   from sklearn.model_selection import train_test_split
   
   salary_table = pd.read_csv('Position_Salaries.csv')
   numeric_table = salary_table.apply(pd.to_numeric, errors='coerce')
   
   levels = torch.Tensor(numeric_table.values[:, 1]).unsqueeze(dim=1)
   salaries = torch.Tensor(numeric_table.values[:, 2]).unsqueeze(dim=1)
   
   print(levels)
   print(salaries)

   The positional column selection deliberately follows the source's layout. Conversion can leave unusable values where numeric parsing fails; it does not repair them. A text position-name column is not an input to this network.

   Inspect the printed tensors before proceeding: each example should have a position-level input and its corresponding salary target. For a closer comparison with the CSV, display both tables:

   print(salary_table)
   print(numeric_table)

   Stop here if the selected columns contain missing or unusable values. Netguru treats imputation as a modelling decision, not an automatic cleaning formality; this walkthrough doesn't choose that policy for your data.

3. Separate training, validation and final testing

   This block reserves test rows first, then splits the remaining development data into training and validation sets.

   development_x, test_x, development_y, test_y = train_test_split(
       levels, salaries, test_size=0.2
   )
   
   training_x, validation_x, training_y, validation_y = train_test_split(
       development_x, development_y, test_size=0.2
   )
   
   print(len(training_x), len(validation_x), len(test_x))

   The test set is not a progress dashboard. Treat model evaluation and validation as separate decisions: validation guides changes, testing provides the final check. This corrects the source example's repeated test-set checks during training, which would otherwise expose the final evaluation data to model-selection decisions.

   Check the printed lengths before training. The second split operates only on the development rows, not the original complete dataset. Keep these partitions fixed while comparing configurations: changing both the model and the evaluation rows makes the comparison harder to interpret. Netguru notes that a single split can introduce evaluation variance.

4. Learn scaling from training rows alone

   This block calculates a centre and spread using only training tensors, then applies that scaling to training and validation data.

   def training_scale(values):
       centre = values[0] * 0
       for value in values:
           centre = centre + value
       centre = centre / len(values)
   
       variance = centre * 0
       for value in values:
           variance = variance + (value - centre) ** 2
       spread = (variance / len(values)) ** 0.5
   
       if spread == 0:
           spread = 1
       return centre, spread
   
   level_centre, level_spread = training_scale(training_x)
   salary_centre, salary_spread = training_scale(training_y)
   
   training_inputs = (training_x - level_centre) / level_spread
   training_targets = (training_y - salary_centre) / salary_spread
   validation_inputs = (validation_x - level_centre) / level_spread
   validation_targets = (validation_y - salary_centre) / salary_spread

   I prefer explicit arithmetic here because the training-only boundary is visible. This changes ProjectPro's scaling-before-splitting order to prevent held-out rows from influencing preprocessing. Retain the salary centre and spread: predictions must return to salary units before their errors are meaningful to a reader.

   Inputs and targets have separate scaling parameters because position level and salary are different quantities. Validation data reuses those parameters; it does not receive a separately fitted scaler. The same rule applies if you replace this arithmetic with the source's StandardScaler: fit on training data, then apply the learned transformation to held-out data.

5. Construct the regression network

   Two 64-neuron hidden layers, squared-error loss and the Adam optimizer define the source example's configuration.

   model = nn.Sequential(
       nn.Linear(1, 64),
       nn.ReLU(),
       nn.Linear(64, 64),
       nn.ReLU(),
       nn.Linear(64, 1)
   )
   
   loss_fn = nn.MSELoss()
   update_weights = torch.optim.Adam(
       model.parameters(), lr=0.01
   )
   training_epochs = 100
   
   print(model)

   The final layer produces a quantity rather than a class label. The settings are an example configuration, not evidence that these layers or this learning rate are optimal for your data.

   Inspect the printed structure against the tensors: one input value enters the network, the hidden layers each contain 64 neurons, and one predicted value comes out. ReLU supplies non-linearity, allowing the network to learn more than a linear relationship between position level and salary.

6. Train while watching validation loss

   This loop updates weights on training rows and reports validation loss without updating weights from validation examples.

   for epoch in range(training_epochs):
       model.train()
       training_predictions = model(training_inputs)
       training_loss = loss_fn(training_predictions, training_targets)
   
       update_weights.zero_grad()
       training_loss.backward()
       update_weights.step()
   
       model.eval()
       with torch.inference_mode():
           validation_predictions = model(validation_inputs)
           validation_loss = loss_fn(
               validation_predictions, validation_targets
           )
   
       if epoch % 10 == 0:
           print(
               f"Epoch {epoch}: "
               f"training={training_loss.item()} "
               f"validation={validation_loss.item()}"
           )

   Falling training loss alone doesn't establish generalisation. Use validation behaviour to decide whether to change capacity, training duration or data preparation. This loop uses a fixed duration; it doesn't implement early stopping or restore a best checkpoint.

   Each epoch passes the complete training tensor through the model. Validation then performs prediction and loss calculation only. Its examples inform your assessment, but they don't participate in the optimizer's weight update.

7. Measure held-out error in salary units

   Once the configuration is settled, this block scales the test inputs, predicts salaries and reports mean absolute error on the original salary scale.

   model.eval()
   with torch.inference_mode():
       test_inputs = (test_x - level_centre) / level_spread
       scaled_test_predictions = model(test_inputs)
       predicted_salaries = (
           scaled_test_predictions * salary_spread + salary_centre
       )
       absolute_error = nn.L1Loss()(predicted_salaries, test_y)
   
   print(f"Held-out salary MAE: {absolute_error.item()}")

   The printed result comes from your run; no accuracy or error figure is promised here. Don't repeatedly change the model in response to this number and continue describing the same rows as an untouched final test.

   Don't compare this number directly with the training log: training and validation report squared error on scaled targets, whereas this result reports absolute error in salary units. Compare the final MAE with the acceptance threshold you defined for the application.

8. Plot predictions without mixing scales

   This block predicts across the available position levels and plots both predictions and observations in salary units.

   model.eval()
   with torch.inference_mode():
       plot_inputs = (levels - level_centre) / level_spread
       plot_predictions = (
           model(plot_inputs) * salary_spread + salary_centre
       )
   
   plt.plot(levels, salaries, label='Actual Data', alpha=0.7)
   plt.plot(levels, plot_predictions, label='Model Predictions')
   plt.xlabel('Position Level')
   plt.ylabel('Salary')
   plt.title('Position Level vs Salary')
   plt.legend()
   plt.show()

   This corrects the source's final plot, which passes raw inputs into a network trained on scaled values. The chart includes training rows, so use it for inspection — not as a replacement for held-out error.

   The sequence is the same for plotting and testing: scale the position level, run the network, then undo salary scaling. Check that sequence before interpreting an unexpected chart as evidence that the architecture needs changing.

Read the training and validation losses together

Netguru's guide gives these patterns for interpreting training and validation behaviour.

These patterns guide the next experiment; they don't establish that the salary error meets your application's requirements. Keep that acceptance threshold separate from whether the training process appears healthy.

The remedies address different mechanisms. Netguru describes L2 regularisation as penalising large weights, while dropout deactivates neurons during training. Early stopping instead watches validation loss and halts training when improvement stops. If you introduce early stopping, retain the best checkpoint rather than assuming the final epoch produced the best model.

A salary notebook is not a production compensation model

This example uses position level alone. I wouldn't treat its output as sufficient evidence for compensation decisions: the sources identify data quality, representativeness and bias as development concerns. A clean split doesn't resolve those issues.

Nor does training-only scaling prevent every kind of leakage. Netguru identifies duplicates crossing dataset boundaries as another source. Its guide also rejects random splitting for time-series data, recommending chronological splits with a gap to prevent look-ahead leakage. If your salary task depends on time, revisit the split strategy before reusing this notebook unchanged.

Before you train a neural network for an application, compare it with an appropriate simpler approach; ProjectPro names linear regression for quantity prediction. Keep this walkthrough at notebook evaluation until you've addressed production concerns such as serving, model versioning, monitoring and retraining. Those are additional engineering work, not consequences of a successful training loop.