Data preparation is a big part of applied machine learning.
Correctly preparing your training data can mean the difference between mediocre and extraordinary results, even with very simple linear algorithms.
Performing data preparation operations, such as scaling, is relatively straightforward for input variables and has been made routine in Python via the Pipeline scikit-learn class.
On regression predictive modeling problems where a numerical value must be predicted, it can also be critical to scale and perform other data transformations on the target variable. This can be achieved in Python using the TransformedTargetRegressor class.
In this tutorial, you will discover how to use the TransformedTargetRegressor to scale and transform target variables for regression using the scikit-learn Python machine learning library.
After completing this tutorial, you will know:
The importance of scaling input and target data for machine learning.
The two approaches to applying data transforms to target variables.
How to use the TransformedTargetRegressor on a real regression dataset.
Kick-start your project with my new book Data Preparation for Machine Learning, including step-by-step tutorials and the Python source code files for all examples.
Let’s get started.
How to Transform Target Variables for Regression With Scikit-Learn Photo by Don Henise, some rights reserved.
Tutorial Overview
This tutorial is divided into three parts; they are:
Importance of Data Scaling
How to Scale Target Variables
Example of Using the TransformedTargetRegressor
Importance of Data Scaling
It is common to have data where the scale of values differs from variable to variable.
For example, one variable may be in feet, another in meters, and so on.
Some machine learning algorithms perform much better if all of the variables are scaled to the same range, such as scaling all variables to values between 0 and 1, called normalization.
This effects algorithms that use a weighted sum of the input, like linear models and neural networks, as well as models that use distance measures such as support vector machines and k-nearest neighbors.
As such, it is a good practice to scale input data, and perhaps even try other data transforms such as making the data more normal (better fit a Gaussian probability distribution) using a power transform.
This also applies to output variables, called target variables, such as numerical values that are predicted when modeling regression predictive modeling problems.
For regression problems, it is often desirable to scale or transform both the input and the target variables.
Scaling input variables is straightforward. In scikit-learn, you can use the scale objects manually, or the more convenient Pipeline that allows you to chain a series of data transform objects together before using your model.
The Pipeline will fit the scale objects on the training data for you and apply the transform to new data, such as when using a model to make a prediction.
The challenge is, what is the equivalent mechanism to scale target variables in scikit-learn?
Want to Get Started With Data Preparation?
Take my free 7-day email crash course now (with sample code).
Click to sign-up and also get a free PDF Ebook version of the course.
How to Scale Target Variables
There are two ways that you can scale target variables.
The first is to manually manage the transform, and the second is to use a new automatic way for managing the transform.
Manually transform the target variable.
Automatically transform the target variable.
1. Manual Transform of the Target Variable
Manually managing the scaling of the target variable involves creating and applying the scaling object to the data manually.
It involves the following steps:
Create the transform object, e.g. a MinMaxScaler.
Fit the transform on the training dataset.
Apply the transform to the train and test datasets.
Invert the transform on any predictions made.
For example, if we wanted to normalize a target variable, we would first define and train a MinMaxScaler object:
1
2
3
4
...
# create target scaler object
target_scaler=MinMaxScaler()
target_scaler.fit(train_y)
We would then transform the train and test target variable data.
1
2
3
4
...
# transform target variables
train_y=target_scaler.transform(train_y)
test_y=target_scaler.transform(test_y)
Then we would fit our model and use the model to make predictions.
Before the predictions can be used or evaluated with an error metric, we would have to invert the transform.
1
2
3
4
...
# invert transform on predictions
yhat=model.predict(test_X)
yhat=target_scaler.inverse_transform(yhat)
This is a pain, as it means you cannot use convenience functions in scikit-learn, such as cross_val_score(), to quickly evaluate a model.
2. Automatic Transform of the Target Variable
An alternate approach is to automatically manage the transform and inverse transform.
This can be achieved by using the TransformedTargetRegressor object that wraps a given model and a scaling object.
It will prepare the transform of the target variable using the same training data used to fit the model, then apply that inverse transform on any new data provided when calling predict(), returning predictions in the correct scale.
To use the TransformedTargetRegressor, it is defined by specifying the model and the transform object to use on the target; for example:
Later, the TransformedTargetRegressor instance can be fit like any other model by calling the fit() function and used to make predictions by calling the predict() function.
1
2
3
4
...
# use the target transform wrapper
wrapped_model.fit(train_X,train_y)
yhat=wrapped_model.predict(test_X)
This is much easier and allows you to use helpful functions like cross_val_score() to evaluate a model
Now that we are familiar with the TransformedTargetRegressor, let’s look at an example of using it on a real dataset.
Example of Using the TransformedTargetRegressor
In this section, we will demonstrate how to use the TransformedTargetRegressor on a real dataset.
We will use the Boston housing regression problem that has 13 inputs and one numerical target and requires learning the relationship between suburb characteristics and house prices.
We can confirm that the dataset can be loaded correctly as a NumPy array and split it into input and output variables.
The complete example is listed below.
1
2
3
4
5
6
7
8
# load and summarize the dataset
from numpy import loadtxt
# load data
dataset=loadtxt('housing.csv',delimiter=",")
# split into inputs and outputs
X,y=dataset[:,:-1],dataset[:,-1]
# summarize dataset
print(X.shape,y.shape)
Running the example prints the shape of the input and output parts of the dataset, showing 13 input variables, one output variable, and 506 rows of data.
1
(506, 13) (506,)
We can now prepare an example of using the TransformedTargetRegressor.
A naive regression model that predicts the mean value of the target on this problem can achieve a mean absolute error (MAE) of about 6.659. We will aim to do better.
In this example, we will fit a HuberRegressor object and normalize the input variables using a Pipeline.
Next, we will define a TransformedTargetRegressor instance and set the regressor to the pipeline and the transformer to an instance of a MinMaxScaler object.
Running the example evaluates the model with normalization of the input and output variables.
Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.
In this case, we achieve a MAE of about 3.1, much better than a naive model that achieved about 6.6.
1
Mean MAE: 3.191
We are not restricted to using scaling objects; for example, we can also explore using other data transforms on the target variable, such as the PowerTransformer, that can make each variable more-Gaussian-like (using the Yeo-Johnson transform) and improve the performance of linear models.
By default, the PowerTransformer also performs a standardization of each variable after performing the transform.
The complete example of using a PowerTransformer on the input and target variables of the housing dataset is listed below.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# example of power transform input and output variables for regression.
from numpy import mean
from numpy import absolute
from numpy import loadtxt
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.pipeline import Pipeline
from sklearn.linear_model import HuberRegressor
from sklearn.preprocessing import PowerTransformer
from sklearn.compose import TransformedTargetRegressor
Running the example evaluates the model with a power transform of the input and output variables.
Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.
In this case, we see further improvement to a MAE of about 2.9.
1
Mean MAE: 2.926
Further Reading
This section provides more resources on the topic if you are looking to go deeper.
It provides self-study tutorials with full working code on: Feature Selection, RFE, Data Cleaning, Data Transforms, Scaling, Dimensionality Reduction,
and much more...
Bring Modern Data Preparation Techniques to Your Machine Learning Projects
Hi Jason – is there any mathematical basis to use the testing metric on the inverse transformed? For example let’s say I have a log log model – and get a higher r2 in log space vs the inverse transform r2. Would this be making a critical error?
Okay thank you – I have a situation where the r2 in a log-log model is way higher vs applying the TargetTransformer with the same function on the same data. Just having trouble deciding which metric should be used/tuned against.
Hi Jason. Thank you for this useful post. How do you interpret the scaled variables? Is it necessary for transform back to the original scale? Finally, how do you decide whether or not to scale the target variable?
Yes, typically this is a good approach when using error metrics like MSE/RMSE/MAE.
There are heuristics for choosing, but in practice, I test a suite of transforms and use whatever best minimizes error with a given dataset + model + hyperparameters.
Hi Jason,
Very helpful post. I have a question regarding my dataset. In my dataset, all the variables (independents and dependent) contain values in a range [-1 to 1]. Do i still need to normalize them to another scale? If not, will this affect the interpretation of the evaluation scores?
Regards
Thank you for this great tutorial! As well as for ‘Regression Tutorial with the Keras Deep Learning Library in Python!’
I understood that I learn with ‘cross_val_score’ about the performance of my model.
But what if I want to improve my model? Or just create it, fit it and make predictions? Especially how can I pass fitParams to the model?
Your code:
pipeline = Pipeline(steps=[(‘normalize’, MinMaxScaler()), (‘model’, HuberRegressor())])
# prepare the model with target scaling
model = TransformedTargetRegressor(regressor=pipeline, transformer=MinMaxScaler())
# this is what I want to do basically
X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X, y, test_size=0.3)
# I know this is wrong!
# history = model.fit(X_train,Y_train,[‘epochs=50’])
# This works, but I would like to get better results
y_resp = model.predict(X_test)
Sorry, this is perhaps a very basic question, but I have no idea how to solve this.Thank you for this great tutorial! As well as for ‘Regression Tutorial with the Keras Deep Learning Library in Python!’
I understood that I learn with ‘cross_val_score’ about the performance of my model.
But what if I want to improve my model? Or just create it, fit it and make predictions? Especially how can I pass fitParams to the model?
Your code:
pipeline = Pipeline(steps=[(‘normalize’, MinMaxScaler()), (‘model’, HuberRegressor())])
# prepare the model with target scaling
model = TransformedTargetRegressor(regressor=pipeline, transformer=MinMaxScaler())
# this is what I want to do basically
X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X, y, test_size=0.3)
# I know this is wrong!
# history = model.fit(X_train,Y_train,[‘epochs=50’])
# This works, but I would like to get better results
y_resp = model.predict(X_test)
Sorry, this is perhaps a very basic question, but I have no idea how to solve this.Thank you for this great tutorial! As well as for ‘Regression Tutorial with the Keras Deep Learning Library in Python!’
I understood that I learn with ‘cross_val_score’ about the performance of my model.
But what if I want to improve my model? Or just create it, fit it and make predictions? Especially how can I pass fitParams to the model?
Your code:
pipeline = Pipeline(steps=[(‘normalize’, MinMaxScaler()), (‘model’, HuberRegressor())])
# prepare the model with target scaling
model = TransformedTargetRegressor(regressor=pipeline, transformer=MinMaxScaler())
# this is what I want to do basically
X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X, y, test_size=0.3)
# I know this is wrong!
# history = model.fit(X_train,Y_train,[‘epochs=50’])
# This works, but I would like to get better results
y_resp = model.predict(X_test)
Sorry, this is perhaps a very basic question, but I have no idea how to solve this. Maybe I missed a post from you
Thank you yery much, Jason and sorry for inserting my question twice. It was a bit hectic here.
I thought I could improve my results by set a training epochs argument, what obviously was wrong for this kind of model. I was able to improve my results a bit meanwhile.
Would you recommend one of your books especially for a regression problem e.g. comparable to the Boston Housing Dataset problem (various inputs, one or two analogous outputs)?
hello Jason, thanks . I’m actually working on a project’s client life value (regression).
My dataset is not linear. what do I need to do to solve this problem
Hi Jason, thanks for the tutorial! Some say it’s not appropriate to scale your target variable and do not specify in which cases (you suggest that it is important to do so especially in regression problems).
Best regards,
Marina
First of all, thanks for this interesting tutorial, which led me to by you book; please keep up the good work!
I’m trying to implement a GridsearchCV procedure for a 12 input two output keras regression model with tensorflow backend:
def model_opt(n_hidden=1, n_units=32, input_shape=[12]):
model = keras.models.Sequential()
# The Input Layer :
model.add(InputLayer(input_shape=input_shape))
model.add(Dropout(.2))
# NN_model.add(BatchNormalization())
# The Hidden Layers :
for layer in range(n_hidden):
model.add(Dense(n_units, kernel_initializer=’normal’,activation=’relu’))
model.add(Dropout(.5))
# NN_model.add(BatchNormalization())
# The Output Layer :
model.add(Dense(2, kernel_initializer=’normal’,activation=’linear’))
# Compile the network :
model.compile(loss=’mse’, optimizer=’adam’, metrics=[‘mse’])
return(model)
I’m trying to embedded the model in a pipeline, and perform target transformtion, as you show above;
ValueError: Invalid parameter mlp for estimator TransformedTargetRegressor(regressor=Pipeline(steps=[(‘scaler’,
StandardScaler()),
(‘mlp’,
)]),
transformer=StandardScaler()). Check the list of available parameters with estimator.get_params().keys().
Off-hand mixing keras + multi-output + grids search sounds challenging, it would be easier to run any grid search manually with for loops via the Keras API directly.
I don’t have an obvious fix for you sorry – I’ve not tried tuning keras model hyperparameters with a pipeline. Again, my gut tells me to run the search manually.
Hi Jason,
Is it possible to use TransformedTargetRegressor() with a classification model?
If not, is there any way to transform multiple target variables in the classification problem using sklearn pipeline?
Thanks very much for your article. I want to classify a numeric target variable based quantiles and then use a classifier. In sklearn you have pipline and can use this pipeline for cross-validation avoiding any data leakage. Can I use TransformedTargetRegressor for this purpose? – But the problems seems that it always transforms it then back, right? Is there any other method in sklearn allowing me that to do?
Thank you for this article. My question is when to re-scale to the original scale. Should you use the transformed or the untransformed y in the grid search for hyperparameter tuning? Should the CV score be based on the transformed y?
Thanks!
Hi Ueberti…This is a great question! The benefit and reason for utilizing neural networks is that the underlying functional mapping cannot be obtained in a closed form solution. If you had such a function, neural networks would not offer much value.
Hi Jason – is there any mathematical basis to use the testing metric on the inverse transformed? For example let’s say I have a log log model – and get a higher r2 in log space vs the inverse transform r2. Would this be making a critical error?
Only that any error calculated (e.g. MSE/MAE) would have the same units as the target variable and be easier to interpret by domain experts.
I don’t believe it is as important/required if you are calculating r^2.
Okay thank you – I have a situation where the r2 in a log-log model is way higher vs applying the TargetTransformer with the same function on the same data. Just having trouble deciding which metric should be used/tuned against.
If r2 is your chosen metric, explore all models/transforms that maximize that score.
Hi Jason. Thank you for this useful post. How do you interpret the scaled variables? Is it necessary for transform back to the original scale? Finally, how do you decide whether or not to scale the target variable?
Yes, typically this is a good approach when using error metrics like MSE/RMSE/MAE.
There are heuristics for choosing, but in practice, I test a suite of transforms and use whatever best minimizes error with a given dataset + model + hyperparameters.
How do I transform predictions back to the original scale in production?
Hi Jason,
Very helpful post. I have a question regarding my dataset. In my dataset, all the variables (independents and dependent) contain values in a range [-1 to 1]. Do i still need to normalize them to another scale? If not, will this affect the interpretation of the evaluation scores?
Regards
Try modeling with and without scaling and compare the performance of the resulting models.
What a nice piece. Its really helpful. Thank you.
Thanks!
I hope it helps on your project.
Thank you for this great tutorial! As well as for ‘Regression Tutorial with the Keras Deep Learning Library in Python!’
I understood that I learn with ‘cross_val_score’ about the performance of my model.
But what if I want to improve my model? Or just create it, fit it and make predictions? Especially how can I pass fitParams to the model?
Your code:
pipeline = Pipeline(steps=[(‘normalize’, MinMaxScaler()), (‘model’, HuberRegressor())])
# prepare the model with target scaling
model = TransformedTargetRegressor(regressor=pipeline, transformer=MinMaxScaler())
# this is what I want to do basically
X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X, y, test_size=0.3)
# I know this is wrong!
# history = model.fit(X_train,Y_train,[‘epochs=50’])
# This works, but I would like to get better results
y_resp = model.predict(X_test)
Sorry, this is perhaps a very basic question, but I have no idea how to solve this.Thank you for this great tutorial! As well as for ‘Regression Tutorial with the Keras Deep Learning Library in Python!’
I understood that I learn with ‘cross_val_score’ about the performance of my model.
But what if I want to improve my model? Or just create it, fit it and make predictions? Especially how can I pass fitParams to the model?
Your code:
pipeline = Pipeline(steps=[(‘normalize’, MinMaxScaler()), (‘model’, HuberRegressor())])
# prepare the model with target scaling
model = TransformedTargetRegressor(regressor=pipeline, transformer=MinMaxScaler())
# this is what I want to do basically
X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X, y, test_size=0.3)
# I know this is wrong!
# history = model.fit(X_train,Y_train,[‘epochs=50’])
# This works, but I would like to get better results
y_resp = model.predict(X_test)
Sorry, this is perhaps a very basic question, but I have no idea how to solve this.Thank you for this great tutorial! As well as for ‘Regression Tutorial with the Keras Deep Learning Library in Python!’
I understood that I learn with ‘cross_val_score’ about the performance of my model.
But what if I want to improve my model? Or just create it, fit it and make predictions? Especially how can I pass fitParams to the model?
Your code:
pipeline = Pipeline(steps=[(‘normalize’, MinMaxScaler()), (‘model’, HuberRegressor())])
# prepare the model with target scaling
model = TransformedTargetRegressor(regressor=pipeline, transformer=MinMaxScaler())
# this is what I want to do basically
X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X, y, test_size=0.3)
# I know this is wrong!
# history = model.fit(X_train,Y_train,[‘epochs=50’])
# This works, but I would like to get better results
y_resp = model.predict(X_test)
Sorry, this is perhaps a very basic question, but I have no idea how to solve this. Maybe I missed a post from you
Best Regards
Matthias
More on how to improve a model:
https://machinelearningmastery.com/machine-learning-performance-improvement-cheat-sheet/
More on how to make predictions:
https://machinelearningmastery.com/make-predictions-scikit-learn/
Thank you yery much, Jason and sorry for inserting my question twice. It was a bit hectic here.
I thought I could improve my results by set a training epochs argument, what obviously was wrong for this kind of model. I was able to improve my results a bit meanwhile.
Would you recommend one of your books especially for a regression problem e.g. comparable to the Boston Housing Dataset problem (various inputs, one or two analogous outputs)?
Best Regards
Matthias
Not really. I recommend this process to get the most out of your model:
https://machinelearningmastery.com/start-here/#process
Thanks Jason for the help. As always, everything is explained very well.
Best Regards
Matthias
Thanks.
hello Jason, thanks . I’m actually working on a project’s client life value (regression).
My dataset is not linear. what do I need to do to solve this problem
I recommend testing a suite of different regression algorithms and discover what works best.
Hi, Again Jason 🙂
I am using feedforward neural network for regression project.
Is it recommended to use Minmaxscaler than StandardScaler for Target (Y) when we have a clear boundary? let say 100 to 8000?
Now I am just doing this (Y= Y/8000) and I am getting good results.
Thank you again for helping us with your answers, helpful blog and books.
Yes.
Hi Jason, thanks for the tutorial! Some say it’s not appropriate to scale your target variable and do not specify in which cases (you suggest that it is important to do so especially in regression problems).
Best regards,
Marina
Thanks.
For keras/tensorflow type models, its done manually and the TransformedTargetRegressor cant be used?
Manually.
Hi Jason,
I ran into this article and I just wanted to get your thoughts on this.
Is R squared really a useless metric in practice?
https://data.library.virginia.edu/is-r-squared-useless/
Regards,
Sam.
I don’t have strong opinions on r^2, perhaps contact the authors directly about your concerns.
Thank you!
You’re welcome.
Hi Jason,
Thanks for the tutorial.
Can we use same TransformedTargetRegressor instance with LSTM sequential model? If so, how could we approach adapt the current code?
No, I don’t think so. You must prepare the data manually I believe.
Thanks!
You’re welcome.
cant we use a custom transformer class that transform the target variable inside the pipeline ?
Yes, see this tutorial:
https://machinelearningmastery.com/create-custom-data-transforms-for-scikit-learn/
Dear Jason,
First of all, thanks for this interesting tutorial, which led me to by you book; please keep up the good work!
I’m trying to implement a GridsearchCV procedure for a 12 input two output keras regression model with tensorflow backend:
def model_opt(n_hidden=1, n_units=32, input_shape=[12]):
model = keras.models.Sequential()
# The Input Layer :
model.add(InputLayer(input_shape=input_shape))
model.add(Dropout(.2))
# NN_model.add(BatchNormalization())
# The Hidden Layers :
for layer in range(n_hidden):
model.add(Dense(n_units, kernel_initializer=’normal’,activation=’relu’))
model.add(Dropout(.5))
# NN_model.add(BatchNormalization())
# The Output Layer :
model.add(Dense(2, kernel_initializer=’normal’,activation=’linear’))
# Compile the network :
model.compile(loss=’mse’, optimizer=’adam’, metrics=[‘mse’])
return(model)
I’m trying to embedded the model in a pipeline, and perform target transformtion, as you show above;
estimators = []
estimators.append(( ‘scaler’, sklearn.preprocessing.StandardScaler() ))
estimators.append(( ‘mlp’, keras.wrappers.scikit_learn.KerasRegressor(model_opt) ))
pipeline = Pipeline(estimators)
pipe_y = sklearn.compose.TransformedTargetRegressor(regressor=pipeline,
transformer=sklearn.preprocessing.StandardScaler() )
hyper_param = {‘mlp__n_hidden’: [1,2,3,4],
‘mlp__n_units’: np.power(2, np.arange(5,10))
}
rsCV = RandomizedSearchCV(pipe_y, hyper_param, n_iter=10, cv=5, refit=True, random_state=1234)
rsCV.fit(Xtrain2, ytrain2, mlp__epochs=200, mlp__batch_size=8, mlp__callbacks=callbacks_list);
but it just giving me the error:
ValueError: Invalid parameter mlp for estimator TransformedTargetRegressor(regressor=Pipeline(steps=[(‘scaler’,
StandardScaler()),
(‘mlp’,
)]),
transformer=StandardScaler()). Check the list of available parameters with
estimator.get_params().keys().Do you have an idea to how I get this to work?
Thank you very much!
Best,
Sebastian
Hi again,
I fixed it by changing the ‘hyper_param’ object to
hyper_param = {‘regressor__mlp__n_hidden’: (1,2,3,4),
‘regressor__mlp__n_units’: (32, 64, 128, 256, 512)
}
It is appenty a sklearn bug…
Well done Sebastian!
Thanks Sebastian!
Off-hand mixing keras + multi-output + grids search sounds challenging, it would be easier to run any grid search manually with for loops via the Keras API directly.
I don’t have an obvious fix for you sorry – I’ve not tried tuning keras model hyperparameters with a pipeline. Again, my gut tells me to run the search manually.
Thank you so much for this wonderful article.
You’re welcome.
Hi Jason is there a way to include inverseTransform in the pipeline so that the MSE is in a linear scale?
It inverts the transform for you when you make predictions.
Nice article Jason,
Just double-checking the line where you mention:
“then apply that inverse transform on any new data provided when calling fit(), returning predictions in the correct scale”
You meant predict(), not fit(), right?
Yes, thanks. Fixed.
Hi Jason,
Nice tutorial. Is there a way to use transformed column CLASSIFIER?
Sorry, I don’t understand your question. Could you please rephrase or elaborate?
Hi Jason,
Is it possible to use TransformedTargetRegressor() with a classification model?
If not, is there any way to transform multiple target variables in the classification problem using sklearn pipeline?
No, it is for regression.
You can use the label encoder on the target variable directly.
How can I use transformer for target like exponential and log(y+1) ? Are these available in power transformer?
See this tutorial:
https://machinelearningmastery.com/power-transforms-with-scikit-learn/
Hi Jason,
Thanks very much for your article. I want to classify a numeric target variable based quantiles and then use a classifier. In sklearn you have pipline and can use this pipeline for cross-validation avoiding any data leakage. Can I use TransformedTargetRegressor for this purpose? – But the problems seems that it always transforms it then back, right? Is there any other method in sklearn allowing me that to do?
Thanks ahead!
Yes, but that should not be a problem if you already fit the model and just use it for prediction.
are we here separately transforming the target variable in training and testing? if not, why?
Yes. Separately, but the transformer is only fitted using the training data.
Thank you for this article. My question is when to re-scale to the original scale. Should you use the transformed or the untransformed y in the grid search for hyperparameter tuning? Should the CV score be based on the transformed y?
Thanks!
Hi ML…You should normalize data and use it for training. Then revert the predictions to the original scale and determine the score.
Do you know how to take a trained neural network and extract the non-linear function from it?. I need to extract the function to use in PSO optimizer.
the trained network:
regressor = Sequential()
regressor.add(Dense(units = 7, activation = ‘relu’, input_dim = 5))
#regressor.add(Dense(units = 4, activation = ‘relu’))
regressor.add(Dense(units = 1, activation = ‘linear’))
otimizador = keras.optimizers.Adam(lr = 0.001, decay = 0.0001, clipvalue = 0.3)
regressor.compile(optimizer = otimizador, loss = ‘mean_absolute_error’,
metrics = [‘mean_absolute_error’])
regressor.fit(X_train, y_train, epochs = 120)
prev = regressor.predict(X_test)
Hi Ueberti…This is a great question! The benefit and reason for utilizing neural networks is that the underlying functional mapping cannot be obtained in a closed form solution. If you had such a function, neural networks would not offer much value.
how to use power transformations on spectral NIR data, where-in target is continuous variable and independent variables are multivariate-wavelengths?
Hi yogini…The following paper is a great starting point for this topic:
https://www.sciencedirect.com/science/article/pii/S0165993622002874