Self-Normalizing Net for Numeric Data
Perform classification or regression on numeric data
Resource retrieval
Resource retrieval
Get the uninitialized net (there are no pre-trained nets in this model):
In[]:=
NetModel["Self-Normalizing Net for Numeric Data"]
Out[]=
NetModel parameters
NetModel parameters
This model consists of a family of individual nets, each identified by a specific parameter combination. Inspect the available parameters:
In[]:=
NetModel["Self-Normalizing Net for Numeric Data","ParametersInformation"]
Out[]=
Pick a non-default model by specifying the parameters:
In[]:=
NetModel[{"Self-Normalizing Net for Numeric Data","TaskType""Regression","Depth"5,"DropoutProbability"0.02}]
Out[]=
Check the default parameter combination:
In[]:=
NetModel["Self-Normalizing Net for Numeric Data","DefaultVariant"]
Out[]=
{Self-Normalizing Net for Numeric Data,TaskTypeClassification,Depth8,Width50,DropoutProbability0.01}
Basic usage
Basic usage
Classification on numerical data: In this example, we use an eight-layer self-normalizing network to perform classification on the UCI Letter dataset. First, obtain the training and test data:
In[]:=
train=ResourceData["Sample Data: UCI Letter","TrainingData"];test=ResourceData["Sample Data: UCI Letter","TestData"];
View two random training examples:
In[]:=
RandomSample[train,2]
Out[]=
{{2,4,3,2,2,9,7,3,5,10,5,7,2,8,4,9}B,{4,10,5,8,6,8,7,7,6,7,6,6,2,8,7,10}B}
Self-normalizing nets assume that the input data has a mean of 0 and variance of 1. Standardize the test and training data:
In[]:=
extractor=FeatureExtraction[N@Keys[train],"StandardizedVector"];trainStandardized=extractor[Keys[train]]Values[train];testStandardized=extractor[Keys[test]]Values[test];
Get the training net:
In[]:=
net=NetModel["Self-Normalizing Net for Numeric Data"]
Out[]=
Specify a decoder for the net:
In[]:=
dec=NetDecoder[{"Class",Union@Values[train]}]
Out[]=
Train the net for 150 rounds, leaving 5% of the data for a validation set:
In[]:=
trainedNet=NetTrain[NetReplacePart[net,"Output"dec],trainStandardized,ValidationSet->Scaled[0.05],MaxTrainingRounds150]
Out[]=
Obtain the accuracy of the trained net on the standardized test set:
In[]:=
ClassifierMeasurements[trainedNet,testStandardized,"Accuracy"]
Out[]=
0.966
Compare the accuracy against all the methods in :
In[]:=
Dataset@ReverseSort@AssociationMap[ClassifierMeasurements[Classify[train,Method#],test,"Accuracy"]&,{"RandomForest","NaiveBayes","SupportVectorMachine","NearestNeighbors","LogisticRegression","GradientBoostedTrees","DecisionTree","Markov"}]
Out[]=
Obtain a random sample of standardized test data:
In[]:=
sample=RandomSample[Thread[Keys[testStandardized]->Values[testStandardized]],2]
Out[]=
{{-1.58254,-2.12752,-1.55617,-2.3716,-1.59663,0.0529081,2.77983,-0.968956,-0.910091,-0.517626,1.33565,0.0351728,-1.30155,-0.221894,-1.43196,0.127078}T,{-1.05855,-0.614119,-1.05697,-1.04567,-1.14046,0.0529081,-0.219573,-1.70656,0.767206,1.89625,-0.17837,0.0351728,-1.30155,-0.221894,-1.04305,-0.496616}I}
Test the trained net on a sample of standardized test data:
In[]:=
trainedNet[Keys[sample]]
Out[]=
{T,I}
Improving accuracy of the classifier net
Improving accuracy of the classifier net
Using the same example for data, first we obtain the dataset:
In[]:=
train=ResourceData["Sample Data: UCI Letter","TrainingData"];test=ResourceData["Sample Data: UCI Letter","TestData"];
Standardize the test and training data in similar fashion:
In[]:=
extractor=FeatureExtraction[N@Keys[train],"StandardizedVector"];trainStandardized=extractor[Keys[train]]Values[train];testStandardized=extractor[Keys[test]]Values[test];
Get the training net:
In[]:=
net=NetModel["Self-Normalizing Net for Numeric Data"]
Out[]=
Specify a decoder for the net:
In[]:=
dec=NetDecoder[{"Class",Union@Values[train]}]
Out[]=
To improve the final accuracy, it is possible to average multiple trained networks obtained from different training runs. The following function runs multiple trainings and creates an ensemble network, which averages the outputs of the trained nets:
Specify the number of nets in the ensemble and create the ensemble network:
Obtain the accuracy of the ensemble network:
Compare it with the accuracy of the individual nets:
Classification on nominal data
Classification on nominal data
In this example, we use an eight-layer self-normalizing network to perform classification on the Mushroom Classification dataset. First, obtain the training and test data:
View two random training examples:
To standardize this data, we first need to convert all the nominal input classes into indicator vectors:
Then standardize the numeric vectors:
Create the standardized test and training dataset:
Get the training net:
Specify a decoder for the net:
Train the net for 150 rounds, leaving 5% of the data for a validation set:
Obtain a sample of standardized test data and view the actual class labels:
Test the trained net on a sample of standardized test data:
Regression on numerical data
Regression on numerical data
In this example, we use an eight-layer self-normalizing network to predict the median value of properties in a neighborhood of Boston, given some features of the neighborhood. First, obtain the training and test data:
View two random training examples:
Self-normalizing nets assume that the input data has a mean of 0 and variance of 1. Standardize the test and training data:
Get the training net:
Train the net for 250 rounds leaving 7% of the data for a validation set and return both the trained net and the lowest validation loss:
Compute the test-set standard deviation:
Obtain a sample of standardized test data and view the actual class labels:
Test the trained net on a sample of standardized test data:
Regression on nominal data
Regression on nominal data
Create a dataset of the average monthly temperature (in degrees Celsius) as a function of the city, the year and the month:
View two random examples:
Split the data into training (80%) and test (20%) sets:
To standardize this data, we need to convert all the nominal classes to a indicator vectors:
Then standardize the numeric vector:
Create the standardized test and training dataset:
Get the training net:
Train the net for 1000 rounds leaving 7% of the data for a validation set and return both the trained net and the lowest validation loss:
Compute the test-set standard deviation:
Obtain a sample of standardized test data and view the actual class labels:
Test the trained net on a sample of standardized test data:
Net information
Net information
Obtain the layer type counts:
Display the summary graphic:
Export to MXNet
Export to MXNet
Represent the MXNet net as a graph: