Sketch-RNN Trained on QuickDraw Data
Generate hand-drawn sketches
Resource retrieval
Resource retrieval
Get the pre-trained net:
In[]:=
NetModel["Sketch-RNN Trained on QuickDraw Data"]
Out[]=
NetModel parameters
NetModel parameters
This model consists of a family of individual nets, each identified by a specific parameter combination. Inspect the available parameters and their default values:
In[]:=
NetModel["Sketch-RNN Trained on QuickDraw Data","ParametersInformation"]
Out[]=
Pick a non-default net by specifying the parameters:
In[]:=
NetModel[{"Sketch-RNN Trained on QuickDraw Data","Object""Chair"}]
Out[]=
NetChain
Pick a non-default uninitialized net:
In[]:=
NetModel[{"Sketch-RNN Trained on QuickDraw Data","Object""Flower"},"UninitializedEvaluationNet"]
Out[]=
NetChain
Check the default parameter combination:
In[]:=
NetModel["Sketch-RNN Trained on QuickDraw Data","DefaultVariant"]
Out[]=
{Sketch-RNN Trained on QuickDraw Data,ObjectCat}
Evaluation function
Evaluation function
Define an evaluation function to generate a sketch from a fixed initial condition using temperature sampling:
In[]:=
drawSketch[obj_,temp_:0.01,maxLen_:300]:=Block[{stateObject,lastPos,pos,stroke,segments,time,lastAction,action,offset},stateObject=NetStateObject@NetModel[{"Sketch-RNN Trained on QuickDraw Data","Object"obj}];lastPos=pos={0,0};stroke={0,0,0,0,0};segments=Table[{},maxLen];time=0;action=1;While[time++<maxLen,lastAction=action;stroke=stateObject[{stroke},{"RandomSample","Temperature"N@temp}];offset=stroke[[;;2]];action=First@Ordering[stroke[[3;;]],-1];lastPos=pos;pos+=offset*{1,-1};Switch[lastAction,1,segments[[time]]={lastPos,pos},3,Break[]]];segments];
In[]:=
netevaluate[obj_,temp_:0.01,maxLen_:300]:=Graphics@Line@drawSketch[obj,temp,maxLen]
Basic usage
Basic usage
Generate four sketches of a cat:
In[]:=
Table[netevaluate["Cat"],4]
Out[]=
,
,
,
The third optional argument is a “temperature” parameter that regulates sampling. A higher temperature increases the variability in the output, increasing the probability of sampling less likely strokes:
In[]:=
Table[netevaluate["Cat",0.4],4]
Out[]=
,
,
,
Very high temperature settings are equivalent to sampling from a flat distribution:
In[]:=
Table[netevaluate["Elephant",3],4]
Out[]=
,
,
,
Very low temperature settings further increase the probability of extracting more likely strokes. Sampling at zero temperature is equivalent to always picking the stroke with maximum probability, and the function produces the same sketch every time:
In[]:=
Table[netevaluate["Chair",0],4]
Out[]=
,
,
,
Visualize the sequence of pen strokes
Visualize the sequence of pen strokes
Create a function which displays an animation showing the object being drawn from the sequence of pen strokes chosen by the network:
Sketch representation
Sketch representation
The model works like a language model, reading a sequence of pen strokes and predicting the next one:
Decoder properties
Decoder properties
Extract the “Function” net decoder:
Inspect the associated Function:
This decoder supports properties. The default one, “Decision”, returns the pen stroke with highest associated probability:
It is possible to perform temperature sampling from the output distribution with the property “RandomSample”. In this case, the evaluation will return different results every time:
The property “Distribution” returns the full probability distribution for the stroke direction and the pen state. The stroke direction distribution is a mixture of two-dimensional normal distributions, while the distribution of the pen state is a categorical distribution with three probability values for 1, 2 and 3:
Net information
Net information
Inspect the number of parameters of all arrays in the net:
Obtain the total number of parameters:
Obtain the layer type counts:
Display the summary graphic:
Export to MXNet
Export to MXNet
Get the size of the parameter file:
Represent the MXNet net as a graph: