Basic Examples

Group the data by the first element in each list, then get the last element of each list, and then compute the
Mean
of each grouping:
[◼]
MapReduceOperator
[FirstLast,Mean][{{a,x},{b,v},{a,y},{a,z},{b,w}}]
In[]:=
a
1
3
(x+y+z),b
v+w
2

Out[]=
Group a list of integers according to their parity, compute their square roots, and display the results as column vectors:
[◼]
MapReduceOperator
[PositiveSqrt,MatrixForm][Range[-4,4]]
In[]:=
False
2

3

2

0
,True
1
2
3
2

Out[]=

Scope

Use
MapReduceOperator
with slot functions to create an operator that works on lists of associations (optionally wrapped in
Dataset
) and then apply it to the Titanic dataset:
[◼]
MapReduceOperator
[(#sex&)(#survived&),Counts][ExampleData[{"Dataset","Titanic"}]]
In[]:=
female
True
339
False
127
male
True
161
False
682
Out[]=
The same query as above, but with the reducer function
Counts
run in parallel on the normal form of the dataset:
[◼]
MapReduceOperator
[(#sex&)(#survived&),Counts,"Parallelize"True][Normal@ExampleData[{"Dataset","Titanic"}]]
In[]:=
femaleTrue339,False127,maleTrue161,False682
Out[]=

Options

"Parallelize"
can take on the value
True
or
False
. If it is set to anything else, it is treated as if it were
False
. “Parallelize” will produce error messages if the data is wrapped in
Dataset
:
[◼]
MapReduceOperator
[(#sex&)(#survived&),Counts,"Parallelize"True][ExampleData[{"Dataset","Titanic"}]]
In[]:=
Counts
:The argument RowBox[{Dataset,RowBox[{[,RowBox[{,2,}],]}]}] is not a list.
ParallelCombine
:
cannot be parallelized; proceeding with sequential evaluation.
female
True
339
False
127
male
True
161
False
682
Out[]=
Applying Normal to the dataset will fix this problem:
[◼]
MapReduceOperator
[(#sex&)(#survived&),Counts,"Parallelize"True][Normal@ExampleData[{"Dataset","Titanic"}]]
In[]:=
femaleTrue339,False127,maleTrue161,False682
Out[]=

Applications

Get histograms of the number of dates between the diagnosis of AIDS and death of the patient broken down by Australian state:
With{died=Select[ExampleData[{"Statistics","AustraliaAIDS"}],#[[5]]==="D"&]},
[◼]
MapReduceOperator
[(#[[1]]&)(#[[4]]-#[[3]]&),Histogram[#,Automatic,"PDF",PlotRange{{0,2500},Automatic}]&][died]​​
In[]:=
NSW,other,QLD,VIC
Out[]=
Group the planets according to the number of moons and compute for each grouping the median volume of the planets (assuming they are spherical):
[◼]
MapReduceOperator
(Length[#Moons]&)
4
3
π#Radius
3
&,Median[ExampleData[{"Dataset","Planets"}]]
In[]:=
0
4.946×10
11
km
3
1
1.0813945×10
12
km
3
2
1.626×10
11
km
3
63
1.3864×10
15
km
3
61
7.887×10
14
km
3
27
6.756×10
13
km
3
13
6.200×10
13
km
3
Out[]=
Compute the fraction surviving (
True
) and dying on the Titanic as a function of their cabin class and sex and do so in parallel using a composition of a
ResourceFunction
and a built-in function:
[◼]
MapReduceOperator
[({#class,#sex}&)(#survived&),ResourceFunction["Proportions"]/*N,"Parallelize"True][Normal@ExampleData[{"Dataset","Titanic"}]]
In[]:=
{1st,female}True0.965278,False0.0347222,{1st,male}True0.340782,False0.659218,{2nd,male}False0.853801,True0.146199,{2nd,female}True0.886792,False0.113208,{3rd,male}False0.84787,True0.15213,{3rd,female}True0.490741,False0.509259
Out[]=
Compute the mean age of passengers on the Titanic broken down by class and deleting data for which age is
Missing
:
[◼]
MapReduceOperator
[(#class&)(#age&),(Select[FreeQ[#,_Missing]&]/*Mean)][ExampleData[{"Dataset","Titanic"}]]
In[]:=
1st
39.1655
2nd
29.5172
3rd
24.8483
Out[]=

Possible Issues