Update WrapRec for 2020
Appears that there are some issues with us supporting WrapRec that need to be resolved before it can be used for teaching:
-
Update documentation on Brightspace
from datasets import CS4065_Dataset from wraprec import PyWrapRec config_file_path = CS4065_Dataset.get_wraprec_sample_data()
should be replaced with
from datasets import CS4065_Dataset from wraprec import PyWrapRec cs4065_dataset = CS4065_Dataset() config_file_path = cs4065_dataset.get_wraprec_sample_data()
-
Executing
!mono /home/student/wraprec/WrapRec.exe /home/student/data/cs4065/wraprec_sample_data3/sample.xml
inside the notebook using the defaultsample.xml
file produces the following output:Case 1 of 4: ---------------------------------------- Loading DataContainer 'ml100k'... Loading data 'ml100k-all' into container... Setuping evaluation context 'eval'... Setuping model 'mf'... Error in expriment 'ml100k', model 'mf', split 'ml100k-cv-fold1': Cannot resolve MmlRecommender: Could not load file or assembly 'MyMediaLite.dll' or one of its dependencies at (wrapper managed-to-native) System.Reflection.Assembly:LoadFrom (string,bool) at System.Reflection.Assembly.LoadFrom (System.String assemblyFile) <0x7f51a9926860 + 0x0000e> in <filename unknown>:0 at WrapRec.Utils.Helpers.ResolveType (System.String typeName) <0x4085a920 + 0x00074> in <filename unknown>:0 at WrapRec.Models.MmlRecommender.Setup () <0x4086be30 + 0x0005f> in <filename unknown>:0 at WrapRec.Models.MmlRecommender.Setup () <0x4086be30 + 0x00583> in <filename unknown>:0 at WrapRec.Core.Experiment.Setup () <0x408644f0 + 0x0010b> in <filename unknown>:0 at WrapRec.Core.ExperimentManager.RunSingleExperiment (WrapRec.Core.Experiment e) <0x40863ef0 + 0x000a7> in <filename unknown>:0 Case 2 of 4: ---------------------------------------- <OTHER 3 CASES> Experiments are executed: 0 succeeded, 4 failed. Results are stored in /home/student/data/cs4065/wraprec_sample_data3/results Total running time: 0 seconds.
When using the python wrapper the students are provided with (
PyWrapRec
) the error messages are not shown to the user and the program executes producing the output csv files. It is unclear whether the above error messages can be safely ignored or should be fixed. -
When replacing the second model in
sample.xml
with<model id="bmf" class="WrapRec.Models.MmlRecommender"> <parameters ml-class="MyMediaLite.dll:MyMediaLite.RatingPrediction.ItemKNN" K="3" /> </model>
executing the command via
mono
(as above) completes (with 0 successes as above) and outputs are produced. Executing the command via the python wrapper leads to the notebook hanging. -
When replacing the second model in
sample.xml
with<model id="libfm" class="WrapRec.Models.LibFmWrapper"><parameters libFmPath="libfm.net.exe" task="r" dim="1-1-10" method="sgd" iter="30" learn_rate="0.02"/></model>
and
<experiment id="ml100k" models="mf,bmf" splits="ml100k-cv" evalContext="eval" />
with
<experiment id="ml100k" models="mf,libfm" splits="ml100k-cv" evalContext="eval" />
the following output is produced:
Case 2 of 4: ---------------------------------------- Setuping evaluation context 'eval'... Setuping model 'libfm'... Calculating split 'ml100k-cv-fold1' statistics... Calculating dataContainer 'ml100k' statistics... Experiment Id: ml100k Split Id: ml100k-cv-fold1 Model Id: libfm Model Parameteres: task:r dim:1,1,10 method:sgd iter:30 learn_rate:0.02 Training... Creating LibFm train and test files... Training and testing with LibFm... Running process: libfm.net.exe -task r -dim 1,1,10 -method sgd -iter 30 -learn_rate 0.02 -save_model train.model -train train.libfm -test test.libfm -out test.out Error in expriment 'ml100k', model 'libfm', split 'ml100k-cv-fold1': ApplicationName='libfm.net.exe', CommandLine='-task r -dim 1,1,10 -method sgd -iter 30 -learn_rate 0.02 -save_model train.model -train train.libfm -test test.libfm -out test.out', CurrentDirectory='', Native error= Cannot find the specified file at System.Diagnostics.Process.Start_noshell (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) <0x41be2f10 + 0x00fe3> in <filename unknown>:0 at System.Diagnostics.Process.Start_common (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process) <0x41be2d60 + 0x000db> in <filename unknown>:0 at System.Diagnostics.Process.Start () <0x41be2ce0 + 0x0004b> in <filename unknown>:0 at (wrapper remoting-invoke-with-check) System.Diagnostics.Process:Start () at WrapRec.Models.LibFmWrapper+<>c__DisplayClass11.<Train>b__e () <0x41be2bf0 + 0x0001b> in <filename unknown>:0 at MyMediaLite.Wrap.MeasureTime (System.Action t) <0x41bca510 + 0x00023> in <filename unknown>:0 at WrapRec.Models.LibFmWrapper.Train (WrapRec.Data.Split split) <0x41bdee10 + 0x0082f> in <filename unknown>:0 at WrapRec.Core.Experiment.<Run>b__0 () <0x41bdeda0 + 0x0001e> in <filename unknown>:0 at MyMediaLite.Wrap.MeasureTime (System.Action t) <0x41bca510 + 0x00023> in <filename unknown>:0 at WrapRec.Core.Experiment.Run () <0x41bdebb0 + 0x000c3> in <filename unknown>:0 at WrapRec.Core.ExperimentManager.RunSingleExperiment (WrapRec.Core.Experiment e) <0x41bcaf80 + 0x0018c> in <filename unknown>:0
and the training does not complete.
-
For the cases where the training does complete the
results
pandas DataFrame contains partially wrong values (row 3):