Perform Batch Simulation and Comparison of Schedulers
An important purpose of the Vehicle Routing Scheduler Library is to allow user to analyze and compare different schedulers on the problem that researchers and developers are trying to tackle. For this purpose, the library provides batch simulation and comparison tools which allow user to run specific number of simulations for selected set of schedulers on selected set of problem instances.
- Go to Performance View tab.
- Select Algorithm Performance Comparison accordion panel as shown in Figure.
- Click Run Comparison to launch simulation run configuration dialog as shown in Figure.
- Select schedulers in Schedulers tab, i.e. SIM, SGMA, CAMA and SAM solution methods.
- Select problem instances in Benchmarks tab, i.e. E-n101-k8, A-n54-k7, B-n41-k6 and B-n68-k9 problem instances as shown in Figure.
- Set the number of simulation runs in Simulation Configuration tab to run each selected scheduler on each selected problem instance.
- Click OK button to start simulation run process as shown in Figure.
- Information about batch simulation runs progress is provided as shown in Figure.
Simulation Runs: total number of simulation run (30)
Heuristic Count: selected number of schedulers (4)
Problem Instance Count: selected number of problem instances (4)
Running Simulation Index: current simulation run (#0)
Running Heuristic: current running scheduler (SIM)
Running Problem: current problem instance (B-n68-k9)
Start Time: batch simulation start time (16:04:46)
Time Span: elapsed time (00:00:05)
Running State: iteration run of the running scheduler (SIM, Adaptive MA, 26)
- Results will be shown after batch simulation run has finished as shown in Figure.
Avg Value: average solution value (average total distance travelled in CVRP) across multiple simulation run
Avg Conv Iter: average value for iteration
Avg Conv Time: average value for time in seconds
Best Value: best solution value among simulation runs
Best Conv Iter: iteration at which best value is obtained
Best Conv Time: time in seconds at which best value is obtained
- Repeat step 3 to 9 to perform batch simulation and comparison between 15 selected schedulers on selected problem instance (E-n76-k8) for 30 simulation runs. From results shown in Figure, meta-heuristics algorithms usually perform better than simple heuristics algorithms.
It is important to take note that before batch simulation performance, the tuning of parameters in selected scheduler has to be done. For instance, schedulers such as SAM and CAMA run in maximum 100 generations by default, which is too short for serious algorithm comparison. User can refer to Different Solution Methods section to tune parameters of scheduler in Algorithm View tab before batch simulation.