***********************************************************
Puddle World using Ordinary Bellman Equation
***********************************************************

Options used when running:

1: Puddle World
20: Order --- this means we will have 20^2 = 400 weights to tune
1: No value averaging
0: Ordinary Bellman
100: Number of interations of value iteration
16: Number of threads used by value iteration (set to your number of cores)
16: Number of threads used to evaluate a policy (set to your number of cores)
1: Evaluate policy after every iteration
0: Print results to file
out_Ordinary_PuddleWorld.txt: The file to print the results to

Console output:

Select environment:
[0] Mountain Car
[1] Puddle World
[2] Acrobot
[3] Cart Pole
[4] Pendulum Swing-Up and Balance
[5] Bicycle
Selection: 1
Enter the order (state is 2 dimensional): 20
Select value averaging:
[0] Yes
[1] No
Selection: 1
Select maxQ Variant:
[0] Ordinary Bellman
[1] Consistent Bellman
Selection: 0
Enter number of iterations of value iteration: 100
Enter number of threads for value-iteration (we suggest # available cores): 16
Enter number of threads for policy evaluation (we suggest # available cores): 16
Evaluate policy every how-many iterations? 1
Select Print results to file:
[0] Yes
[1] No
Selection: 0
Enter filename: out_Consistent_PuddleWorld.txt

Initially: Undiscounted return = -1067.88
Iteration: 1	Undiscounted return = -1000.35
Iteration: 2	Undiscounted return = -1000.03
Iteration: 3	Undiscounted return = -1000
Iteration: 4	Undiscounted return = -1000
Iteration: 5	Undiscounted return = -999.252
Iteration: 6	Undiscounted return = -985.562
Iteration: 7	Undiscounted return = -985.746
Iteration: 8	Undiscounted return = -946.789
Iteration: 9	Undiscounted return = -934.584
Iteration: 10	Undiscounted return = -799.777
Iteration: 11	Undiscounted return = -886.378
Iteration: 12	Undiscounted return = -567.323
Iteration: 13	Undiscounted return = -349.061
Iteration: 14	Undiscounted return = -58.865
Iteration: 15	Undiscounted return = -73.809
Iteration: 16	Undiscounted return = -45.089
Iteration: 17	Undiscounted return = -37.47
Iteration: 18	Undiscounted return = -37.752
Iteration: 19	Undiscounted return = -37.99
Iteration: 20	Undiscounted return = -37.954
Iteration: 21	Undiscounted return = -36.167
Iteration: 22	Undiscounted return = -37.254
Iteration: 23	Undiscounted return = -35.935
Iteration: 24	Undiscounted return = -37.333
Iteration: 25	Undiscounted return = -35.662
Iteration: 26	Undiscounted return = -35.498
Iteration: 27	Undiscounted return = -35.422
Iteration: 28	Undiscounted return = -35.33
Iteration: 29	Undiscounted return = -35.226
Iteration: 30	Undiscounted return = -35.1522
Iteration: 31	Undiscounted return = -34.9514
Iteration: 32	Undiscounted return = -34.949
Iteration: 33	Undiscounted return = -34.9417
Iteration: 34	Undiscounted return = -34.865
Iteration: 35	Undiscounted return = -34.781
Iteration: 36	Undiscounted return = -34.9415
Iteration: 37	Undiscounted return = -34.8183
Iteration: 38	Undiscounted return = -34.2071
Iteration: 39	Undiscounted return = -35.0182
Iteration: 40	Undiscounted return = -35.0102
Iteration: 41	Undiscounted return = -34.5336
Iteration: 42	Undiscounted return = -34.8094
Iteration: 43	Undiscounted return = -34.7635
Iteration: 44	Undiscounted return = -34.9028
Iteration: 45	Undiscounted return = -34.8142
Iteration: 46	Undiscounted return = -34.8219
Iteration: 47	Undiscounted return = -34.7795
Iteration: 48	Undiscounted return = -34.704
Iteration: 49	Undiscounted return = -34.9942
Iteration: 50	Undiscounted return = -34.8188
Iteration: 51	Undiscounted return = -34.9717
Iteration: 52	Undiscounted return = -34.7568
Iteration: 53	Undiscounted return = -34.7765
Iteration: 54	Undiscounted return = -34.989
Iteration: 55	Undiscounted return = -34.7514
Iteration: 56	Undiscounted return = -34.809
Iteration: 57	Undiscounted return = -34.8284
Iteration: 58	Undiscounted return = -34.8638
Iteration: 59	Undiscounted return = -34.9249
Iteration: 60	Undiscounted return = -34.6902
Iteration: 61	Undiscounted return = -34.6086
Iteration: 62	Undiscounted return = -34.7346
Iteration: 63	Undiscounted return = -34.8381
Iteration: 64	Undiscounted return = -34.7851
Iteration: 65	Undiscounted return = -34.8627
Iteration: 66	Undiscounted return = -34.7168
Iteration: 67	Undiscounted return = -34.7819
Iteration: 68	Undiscounted return = -34.852
Iteration: 69	Undiscounted return = -34.6591
Iteration: 70	Undiscounted return = -34.764
Iteration: 71	Undiscounted return = -34.8415
Iteration: 72	Undiscounted return = -34.9318
Iteration: 73	Undiscounted return = -34.7874
Iteration: 74	Undiscounted return = -34.8195
Iteration: 75	Undiscounted return = -34.6106
Iteration: 76	Undiscounted return = -34.7165
Iteration: 77	Undiscounted return = -35.0429
Iteration: 78	Undiscounted return = -34.819
Iteration: 79	Undiscounted return = -34.9363
Iteration: 80	Undiscounted return = -34.7673
Iteration: 81	Undiscounted return = -34.779
Iteration: 82	Undiscounted return = -34.9043
Iteration: 83	Undiscounted return = -34.9149
Iteration: 84	Undiscounted return = -34.95
Iteration: 85	Undiscounted return = -34.6733
Iteration: 86	Undiscounted return = -34.707
Iteration: 87	Undiscounted return = -34.7828
Iteration: 88	Undiscounted return = -34.792
Iteration: 89	Undiscounted return = -34.7215
Iteration: 90	Undiscounted return = -34.9443
Iteration: 91	Undiscounted return = -34.8304
Iteration: 92	Undiscounted return = -34.5998
Iteration: 93	Undiscounted return = -34.7258
Iteration: 94	Undiscounted return = -34.7088
Iteration: 95	Undiscounted return = -34.7645
Iteration: 96	Undiscounted return = -34.723
Iteration: 97	Undiscounted return = -34.8357
Iteration: 98	Undiscounted return = -34.9194
Iteration: 99	Undiscounted return = -34.8873
Iteration: 100	Undiscounted return = -34.887
Printing results to out_Consistent_PuddleWorld.txt

