***********************************************************
Acrobot using Ordinary Bellman Equation
***********************************************************

Options used when running:

2: Acrobot
30: Order --- this means we will have 30^4 = 810,000 weights to tune
1: No value averaging
0: Ordinary Bellman
100: Number of interations of value iteration
16: Number of threads used by value iteration (set to your number of cores)
16: Number of threads used to evaluate a policy (set to your number of cores)
1: Evaluate policy after every iteration
0: Print results to file
out_Ordinary_Acrobot.txt: The file to print the results to

Console output:

Select environment:
[0] Mountain Car
[1] Puddle World
[2] Acrobot
[3] Cart Pole
[4] Pendulum Swing-Up and Balance
[5] Bicycle
Selection: 2
Enter the order (state is 4 dimensional): 30
Select value averaging:
[0] Yes
[1] No
Selection: 1
Select maxQ Variant:
[0] Ordinary Bellman
[1] Consistent Bellman
Selection: 0
Enter number of iterations of value iteration: 100
Enter number of threads for value-iteration (we suggest # available cores): 16
Enter number of threads for policy evaluation (we suggest # available cores): 16
Evaluate policy every how-many iterations? 1
Select Print results to file:
[0] Yes
[1] No
Selection: 0
Enter filename: out_Ordinary_Acrobot.txt

Initially: Episode duration = 200
Iteration: 1	Episode duration = 200
Iteration: 2	Episode duration = 200
Iteration: 3	Episode duration = 200
Iteration: 4	Episode duration = 200
Iteration: 5	Episode duration = 122
Iteration: 6	Episode duration = 200
Iteration: 7	Episode duration = 200
Iteration: 8	Episode duration = 200
Iteration: 9	Episode duration = 200
Iteration: 10	Episode duration = 200
Iteration: 11	Episode duration = 200
Iteration: 12	Episode duration = 16.8
Iteration: 13	Episode duration = 26.8
Iteration: 14	Episode duration = 16.6
Iteration: 15	Episode duration = 16.6
Iteration: 16	Episode duration = 21.6
Iteration: 17	Episode duration = 21.6
Iteration: 18	Episode duration = 16.4
Iteration: 19	Episode duration = 17.4
Iteration: 20	Episode duration = 18
Iteration: 21	Episode duration = 18.4
Iteration: 22	Episode duration = 18.4
Iteration: 23	Episode duration = 16.2
Iteration: 24	Episode duration = 16.2
Iteration: 25	Episode duration = 18.8
Iteration: 26	Episode duration = 18.8
Iteration: 27	Episode duration = 18.8
Iteration: 28	Episode duration = 18.8
Iteration: 29	Episode duration = 18.8
Iteration: 30	Episode duration = 18.8
Iteration: 31	Episode duration = 18.8
Iteration: 32	Episode duration = 18.8
Iteration: 33	Episode duration = 17.8
Iteration: 34	Episode duration = 20.4
Iteration: 35	Episode duration = 20.4
Iteration: 36	Episode duration = 19.8
Iteration: 37	Episode duration = 19.8
Iteration: 38	Episode duration = 19.8
Iteration: 39	Episode duration = 19.8
Iteration: 40	Episode duration = 19.8
Iteration: 41	Episode duration = 19.8
Iteration: 42	Episode duration = 18.4
Iteration: 43	Episode duration = 18.4
Iteration: 44	Episode duration = 18.4
Iteration: 45	Episode duration = 18.4
Iteration: 46	Episode duration = 19.2
Iteration: 47	Episode duration = 17.8
Iteration: 48	Episode duration = 17.8
Iteration: 49	Episode duration = 17.8
Iteration: 50	Episode duration = 17.8
Iteration: 51	Episode duration = 17.8
Iteration: 52	Episode duration = 17.8
Iteration: 53	Episode duration = 17.8
Iteration: 54	Episode duration = 17.8
Iteration: 55	Episode duration = 17.8
Iteration: 56	Episode duration = 17.8
Iteration: 57	Episode duration = 17.8
Iteration: 58	Episode duration = 17.8
Iteration: 59	Episode duration = 17.8
Iteration: 60	Episode duration = 17.8
Iteration: 61	Episode duration = 17.8
Iteration: 62	Episode duration = 17.8
Iteration: 63	Episode duration = 17.8
Iteration: 64	Episode duration = 17.8
Iteration: 65	Episode duration = 17.8
Iteration: 66	Episode duration = 17.8
Iteration: 67	Episode duration = 17.8
Iteration: 68	Episode duration = 17.8
Iteration: 69	Episode duration = 17.8
Iteration: 70	Episode duration = 17.8
Iteration: 71	Episode duration = 17.8
Iteration: 72	Episode duration = 17.8
Iteration: 73	Episode duration = 17.8
Iteration: 74	Episode duration = 17.8
Iteration: 75	Episode duration = 17.8
Iteration: 76	Episode duration = 17.8
Iteration: 77	Episode duration = 17.8
Iteration: 78	Episode duration = 17.8
Iteration: 79	Episode duration = 17.8
Iteration: 80	Episode duration = 17.8
Iteration: 81	Episode duration = 17.8
Iteration: 82	Episode duration = 17.8
Iteration: 83	Episode duration = 17.8
Iteration: 84	Episode duration = 17.8
Iteration: 85	Episode duration = 17.8
Iteration: 86	Episode duration = 17.8
Iteration: 87	Episode duration = 17.8
Iteration: 88	Episode duration = 17.8
Iteration: 89	Episode duration = 17.8
Iteration: 90	Episode duration = 17.8
Iteration: 91	Episode duration = 17.8
Iteration: 92	Episode duration = 17.8
Iteration: 93	Episode duration = 17.8
Iteration: 94	Episode duration = 17.8
Iteration: 95	Episode duration = 17.8
Iteration: 96	Episode duration = 17.8
Iteration: 97	Episode duration = 17.8
Iteration: 98	Episode duration = 17.8
Iteration: 99	Episode duration = 17.8
Iteration: 100	Episode duration = 17.8
Printing results to out_Ordinary_Acrobot.txt

