# Administrivia - A01 due tomorrow at 1700 (12 at least partial submissions so far); `readme.txt` is important! - Exam 1 scheduled for Thursday, 25 September, 1900, HAS 134. # Today - Hill-climbing - Simulated annealing - Local beam search - Genetic algorithms - Example — Expected-value navigation # Milgram Joe Smith Stockbroker Boston, MA - letter experiment - 296 letters - 22% reached target - median chain length = 6 - uninformed? - other greatest hits include obedience, lost letters, looking up, etc. - Milgram's subjects were using a local, heuristic search # problem space generally, to locate a target node in a large network whose topology is known only locally, it's not uninformed search or (global) heuristic search Widely applicable, including: - WWW browsing - p2p file sharing - ad-hoc wireless - multi-agent systems (but, for most of these, better solutions exist if you can see the whole system) # recall How do we evaluation search strategies? # O(b^d) O(b^d) is a big deal for all but very small b and d - b = 10, 10k nodes/sec; 1kB/node - d = 10 ; time = 3 hours; space = 10 TB - d = 12 ; time = 13 days; space= 1 PB - d = 14 ; time = 3.5 years; space = 100 PB how can we ease up on completeness and optimality in the interest of improving time and space complexity? # back to robot world see ppt: track only current location (and possibly goal), but ignore all else # AI Jeopardy - this type of search algorithm only keeps track of a single current state - local search - these states have higher values than all their successors - local maxima - these successors have the same values as the current state - sideways moves # when/how can we use local search? many search problems only require finding a goal state, not the path to that goal state - Actual physical (vs. virtual) navigation - Configuration problems — e.g., determining good compiler parameter settings - Design problems — e.g., VLSI layout or oil pipeline network design "optimization problems" though that usually refers to *numerical* optimization # example To the board, convex polygonal obstacles, "hill climbing" - Have we seen a search equivalent to this before? Also: TSP, any C.O.P. # exercise Knapsack: pack items of different sizes and values into a fixed-size container. attempting to maximize *value* of things packed What's a reasonable heuristics for knapsack via hill climbing? Space for 15 lbs of stuff. Five boxes: - 12 lbs / $4 - 4 lbs / $10 - 2 lbs / $1 - 2 lbs / $2 - 1 lbs / $1 Packing the bag exactly full (12 + 2 + 1) has a value of $4 + $2 + $1 = $7. Q1a. Define a heuristic of your choice, and use hill climbing. What's the final value of the bag? Q1b. What's the optimal solution and value? Q2a. Assume unlimited numbers of each box are available. Using hill climbing, what's the final value of the bag? Q2b. What's the optimal solution and value? # why do we ever keep track of prior states, then? - efficient backtracking (e.g., concave obstacles) - loop avoidance - perhaps the problem requires it (R.T.) # why do we ever use local search, then? low memory requirements (usually constant) effective (an often find reasonable solutions in extremely large state spaces) # challenges for hill climbing local maxima: once a local maximum is reached, there is no way to backtrack or move out of that maximum ridges: can produce a series of local maxima plateaux: hillclimbing can have difficult time finding its way off of a flat portion of the state space landscape # some landscapes see ppts > ...many real problems have a landscape that looks more like a widely scattered family of balding porcupines on a flat floor, with miniature porcupines living on the tip of each porcupine needle, ad infinitum. (Russell and Norvig) # variants of local hill-climbing stochastic hill-climbing - select randomly from all moves that improve the value of the evaluation function random-restart hill-climbing - conducts a series of hill-climbing searches, starting from random positions - keep best result - very frequently used general method in AI # simulated annealing similar to hill-climbing, however, picks from among valid moves at random - always accepts if it is better than current state - may accept (with p<1), if it is not better - probability p decreases as search progresses board: example of ball rolling down hill on a bumpy landscape # local beam search similar to hill-climbing, but keeps track of k current states rather than just a single current state Analogy: - hill-climbing — "...trying to find the top of Mt. Everest in a thick fog while suffering amnesia." - local beam search — "Doing this with several friends, each of whom has a short-range radio and an altimeter." variant: stochastic beam search — select successors at random weighted by value # genetic algorithms a variant of stochastic beam search where new states are generated by combining existing states basic components: - population — a set of states, initially generated randomly - fitness function — an evaluation function - reproduction — a method for generating new states from pairs of old ones (e.g., crossover) - mutation — a method for randomly modifying states to create variation in the population # Stuff we didn't get to :( ## n-Queens example population represented by queen position fitness: # non-attacking pairs create new population by: mating: chosen proportional to fitness (elitism?) crossover: chosen at random mutation: small at random repeat until good-enough solution found ## Recall Milgram Joe Smith Stockbroker Boston, MA ## (one possible) generalization: expected value navigation problem: locate a target node in a large graph whose topology is known only locally at each node, minimize expected path length to target: (see ppt) ## close enough for gov't work hard to compute: need > (1) a list of nodes that have already seen the message, (2) the target node and its at- tributes, (3) immediate neighbors, their attributes, and degree, (4) known (or estimated) homophily structure of the graph, in other words, a statistical relationship between node similarity and link formation but we can estimate using only (1) and (2), since we need only relative values ## searching in continuous spaces previous robot example had a little bit of a lie: states have to be discrete what if they're not? - discretize - or... ## gradient descent - follow the derivative (if differentiable: this is newton's method!) - or compute it empirically for a small delta (this is the same as discretizing) - beyond the scope of this course, take an ML or numerical optimization course to learn more ## constrained optimization - linear programming (constraints are linear inequalities; form a convex set; objective function is linear) - as above, BTSOTC