Game description: the aim is to beat the dealer's hand by either getting a better hand, without exceeding 21, or by the dealer's hand going bust.
Search and evaluation. traditional tree-search algorithms such as -search and a * search have been successfully applied in many task domains, for instance board games such as checkers and chess e.g., [1, 2, 3] ; . starting from the current state of such a game, a tree is developed that reflects the various possible moves. if we were able to expand this tree up to the level of final states, an optimal strategy could be found for any state using minimax backpropagation. however, almost all games require a game tree that cannot be completely constructed. therefore, search techniques rely heavily on the existence of an evaluation function: instead of expanding the game tree until the end of the game is reached, we expand it just a few e.g., 5 to 10 ; moves ahead and then estimate the value of the non-final ; states obtained using an evaluation function which is usually heuristic [4, 5]. research in the combinatorial games domain has shown that obtaining a sufficiently accurate evaluation function can be difficult if not impossible ; and or requires a substantial amount of parameter tuning [6]. monte-carlo techniques. to address the problems outlined above, monte-carlo sampling is used in many combinatorial games as an evaluation function [7, 8, 9, 10]. the idea of monte-carlo sampling is basically to perform a large number of randomized ; simulations, and then analyse the results of these simulations statistically in order to approximate some outcome which cannot be computed exactly. monte-carlo techniques were first applied to compute integrals and subsequently became very popular in physics and computer science e.g., reinforcement learning [11] ; . in combinatiorial games, monte-carlo sampling can be applied without including any domain knowledge, except for the rules of the game at hand. it evaluates a state s by playing 1.