Given a European derivative security with an arbitrary payoff function and a corresponding set of underlying securities on which the derivative security is based, we solve the dynamic replication problem: find a self-financing dynamic portfolio strategy involving only the underlying securities that most closely approximates the payoff function at maturity. By applying stochastic dynamic programming to the minimization of a mean-squared-error loss function under Markov state-dynamics, we derive recursive expressions for the optimal-replication strategy that are readily implemented in practice. The approximation error or of the optimal-replication strategy is also given recursively and may be used to quantify the degree of market incompleteness. To investigate the practical significance of these -arbitrage strategies examples including path-dependent options and options on assets with stochastic volatility and jumps.