Echo State Network (ESN) owns simple network structure and is coupled with a time parameter and thus it shows important theoretical and application values in time series forecasting. In this study, we propose to optimize the output weight matrix by Adaptive Backtracking Search optimization Algorithm (ABSA) to overcome overfitting problem caused by linear regression algorithm. ABSA adopts adaptive mutation factor strategy to replace the strategy of randomly given mutation factor in standard BSA to achieve the balance between convergence accuracy and convergence rate. Experimental results show that the ESN optimized by ABSA outperforms the basic ESN without optimization and the ESNs optimized by other EAs.