The research of heterogeneous data integration based on middleware is a hot research in this area. Considering the querying efficiency and the ease of use, classified the heterogeneous data source three types based on the data's format and saved method, using xml schema to rebuild each data source, construct a logic database. Take the XQuery processor as a part of middleware to process the xml data specially, then designed uniform interface to operate the data source refer to SQL and JDBC for the user. Based on this idea, designed and implemented a heterogeneous data integration middleware, the experiment showed that the middleware can deal with searching and updating data, with simple using and good expansibility.