The Automatic Identification System (AIS) can provide the time stamp, latitude and longitude, heading angle, speed, and other data information of ships. In light of multi-dimensional ship trajectories and the demand for accuracy and timeliness in trajectory prediction, this study proposes a calculation method for trajectory similarity based on image detection and matching. To be specific, the trajectory data of all fishing boats are first visualized, and then the Oriented FAST and Rotated BRIEF (ORB) algorithm and Brute-Force (BF) matching are used to calculate the similarity between trajectory pictures for classifying fishing boat trajectories. The experimental results show that the proposed method has high accuracy and can be easily implemented and is superior to the traditional methods in the processing efficiency and speed of trajectory data.