The SEAT algorithm aims at classifying different fish species using relative frequency response acquired by downward looking echosounders operating at multiple frequencies. The performance of the system has been being evaluated on three Dutch freezer trawlers: Fishing Vessel (FV) SCH302 Willem van der Zwan, FV SCH6 Alida and FV SCH24 Afrika. One of these vessels (FV Alida SCH6) operates a mixture of SIMRAD EK60 & EK80 software while others operate only Simrad EK80. As reported by these vessels, the classification accuracy of the SEAT software has been reduced particularly at the later stages of the project. To investigate this problem these vessels collected acoustic data in close range of each other while targeting herring in the summer of 2017 at east of Shetland isles. Using this dataset together with calibration data, a statistical comparison was conducted. Furthermore, potential discrepancies between Simrad EK60 and EK80 systems were examined using data collected during herring assessment survey HERAS of Fishing Research Vessel (FRV) TRIDENS in July 2017. This dataset included recordings of both systems in alternating mode enabling a ping to ping comparison. It was found that two serious software bugs were likely to have influenced the calibration procedure of the EK80 software. One of these impacted the comparison of HERAS FRV Tridens records and lead to Sv gain offsets of 1.76 dB. After the correction, the measured acoustic intensities were comparable between EK60 and EK80 implying that the interchangeable application of these tools on board SCH6 should not affect species classification and measurements should be similar between vessels either using EK60 or EK80 given the instruments are calibrated correctly. The calculated relative frequency responses from the acoustic recordings of these three fishing vessels showed that FV Willem van der Zwan SCH302 and FV Alida SCH6 were found reasonably coherent, but FV Afrika SCH24 was different. These differences are associated with lower mean backscatter values of the 38 kHz channel.Similar analysis conducted in the earlier phases of this project where frequency response calculated from data collected by FV Alida SCH6 to investigate the discrepancies in the received horse mackerel frequency response and its expected fingerprints (Fassler, 2016 Annex 1). His results showed that the contribution of the 120 kHz data on the classification of varied with location and increased above latitude 52. In addition, this contribution was much lower for shoals detected in the English Channel (Fassler, 2016). Fassler (2016) also suggested that water pressure may affect the morphology of swimbladdered species and may explain the variability between shoals detected on the Atlantic Ocean and in the English Channel. The depth related effects found in different cases suggests that water depth has to be accounted for as an additional variable for each location. As suggested by Fassler (2016), these results may gain significance when the number of datasets increases.The results of the investigations presented here show that further post-processing of calibration records may improve the data quality hence the classification outputs. Particularly the unexpected reduction in the classification performance after 2016 can be improved by rolling back all the SEAT settings to an original state followed by proper calibrations settings. It is also recommended to maintain the latest software versions to ensure equipment are operating efficiently and consistent across the vessels. Regular tests with vessels fishing in close range as in the case of the summer of 2017 is a useful approach to test species recognition and to compare overall performance of the classification algorithm.