Abstract

As an economically important species in aquaculture, quality classification and meat yield assessment of oysters are crucial for industrial efficiency. Traditional manual assessment methods are inefficient and subjective. While computer vision-based approaches have been explored for oyster weight estimation, they primarily rely on manually measured morphological parameters and often overlook valuable visual appearance features inherent in the raw images. Furthermore, weight alone is an insufficient indicator of meat content, as large shells may contain little meat. To address these limitations, this study pioneers a multimodal oyster meat yield prediction model that synergistically combines shape and appearance features for quality grading. Specifically, a segmentation network extracts shape parameters and appearance image data, constructing a multimodal dataset. A dual-branch feature extraction architecture is designed: the appearance branch utilizes self-attention mechanisms to capture pixel-level interactions, while the shape branch employs variational autoencoders (VAE) to map features into robust latent representations. These modality-specific features are concatenated and processed through a Multilayer Perceptron (MLP) to directly predict meat yield. Experimental results demonstrate that the proposed multimodal fusion approach, which comprehensively leverages both morphological and visual characteristics, establishes significantly more robust and accurate mapping relationships than unimodal models relying solely on shape or appearance. The model effectively captures complementary information and adaptively modulates cross-modal influences, thereby enhancing prediction accuracy (R²=0.9567). The key advantages of the proposed method lie in its ability to overcome the limitations of manual feature measurement and unimodal analysis by automatically extracting and fusing richer information and achieve superior prediction performance crucial for practical quality grading applications in oyster aquaculture.

References

• Asha, K. K., Anandan, R., Mathew, S., & Lakshmanan, P. T. (2014). Biochemical profile of oyster Crassostrea madrasensis and its nutritional attributes. The Egyptian Journal of Aquatic Research, 40(1), 35-41.
• Yearbook F. Fishery and aquaculture statistics 2016 [J]. FAO: Rome, Italy, 2019.
• Botta, R., Asche, F., Borsum, J. S., & Camp, E. V. (2020). A review of global oyster aquaculture production and consumption. Marine Policy, 117, 103952.
• Mizuta, D. D., & Wikfors, G. H. (2019). Seeking the perfect oyster shell: a brief review of current knowledge. Reviews in Aquaculture, 11(3), 586-602.
• Botta, R., Asche, F., Borsum, J. S., & Camp, E. V. (2020). A review of global oyster aquaculture production and consumption. Marine Policy, 117, 103952.
• Nayar, K. N., Mahadevan, S., & Muthiah, P. (1987). Economics of oyster culture. CMFRI Bulletin-Oyster culture-status and prospects, 38, 67-70.
• Van In, V., & O'Connor, W. (2024). Blue Economy: Valuing the Carbon Sequestration Potential in Oyster Aquaculture.
• Lapico A, Sankupellay M, Cianciullo L, et al. Using image processing to automatically measure pearl oyster size for selective breeding [C]. In 2019 Digital Image Computing: Techniques and Applications (DICTA), 2019: 1–8.
• VG N, Hareesh K. Quality inspection and grading of agricultural and food products by computer vision-a review [J]. International journal of computer applications, 2010, 975: 8887.
• Narendra, V.G., Hareesh, K.S., 2010. Quality inspection and grading of agricultural and food products by computer vision-a review. Int. J. Comput. Appl. 2 (1), 43–65.
• Ye, X., Liu, Y., Zhang, D., Hu, X., He, Z., Chen, Y., 2023. Rapid and accurate crayfish sorting by size and maturity based on improved YOLOv5. Appl. Sci. 13 (15), 8619
• Yuan, B., Cui, Y., Liu, W., Sheng, W., Xu, H., & Yang, L. (2023). Consumer preferences for oyster trait attributes in China: A choice experiment. Aquaculture, 571, 739471.
• Parr, M. B., Byler, R. K., Diehl, K. C., & Hackley, C. R. (1995). Machine vision based oyster meat grading and sorting machine. Journal of Aquatic Food Product Technology, 3(4), 5-24.
• Lee, D. J., Xu, X., Lane, R. M., & Zhan, P. (2004, December). Shape analysis for an automatic oyster grading system. In Two-and Three-Dimensional Vision Systems for Inspection, Control, and Metrology II (Vol. 5606, pp. 27-36). SPIE.
• Zhang L, Du X, Guo S. Sse-ppl: a machine vision technology to detect aquatic product quality [C].In International Conference on Physics, Photonics, and Optical Engineering (ICPPOE 2024), 2025:665–671.
• Xiao L, Yang X, Lan X, et al. Towards Visual Grounding: A Survey [J]. arXiv preprint arXiv:2412.20206, 2024.
• Gümüş, B., Balaban, M. Ö., & ÜNLÜSAYIN, M. (2011). Machine vision applications to aquatic foods: a review. Turkish Journal of Fisheries and Aquatic Sciences, 11(1).
• Antony, M. A., & Kumar, R. S. (2021, March). A comparative study on predicting food quality using machine learning techniques. In 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS) (Vol. 1, pp. 1771-1776). IEEE.
• Sun D-W. Inspecting pizza topping percentage and distribution by a computer vision method [J].Journal of food engineering, 2000, 44 (4): 245–249.
• Kuswantori A, Suesut T, Tangsrirat W, et al. Fish detection and classification for automatic sorting system with an optimized YOLO algorithm [J]. Applied Sciences, 2023, 13 (6): 3812.
• Li D, Wang Q, Li X, et al. Recent advances of machine vision technology in fish classification [J]. ICES Journal of Marine Science, 2022, 79 (2): 263–284.
• Yu X, Wang Y, Liu J, et al. Non-contact weight estimation system for fish based on instance seg mentation [J]. Expert systems with applications, 2022, 210: 118403.
• Zhang, L., Wang, J., & Duan, Q. (2020). Estimation for fish mass using image analysis and neural network. Computers and Electronics in Agriculture, 173, 105439.
• Zhang, T., Yang, Y., Liu, Y., Liu, C., Zhao, R., Li, D., & Shi, C. (2024). Fully automatic system for fish biomass estimation based on deep neural network. Ecological Informatics, 79, 102399.
• Peng, B. (2019). Application of marine remote sensing technology in the development of fishery economy. Journal of Coastal Research, 94(SI), 783-787.
• Lim L-S L-S, Liew K-S, Yap T-K, et al. Length-weight relationship and relative condition factor of pearl oyster, Pinctada fucata martensii, cultured in the Tieshangang Bay of the Beibu Gulf, Guangxi Province, China [J]. Borneo Journal of Marine Science and Aquaculture (BJoMSA), 2020, 4 (1):24–27.
• Singh Y T. Relationships between environmental factors and biological parameters of Asian wedge clam, Donax scortum, morphometric analysis, length-weight relationship and condition index: a first report in Asia [J]. Journal of the Marine Biological Association of the United Kingdom, 2017,97 (8): 1617–1633.
• Yoshizumi, T., Gondolesi, G. E., Bodian, C. A., Jeon, H., Schwartz, M. E., Fishbein, T. M., ... & Emre, S. (2003, June). A simple new formula to assess liver weight. In Transplantation proceedings (Vol. 35, No. 4, pp. 1415-1420). Elsevier.
• Ji, X., Dahlgren, R. A., & Zhang, M. (2016). Comparison of seven water quality assessment methods for the characterization and management of highly impaired river systems. Environmental monitoring and assessment, 188, 1-16.
• Liu, X., Du, K., Zhang, C., Luo, Y., Sha, Z., & Wang, C. (2023). Precision feeding system for largemouth bass (Micropterus salmoides) based on multi-factor comprehensive control. Biosystems Engineering, 227, 195-216.
• Harvey, B. C., & Railsback, S. F. (2007). Estimating multi-factor cumulative watershed effects on fish populations with an individual-based model. Fisheries, 32(6), 292-298.
• Dame R F. Comparison of various allometric relationships in intertidal and subtidal American oys ters [J]. Fishery Bulletin, 1972, 70 (4): 1121–1126.
• Pineda-Metz S E, Merk V, Pogoda B. A machine learning model and biometric transformations to facilitate European oyster monitoring [J]. Aquatic Conservation: Marine and Freshwater Ecosys tems, 2023, 33 (7): 708–720.
• Gimin R, Mohan R, Thinh L, et al. The relationship of shell dimensions and shell volume to live weight and soft tissue weight in the mangrove clam, Polymesoda erosa (Solander, 1786) from northern Australia [J]. NAGA, WorldFish Center Quarterly, 2004, 27 (3): 32–35.
• Vu S V, Knibb W, Nguyen N T, et al. First breeding program of the Portuguese oyster Crassostrea angulata demonstrated significant selection response in traits of economic importance [J]. Aquaculture, 2020, 518: 734664
• Vu S V, Knibb W, Gondro C, et al. Genomic prediction for whole weight, body shape, meat yield, and color traits in the Portuguese oyster Crassostrea angulata [J]. Frontiers in Genetics, 2021, 12:661276.
• Zhao, F., Hao, J., Zhang, H., Yu, X., Yan, Z., & Wu, F. (2024). Quality recognition method of oyster based on U-net and random forest. Journal of Food Composition and Analysis, 125, 105746.
• Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale [J]. arXiv preprint arXiv:2010.11929, 2020.