Real-time Integration Technology of Large-scale Heterogeneous Data Based on Big Data and Artificial Intelligence

Authors

  • Jiawei Zhang Beijing Normal University- Hong Kong Baptist University United International College, Zhuhai 519000, Guangdong, China

Abstract


In today’s era of increasingly sophisticated network communication technology, large-scale heterogeneous data are emerging, and people’s requirements for data integration technology are increasing. To address this challenge, many scholars have begun to combine big data with artificial intelligence to integrate large-scale heterogeneous data in real time. Since the neural network can fully take into account the characteristics of the data, it has strong upgrade ability and fault resistance, and does not need to know the real-time data integration for specific instance training in advance. Hence, it is able to deal with the real-time integration problem of large-scale heterogeneous data. In order to increase the speed and effectiveness of data integration, this study presents a neural network based on big data and artificial intelligence to assess large-scale heterogeneous data real-time
integration technology. According to this study’s experimental findings, the particle swarm-BP neural network has an error of about 0.020 and the BP neural network has an error of about 0.021 in terms of training results for the three techniques. The enhanced particle swarm-BP neural network technique has an inaccuracy of 0.15 to 0.2; hence, with the enhanced particle swarm-BP neural network technique, there is significantly less error. Also, the improved algorithm requires much less training time than the other two algorithms in the range of 114ms-121ms, indicating that the algorithm has better integration efficiency. Therefore, this method is very effective for the real-time integration of large-scale heterogeneous data.

Keywords: heterogeneous data, neural networks, artificial intelligence, big data

Cite As

J. Zhang, "Real-time Integration Technology of Large-scale Heterogeneous Data Based on Big Data and Artificial Intelligence", Engineering Intelligent Systems, vol. 33 no. 2, pp. 179-188, 2025.



Downloads

Published

2025-03-01

Issue

Vol. 33 No. 2 (2025): International Journal of Engineering Intelligent Systems

Section

Article

License

The submission of a paper implies that, if accepted for publication, it will not be published elsewhere in the same form, in any language, without the prior consent of the publisher. Before publication, authors are requested to assign copyright to CRL Publishing Ltd. This allows CRL to sanction photocopying, and to authorize the reprinting of issues or volumes according to demand. Authors' traditional rights will not be jeopardized by assigning Copyright in this way, as they retain the right to reuse the material following publication, and to veto third-party publication.