An Efficient Management Technique for Optimization of Compute, Storage, and Communication in Big Data Processing
| International Journal of Electronics and Communication Engineering |
| © 2025 by SSRG - IJECE Journal |
| Volume 12 Issue 1 |
| Year of Publication : 2025 |
| Authors : Vijay Kumar Vasantham, D. Haritha |
10.14445/23488549/IJECE-V12I1P105
How to Cite?
Vijay Kumar Vasantham, D. Haritha, "An Efficient Management Technique for Optimization of Compute, Storage, and Communication in Big Data Processing," SSRG International Journal of Electronics and Communication Engineering, vol. 12, no. 1, pp. 72-82, 2025. Crossref, https://doi.org/10.14445/23488549/IJECE-V12I1P105
Abstract:
There are specific challenges like handling a variety of datasets, velocity, and largest, high-dimensional datasets to be handled efficiently in big data processing and optimization. In this regard, three crucial aspects were considered: computing, storage, and communication in big data optimization. To achieve accurate and timely data assessment and analysis, these three aspects are focused on the extraction of insights for the rest to be assured for future endeavours. The strategies used in the computing aspect enable the provision of fast data access, effective allocation of resources, and parallel processing. The approaches considered are distributed frameworks, in-memory systems, lazy evaluations, partitioning, and selective algorithms. The second aspect is the computation storage, which requires hardware, software tools, Orchestration tools, Analytics tools, and data management tools for faster access, efficient data movements, enhanced scalability, and effective security. The third aspect of big data optimization is communication, in which the approaches that were considered related to infrastructure, serialization, protocols, and notification/alerting systems to experience less overhead, proactive issue detection, and effective data exchange. This integrated aspect leads to efficient, informed decision-making based on insights and significant analysis. The customized framework for computing, storage, and communication services in big data optimization ensures better accuracy, scalability, processing efficiency, and cost-effectiveness against existing approaches.
Keywords:
Big Data Optimization, Communication Services, Computation storage, Communication Management, Efficiency and Accuracy, Resource Allocation, Workload Distribution.
References:
[1] Chandrima Roy, Siddharth Swarup Rautaray, and Manjusha Pandey, “Big Data Optimization Techniques: A Survey,” International Journal of Information Engineering and Electronic Business, vol. 10, no. 4, pp. 41-48, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Mengxuan Wu, Jingjing Jiang, and Lijuan Wang, “Research on the Optimization Algorithm of Big Data Computing System,” 2021 International Wireless Communications and Mobile Computing (IWCMC), Harbin City, China, pp. 1783-1787, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Hira Zahid et al., “Big Data Analytics in Telecommunications: Literature Review and Architecture Recommendations,” IEEE/CAA Journal of Automatica Sinica, vol. 7, no. 1, pp. 18-38, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Agnieszka Smalec, “Big Data as a Tool Helpful in Communication Management,” Procedia Computer Science, vol. 192, pp. 5156-5165, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Arezou Naghib et al., “A Comprehensive and Systematic Literature Review on the Big Data Management Techniques in the Internet of Things,” Wireless Networks, vol. 29, pp. 1085-1144, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Madhavi Vaidya, Shrinivas Deshpande, and Vilas Thakare, “Design and Analysis of Large Data Processing Techniques,” International Journal of Computer Applications, vol. 100, no. 8, pp. 24-28, 2014.
[Google Scholar] [Publisher Link]
[7] Sandeep Dasari, and Rajesh Kaluri, “Big Data Analytics, Processing Models, Taxonomy of Tools, V’s, and Challenges: State-of-Art Review and Future Implications,” Wireless Communications and Mobile Computing, vol. 2023, pp. 1-14, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Rahat Iqbal et al., “Big Data analytics and Computational Intelligence for Cyber–Physical Systems: Recent Trends and State of the Art Applications,” Future Generation Computer Systems, vol. 105, pp. 766-778, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Ahmed Hadi Ali AL-Jumaili et al., “Big Data Analytics Using Cloud Computing Based Frameworks for Power Management Systems: Status, Constraints, and Future Recommendations,” Sensors, vol. 23, no. 6, pp. 1-37, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Qingqing Chang, Shah Nazir, and Xia Li, “Decision-Making and Computational Modeling of Big Data for Sustaining Influential Usage,” Scientific Programming, vol. 2022, pp. 1-15, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Shubham Upadhyay et al., “Analytics and Storage of Big Data,” Proceedings of the International Semantic Intelligence Conference (ISIC 2021), New Delhi, India, pp. 202-210, 2021.
[Google Scholar] [Publisher Link]
[12] Cheng Luo, “Computer Data Storage and Management Platform Based on Big Data,” Journal of Physics: Conference Series, vol. 2066, pp. 1-6, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Blend Berisha, Endrit Meziu, and Isak Shabani, “Big Data Analytics in Cloud Computing: An Overview,” Journal of Cloud Computing, vol. 11, pp. 1-10, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Amanpreet Kaur Sandhu, “Big Data with Cloud Computing: Discussions and Challenges,” Big Data Mining and Analytics, vol. 5, no. 1, pp. 32-40, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Mohan Naik Ramachandra et al., “An Efficient and Secure Big Data Storage in Cloud Environment by Using Triple Data Encryption Standard,” Big Data and Cognitive Computing, vol. 6, no. 4, pp. 1-20, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Neelay Jagani, and Parthil Jagani, “Big Data in Cloud Computing: A Literature Review,” International Journal of Engineering Applied Sciences and Technology, vol. 5, no. 11, pp. 185-191, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Christos Stergiou, and Kostas E. Psannis, “Algorithms for Big Data in Advanced Communication Systems and Cloud Computing,” 2017 IEEE 19th Conference on Business Informatics (CBI), Thessaloniki, Greece, pp. 196-201, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Akansha Gautam, and Indranath Chatterjee, “Big Data and Cloud Computing: A Critical Review,” International Journal of Operations Research and Information Systems, vol. 11, no. 3, pp. 19-38, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Lingqi Xue, “Financial Big Data Based on Internet of Things and Wireless Network Communication,” Wireless Communications and Mobile Computing, vol. 2021, pp. 1-12, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Mahzad Mahdavisharif, Shahram Jamali, and Reza Fotohi, “Big Data-Aware Intrusion Detection System in Communication Networks: A Deep Learning Approach,” Journal of Grid Computing, vol. 19, 2021.
[CrossRef] [Google Scholar] [Publisher Link]