SNAPLOGIC AUTOMATION AS A CATALYST FOR ETL AND MACHINE LEARNING WORKFLOWS
DOI:
https://doi.org/10.20544/uklop.2026.611Keywords:
Automation, ETL, SnapLogic, Machine Learning, Workflows, Enterprise, AIAbstract
Introduction: This study explores the use of SnapLogic automation to modernize Extract, Transform, and Load (ETL) processes and support scalable machine learning (ML) workflows. Traditional ETL pipelines are slowed by manual intervention, inefficiency, and limited scalability.
Methods: Our methodology followed an applied research design, focusing on the practical implementation of SnapLogic automation for ETL and machine learning workflows. Automated pipelines were built to handle data preparation, model deployment, and performance monitoring, and were tested with both synthetic and real-world datasets.
Results: SnapLogic automation significantly improved efficiency, reduced development time, minimized manual errors, and enabled real-time processing of large, dynamic workloads. Continuous monitoring and automated lifecycle management enhanced system reliability and resilience.
Discussion: These findings demonstrate that SnapLogic directly addresses the limitations of traditional ETL by streamlining workflows and bridging the gap between data engineering and ML. While the platform may involve a higher upfront investment, it optimizes long-term costs for large organizations.
Downloads
References
1. Gour, Vishal, S. S. Sarangdevot, Govind Singh Tanwar, and Anand Sharma. "Improve performance of extract, transform and load (ETL) in data warehouse." Int. Journal on Comp. Sci. and Eng 2, no. 3 (2010): 786-789.
2. Chalmers, Neil, Stephen Gough, and William Scott. Sustainable development and learning: Framing the issues. Routledge, 2003.
3. Bomma, Hari Prasad. "Revolutionizing ETL with AI Powered Automation." IJLRP-International Journal of Leading Research Publication 5, no. 2 (2024).
4. Mishra, Sarbaree. "Automating the data integration and ETL pipelines through machine learning to handle massive datasets in the enterprise." International Journal of Emerging Research in Engineering and Technology 1, no. 2 (2020): 69-78.
https://doi.org/10.63282/3050-922X.IJERET-V1I2P109
5. Ravi, Chandran. "ETL (Extract, Transform & Load) Automation." International Journal of Emerging Trends in Computer Science and Information Technology 6, no. 1 (2025): 52-55.
https://doi.org/10.63282/3050-9246.IJETCSIT-V6I1P106
6. Kumar, G. Sunil Santhosh, and M. Rudra Kumar. "Dimensions of automated etl management: A contemporary literature review." In 2022 International Conference on Automation, Computing and Renewable Systems (ICACRS), pp. 1292-1297. IEEE, 2022.
https://doi.org/10.1109/ICACRS55517.2022.10029274
7. Trajkovska, Aneta, Tome Dimovski, Ramona Markoska, and Zoran Kotevski. "Automation and Monitoring on Integration ETL Processes while Distributing Data." (2023): 212-219.
8. Zheng, Mei, Jianhua Ren, Linlin Sun, and Hui Ding. "The Process of Integration." In 2010 International Conference on E-Product E-Service and E-Entertainment, pp. 1-5. IEEE, 2010.
https://doi.org/10.1109/ICEEE.2010.5660097
9. Maréchal, François, and Boris Kalitventzeff. "Process integration: Selection of the optimal utility system." Computers & Chemical Engineering 22 (1998): S149-S156.
https://doi.org/10.1016/S0098-1354(98)00049-0
10. Hussain, Sadath, and Mohan Aradhya. "Big Data Integration & Transformation: A Comparative Analysis of SnapLogic and AWS Glue." In 2024 8th International Conference on Computational System and Information Technology for Sustainable Solutions (CSITSS), pp. 1-6. IEEE, 2024.
https://doi.org/10.1109/CSITSS64042.2024.10816957
11. Khan, Bilal, Saifullah Jan, Wahab Khan, and Muhammad Imran Chughtai. "An Overview of ETL Techniques, Tools, Processes and Evaluations in Data Warehousing." Journal on Big Data 6 (2024).
https://doi.org/10.32604/jbd.2023.046223
12. Kommera, Adisheshu Reddy. "Future of enterprise integrations and iPaaS (Integration Platform as a Service) adoption." Neuroquantology 13, no. 1 (2015): 176-186.
13. Clemens-Meyer, Francois HLR, Mathieu Lepot, Frank Blumensaat, Dominik Leutnant, and Guenter Gruber. "Data validation and data quality assessment." Metrology in Urban Drainage and Stormwater Management: Plug and Pray, edited by: Bertrand-Krajewski, J.-L., Clemens-Meyer, F., and Lepot, M., IWA Publishing (2021): 327-390.
https://doi.org/10.2166/9781789060119_0327
14. Dasu, Tamraparni, and Theodore Johnson. Exploratory data mining and data cleaning. John Wiley & Sons, 2003.
https://doi.org/10.1002/0471448354
15. Côté, Pierre-Olivier, Amin Nikanjam, Nafisa Ahmed, Dmytro Humeniuk, and Foutse Khomh. "Data cleaning and machine learning: a systematic literature review." Automated Software Engineering 31, no. 2 (2024): 54.
https://doi.org/10.1007/s10515-024-00453-w
16. Patel, Paraskumar. "Challenges and solutions in cross-platform ML systems integration." Journal of Artificial Intelligence & Cloud Computing 2, no. 4 (2023): 1-5.
