Learning objectives
By the end of the day
- Generate and optimise Spark, Python and SQL scripts
- Speed up the building of reliable pipelines
- Keep control of the quality and maintainability of generated code
Programme
Course outline
- Frame a pipeline and generate the code skeleton
- Assisted ETL / ELT transformations (Spark, Python, SQL)
- Optimisation and refactoring of generated code
- Best practices for review and security
- Workshop on participants' real pipeline
Audience and prerequisites
Who it is for
Audience: Data Engineers
Prerequisites: Familiarity with Python / SQL and an orchestrator. Regular use of a GenAI tool recommended.
Practical details
Format and funding
- On-site or remote, open sessions or in-house.
- 100% hands-on, on real cases brought by participants.
- Assessment of learning and certificate of completion.
- Qualiopi-certified training body: course eligible for OPCO funding.
This module is part of the Visian AI Training programme : 6 roles, 21 modules, taken individually or as a track.