Programme Guide
Overview
The Imperial College and Corndel Data Engineering Programme will develop the
knowledge, skills and behaviours you need to be an effective data engineer in the
modern digital workplace. By completing this programme, you will also achieve the
Level 5 Data Engineer Apprenticeship Standard.
The apprenticeship concerns itself with the application of the knowledge, skills and
behaviours required to be an effective data engineering. The Corndel Diploma
combines these requirements with a focus on understanding, analysis and application
of fundamental data-focused and modern workplace tools, techniques, and theories,
as well as techniques for how to effectively build critical data pipelines. The programme
leads you through a series of units, modules and tasks to efficiently meet all these
requirements.
Programme Structure
The Corndel Diploma is made up of the 10 units listed below. We start by building a
key understanding of data engineering fundamentals such as cloud, data storage,
and developing our skills in SQL and Python. We then gain practical knowledge in
collaboration and requirements to ensure our data products directly fulfil stakeholder
needs. We build practical understanding of data pipelines and automation, followed
by best practice in data security. Once we've mastered these skills, we further
explore software best practice, unstructured data and AI, before working on
delivering data engineering pipelines into production, the programme ends with a
focus on horizon scanning, continuous improvement, analytics, and ensuring our
skills remain relevant in a constantly changing digital landscape.
Units
Unit 01: Discovery - Data engineering in your organisation
Unit 02: Data engineering fundamentals
Unit 03: Collaboration & Requirements gathering
Unit 04: Data pipelines and data quality
Unit 05: Automating data pipelines
Unit 06: Data security and governance
Unit 07: Software development best practices
Unit 08: Unstructured data and AI
Unit 09: Delivering into production
Unit 10: Continuous improvement, innovation and analytics
Each unit is divided into a number of modules. These are:
Unit 1
1.1 The Uses of Cloud Computing Platforms
1.2 Data engineering in your organisation
Unit 2
2.1 Core SQL and Python for Data Engineering
2.2 Data normalisation for data integrity
2.3 Advanced SQL for Data Engineering
2.4 Data store types and optimisation
2.5 The data warehouse: data modelling, data lakes, data marts
Unit 3
3.1 Collaboration: technical and non-technical
3.2 Data product process discovery
3.3 Fulfilling User Requirements
Unit 4
4.1 Ensuring Data Quality
4.2 Advanced Python for data engineering
4.3 Advanced Data Management: Ingest, Manipulate, and Validate (part 1)
4.4 Advanced Data Management: Ingest, Manipulate, and Validate (part 2)
4.5 Landscape of data integration tools, on-prem, cloud, and more
4.6 Comprehensive ETL
Unit 5
5.1 Automation for workload reduction
5.2 Data Engineering Project Milestone
Unit 6
6.1 Data security and governance best practices (part 1)
6.2 Data security and governance best practices (part 2)
6.3 Batch Processing: OLTP data source integration
Unit 7
7.1 Software development best practice
Unit 8
8.1 Working with structured, semi-structured, and unstructured data
8.2 Develop algorithms to extract structured data from unstructured
Unit 9
9.1 Incident Response, Troubleshooting, and Root Cause Analysis
9.2 From Prototype to Production: Implementing Data Solutions
9.3 Streaming tools and frameworks
9.4 Designing Effective Data Product Guides
9.5 SQL Skills Knowledge Boost
9.6 Data Engineering Project Milestone
9.7 Identifying and Implementing Sustainable Solutions
9.8 Spark based tools and techniques
9.9 Monitoring tools and frameworks
Unit 10
10.1 Reducing Technical Debt
10.2 Data Engineering Project Milestone
10.3 Testing tools and frameworks
10.4 Scanning the latest technology in data science, data engineering, and AI
10.5 Python Skills Knowledge Boost
10.6 API data source integration best practice
10.7 Automating Descriptive, Predictive and Prescriptive Data Analysis
10.8 Data Engineering Project Milestone
10.9 Analytics Engineering
10.10 Staying at the Cutting Edge in Data Engineering
10.11 Compile Project Evaluation Report - Draft Mapping Appendix
10.12 Selling your Data Product to Senior Stakeholders
10.13 Compile Professional Discussion Notes
10.14 Articulating your data product to Senior Technical Stakeholders
10.15 Data Engineering Project Milestone
Your Apprenticeship and End-Point Assessment (EPA)
To pass the programme you will need to successfully complete EPA. For more
information about EPA please read the EPA Overview found in Useful Documents at
the top of your Learning Plan in Aptem.
Personal development, behaviours and attitudes (PDBA)
As a regulated training provider, we have a responsibility to support learners’
personal development, behaviours and attitudes. This requirement is broken down
into five key elements:
P1 Fundamental British values: democracy, the rule of law, individual
liberty and mutual respect and tolerance of different faiths and
beliefs.
P2 Equality and diversity: fairness, protected characteristics, gender,
valuing diversity.
P3 Personal development: behaviours, attitudes to learning, career
development, healthy living.
P4 Keeping safe: e-safety, health and safety, staff welfare.
P5 Safeguarding: extremism, prevent/channel, well-being.
We meet these elements throughout the programme, through the following learning
activities:
• CPD tasks. Where relevant, you will be asked to answer PDBA related
questions. These will be tagged accordingly, for example, “PDBA”;
• Workshops. An element of PDBA is delivered in Corndel workshops; and
• 121s. Where a topic relevant to PDBA comes up in a 121 session, your PDE
will ensure it is discussed from a PDBA perspective.
