Data Engineering Best Practices

Table of Contents

1. Overview of the Business Problem Statement
2. A Data Engineer's Journey – Background Challenges
3. A Data Engineer's Journey – IT's Vision and Mission
4. Architecture Principles
5. Architecture Framework – Conceptual Architecture Best Practices
6. Architecture Framework – Logical Architecture Best Practices
7. Architecture Framework – Physical Architecture Best Practices
8. Software Engineering Best Practice Considerations
9. Key Considerations for Agile SDLC Best Practices
10. Key Considerations for Quality Testing Best Practices
11. Key Considerations for IT Operational Service Best Practices
12. Key Considerations for Data Service Best Practices
13. Key Considerations for Management Best Practices
14. Key Considerations for Data Delivery Best Practices
15. Other Considerations – Measures, Calculations, Restatements, and Data Science Best Practices
16. Machine Learning Pipeline Best Practices and Processes
17. Takeaway Summary – Putting It All Together
18. Appendix and Use Cases