86 lines
2.7 KiB
Markdown
86 lines
2.7 KiB
Markdown
<a name="readme-top"></a>
|
|
|
|
<br />
|
|
|
|
<h1 align="center">🧪 Data Lab</h1>
|
|
|
|
<p align="center">
|
|
<img height="250" src="https://git.camcodes.dev/Cameron/Data_lab/raw/branch/main/images/datalab.jpg" alt="DataLab" />
|
|
</p>
|
|
|
|
<h4 align="center">Learning • Experimentation • Data Engineering Projects</h4>
|
|
|
|
> [!NOTE]
|
|
> Code, datasets, and designs may change as I refine implementations and adopt best practices.
|
|
|
|
---
|
|
|
|
<!-- TABLE OF CONTENTS -->
|
|
<details open>
|
|
<summary>Table of Contents</summary>
|
|
<ol>
|
|
<li><a href="#about">About</a></li>
|
|
<li><a href="#projects">Projects</a></li>
|
|
<li><a href="#tools--technologies">Tools & Technologies</a></li>
|
|
<li><a href="#contact">Contact Me</a></li>
|
|
</ol>
|
|
</details>
|
|
|
|
---
|
|
|
|
<a id="about"></a>
|
|
## About
|
|
|
|
**Data Lab** is a personal sandbox for building and iterating on **data engineering projects**.
|
|
|
|
The focus is on:
|
|
- Designing reliable data pipelines
|
|
- Applying analytics engineering principles
|
|
- Working with batch data, lakehouse patterns, and validation
|
|
- Learning by building realistic systems rather than just toys
|
|
|
|
Projects range from small experiments to end-to-end pipelines using production-style tools.
|
|
|
|
<p align="right">(<a href="#readme-top">back to top</a>)</p>
|
|
|
|
---
|
|
|
|
<a id="projects"></a>
|
|
|
|
# Projects
|
|
|
|
*Each project will be in its own Repo.*
|
|
|
|
## [Banking Transaction Pipeline](https://git.camcodes.dev/Cameron/Banking_Data_Pipeline)
|
|
- Spark-based ETL pipeline
|
|
- Bronze → Silver → Gold lakehouse design
|
|
- Data quality enforcement and validation
|
|
|
|
|
|
<p align="right">(<a href="#readme-top">back to top</a>)</p>
|
|
|
|
---
|
|
|
|
## Tools & Technologies
|
|
|
|
- **Languages:** Python, SQL, Java
|
|
- **Processing:** Apache Spark (PySpark)
|
|
- **Storage:** S3 storage
|
|
- **Data Formats:** Parquet, Delta-style layouts
|
|
- **Databases:** PostgreSQL, SQLite (project-dependent)
|
|
- **Visualization:** Tableau / Power BI
|
|
|
|
*Tools may expand as new projects are added.*
|
|
|
|
|
|
---
|
|
|
|
<a id="contact"></a>
|
|
## 💬 Connect With Me
|
|
|
|
|
|
<span>[](https://www.linkedin.com/in/cameron-css/) [](https://CamDoesData.com) [](https://www.kaggle.com/cameronseamons) [](mailto:CameronSeamons@gmail.com) [](https://drive.google.com/file/d/1YaM4hDtt2-79ShBVTN06Y3BU79LvFw6J/view?usp=sharing)</span>
|
|
|
|
|
|
<p align="right">(<a href="#readme-top">back to top</a>)</p>
|
|
|