The Virtues of a Programming your ETL Pipeline using a Fluent Interface

Introduction: The Fluent Interface Pattern

This week marked a new chapter in my career. I started a new job as a Data Engineer for Synaptiq building ETL pipelines that feed into ML and AI models. Looking back at my (nearly) year long career at Sequoia (A First American Startup), I cant help but feel grateful for the opportunity to have worked with such a talented team. I learned a lot about Data Engineering. One of the most underrated things I learned in that role came in the form of a design preference from my Team lead. Often he would ask me to write additional transforms and apply them in pipelines using the Fluent API

Continue reading »

Fablr package development with Poetry- Dependency Management, Versioning and Publishing.

Introduction:

A short while ago I wrote an article detailing how to create, manage, and build your own development container using Docker, VSCode and GitHub Codespaces, as well as some cool tweaks that allow you to auto-install extensions, etc. In that article, I was working on a toy package I …

Continue reading »

Development Containers in VSCode and Github Codespaces!

Development Containers

Intro

If you're involved in a collaborative project, you're likely familiar with dev containers. They serve as a crucial tool for ensuring consistency in development environments across team members. This means you're guaranteed to be working with the same Python version, have the necessary drivers, and more.

In my current role, we leverage dev containers in conjunction with Poetry not only to enforce a standardized development environment but also to establish a uniform Python virtual environment.

Continue reading »

Creating a markdown blog using ipynb and Pelican!

Welcome to my new blog!

For those of you that don't know me or stumbled upon my new blog, let me introduce myself. I'm Jesus (Jesse) Caro, and I have been working in the data space for about 7 years as of 2023. When I started out of grad school, my primary language was R. I had a great blog written in R (powered by blogdown

Continue reading »