Skip to content Skip to sidebar Skip to footer
Showing posts with the label Luigi

Luigi Pipeline Beginning In S3

My initial files are in AWS S3. Could someone point me how I need to setup this in a Luigi Task? I … Read more Luigi Pipeline Beginning In S3

Luigi: How To Pass Different Arguments To Leaf Tasks?

This is my second attempt at understanding how to pass arguments to dependencies in Luigi. The firs… Read more Luigi: How To Pass Different Arguments To Leaf Tasks?

When A New File Arrives In S3, Trigger Luigi Task

I have a bucket with new objects getting added at random intervals with keys based on their time of… Read more When A New File Arrives In S3, Trigger Luigi Task

Luigi Pipeline Beginning In S3

My initial files are in AWS S3. Could someone point me how I need to setup this in a Luigi Task? I … Read more Luigi Pipeline Beginning In S3