Fig 1: A screenshot of Luigi's web UI, of a real-world (although rather simple) workflow implemented in Luigi:
Update May 5, 2016: Most of the below material is more or less outdated. Our latest work has resulted in the SciLuigi helper library, which we have used in production and will be focus of further developments.
In the Bioclipse / Pharmaceutical Bioinformatics group at Dept of Pharm. Biosciences att UU, we are quite heavy users of Spotify's Luigi workflow library, to automate workflows, mainly doing Machine Learning heavy lifting.
Some people have asked for our experiences using Luigi. While we have yet to write up a comprehensive summary and review with conclusion, I thought to gather links to the resources already available where we discuss just that. So, find the selection of links below, in approximate order of recency and relevance:
- Wrote a small helper library, "luigi's monkey wrench", to help with complex dependency graphs.
- Blogged a bit about the tool here.
- Held a Comprehensive tutorial on building workflows with Luigi. All materials available.
- Held a short 15 min presentation about our experiences at the eInfraMPS2015 workshop. See:
- Wrote a long post at medium.com describing some workaround we figured we needed with Luigi.
A few external links
- Radhouane Aniba created a nice little on writing bioinformatics workflows in "vanilla Luigi" at CodersCrowd
- Discussion on the above tutorial on BioStars
- A nice post on Luigi in bioinformatics by Fabio Barteri, aka ATCGeek (twitter; @linusgeek)