Basically you define a job with some tasks, the relations between tasks and the framework took care to execute them in the right order (topological sorting and so on).
Parallelism and concurrency are achieved with clojure.core.async.
Using datalog as query language gives some help managing jobs and tasks relations: it's almost like having a graph db.
Of course we are going to release it #opensource when ready.
I can write about it a little bit more if anyone is interested.
@schaueho yes, the idea was to build something like Onyx, we really like the way you describe jobs and tasks, but we also wanted something "smaller", with almost no external dependencies.
We are not on Onyx level at all right now, but we'll work on it. 😊
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!