Group by + fill gaps

[LeaVerou]

• One of the reasons tracking applications are so hard to do with Mavo is that the kinds of temporal visualizations required (e.g. calendars, timelines, etc.) require both grouping (by time of day, date, month, year, etc.) but also filling in any gaps between the groups. Not to mention that by default this does not follow the order we'd want.
• While most use cases around filling gaps are temporal, there are a few with plain numbers.
• There are also entirely non-ordered cases too, e.g. a select menu property where some of the options are unused but we still want to display them.

I wonder what's the best way to deal with this in Mavo.

There are two framings:

• This is just regular grouping, but we need to "fill in the holes" and provide some additional values whose groups will be empty
• This is grouping where the groups are externally provided (which could also provide a workaround for Group by list-valued properties is currently broken #387)

Option 1: Holefilling option for groupBy() and by

This is probably the easiest for authors, but also the least flexible solution, bordering on overfitting.
We could add a third argument for options, one of which would be the granularity to use when filling holes, e.g.

groupBy(entry, entry.date, fill: "day")

entry by date with fill: "day"

Option 2: Option to provide the groups and their order + way to specify the range

Here we decompose the problem into its essential components, and solve them separately.

2.2 Way to specify the range

Option 2.1a: New function to fill holes

We could have a function (name ideas: expand_list()/list_expand(),fill_gaps()) that takes a list (and an optional granularity) and returns an ordered list with any holes filled in.

Pros

• Nicer syntax for our primary use case: expand_list(list)

Cons

• New function to learn
• More involved for generating a range from a min and max: expand_list(list(min, max))

Option 2.1b: Extend range()

We currently have a range() function (and corresponding operator) that only works with numbers. We can easily extend it to work with dates and times.

We could try to auto-detect the granularity needed, and accept a step argument to set it explicitly. Options for that:

• Keywords, e.g. "day"
• Generalized date notations like those used in <time> properties, e.g. YYYY-MM, HH:mm etc.

Pros:

• Natural extension of existing syntax that people may even try on their own

Cons:

• Quite involved for our primary use case: range(min(entries), max(entries)), also would need to iterate over entries twice. OTOH we could introduce a range(list) syntax too.

2.2 Option to provide the groups and their order

Option 2.2a

A third argument:

groupBy(entry, entry.date, dates)

entry by date using dates

Pros:

• More natural syntax

Cons:

• What if we want to add more params in the future?

Option 2.2b

A third options argument.

groupBy(entry, entry.date, groups: dates)

entry by date with groups: dates

Pros:

• More room to grow in the future
• More readable?

Cons:

• Clumsier

Recommendation

I’m currently learning towards Option 2 with Option 2.1a (with the list single argument overloading) + Option 2.2a (low confidence, can easily be swayed towards 2.2b).