Fix grouping by lineage.

[everaldorodrigo]

Context

Getting HTTP 500 error in queries as:

/genomics/prevalence-by-location-all-lineages?location_id=USA&other_threshold=0.03&nday_threshold=5&ndays=60&window=60&ndays=60&cumulative=true&min_date=2022-01-22&max_date=2023-10-22

Solution

It was missing the column lineage in the dataframe after grouping by lineage.
The solution consists in adding the lineage column to data frame after the grouping by.
Also was considered the group_keys behavior changes from pandas 1.4 (1) to 1.5 (2).

Suggestion

Each transformation in the dataframe could be a testable function.

References

(1) https://pandas.pydata.org/pandas-docs/version/1.4/reference/api/pandas.DataFrame.groupby.html
(2) https://pandas.pydata.org/pandas-docs/version/1.5/reference/api/pandas.DataFrame.groupby.html

[everaldorodrigo]

Probably we also could remove this drop:

• https://github.com/outbreak-info/outbreak.api/blob/master/web/handlers/genomics/util.py#L41

And change here to False:

• https://github.com/outbreak-info/outbreak.api/blob/master/web/handlers/v2/genomics/prevalence_all_lineages_by_location.py#L108

[mindoftea]

In order to fix the issue with all lineages being reported under "other", we need to update the get_major_lineage_prevalence function to consider the min_date and max_date, if available

[everaldorodrigo]

The function get_major_lineage_prevalence was updated to consider the min_date and max_date, if available.

Does this make sense, even if the ElasticSearch query is considering the min_date and max_date?