\n",
- " ![](\"img/peak_swe_example.png\")
\n",
+ " ![](\"../img/peak_swe_example.png\")
\n",
"
\n",
"\n",
"_Example daily SWE at a single site, showing two important periods in snow processes: accumulation (before peak) and ablation (after peak). The vertical line marks peak SWE._"
@@ -977,6 +721,7 @@
},
{
"cell_type": "markdown",
+ "id": "add14a79",
"metadata": {},
"source": [
"### 5.1 Comparing Modeled and Observed Peak SWE at All Sites in the Watershed\n",
@@ -990,28 +735,39 @@
{
"cell_type": "code",
"execution_count": null,
+ "id": "2e15a789",
"metadata": {},
"outputs": [],
"source": [
"# isolate the columns associated with observations and model predictions.\n",
"# these will be inputs to our same-day comparison function.\n",
- "obs_cols = sorted([col for col in combined_df.columns if col.endswith('SNTL')])\n",
- "mod_cols = sorted([col for col in combined_df.columns if col.endswith('PFCONUS1')])"
+ "obs_swe_cols = obs_df.columns.tolist()\n",
+ "mod_swe_cols = model_df.columns.tolist()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3854f098",
+ "metadata": {},
+ "source": [
+ "Use the `modeled_swe_at_observed_peak` utility function to perform the same-day SWE comparison."
]
},
{
"cell_type": "code",
"execution_count": null,
+ "id": "ed342496",
"metadata": {},
"outputs": [],
"source": [
- "# compute the same-day SWE comparison during the observed peak SWE for each of the observation and modeled sites.\n",
- "df_observed_peak = utils.modeled_swe_at_observed_peak(combined_df, obs_cols, mod_cols)\n",
+ "# compute the same-day SWE comparison during the observed peak SWE for each of the observation and modeled sites\n",
+ "df_observed_peak = snow_utils.modeled_swe_at_observed_peak(obs_df, model_df)\n",
"df_observed_peak"
]
},
{
"cell_type": "markdown",
+ "id": "7e0c6dea",
"metadata": {},
"source": [
"##### đź“Š Visualize the amount of SWE on **the day of observed peak SWE occurs** for both the model and observations at each station"
@@ -1020,6 +776,7 @@
{
"cell_type": "code",
"execution_count": null,
+ "id": "297f8608",
"metadata": {},
"outputs": [],
"source": [
@@ -1060,6 +817,7 @@
},
{
"cell_type": "markdown",
+ "id": "60d98757",
"metadata": {},
"source": [
"#### đź“‹ Modeled vs Observed Peak SWE Comparison (timing & magnitude) \n",
@@ -1069,16 +827,18 @@
{
"cell_type": "code",
"execution_count": null,
+ "id": "78907563",
"metadata": {},
"outputs": [],
"source": [
"# compute the different-day SWE comparison for each of the observed and modeled sites.\n",
- "df_both_peak = utils.modeled_vs_observed_peak_swe(combined_df, obs_cols, mod_cols)\n",
+ "df_both_peak = snow_utils.modeled_vs_observed_peak_swe(obs_df, model_df)\n",
"df_both_peak"
]
},
{
"cell_type": "markdown",
+ "id": "d5ec9e5d",
"metadata": {},
"source": [
"##### đź“Š Visualize the quantity of peak SWE for both the model and observations at each station"
@@ -1087,6 +847,7 @@
{
"cell_type": "code",
"execution_count": null,
+ "id": "a93b9fa9",
"metadata": {},
"outputs": [],
"source": [
@@ -1125,6 +886,7 @@
},
{
"cell_type": "markdown",
+ "id": "ed021831",
"metadata": {},
"source": [
"### 5.2 Visualizing Model Error for Peak SWE\n",
@@ -1139,6 +901,7 @@
{
"cell_type": "code",
"execution_count": null,
+ "id": "e46b82f8",
"metadata": {},
"outputs": [],
"source": [
@@ -1146,21 +909,14 @@
"df_both_peak['Peak_Date_Diff_Days'] = (df_both_peak['Modeled_Date'] - \n",
" df_both_peak['Observed_Date']).dt.days\n",
"df_both_peak['Peak_SWE_Diff'] = (df_both_peak['Modeled'] - \n",
- " df_both_peak['Observed'])"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "df02795e",
- "metadata": {},
- "outputs": [],
- "source": [
+ " df_both_peak['Observed'])\n",
+ "\n",
"df_both_peak"
]
},
{
"cell_type": "markdown",
+ "id": "b77c566e",
"metadata": {},
"source": [
"##### đź“Š Visualize the error between the modeled and observed peak SWE "
@@ -1169,6 +925,7 @@
{
"cell_type": "code",
"execution_count": null,
+ "id": "29c68cf0",
"metadata": {},
"outputs": [],
"source": [
@@ -1206,6 +963,7 @@
},
{
"cell_type": "markdown",
+ "id": "4f707f0c",
"metadata": {},
"source": [
"The left panel shows the timing error (date difference) and the right panel the magnitude error (SWE difference). When we computed the difference in date and SWE quantity above, we took `modeled - observed` so: \n",
@@ -1218,11 +976,12 @@
},
{
"cell_type": "markdown",
+ "id": "8729d9ff",
"metadata": {},
"source": [
"\n",
+ " \n",
"\n",
"
"
@@ -1230,6 +989,7 @@
},
{
"cell_type": "markdown",
+ "id": "fdabdbd6",
"metadata": {},
"source": [
"#### đź“Š Next, we combine the timing and magnitude errors and plot them together for each station."
@@ -1238,6 +998,7 @@
{
"cell_type": "code",
"execution_count": null,
+ "id": "66105b84",
"metadata": {},
"outputs": [],
"source": [
@@ -1260,11 +1021,12 @@
"vline = hv.VLine(0).opts(color='gray', line_dash='dashed')\n",
"hline = hv.HLine(0).opts(color='gray', line_dash='dashed')\n",
"\n",
- "(scatter * vline * hline).opts(legend_position='top_left', show_grid=True)\n"
+ "(scatter * vline * hline).opts(legend_position='bottom_left', show_grid=True)\n"
]
},
{
"cell_type": "markdown",
+ "id": "285ec901",
"metadata": {},
"source": [
"✏️ **Try changing how we view this plot.** \n",
@@ -1274,313 +1036,292 @@
},
{
"cell_type": "markdown",
+ "id": "f6a20df4",
"metadata": {},
"source": [
- "## 6. Compute and Statistics and Error Metrics \n",
- "The previous section visualized when and where modeled SWE differs from observations, both in terms of peak SWE timing and magnitude. However, visual inspection alone makes it difficult to compare performance across sites or to summarize model behavior in a consistent or quantifiable way. In this section, we compute commonly used statistical error metrics to quantify model performance, allowing us to objectively assess bias, error magnitude, and variability for sites within the watershed. \n",
- "\n",
- "Proposed outline (DTK, Jan 2026):\n",
- "- Summary metrics at a station\n",
- "- Summary metrics at all stations within the watershed\n",
- "- Combined timing and magnitude for all stations within the watershed (Condon metric)\n",
- "- Focus on timing: summary statistics for single station for accumulation & ablation periods (using the new wrapper: `nwm_utils.compute_stats_period()`)\n",
- "- Melt period statistics"
+ "### 5.3 Visualizing Model Error for Melt Period\n",
+ "The function `compute_melt_period_obs_vs_model` summarizes melt behavior for both the observed and modeled SWE time series at each station and for each water year, allowing the two datasets to be compared side by side. For every station-year combination, it identifies the date of peak SWE, then defines the end of the melt period as the first date after that peak when SWE reaches zero and stays at zero for at least `min_zero_days` consecutive days. Using these two dates, it computes the melt period length in days and the average melt rate over the full melt period, expressed in meters per day. The final output is a table that reports these melt timing and melt rate statistics for both observations and model simulations, making it possible to evaluate whether the model melts snow too early or too late, too quickly or too slowly, and how well it reproduces the overall timing and pace of seasonal snow disappearance across sites and years.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "nwm_utils.compute_stats(combined_df, f'CCSS_{my_site_code}_swe_m', f'NWM_{my_site_code}_swe_m')"
- ]
- },
- {
- "cell_type": "markdown",
+ "id": "b2636cad",
"metadata": {},
+ "outputs": [],
"source": [
- "Pearson and Spearman correlations are both close to 1, suggesting a strong relationship between observed and modeled SWE. As shown on the timeseries plot, this strong correlation alone does not indicate a \"good\" model. For example, it does not guarantee accurate timing of key events, such as peak SWE or melt onset. Let's compare these as well. The following code uses `report_max_dates_and_values` function to identify the peak SWE value and the date it occurs for both the observed (CCSS) and modeled (NWM) datasets. "
+ "melt_df_stats = snow_utils.compute_melt_period_obs_vs_model(obs_df, model_df, min_zero_days=10)\n",
+ "melt_df_stats"
]
},
{
"cell_type": "markdown",
+ "id": "9cc72756",
"metadata": {},
"source": [
- "🧠Reflect
\n", - "Looking at the two plots, what could be some reasons for the model having simulated peak SWE both earlier and less than the observed peak SWE? Perhaps try changing the my_site_code from earlier in the notebook to rerun nwm_utils.comparison_plots() to see the timeseries for a different station to look at the peak magnitude and timing. \n",
+ "
Looking at the two plots, what could be some reasons for the model having simulated peak SWE both earlier and less than the observed peak SWE? Perhaps try changing the my_site_code from earlier in the notebook to rerun plot_utils.comparison_plots() to see the timeseries for a different station to look at the peak magnitude and timing. \n",
"\n",
"
What happens if you change the year that is plotted?
✏️ Try modifying the bar plot code from bar1 = year1.hvplot.bar to bar1 = year2.hvplot.bar. Don't forget to change the title!
\n",
- "
"
+ "From a snow hydrologist’s perspective, this table is very useful because it captures three core aspects of seasonal snow behavior: **peak timing and magnitude**, **melt-out timing**, and **average melt rate**. The timeline or dumbbell view is often the most intuitive for timing diagnostics, while the 1:1 scatter is the best overall summary of performance."
]
},
{
"cell_type": "code",
"execution_count": null,
- "metadata": {
- "tags": []
- },
+ "id": "a2b7829c",
+ "metadata": {},
"outputs": [],
"source": [
- "summary_table = nwm_utils.report_max_dates_and_values(combined_df, f'CCSS_{my_site_code}_swe_m', f'NWM_{my_site_code}_swe_m')\n",
- "summary_table"
+ "plot_utils.plot_scatter_melt_metrics(melt_df_stats)"
]
},
{
"cell_type": "markdown",
+ "id": "5359941d",
"metadata": {},
"source": [
- "### Summary Metrics at Multiple Sites"
+ "While site-level comparisons provide useful detail, it is often more informative to evaluate model behavior at the **process level** to understand systematic tendencies across the watershed. In this step, we assess whether the model tends to melt snow earlier or later than observed by analyzing the difference in melt period length (model minus observed). Positive differences indicate that the model retains snow longer (later melt), while negative differences indicate earlier melt. The following visualization summarizes this behavior across all stations and water years, highlighting both the direction and magnitude of melt timing bias, and providing an overall assessment of whether the model systematically accelerates or delays snowmelt processes."
]
},
{
"cell_type": "code",
"execution_count": null,
+ "id": "1b7b75be",
"metadata": {},
"outputs": [],
"source": [
- "site_codes = ['DAN', 'HRS', 'KIB', 'PDS', 'SLI', 'TUM', 'WHW']\n",
- "\n",
- "rows = []\n",
- "\n",
- "for site in site_codes:\n",
- " obs_col = f'CCSS_{site}_swe_m'\n",
- " mod_col = f'NWM_{site}_swe_m'\n",
- "\n",
- " stats_table = nwm_utils.compute_stats(combined_df, obs_col, mod_col)\n",
- "\n",
- " rows.append({\n",
- " 'Station': site,\n",
- " 'Mean_Obs': stats_table.loc['observed', 'Mean'],\n",
- " 'Mean_Mod': stats_table.loc['modeled', 'Mean'],\n",
- " 'Bias_m': stats_table.loc['Bias (Modeled - Observed)', 'Mean'],\n",
- " 'Pearson_r': stats_table.loc['Pearson Correlation', 'Mean'],\n",
- " 'Spearman_r': stats_table.loc['Spearman Correlation', 'Mean'],\n",
- " 'NSE': stats_table.loc['Nash-Sutcliffe Efficiency (NSE)', 'Mean'],\n",
- " 'KGE': stats_table.loc['Kling-Gupta Efficiency (KGE)', 'Mean']\n",
- " })\n",
- "\n",
- "stats_AllStations = pd.DataFrame(rows)\n",
- "\n",
- "print('All Stations Statistics Summary:')\n",
- "stats_AllStations"
+ "# Process-level assessment\n",
+ "plot_utils.plot_melt_bias_summary(\n",
+ " melt_df_stats, # Pandas DataFrame containing site melt statistics\n",
+ " \"Melt_Period_Days_Obs\", # Name of the column corresponding with the observation data\n",
+ " \"Melt_Period_Days_Mod\", # Name of the column corresponding with the modeled data\n",
+ " \"Melt Period Length (Obs vs Model)\" # Title of the plot\n",
+ ")"
]
},
{
- "cell_type": "code",
- "execution_count": null,
+ "cell_type": "markdown",
+ "id": "809e568f",
"metadata": {},
- "outputs": [],
"source": [
- "stats_AllStations.hvplot.bar(\n",
- " x='Station',\n",
- " y='NSE',\n",
- " rot=45,\n",
- " ylabel='Nash–Sutcliffe Efficiency',\n",
- " title='NSE by Station',\n",
- " height=400,\n",
- " width=600,\n",
- " bar_width=0.5\n",
- ")\n"
+ "## 6. Compute and Statistics and Error Metrics \n",
+ "The previous section visualized when and where modeled SWE differs from observations, both in terms of peak SWE timing and magnitude. However, visual inspection alone makes it difficult to compare performance across sites or to summarize model behavior in a consistent or quantifiable way. In this section, we compute commonly used statistical error metrics to quantify model performance, allowing us to objectively assess bias, error magnitude, and variability for sites within the watershed. "
]
},
{
"cell_type": "code",
"execution_count": null,
+ "id": "7bff12bf",
"metadata": {},
"outputs": [],
"source": [
- "stats_summary.hvplot.scatter(\n",
- " x='Station',\n",
- " y='Bias_m',\n",
- " size=100,\n",
- " rot=45,\n",
- " ylabel='Bias (m)',\n",
- " title='Mean SWE Bias by Station'\n",
- ")\n"
+ "# Check for mismatched site columns between obs and model data\n",
+ "for col in obs_df.columns:\n",
+ " if col not in model_df.columns:\n",
+ " print(f\"{col} missing in model data\")"
]
},
{
"cell_type": "markdown",
+ "id": "64161df0",
"metadata": {},
"source": [
- "### Combine Magnitude (absolute relative bias) and Timing (Spearman's rho) metrics using the Condon metric (and with all stations, a Condon diagram)"
+ "### 6.1 Compute Metrics"
]
},
{
- "cell_type": "code",
- "execution_count": null,
+ "cell_type": "markdown",
+ "id": "21c8ca67",
"metadata": {},
- "outputs": [],
"source": [
- "bias1 = evaluation_metrics.bias(combined_df.CCSS_TUM_swe_m, combined_df.NWM_TUM_swe_m)\n",
- "bias1"
+ "Using the modeled and observed dataframes, we can compute summary metrics. The following is some examples demonstrating how to use individual metrics of interest for a specific site, referenced as `my_site_code`."
]
},
{
"cell_type": "code",
"execution_count": null,
+ "id": "9d0f5c1d",
"metadata": {},
"outputs": [],
"source": [
- "abs_bias = evaluation_metrics.absolute_relative_bias(combined_df.CCSS_TUM_swe_m, combined_df.NWM_TUM_swe_m)\n",
- "abs_bias"
+ "bias = metric_utils.bias(obs_df.loc[:, f'{my_site_code}'], model_df.loc[:, f'{my_site_code}'])\n",
+ "abs_bias = metric_utils.absolute_relative_bias(obs_df.loc[:, f'{my_site_code}'], model_df.loc[:, f'{my_site_code}'])\n",
+ "srho = metric_utils.spearman_rank(obs_df.loc[:, f'{my_site_code}'], model_df.loc[:, f'{my_site_code}'])\n",
+ "\n",
+ "print(f\"For site {my_site_code}: bias = {bias}, abs_bias = {abs_bias}, srho = {srho}\")"
]
},
{
- "cell_type": "code",
- "execution_count": null,
+ "cell_type": "markdown",
+ "id": "bf7a7432",
"metadata": {},
- "outputs": [],
"source": [
- "srho = evaluation_metrics.spearman_rank(combined_df.CCSS_TUM_swe_m, combined_df.NWM_TUM_swe_m)\n",
- "srho"
+ "Apply it to all sites using the `calculate_metrics` capability of the evaluation framework."
]
},
{
"cell_type": "code",
"execution_count": null,
+ "id": "1a6cc22b",
"metadata": {},
"outputs": [],
"source": [
- "evaluation_metrics.condon(abs_bias, srho)"
+ "metrics_df = evaluation_utils.calculate_metrics(\n",
+ " obs_df, # Pandas DataFrame containing the time series observations. \n",
+ " model_df, # Pandas DataFrame containing the time series model calculations.\n",
+ " metadata_df, # Pandas DataFrame containing location and site attribute information.\n",
+ " metrics_list=None, # List of metrics to calculate. None indicates that all metrics will be computed.\n",
+ " write_csv=False, # Indicates the the output should not be saved to CSV (default = False)\n",
+ " csv_path=None, # Indicates the the output should not be saved to CSV (default = None)\n",
+ ")\n",
+ "metrics_df"
]
},
{
"cell_type": "markdown",
+ "id": "0838c8f5",
"metadata": {},
"source": [
- "🧠Reflect
\n", - "You now have several performance metrics: Bias, Pearson Correlation, Spearman Correlation, NSE, and KGE. If you had to pick just one metric to summarize model performance, which would you choose—and why? As you review the results, compare the peak flow amounts and the timing of snowmelt onset. Do you see any significant differences? Which dataset indicates an earlier melt?
\n", - "\n",
- "
\n"
+ "Look at plots of summary statistics for each station. Here we look at Bias and NSE for each station in the watershed:"
]
},
{
- "cell_type": "markdown",
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "d58febb4",
"metadata": {},
+ "outputs": [],
"source": [
- "### Focus on Timing: Melt Period Metrics\n",
- "Compare the average melt rate over the full melt period. "
+ "# Bias scatter\n",
+ "scatter = metrics_df.hvplot.scatter(\n",
+ " x='site_id',\n",
+ " y='bias',\n",
+ " size=100,\n",
+ " rot=45,\n",
+ " ylabel='Bias (m)',\n",
+ " title='Mean SWE Bias by Station'\n",
+ ")\n",
+ "\n",
+ "hline = hv.HLine(0).opts(color='black', line_dash='dashed', line_width=1)\n",
+ "\n",
+ "scatter * hline"
]
},
{
- "cell_type": "markdown",
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "988fcb43",
"metadata": {},
+ "outputs": [],
"source": [
- "The following function computes the melt period length by identifying the first date after the peak SWE when SWE drops to zero and remains at zero for at least (`min_zero_days`) consecutive days. This is used to define the end of the melt period. Finally, the function calculates the average melt rate, which represents the rate at which snow disappeared, expressed in meters per day, over the full melt period."
+ "# NSE histogram\n",
+ "metrics_df.hvplot.bar(\n",
+ " x='site_id',\n",
+ " y='nse',\n",
+ " rot=45,\n",
+ " ylabel='Nash–Sutcliffe Efficiency',\n",
+ " title='NSE by Station',\n",
+ " height=400,\n",
+ " width=600,\n",
+ " bar_width=0.5\n",
+ ")\n"
]
},
{
- "cell_type": "code",
- "execution_count": null,
+ "cell_type": "markdown",
+ "id": "d0ee45fb",
"metadata": {},
- "outputs": [],
"source": [
- "melt_stats_df = utils.compute_melt_period_statistics(combined_df)\n",
- "melt_stats_df.head()\n",
- "melt_stats_df"
+ "🧠Reflect
\n", - "\n",
- " What is the modeled SWE on the date when the observed SWE reaches its peak?
\n",
- " ✏️ Use the code snippet below to find the answer.\n",
- "
\n",
- " \n",
- " # Find date of the peak SWE from observed data\n",
- " date_obs_max = combined_df['CCSS_HRS_swe_m'].idxmax()\n",
- "\n",
- " # Get corresponding value of modeled data on that date\n",
- " value_mod_at_max_obs = combined_df.loc[date_obs_max, 'NWM_HRS_swe_m']\n",
- " \n",
+ "
"
]
},
{
"cell_type": "code",
"execution_count": null,
- "metadata": {
- "tags": []
- },
+ "id": "9fe75736",
+ "metadata": {},
"outputs": [],
"source": [
- "observed_melt_period = nwm_utils.compute_melt_period(combined_df[f'CCSS_{my_site_code}_swe_m'])\n",
- "observed_melt_period"
+ "plot_utils.comparison_plots(obs_df, model_df, f'{my_site_code}', f'{my_site_code}', site_label=None)\n",
+ "\n",
+ "# Change the site code to see other Snotel Stations --> e.g., '688:CO:SNTL'\n",
+ "#plot_utils.comparison_plots(obs_df, model_df, '688:CO:SNTL', '688:CO:SNTL', site_label=None)"
]
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "tags": []
- },
- "outputs": [],
+ "cell_type": "markdown",
+ "id": "a2d295d2",
+ "metadata": {},
"source": [
- "modeled_melt_period = nwm_utils.compute_melt_period(combined_df[f'NWM_{my_site_code}_swe_m'])\n",
- "modeled_melt_period"
+ "🧠Reflect
\n", + "\n", + " If you recall from earlier, we plotted the timeseries of our selected station. Replot it below. Do the metrics make sense given the visual comparison between modeled and observed? For example, when you look at the timeseries, is the model consistently predicting SWE to be higher or lower than observations? Does this align with the Bias sign (+ or -)?\n", + "
\n", + "\n",
+ "
"
]
},
{
- "cell_type": "code",
- "execution_count": null,
+ "cell_type": "markdown",
+ "id": "ced951d0",
"metadata": {},
- "outputs": [],
"source": [
- "accum_months = [10, 11, 12, 1, 2, 3]\n",
- "ablation_months = [4, 5, 6]\n",
- "\n",
- "accum_stats = nwm_utils.compute_stats_period(\n",
- " combined_df,\n",
- " f'CCSS_{my_site_code}_swe_m',\n",
- " f'NWM_{my_site_code}_swe_m',\n",
- " accum_months\n",
- ")\n",
- "\n",
- "accum_stats"
+ "The metrics above are computed over the entire snow season, including both accumulation and ablation periods. If desired, you can subset the data and recompute the metrics to examine differences between these phases. For example, if we define the **ablation period** as April through June, the following code computes statistics for data within those months. This approach provides a more detailed understanding of how well the model represents snow accumulation versus melt processes and where performance differences may occur."
]
},
{
"cell_type": "code",
"execution_count": null,
+ "id": "ba082688",
"metadata": {},
"outputs": [],
"source": [
+ "ablation_months = [4, 5, 6] # 4:April, 5:May, 6:June. \n",
+ "\n",
+ "# Create subsets of the aligned dataframes that exclude the ablation months\n",
+ "obs_df_ablation = obs_df[obs_df.index.month.isin(ablation_months)]\n",
+ "model_df_ablation = model_df[model_df.index.month.isin(ablation_months)]\n",
"\n",
- "ablation_stats = nwm_utils.compute_stats_period(\n",
- " combined_df,\n",
- " f'CCSS_{my_site_code}_swe_m',\n",
- " f'NWM_{my_site_code}_swe_m',\n",
- " ablation_months\n",
+ "# Compute metrics for the ablation period\n",
+ "metrics_df_ablation = evaluation_utils.calculate_metrics(\n",
+ " obs_df_ablation,\n",
+ " model_df_ablation,\n",
+ " metadata_df,\n",
+ " metrics_list=None,\n",
+ " write_csv=False,\n",
+ " csv_path=None,\n",
")\n",
"\n",
- "ablation_stats"
+ "metrics_df_ablation"
]
},
{
"cell_type": "markdown",
+ "id": "aff0fac4",
"metadata": {},
"source": [
- "🧠Reflect
\n", + "You now have several performance metrics: Bias, Pearson Correlation, Spearman Correlation, NSE, and KGE. If you had to pick just one metric to summarize model performance, which would you choose—and why?
\n", + "\n",
- "
"
+ "### 6.2 Combining Magnitude (Absolute Relative Bias) and Timing (Spearman’s Rho) Using the Condon Metric "
]
},
{
- "cell_type": "code",
- "execution_count": null,
+ "cell_type": "markdown",
+ "id": "53396d9b",
"metadata": {},
- "outputs": [],
"source": [
- "nwm_utils.comparison_plots(combined_df, f'CCSS_{my_site_code}_swe_m', f'NWM_{my_site_code}_swe_m')\n"
+ "One way to learn more about the model performance is to combine metrics that tell us about different aspects of model behavior—such as timing, variability, and magnitude—rather than relying on a single summary measure.\n",
+ "\n",
+ "The Condon diagram separates model performance into quadrants based on two metrics: **Spearman’s rho** (shape/time agreement) and **relative bias** (magnitude error). The horizontal line at 0.5 distinguishes whether the model captures the temporal pattern well (above 0.5 = good shape), while the vertical line is traditionally placed at a relative bias of 1.0, which represents a 100% error. This means the model’s total error is as large as the observed signal itself. This threshold has a clear physical interpretation and is used in the original Condon framework to distinguish acceptable vs. large bias. "
]
},
{
"cell_type": "code",
"execution_count": null,
+ "id": "fb9d46d7",
"metadata": {},
"outputs": [],
- "source": []
+ "source": [
+ "plot_utils.plot_condon_diagram(metrics_df, variable=\"SWE\")"
+ ]
}
],
"metadata": {
diff --git a/src/cssi_evaluation/utils/evaluation_utils.py b/src/cssi_evaluation/utils/evaluation_utils.py
index fabb7de..3e2ce96 100644
--- a/src/cssi_evaluation/utils/evaluation_utils.py
+++ b/src/cssi_evaluation/utils/evaluation_utils.py
@@ -118,10 +118,23 @@ def calculate_metrics(
# Initialize empty metrics DataFrame to store calculated comparison metrics.
metrics_df = initialize_metrics_df(obs_metadata_df, metrics_list)
- num_sites = obs_data_df.shape[1] - 1 # first column is 'date'
+ # Ensure datetime index for obs_data_df
+ if "date" in obs_data_df.columns:
+ obs_data_df["date"] = pd.to_datetime(obs_data_df["date"])
+ obs_data_df = obs_data_df.set_index("date")
+ else:
+ obs_data_df.index = pd.to_datetime(obs_data_df.index)
+
+ # --- Ensure datetime index for model_data_df ---
+ if "date" in model_data_df.columns:
+ model_data_df["date"] = pd.to_datetime(model_data_df["date"])
+ model_data_df = model_data_df.set_index("date")
+ else:
+ model_data_df.index = pd.to_datetime(model_data_df.index)
+
+ site_cols = obs_data_df.columns
- for i in range(num_sites):
- site_id = obs_data_df.columns[(i + 1)]
+ for site_id in site_cols:
obs_data = obs_data_df.loc[:, [site_id]].to_numpy()
model_data = model_data_df.loc[:, [site_id]].to_numpy()
diff --git a/src/cssi_evaluation/variables/snow_utils.py b/src/cssi_evaluation/variables/snow_utils.py
index ad0a1d9..109f694 100644
--- a/src/cssi_evaluation/variables/snow_utils.py
+++ b/src/cssi_evaluation/variables/snow_utils.py
@@ -82,94 +82,6 @@ def modeled_swe_at_observed_peak(obs_df: pd.DataFrame, model_df: pd.DataFrame) -
return pd.concat(results)
-######### ORIGINAL CUAHSI FUNCTION BELOW:
-# def modeled_swe_at_observed_peak(
-# df: pd.DataFrame, obs_swe_cols: list[str], mod_swe_cols: list[str]
-# ) -> pd.DataFrame:
-# """
-# Extract modeled SWE values on the dates of observed peak (maximum) SWE.
-
-# This function evaluates model performance by comparing observed peak SWE
-# to the modeled SWE on the same calendar date. For each station and water year,
-# the date of maximum observed SWE is identified, and the modeled SWE value
-# at that date is extracted.
-
-# Parameters
-# ==========
-# df: pandas.DataFrame
-# A pandas dataframe containing columns associated with modeled and observed SWE. The
-# dataframe must have an datetime[64] index.
-# obs_swe_cols: list[str]
-# Names of the columns associated with observed SWE
-# mod_swe_cols: list[str]
-# Names of the columns associated with modeled SWE
-
-# Returns
-# =======
-# df: pandas.DataFrame
-# A dataframe containing observed max observed SWE and the modeled SWE at the index of the
-# maximum observed. The format of the DataFrame will be:
-
-# Observed Modeled Water_Year Station
-# 🧠Reflect
\n", - "\n", - " If you recall from earlier, we plotted the timeseries of out selected station. Replot it below. Do the metrics make sense given the visual comparison between modeled and observed? For example, when you look at the timeseries, is the model consistently predicting SWE to be higher or lower than observations? Does this align with the Bias sign (+ or -)?\n", - "
\n", - "