Load candidates into Slick

Explore the Eight CMP Comparison Cases in Slick

Slick is an interactive tool for exploring MSE results. The simulation is run in jmMSE; this repository converts the resulting performance table into a Slick file.

1 Open the current file

1.1 In the hosted app

Download the current 500-draw jm_candidates_500.slick.
Open the Blue Matter Slick app.
If the app has stopped, select Restart app.
Under Load your MSE Results, select the downloaded .slick file.
When the Overview appears, use the MP and OM filters to explore the results. The current file opens with All CMPs selected, so all eight annual-limit variants should be visible immediately.

If only HS+20 and PR+20 appear, open the MPs filter and select All CMPs. That indicates either that the Sensitivity cases (HS+20 and PR+20) preset is active or that an older copy of the same-named file was loaded. The current file contains eight CMPs. Its SHA-256 fingerprint is recorded in the traceability page for the published branch.

The hosted app accepts file uploads but does not currently provide a field for loading a file directly from a GitHub URL.

1.2 From R

Show implementation detail

install.packages("Slick") # needed only once
library(Slick)

url <- paste0(
  "https://raw.githubusercontent.com/SPRFMO/jmMSE26/",
  "main/",
  "output/jm_candidates_500.slick"
)
file <- tempfile(fileext = ".slick")
download.file(url, file, mode = "wb")

slick <- readRDS(file)
Check(slick)
App(slick = slick)

1.3 Make it a selectable case study

Slick’s case-study menu is populated from the Blue-Matter/SlickLibrary repository. To request direct integration, copy the file to Slick_Objects/SPRFMO_Jack_Mackerel.slick in a fork of that repository and submit a pull request. Once the SlickLibrary maintainers merge it, SPRFMO Jack Mackerel will appear in the hosted app’s case-study menu. See the official contribution instructions.

2 What is needed

The conversion can use either:

the table of performance results produced by jmMSE; or
a saved jmMSE results object that already contains those performance results.

The results must include the eight CMPs listed below, the operating-model and stock names, simulation year and replicate, and the calculated performance measures. At a minimum, the results must include spawning biomass relative to $SB_{MSY}$, fishing mortality relative to $F_{MSY}$, and catch. Interannual catch change (IACC) is included when available. The published Slick file also requires vulnerable biomass relative to its 2025 level and relative to equilibrium vulnerable biomass at the fishing mortality that produces MSY.

The current published comparison uses the completed 500-posterior-draw runs through 2050. The eight CMPs are retained across the refinement and robustness archives.

model/candidates/reference/: HS+20, HS-20, PR+20, and PR-20; and
model/candidates/additional-change-limits/reference/: HSsym, HS-30, PRsym, and PR-30.

The augmented six-metric tables used for the published Slick file are retained in this repository as output/candidate-performance-500/reference/performance_with_vb.rds and output/candidate-performance-500/robustness/performance_with_vb.rds.

These are reference-operating-model results. They are suitable for checking the candidate calculations and comparing the rule and data contrasts, but they are not a substitute for robustness testing across the full set of operating models.

The corresponding nine-OM robustness results are stored in jmMSE/model/candidates/robustness/ and jmMSE/model/candidates/additional-change-limits/robustness/. Together they contain all eight CMPs, 500 posterior draws, and annual results through 2050 for every operating model.

3 CMPs included

HS-20 (MP43) and PR-20 (MP44) are the focal cases. HS+20 (MP29) and PR+20 (MP32) are retained as sensitivity cases, and four additional variants test symmetric and 30%-decrease constraints within the same two HCR families.

CMP	Role in the comparison
HS+20 (MP29)	Sensitivity; hockey-stick; -15%/+20% annual limits; trigger 2.125.
HS-20 (MP43)	Focal; hockey-stick; -20%/+15% annual limits; trigger 1.9375.
HSsym (MP45)	Comparison; hockey-stick; -15%/+15% annual limits; trigger 2.0625.
HS-30 (MP47)	Comparison; hockey-stick; -30%/+20% annual limits; trigger 2.0000.
PR+20 (MP32)	Sensitivity; power ramp; -15%/+20% annual limits; trigger 1.640625.
PR-20 (MP44)	Focal; power ramp; -20%/+15% annual limits; trigger 1.6015625.
PRsym (MP46)	Comparison; power ramp; -15%/+15% annual limits; trigger 1.6250.
PR-30 (MP48)	Comparison; power ramp; -30%/+20% annual limits; trigger 1.578125.

The MP filter includes one-click selections for the focal pair, the +20 sensitivity pair, each HCR family, each annual-limit design, all CMPs, and each individual CMP. All eight CMPs were refined under reference OM11 toward a mean dynamic Kobe-green probability of 0.60 over 2041–2050 using the same 500 posterior draws. The resulting probabilities range from 0.592 to 0.610.

4 Run or extend the reference comparison

From the jmMSE folder on branch dev_jim:

Rscript run_candidate_mps.R \
  --candidates=tun29,tun43,tun45,tun47,tun32,tun44,tun46,tun48 \
  --mode=reference \
  --cores=1

The candidate codes come from candidate_mps.csv. Additional hockeystick or powerramp trials can be included for contrast, for example:

Rscript run_candidate_mps.R \
  --candidates=tun25,tun29,tun32 \
  --mode=reference \
  --cores=1

The runner uses the recorded definitions in candidate_mps.csv together with the original estimator and rule functions in jmMSE. Shortcut-assessment errors are generated from the original error model using fixed seeds so the comparison can be repeated. Each CMP starts from the same fixed random stream within an operating model, so adding or reordering CMPs does not change the other CMP results. Moving to a newer combined-index implementation should be treated as a method change requiring the operating models to be rebuilt and the candidates to be retuned and revalidated.

The validated reference run used R 4.6.1 with FLCore 2.6.30, FLFishery 0.4.0, mse 2.4.8.9004, msemodules 0.2.2, and FLjjm 0.3.4. The run folder contains the complete sessionInfo.txt record.

5 Prepare the combined Slick file

From the main jmMSE26 folder:

Show implementation detail

source("R/build_candidate_slick.R")

reference_slick <- build_candidate_slick(
  performance_file =
    "output/candidate-performance-500/reference/performance_with_vb.rds",
  out_file = NULL,
  historical_om_files = candidate_reference_om_files("../jmMSE"),
  require_vb = TRUE
)

The script organizes the results by:

management procedure (one of the eight CMP comparison cases);
operating model and stock;
simulation replicate and year; and
performance measure, including conventional $SB/SB_{MSY}$, the dynamic spawning-biomass ratio used for tuning, $F/F_{MSY}$, plus $VB/VB_{2025}$ and $VB/VB_{\mathrm{MSY}}$.

It also prepares three summary views using the comparison periods already used in jmMSE:

Boxplots show the distribution among simulation replicates in the final year, 2050.
Quilts compare stock status, fishing pressure, catch, and catch stability among operating models and MPs. They also include both vulnerable-biomass ratios. Long-term results use the SC14 tuning window, 2041–2050; short-term catch uses 2026–2030.
Trade-off plots use the same periods to compare conservation, catch, and catch-stability measures directly, including both vulnerable-biomass ratios.

The vulnerable-biomass time series are available from 2024 through 2050; earlier years are intentionally missing. VB/VB[2025] uses each simulated stock’s 2025 vulnerable biomass as its denominator. VB/VB[MSY] uses equilibrium vulnerable biomass at the fishing mortality that produces MSY, not MSY catch.

Kobe quadrant probabilities are calculated from the dynamic spawning-biomass ratio and $F/F_{MSY}$. The dynamic denominator applies the equilibrium $SB_{MSY}/SB_0$ fraction to projected unfished spawning biomass in each year and iteration, matching the SC14 tuning statistic. Conventional $SB/SB_{MSY}$ remains available as a separate time series and boxplot metric. When $F_{MSY}$ is zero, $F/F_{MSY}$ is undefined and is treated as missing. Finite $F/F_{MSY}$ values above 4 are shown at 4 so that a small number of post-collapse numerical values do not make the plots unreadable. This is a display rule only: the original jmMSE performance tables are not changed.

The Slick quilt’s probability of catch below 270 kt is calculated over all valid iteration-year outcomes during 2041–2050. For example, if 10% of iterations are below 270 kt in five of those ten years and all iterations are above it in the other five, the displayed probability is 5%.

The OM metadata is kept in exactly the same order as the OM dimension in each Slick value array. This ensures that Northern and Southern stock results retain the correct labels in the two-stock operating models.

Before saving the Slick file, the script checks that the expected dimensions and labels agree. The technical checks are shown in the code for analysts who maintain the export.

The reference is labelled om11 in Slick. Its source name, h1_0.16, remains in the OM metadata for traceability.

5.1 Add the robustness sets

The robustness operating models and candidate runs can be reproduced from the recorded 500-draw refinement controls and robustness workflow.

Rscript prepare_candidate_robustness_oms.R
Rscript run_candidate_mps.R \
  --candidates=tun29,tun43,tun45,tun47,tun32,tun44,tun46,tun48 \
  --mode=robustness \
  --cores=1

Then build the robustness component and combine it with the reference:

Show implementation detail

source("R/build_candidate_slick.R")

robustness_slick <- build_candidate_slick(
  performance_file =
    "output/candidate-performance-500/robustness/performance_with_vb.rds",
  out_file = NULL,
  historical_om_files = candidate_robustness_om_files("../jmMSE"),
  require_vb = TRUE
)

slick <- build_combined_candidate_slick(
  reference_slick = reference_slick,
  robustness_slick = robustness_slick,
  out_file = "output/jm_candidates_500.slick"
)

The resulting Slick file contains 14 OM–stock series: one reference, five single-stock robustness cases, and eight stock-specific series from four two-stock robustness cases. Slick retains the Southern and Northern results rather than averaging them.

The Set filter provides three main categories:

Reference: om11;
Robustness CJM: the five single-stock robustness OMs; and
Robustness 2-stock: the Southern and Northern results from the four two-stock robustness OMs.

The OM and Stock filters remain available for selecting individual models and stock components. One-click presets provide quick selections for the reference, all CJM cases, the CJM robustness cases, the two-stock robustness cases, all OMs, and each individual alternative OM. Selecting a two-stock OM includes both its Southern and Northern rows.

The dimensions are 500 posterior draws, 14 OM–stock series, eight management procedures, seven performance measures, and 81 annual values from 1970 through 2050. The seven measures are $SB/SB_{MSY}$, dynamic $SB/SB_{MSY}$, $F/F_{MSY}$, catch, IACC, $VB/VB_{2025}$, and $VB/VB_{\mathrm{MSY}}$. Each series uses its own conditioned history through 2025; projections under the eight CMPs begin in 2026.

Do not combine results produced under different package sets without first confirming that the candidate calculations are equivalent.

6 Inspect or launch

Show implementation detail

library(Slick)

slick <- readRDS("output/jm_candidates_500.slick")
Check(slick)
methods::validObject(slick)

dim(Value(Timeseries(slick)))
Metadata(MPs(slick))
Design(slick)

App(slick = slick)

7 What to save with the Slick file

Save the Slick file together with the original performance results, their digital fingerprint, the exact saved versions of jmMSE and jmMSE26, and the versions of the R packages used.

--- title: "Load candidates into Slick" --- {{< include ../_includes/sprfmo-banner.qmd >}} <div class="paper-title">Explore the Eight CMP Comparison Cases in Slick</div> Slick is an interactive tool for exploring MSE results. The simulation is run in `jmMSE`; this repository converts the resulting performance table into a Slick file. ## Open the current file ### In the hosted app 1. [Download the current 500-draw `jm_candidates_500.slick`](https://raw.githubusercontent.com/SPRFMO/jmMSE26/main/output/jm_candidates_500.slick). 2. Open the [Blue Matter Slick app](https://shiny.bluematterscience.com/app/slick). 3. If the app has stopped, select **Restart app**. 4. Under **Load your MSE Results**, select the downloaded `.slick` file. 5. When the Overview appears, use the MP and OM filters to explore the results. The current file opens with **All CMPs** selected, so all eight annual-limit variants should be visible immediately. If only HS+20 and PR+20 appear, open the **MPs** filter and select **All CMPs**. That indicates either that the **Sensitivity cases (HS+20 and PR+20)** preset is active or that an older copy of the same-named file was loaded. The current file contains eight CMPs. Its SHA-256 fingerprint is recorded in the traceability page for the published branch. The hosted app accepts file uploads but does not currently provide a field for loading a file directly from a GitHub URL. ### From R ```{r} #| eval: false install.packages("Slick") # needed only once library(Slick) url <- paste0( "https://raw.githubusercontent.com/SPRFMO/jmMSE26/", "main/", "output/jm_candidates_500.slick" ) file <- tempfile(fileext = ".slick") download.file(url, file, mode = "wb") slick <- readRDS(file) Check(slick) App(slick = slick) ``` ### Make it a selectable case study Slick's case-study menu is populated from the [`Blue-Matter/SlickLibrary`](https://github.com/Blue-Matter/SlickLibrary) repository. To request direct integration, copy the file to `Slick_Objects/SPRFMO_Jack_Mackerel.slick` in a fork of that repository and submit a pull request. Once the SlickLibrary maintainers merge it, **SPRFMO Jack Mackerel** will appear in the hosted app's case-study menu. See the [official contribution instructions](https://slick.bluematterscience.com/articles/DevelopersGuide.html#slick-object-library). ## What is needed The conversion can use either: - the table of performance results produced by `jmMSE`; or - a saved `jmMSE` results object that already contains those performance results. The results must include the eight CMPs listed below, the operating-model and stock names, simulation year and replicate, and the calculated performance measures. At a minimum, the results must include spawning biomass relative to $SB_{MSY}$, fishing mortality relative to $F_{MSY}$, and catch. Interannual catch change (IACC) is included when available. The published Slick file also requires vulnerable biomass relative to its 2025 level and relative to equilibrium vulnerable biomass at the fishing mortality that produces MSY. The current published comparison uses the completed 500-posterior-draw runs through 2050. The eight CMPs are retained across the refinement and robustness archives. - `model/candidates/reference/`: HS+20, HS-20, PR+20, and PR-20; and - `model/candidates/additional-change-limits/reference/`: HSsym, HS-30, PRsym, and PR-30. The augmented six-metric tables used for the published Slick file are retained in this repository as `output/candidate-performance-500/reference/performance_with_vb.rds` and `output/candidate-performance-500/robustness/performance_with_vb.rds`. These are reference-operating-model results. They are suitable for checking the candidate calculations and comparing the rule and data contrasts, but they are not a substitute for robustness testing across the full set of operating models. The corresponding nine-OM robustness results are stored in `jmMSE/model/candidates/robustness/` and `jmMSE/model/candidates/additional-change-limits/robustness/`. Together they contain all eight CMPs, 500 posterior draws, and annual results through 2050 for every operating model. ## CMPs included HS-20 (MP43) and PR-20 (MP44) are the focal cases. HS+20 (MP29) and PR+20 (MP32) are retained as sensitivity cases, and four additional variants test symmetric and 30%-decrease constraints within the same two HCR families. | CMP | Role in the comparison | |---|---| | HS+20 (MP29) | Sensitivity; hockey-stick; -15%/+20% annual limits; trigger 2.125. | | HS-20 (MP43) | **Focal**; hockey-stick; -20%/+15% annual limits; trigger 1.9375. | | HSsym (MP45) | Comparison; hockey-stick; -15%/+15% annual limits; trigger 2.0625. | | HS-30 (MP47) | Comparison; hockey-stick; -30%/+20% annual limits; trigger 2.0000. | | PR+20 (MP32) | Sensitivity; power ramp; -15%/+20% annual limits; trigger 1.640625. | | PR-20 (MP44) | **Focal**; power ramp; -20%/+15% annual limits; trigger 1.6015625. | | PRsym (MP46) | Comparison; power ramp; -15%/+15% annual limits; trigger 1.6250. | | PR-30 (MP48) | Comparison; power ramp; -30%/+20% annual limits; trigger 1.578125. | The MP filter includes one-click selections for the focal pair, the +20 sensitivity pair, each HCR family, each annual-limit design, all CMPs, and each individual CMP. All eight CMPs were refined under reference OM11 toward a mean dynamic Kobe-green probability of 0.60 over 2041–2050 using the same 500 posterior draws. The resulting probabilities range from 0.592 to 0.610. ## Run or extend the reference comparison From the `jmMSE` folder on branch `dev_jim`: ```sh Rscript run_candidate_mps.R \ --candidates=tun29,tun43,tun45,tun47,tun32,tun44,tun46,tun48 \ --mode=reference \ --cores=1 ``` The candidate codes come from `candidate_mps.csv`. Additional hockeystick or powerramp trials can be included for contrast, for example: ```sh Rscript run_candidate_mps.R \ --candidates=tun25,tun29,tun32 \ --mode=reference \ --cores=1 ``` The runner uses the recorded definitions in `candidate_mps.csv` together with the original estimator and rule functions in `jmMSE`. Shortcut-assessment errors are generated from the original error model using fixed seeds so the comparison can be repeated. Each CMP starts from the same fixed random stream within an operating model, so adding or reordering CMPs does not change the other CMP results. Moving to a newer combined-index implementation should be treated as a method change requiring the operating models to be rebuilt and the candidates to be retuned and revalidated. The validated reference run used R 4.6.1 with FLCore 2.6.30, FLFishery 0.4.0, mse 2.4.8.9004, msemodules 0.2.2, and FLjjm 0.3.4. The run folder contains the complete `sessionInfo.txt` record. ## Prepare the combined Slick file From the main `jmMSE26` folder: ```{r} #| eval: false source("R/build_candidate_slick.R") reference_slick <- build_candidate_slick( performance_file = "output/candidate-performance-500/reference/performance_with_vb.rds", out_file = NULL, historical_om_files = candidate_reference_om_files("../jmMSE"), require_vb = TRUE ) ``` The script organizes the results by: - management procedure (one of the eight CMP comparison cases); - operating model and stock; - simulation replicate and year; and - performance measure, including conventional $SB/SB_{MSY}$, the dynamic spawning-biomass ratio used for tuning, $F/F_{MSY}$, plus $VB/VB_{2025}$ and $VB/VB_{\mathrm{MSY}}$. It also prepares three summary views using the comparison periods already used in `jmMSE`: - **Boxplots** show the distribution among simulation replicates in the final year, 2050. - **Quilts** compare stock status, fishing pressure, catch, and catch stability among operating models and MPs. They also include both vulnerable-biomass ratios. Long-term results use the SC14 tuning window, 2041–2050; short-term catch uses 2026–2030. - **Trade-off plots** use the same periods to compare conservation, catch, and catch-stability measures directly, including both vulnerable-biomass ratios. The vulnerable-biomass time series are available from 2024 through 2050; earlier years are intentionally missing. `VB/VB[2025]` uses each simulated stock's 2025 vulnerable biomass as its denominator. `VB/VB[MSY]` uses equilibrium vulnerable biomass at the fishing mortality that produces MSY, not MSY catch. Kobe quadrant probabilities are calculated from the dynamic spawning-biomass ratio and $F/F_{MSY}$. The dynamic denominator applies the equilibrium $SB_{MSY}/SB_0$ fraction to projected unfished spawning biomass in each year and iteration, matching the SC14 tuning statistic. Conventional $SB/SB_{MSY}$ remains available as a separate time series and boxplot metric. When $F_{MSY}$ is zero, $F/F_{MSY}$ is undefined and is treated as missing. Finite $F/F_{MSY}$ values above 4 are shown at 4 so that a small number of post-collapse numerical values do not make the plots unreadable. This is a display rule only: the original `jmMSE` performance tables are not changed. The Slick quilt's probability of catch below 270 kt is calculated over all valid iteration-year outcomes during 2041--2050. For example, if 10% of iterations are below 270 kt in five of those ten years and all iterations are above it in the other five, the displayed probability is 5%. The OM metadata is kept in exactly the same order as the OM dimension in each Slick value array. This ensures that Northern and Southern stock results retain the correct labels in the two-stock operating models. Before saving the Slick file, the script checks that the expected dimensions and labels agree. The technical checks are shown in the code for analysts who maintain the export. The reference is labelled `om11` in Slick. Its source name, `h1_0.16`, remains in the OM metadata for traceability. ### Add the robustness sets The robustness operating models and candidate runs can be reproduced from the recorded 500-draw refinement controls and robustness workflow. ```sh Rscript prepare_candidate_robustness_oms.R Rscript run_candidate_mps.R \ --candidates=tun29,tun43,tun45,tun47,tun32,tun44,tun46,tun48 \ --mode=robustness \ --cores=1 ``` Then build the robustness component and combine it with the reference: ```{r} #| eval: false source("R/build_candidate_slick.R") robustness_slick <- build_candidate_slick( performance_file = "output/candidate-performance-500/robustness/performance_with_vb.rds", out_file = NULL, historical_om_files = candidate_robustness_om_files("../jmMSE"), require_vb = TRUE ) slick <- build_combined_candidate_slick( reference_slick = reference_slick, robustness_slick = robustness_slick, out_file = "output/jm_candidates_500.slick" ) ``` The resulting Slick file contains 14 OM–stock series: one reference, five single-stock robustness cases, and eight stock-specific series from four two-stock robustness cases. Slick retains the Southern and Northern results rather than averaging them. The `Set` filter provides three main categories: - **Reference**: om11; - **Robustness CJM**: the five single-stock robustness OMs; and - **Robustness 2-stock**: the Southern and Northern results from the four two-stock robustness OMs. The OM and Stock filters remain available for selecting individual models and stock components. One-click presets provide quick selections for the reference, all CJM cases, the CJM robustness cases, the two-stock robustness cases, all OMs, and each individual alternative OM. Selecting a two-stock OM includes both its Southern and Northern rows. The dimensions are 500 posterior draws, 14 OM–stock series, eight management procedures, seven performance measures, and 81 annual values from 1970 through 2050. The seven measures are $SB/SB_{MSY}$, dynamic $SB/SB_{MSY}$, $F/F_{MSY}$, catch, IACC, $VB/VB_{2025}$, and $VB/VB_{\mathrm{MSY}}$. Each series uses its own conditioned history through 2025; projections under the eight CMPs begin in 2026. Do not combine results produced under different package sets without first confirming that the candidate calculations are equivalent. ## Inspect or launch ```{r} #| eval: false library(Slick) slick <- readRDS("output/jm_candidates_500.slick") Check(slick) methods::validObject(slick) dim(Value(Timeseries(slick))) Metadata(MPs(slick)) Design(slick) App(slick = slick) ``` ## What to save with the Slick file Save the Slick file together with the original performance results, their digital fingerprint, the exact saved versions of `jmMSE` and `jmMSE26`, and the versions of the R packages used.