Reproduce FAIR xwOBA

Prepared-data contract, public evidence, release audit, and site build

How to audit the FAIR xwOBA v0.1.0 evidence release and reproduce the rolling model from user-supplied prepared data.

The public repository has two reproducibility levels:

1. Evidence reproduction is self-contained. It validates the committed FAIR release, rebuilds every figure and table, runs publication-boundary tests, and renders the three-page site.
2. Model reproduction requires a user-supplied prepared Statcast contact dataset. Raw third-party baseball data and row-level predictions are not redistributed.

The public evidence is immutable development release v0.1.0. It contains aggregate tables plus exactly two predeclared illustrative player-season summaries. It contains no player IDs, full player leaderboard, contact-level predictions, model objects, raw traits, bootstrap draws, or machine-local paths.

Quick verification

From the repository root:

Rscript --vanilla tests/testthat.R
Rscript --vanilla scripts/05_audit_public_release.R
Rscript --vanilla scripts/06_render_site.R

Or use:

make verify

The render helper validates every SHA-256 digest before reading release values, generates the hero/social assets, renders exactly three HTML pages, and writes optional external links only when valid URLs are configured.

Public release downloads

All article values come from results/releases/fair_xwoba_v0_1_0_development_2019_2024.

Table 1: Files owned by the FAIR v0.1.0 release manifest.

File	SHA-256
accuracy_calibration_gates.csv	5325b0744dfb007887b9edd267bdcbf40db482fe8425dc0cb283f03aaef4dc5c
aggregate_model_comparison.csv	e3f020825d125cd4b906f1b66e5ecaa429c220c552d50e3d3b1cc51be8432c9a
annual_model_comparison.csv	c32bd9dbf604c43260e2674530be054195650f9efe23c11ec1d77ed061898012
archetype_examples.csv	37d1267dd9644584c3c21a56ea218a93343fe48f5f7bc5378f72a5d2d1770236
attribute_persistence_decomposition.csv	22cbae26f82b44e33790e72667dfa538481384d3490fc63e96dc5e705e82f27f
lagged_sprint_metric_contrasts.csv	4f7905d434e7cb98c045718651659613561b8fed52deea6e8f4f9840c8f2ce46
model_differences.csv	9694c0d9745889c08282e863600be62abcf181000aca78a0ef8501bf4b2bec6e
neutrality_diagnostics.csv	aba287dcf324aac620543f4816d05bc6c7c83f2a8b073dff7d6a8ed3a0b098b2
origin_persistence_binned.csv	9c3b5b5c03bc35146c5f84ae01b4932846d04b3465981a748602865d72d12244
origin_trait_associations.csv	21ee89484c2ed86feb8ac7055b0b3e9c09e77e043cbaa5956db3e17b482c3747
provenance_receipt.csv	db61c4f07a6411d11b4af7b815ebe25b160ccd6ed0c2f443119fda2b1c0e149f
residual_persistence.csv	2967f4bb4c4e71cfa777d3bccb3210b8562c9b995318f28d5d9c60fe230ad7e9
selected_model_specification.json	685e0dbd627eabeaf2b01fafe38fc55b61641b83a968e98371f7ed03eb8fa32a
sprint_mechanism.csv	4252242405d4d8ef714aa812d5cb8ceee5d74a5fd60b3868758ee2d8f72c2423
sprint_regions.csv	d8c24da74b29e5b392ba742c9d5e0547fbac1a1e6f7b1b0521a895290e9e1ab8
sprint_run_value.csv	317264300b6ec256c73551ff3e62dfa2d3674d14c85d642bf96eec6db7871364
trait_associations.csv	0b9d15b94698c47c0dd5c9bcb5dea303fbb28238afff6e01c808237be9d0f16b
trait_scaling.csv	2356b008e387b59e6f90baf0a76910e40405ecf7b4911a74108cc984a346178c

The manifest itself is available as release_manifest.json. Its source receipts pin the independently verified model, neutrality, Sprint Speed, persistence, and original-study inputs.

Prepared-data interface

The repository does not promise raw-data acquisition. Model reproduction begins from three separate prepared files so prediction data and post-score identity remain physically distinct.

model_contacts.parquet

Required keys and chronology fields:

• season
• game_pk
• at_bat_number
• game_date
• calendar_month

Required target and comparator fields:

• target_index, using zero-based indices 0–4
• actual_woba_value, matching 0, .90, 1.25, 1.60, or 2.00
• statcast_xwoba

Required FAIR inputs:

• launch_speed_mph, launch_angle_deg
• spray_unit_x, spray_unit_y
• hang_time_proxy_s
• temperature_delta_f
• tailwind_exposure, headwind_exposure, crosswind_exposure
• park_canonical_id, roof_status, shift_rule_era
• if_alignment_v1, of_alignment_v1

The file is rejected if it contains player IDs, names, Sprint Speed, age, body measurements, position, handedness, team, fielder identity, or downstream event descriptors.

contact_identity.parquet

Contains the three contact keys plus player_id and player_name. The residual script may not open it until the completed prediction artifact has been written and hashed.

player_traits.parquet

Contains player_id, season, sprint_speed, age, height_in, and weight_lb. It is post-score audit input only.

Synthetic examples with fake names and outcomes are supplied under data/demo/. They exercise the data boundary and pipeline mechanics but are not intended to recreate the published baseball estimates.

Configuration

Copy the checked example configuration and edit only paths and local execution resources:

cp config/fair_xwoba.example.yml config/fair_xwoba.local.yml

The canonical seed, seasons, model parameters, outcome values, calibration rules, and feature contract are frozen. Local configuration may change input locations, output locations, and core count; changing a scientific setting marks the run noncanonical.

Run the numbered sequence:

Rscript --vanilla scripts/00_check_environment.R \
  --config config/fair_xwoba.local.yml

Rscript --vanilla scripts/01_validate_prepared_data.R \
  --config config/fair_xwoba.local.yml

Rscript --vanilla scripts/02_run_rolling_fair.R \
  --config config/fair_xwoba.local.yml

Rscript --vanilla scripts/03_build_residual_audits.R \
  --config config/fair_xwoba.local.yml

The full rolling fit is CPU-intensive. GitHub Actions runs a compact synthetic smoke configuration, not the complete historical training job.

Prediction interface

Private local prediction artifacts distinguish:

• class_probability_value_0_00
• class_probability_value_0_90
• class_probability_value_1_25
• class_probability_value_1_60
• class_probability_value_2_00
• probability_implied_xwoba
• predicted_xwoba
• probability_calibration_method
• prior_scalar_calibration_method
• scalar_calibration_method
• local_probability_adjustment_method
• local_scalar_adjustment_method

Probabilities must be finite, bounded, and sum to one. Final scalar predictions must be finite and lie in \([0,2]\).

Exporting the evidence release

The release exporter accepts four independently verified private aggregate-audit directories:

Rscript --vanilla scripts/04_export_public_release.R \
  --intrinsic-output PATH_TO_VERIFIED_FAIR_OUTPUT \
  --neutrality-output PATH_TO_VERIFIED_NEUTRALITY_OUTPUT \
  --sprint-output PATH_TO_VERIFIED_SPRINT_OUTPUT \
  --attribute-output PATH_TO_VERIFIED_ATTRIBUTE_OUTPUT

Before opening source tables, the exporter verifies canonical manifest hashes and independent verification status. It then:

1. maps the internal candidate ID to the FAIR public alias;
2. filters every comparison to FAIR and Statcast;
3. rebuilds the ten-bin original-study summary from a hash-pinned source;
4. applies the fixed two-player-season selection rule after score freeze;
5. strips IDs and rejects an unexpected selection;
6. writes a new directory rather than overwriting a release; and
7. enumerates every file with SHA-256 in the manifest.

The original run metadata are represented only by a hash receipt because the private object contains machine-local paths.

Publication audit

The audit scans all files Git would publish, including untracked non-ignored files. It fails on:

• serialized data or model formats such as RDS, Parquet, UBJ, or XGBoost binaries;
• private experiment directories;
• row-level prediction or leaderboard filenames;
• files larger than 10 MB without an explicit exception;
• machine-specific home paths;
• credential-like strings;
• missing ignore rules for raw data, artifacts, model outputs, environments, and the rendered site;
• any release file absent from the manifest or with a mismatched SHA-256;
• any named-player output beyond the exact two-row archetype file.

The two illustrative seasons are a narrow exception, not a general permission to publish player-level residuals.

Continuous integration

Every push and pull request performs:

1. dependency setup;
2. synthetic prepared-data smoke validation;
3. unit and integration tests;
4. release and repository-boundary audit;
5. a clean three-page Quarto render;
6. HTML and navigation checks; and
7. upload of the rendered site artifact.

The Pages workflow deploys only after the validation workflow succeeds on main. The repository is intentionally kept local until the publication owner separately authorizes creation of the public GitHub remote.

Site links

config/site.yml records the canonical article URL, public GitHub repository, and Max Gebauer’s main site. The render helper validates the configured HTTP(S) navigation links and writes them into the rendered site. Blank optional links would create no empty navigation elements.

No comments, analytics, RSS feed, external fonts, player photographs, or third-party embeds are loaded.

Licensing and citation

Original code is MIT licensed. The article, original figures, and derived aggregate evidence are CC BY 4.0. Those licenses do not grant redistribution rights for third-party baseball data.

The preferred citation is the article, with the versioned software release included as supporting metadata in CITATION.cff. Development v0.1.0 remains immutable. A future v1.0 is reserved for a successful locked untouched-season confirmation.

Return to What xwOBA Misses or inspect the full model methods.