Skip to content
Prakruti

Validation

Model performance

Experimental research

How well the forecasting model actually performs, measured against earthquakes it never saw. Published whether the answer is flattering or not — a model that cannot beat a constant-rate baseline has demonstrated nothing, and saying so is the point of this page.

Backtest results by region

ETAS etas-mle-v1 — fitted on a training period, scored on a later period it never saw

Historical data
RegionTargetTest periodForecastsOccurredBrierBaselineSkillVerdict
Greece, the Aegean and western TürkiyeM5.0+ / 7d2021-12-29 → 2026-08-2924351 (21%)0.16090.1777+0.094beats baseline
Alaska and the AleutiansM5.0+ / 7d2022-01-03 → 2026-09-20245103 (42%)0.24190.2842+0.149beats baseline
Türkiye, the Caucasus and IranM5.0+ / 7d2022-01-03 → 2026-09-2124571 (29%)0.21290.2159+0.014no better than baseline
California and the Gulf of CaliforniaM5.0+ / 7d2021-11-04 → 2026-06-2424113 (5%)0.05100.0523+0.025no better than baseline
The CaribbeanM5.0+ / 7d2022-01-02 → 2026-09-20245106 (43%)0.25570.2459-0.040no better than baseline
Chile and western ArgentinaM5.0+ / 7d2022-01-03 → 2026-09-21246150 (61%)0.24410.2714+0.101beats baseline
The Himalaya, Tibet and Hindu KushM5.0+ / 7d2022-01-02 → 2026-09-21246112 (46%)0.24820.2499+0.007no better than baseline
Indonesia and the Banda SeaM5.0+ / 7d2022-01-02 → 2026-09-21246238 (97%)0.03170.0318+0.002no better than baseline
Italy and the central MediterraneanM5.0+ / 7d2022-01-08 → 2026-07-3123712 (5%)0.04830.0481-0.004no better than baseline
Japan and the Ryukyu arcM5.0+ / 7d2022-01-02 → 2026-09-21246181 (74%)0.19030.2408+0.210beats baseline
Mexico and Central AmericaM5.0+ / 7d2022-01-01 → 2026-09-19245140 (57%)0.24560.2537+0.032no better than baseline
New ZealandM5.0+ / 7d2022-01-04 → 2026-09-2124540 (16%)0.13350.1775+0.248beats baseline
Peru, Ecuador and ColombiaM5.0+ / 7d2022-01-08 → 2026-09-21245111 (45%)0.24520.2478+0.011no better than baseline
Philippines and TaiwanM5.0+ / 7d2021-12-19 → 2026-08-31245187 (76%)0.18290.1807-0.012no better than baseline
Papua New Guinea, Solomon Islands and VanuatuM5.0+ / 7d2022-01-02 → 2026-09-20246235 (96%)0.04320.0442+0.023no better than baseline
Tonga, Kermadec and FijiM5.0+ / 7d2022-01-02 → 2026-09-20245232 (95%)0.04970.0517+0.039no better than baseline

Brier score is the mean squared error of the forecast probabilities against what happened — lower is better, and 0 is perfect. On its own it is close to meaningless, because a rare event can be forecast well simply by always predicting “no”.

Skill is what the score is worth relative to a constant-rate Poisson model fitted on the same training data. Positive means the model added something; zero or negative means it did not, whatever its raw score looks like.

Training and test periods are strictly separated in time, and each forecast conditions only on events up to its own issue time. Without both rules a backtest reports the model's ability to read the answer sheet.

Calibration — New Zealand

M5.0+ over 7 days, 245 forecasts

Historical data
Reliability
0.00342
lower is better — how far each bin sits from its forecast
Resolution
0.00625
higher is better — how far bins separate from the base rate
Log score
-0.4386
baseline -0.5450

A well-calibrated model puts its points on the diagonal: forecasts of 20% are followed by the event about a fifth of the time. A model can be sharp and badly calibrated, or calibrated and useless — a forecast of the long-run average every day is perfectly calibrated and tells you nothing. That is what resolution measures, and why both are here.

Fitted model parameters

Every stored fit, including the ones that failed their diagnostics

Historical data
RegionEventsμAαcpbn̄vs PoissonStatus
Greece, the Aegean and western Türkiye1,4570.0640.03591.340.00540.9481.340.79+891usable
Alaska and the Aleutians2,1930.1890.01411.540.00501.0820.950.56+1533usable
Türkiye, the Caucasus and Iran1,5560.1320.01291.990.01521.0791.150.53+1269usable
California and the Gulf of California2620.0220.01311.640.00161.0600.990.62+368usable
The Caribbean2,1210.2050.01651.410.00350.9311.250.42+470usable
Chile and western Argentina5,1240.5830.00971.940.02001.1321.250.28+2036usable
The Himalaya, Tibet and Hindu Kush2,6200.3590.00552.020.00971.2371.330.19+840usable
Indonesia and the Banda Sea15,6021.2130.02541.580.01351.0581.260.58+2943usable
Italy and the central Mediterranean1970.0240.00742.230.01811.2391.230.35+158usable
Japan and the Ryukyu arc10,8640.3780.04261.440.02491.0691.260.81+9651usable
Mexico and Central America3,5810.3170.01531.610.00670.9771.160.46+1078usable
New Zealand1,2840.0990.01672.010.03091.1931.190.54+1635usable
Peru, Ecuador and Colombia2,2520.2210.01821.110.00490.8731.160.44+240usable
Philippines and Taiwan6,8310.3740.03121.630.01361.0851.300.70+4176usable
Papua New Guinea, Solomon Islands and Vanuatu14,4550.7420.03121.540.02001.1131.110.76+6157usable
Tonga, Kermadec and Fiji15,2960.7290.03701.400.01601.0091.260.76+3421usable

μ background rate · A aftershock productivity · α magnitude scaling · c, p Omori-Utsu decay · b Gutenberg-Richter slope · n̄ branching ratio, the mean number of directly triggered earthquakes per earthquake over the model's horizon.

“vs Poisson” is the log-likelihood gain over a constant-rate model on the same data. A rejected fit is kept and shown rather than deleted; no forecast is issued from one.

4 of 16 regions fitted an Omori decay of p ≤ 1. Observed aftershock sequences usually sit near p = 1.1, and p ≤ 1 means the sequence has no finite total over infinite time. The likely cause is this implementation: triggering is summed over a bounded 730-day lookback, so the fit never pays the cost of a heavy tail and a shallow decay is cheap. Forecasts remain well defined because they use the same bounded horizon — but the fitted p should not be quoted as “the Omori p-value” for these regions. A longer lookback would very likely return a steeper value.

Greece, the Aegean and western Türkiye: Omori decay p = 0.948 is at or below 1, so the aftershock sequence has no finite total over infinite time. Forecasts remain well defined because triggering is summed over a bounded horizon, but the fitted decay is very shallow.

The Caribbean: Omori decay p = 0.931 is at or below 1, so the aftershock sequence has no finite total over infinite time. Forecasts remain well defined because triggering is summed over a bounded horizon, but the fitted decay is very shallow.

Mexico and Central America: Omori decay p = 0.977 is at or below 1, so the aftershock sequence has no finite total over infinite time. Forecasts remain well defined because triggering is summed over a bounded horizon, but the fitted decay is very shallow.

Peru, Ecuador and Colombia: Omori decay p = 0.873 is at or below 1, so the aftershock sequence has no finite total over infinite time. Forecasts remain well defined because triggering is summed over a bounded horizon, but the fitted decay is very shallow.

Tonga, Kermadec and Fiji: Over all future time the branching ratio would be 8.68 against 0.76 over the model's horizon — a shallow decay, so the sequence total depends heavily on where counting stops.

Forecast track record

Forecasts Prakruti actually issued, scored after their window closed

Historical data
Forecasts resolved
500
Mean Brier score
0.0435
Occurred
154 / 500
The CaribbeanM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 1%none occurred
The CaribbeanM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 11%none occurred
The CaribbeanM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 87%2 occurred
Italy and the central MediterraneanM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 0%none occurred
Italy and the central MediterraneanM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 1%none occurred
Italy and the central MediterraneanM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 17%none occurred
The Himalaya, Tibet and Hindu KushM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 1%none occurred
The Himalaya, Tibet and Hindu KushM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 11%none occurred
The Himalaya, Tibet and Hindu KushM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 91%5 occurred
Türkiye, the Caucasus and IranM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 1%none occurred
Türkiye, the Caucasus and IranM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 8%none occurred
Türkiye, the Caucasus and IranM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 70%1 occurred
Greece, the Aegean and western TürkiyeM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 0%none occurred
Greece, the Aegean and western TürkiyeM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 3%none occurred
Greece, the Aegean and western TürkiyeM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 53%none occurred
Alaska and the AleutiansM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 3%none occurred
Alaska and the AleutiansM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 21%none occurred
Alaska and the AleutiansM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 88%5 occurred
California and the Gulf of CaliforniaM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 0%none occurred
California and the Gulf of CaliforniaM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 3%none occurred
California and the Gulf of CaliforniaM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 23%none occurred
Mexico and Central AmericaM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 2%none occurred
Mexico and Central AmericaM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 21%none occurred
Mexico and Central AmericaM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 96%2 occurred
Peru, Ecuador and ColombiaM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 1%none occurred
Peru, Ecuador and ColombiaM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 16%none occurred
Peru, Ecuador and ColombiaM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 92%1 occurred
Chile and western ArgentinaM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 1%none occurred
Chile and western ArgentinaM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 22%none occurred
Chile and western ArgentinaM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 99%2 occurred
New ZealandM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 0%none occurred
New ZealandM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 5%none occurred
New ZealandM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 58%1 occurred
Tonga, Kermadec and FijiM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 3%none occurred
Tonga, Kermadec and FijiM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 38%none occurred
Tonga, Kermadec and FijiM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 100%12 occurred
Papua New Guinea, Solomon Islands and VanuatuM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 6%none occurred
Papua New Guinea, Solomon Islands and VanuatuM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 56%3 occurred
Papua New Guinea, Solomon Islands and VanuatuM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 100%32 occurred
Philippines and TaiwanM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 1%none occurred
Philippines and TaiwanM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 19%none occurred
Philippines and TaiwanM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 99%3 occurred
Indonesia and the Banda SeaM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 4%none occurred
Indonesia and the Banda SeaM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 53%none occurred
Indonesia and the Banda SeaM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 100%17 occurred
Japan and the Ryukyu arcM7.0+ / 30d2026-10-27 15:49:43 UTCforecast 1%none occurred
Japan and the Ryukyu arcM6.0+ / 30d2026-10-27 15:49:43 UTCforecast 24%1 occurred
Japan and the Ryukyu arcM5.0+ / 30d2026-10-27 15:49:43 UTCforecast 99%5 occurred
The CaribbeanM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 1%none occurred
The CaribbeanM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 11%none occurred
The CaribbeanM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 87%2 occurred
Italy and the central MediterraneanM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 0%none occurred
Italy and the central MediterraneanM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 1%none occurred
Italy and the central MediterraneanM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 17%none occurred
The Himalaya, Tibet and Hindu KushM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 1%none occurred
The Himalaya, Tibet and Hindu KushM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 11%none occurred
The Himalaya, Tibet and Hindu KushM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 91%5 occurred
Türkiye, the Caucasus and IranM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 1%none occurred
Türkiye, the Caucasus and IranM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 8%none occurred
Türkiye, the Caucasus and IranM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 70%1 occurred
Greece, the Aegean and western TürkiyeM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 0%none occurred
Greece, the Aegean and western TürkiyeM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 3%none occurred
Greece, the Aegean and western TürkiyeM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 53%none occurred
Alaska and the AleutiansM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 3%none occurred
Alaska and the AleutiansM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 22%none occurred
Alaska and the AleutiansM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 88%5 occurred
California and the Gulf of CaliforniaM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 0%none occurred
California and the Gulf of CaliforniaM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 3%none occurred
California and the Gulf of CaliforniaM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 23%none occurred
Mexico and Central AmericaM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 2%none occurred
Mexico and Central AmericaM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 21%none occurred
Mexico and Central AmericaM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 96%2 occurred
Peru, Ecuador and ColombiaM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 1%none occurred
Peru, Ecuador and ColombiaM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 16%none occurred
Peru, Ecuador and ColombiaM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 92%1 occurred
Chile and western ArgentinaM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 1%none occurred
Chile and western ArgentinaM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 22%none occurred
Chile and western ArgentinaM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 99%2 occurred
New ZealandM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 0%none occurred
New ZealandM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 5%none occurred
New ZealandM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 58%1 occurred
Tonga, Kermadec and FijiM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 3%none occurred
Tonga, Kermadec and FijiM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 38%none occurred
Tonga, Kermadec and FijiM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 100%11 occurred
Papua New Guinea, Solomon Islands and VanuatuM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 6%none occurred
Papua New Guinea, Solomon Islands and VanuatuM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 56%3 occurred
Papua New Guinea, Solomon Islands and VanuatuM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 100%32 occurred
Philippines and TaiwanM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 1%none occurred
Philippines and TaiwanM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 19%none occurred
Philippines and TaiwanM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 99%3 occurred
Indonesia and the Banda SeaM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 4%none occurred
Indonesia and the Banda SeaM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 53%none occurred
Indonesia and the Banda SeaM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 100%17 occurred
Japan and the Ryukyu arcM7.0+ / 30d2026-10-27 14:49:06 UTCforecast 1%none occurred
Japan and the Ryukyu arcM6.0+ / 30d2026-10-27 14:49:06 UTCforecast 24%1 occurred
Japan and the Ryukyu arcM5.0+ / 30d2026-10-27 14:49:06 UTCforecast 99%5 occurred
The CaribbeanM7.0+ / 30d2026-10-27 13:48:01 UTCforecast 1%none occurred
The CaribbeanM6.0+ / 30d2026-10-27 13:48:01 UTCforecast 11%none occurred
The CaribbeanM5.0+ / 30d2026-10-27 13:48:01 UTCforecast 87%2 occurred
Italy and the central MediterraneanM7.0+ / 30d2026-10-27 13:48:01 UTCforecast 0%none occurred

Source: Prakruti forecast registry · issued forecasts are never rewritten; only the outcome columns are filled in after the window closes