AISimulate - FPM Predictors

Regression average MAPE

—

FPM (KV warmup on) average MAPE

—

FPM (KV warmup off) average MAPE

—

How the predictors work

One online learner and two measured-library predictors.

Regression

Learns from preceding observations per worker. Predict first, then tune on the observed latency.

Input: Previous worker observations
Mode: fpm_regression

Python example

config = replace(
    base_config,
    estimation_mode="fpm_regression",
    fallback_policy="deny",
)
model = RustForwardPassPerfModel.best_available(config)
latency_ms = model.estimate_forward_pass_time_ms(workload)
model.tune_with_fpms(observations)

FPM (KV warmup on)

Predicts from whole-forward measurements using interpolation and supported analytical scaling.

Input: Library collected with KV warmup
Mode: fpm_interpolation

Python example

config = replace(
    base_config,
    estimation_mode="fpm_interpolation",
    fallback_policy="deny",
    systems_paths=(warmup_library_root,),
)
model = RustForwardPassPerfModel.best_available(config)
latency_ms = model.estimate_forward_pass_time_ms(workload)

FPM (KV warmup off)

Uses the same interpolation and analytical scaling with measurements collected without KV warmup.

Input: Library collected without KV warmup
Mode: fpm_interpolation

Python example

config = replace(
    base_config,
    estimation_mode="fpm_interpolation",
    fallback_policy="deny",
    systems_paths=(no_warmup_library_root,),
)
model = RustForwardPassPerfModel.best_available(config)
latency_ms = model.estimate_forward_pass_time_ms(workload)

KV warmup describes how the input library was collected. It is not a prediction-time Boolean.

Examples use replace from dataclasses and RustForwardPassPerfModel from aisimulate_core.sdk. Supply a ForwardPassPerfModelConfig as base_config with matching model, hardware, backend, topology, and worker role; use a separate regression model per worker.

See the configuration API and Gym regression settings for the evaluated regression policy, which differs from SDK defaults.

Summary cards average configuration MAPEs equally. Model table rows are weighted by successful prediction count.

MAPE vs silicon — successful forward-pass predictions

Accuracy grouped by model and configuration. Activate a column heading to sort model groups. Expand a model to inspect configurations.

Loading accuracy matrix…

* Serving layout is unknown; Agg is shown by default. N/A means not applicable.