BQP Orbit Propagation Benchmark: Speed, Uncertainty, and Operator Trust

BQP says that, under a U.S. Space Force and SpaceWERX SBIR effort extending SDA TAP Lab work, its physics AI propagated more than 3,000 space objects in under one second—about 250 times faster than open-source Orekit—by compressing the model. This is a company-reported benchmark. It is not an independently audited result, a universal comparison with Orekit, or proof of operational superiority. The distinction is fundamental. Orbit propagation is not one fixed task: results depend on the state source, epoch, propagation horizon, force model, atmosphere assumptions, numerical tolerances, covariance representation, hardware, software version, and requested output. A meaningful benchmark must make those choices inspectable.

Propagation Is a Decision Input

Propagation estimates where an object may be at future times from a state estimate and a dynamical model. In space domain awareness, those predictions support catalog updates, sensor scheduling, conjunction assessment, anomaly investigation, and maneuver analysis. An operator does not consume a trajectory for its own sake. They use it to decide whether to observe again, correlate an observation, screen a pair, alert an owner, recommend a maneuver, or defer action. Each decision has a different tolerance for approximation. A broad catalog refresh can prioritize speed and coverage. A close-approach decision needs calibrated uncertainty and conservative escalation. A method that is excellent at ranking candidates may be unsuitable as the final probability-of-collision engine.

Latency is therefore a chain, not a kernel runtime. New measurements must be ingested, associated, quality-controlled, and used to update a state. The system then propagates candidate trajectories, computes screening quantities, prioritizes results, presents evidence, receives authority where necessary, and records the disposition. A sub-second model may create value by making a larger portion of that chain responsive, but only if data and workflow delays do not dominate. Teams should report end-to-end latency percentiles, not just an isolated algorithm's best run. They should also show behavior during sensor outages, late data, maneuvers, and a changing space environment.

Accuracy and Covariance Cannot Be Afterthoughts

A predicted position without uncertainty can create false confidence. Covariance describes uncertainty and correlation in an estimated state; it affects the size, orientation, and evolution of the region in which an object may plausibly be. In conjunction work, relative uncertainty and encounter geometry influence risk estimates as much as nominal miss distance. A compressed model can be valuable only if it preserves, calibrates, or conservatively bounds the uncertainty relevant to its use. Point errors should be paired with distributions, residuals, confidence calibration, and failure modes. A low average position error can still hide unacceptable tails, especially when a system ranks rare high-consequence events.

Orekit Is a Reference, Not a Straw Man

Orekit is open-source space-dynamics software with broad capabilities and configurable models. Comparing against it can be useful, but only if configurations are comparable. A general-purpose, high-fidelity propagation setup may solve a different problem from a compressed model tuned for rapid screening. The correct conclusion from a faster narrow workload is that the narrow workload may be worth deploying alongside a reference path—not that one package is categorically superior. Independent replication should disclose versions, compiler and hardware choices, number of objects, horizon, output requirements, accuracy measures, and whether initialization, data movement, and uncertainty calculations are included.

Catalog updates add another complication: the input catalog is never static. Measurements arrive with different quality, sensors have biases, and object identities can be uncertain. A rapid propagator cannot repair a poor state estimate simply by calculating faster. Its best role may be to help decide where new observations are most valuable or which cases warrant more expensive estimation. Evaluation should therefore include residual behavior against later observations, track-maintenance outcomes, and the rate at which the tool correctly prioritizes scarce sensor time. These outcome metrics are more informative than a raw objects-per-second figure.

Conjunction screening is also a funnel. A broad first pass can identify candidate close approaches, followed by refined propagation, covariance analysis, analyst review, owner coordination, and sometimes maneuver planning. Different stages can legitimately use different models. The safety property comes from explicit handoffs and conservative thresholds. If a compressed model is used to discard cases, its false-negative behavior is particularly important. If it is used to prioritize cases, ranking quality and calibration matter. A vendor should say which role it tested rather than allowing a screening benchmark to imply validated collision-avoidance authority.

Physics AI can compress a model by learning or approximating expensive relationships while retaining physical structure. That can reduce computation, but it introduces model-governance questions. What training data and simulation regimes were used? Does the method extrapolate across orbit classes, solar conditions, drag changes, maneuvers, and sparse observations? What monitors detect drift? Can an analyst reproduce a result months later using the same model, data, and configuration? These are not bureaucratic additions. In a contested or safety-sensitive environment, they are prerequisites for an operator to distinguish a useful acceleration from an opaque source of risk.

Operator Trust Is a System Property

Trust grows when a tool is predictable about what it knows and does not know. A useful interface should present the recommendation, the input-data age and provenance, uncertainty, model version, comparable reference result where available, and reasons for escalation. It should support replay after an event and permit users to inspect differences when a model changes. Human authority should be explicit for consequential action. Automation may triage routine cases, but a mission owner needs a defined safe state when data quality falls, confidence is low, or a novel maneuver breaks assumptions.

Reproducibility is an operational capability. A later review must recreate not only a trajectory but the exact data snapshot, constants, force settings, model weights, processor implementation, and threshold that produced it. This allows a team to distinguish a real environmental change from a software regression. It also supports collaboration between government, commercial operators, and independent analysts who may use different tools. Where proprietary models prevent full disclosure, vendors can still provide test vectors, versioned interfaces, uncertainty reports, and independently witnessed evaluations. Trust is earned by making disagreements diagnosable.

Inputs and Force Models Set the Ceiling

Propagation begins with an estimated state, its epoch and covariance, and assumptions about the environment. Relevant effects can include central gravity, nonspherical gravity, atmospheric drag, solar radiation pressure, third-body gravity, maneuvers, attitude, and density or space-weather models, depending on the orbit and time horizon. No acceleration technique can recover information absent from the initial state. Evaluation must therefore separate propagation error from measurement and estimation error. It should disclose orbit regimes, object characteristics, prediction horizons, maneuver treatment, data age, and the force-model fidelity selected for both the candidate method and its reference. A fast result with a simplified model may be appropriate for a first-pass screen, but the simplification must be explicit.

False positives and false negatives have different operational costs. A false positive can consume analyst time, sensor capacity, owner coordination, and perhaps an unnecessary maneuver assessment. A false negative can leave a consequential case unexamined. Screening thresholds should reflect that asymmetry and be tested with representative encounter distributions, not only average trajectory errors. An evaluation can measure recall of cases that a trusted high-fidelity process escalates, precision of alerts, calibration of risk bands, and time saved per analyst disposition. It should include difficult updates such as new observations, planned and unplanned maneuvers, decaying objects, and periods when the catalog is incomplete.

Deployment Architecture and Evaluation Plan

A prudent deployment architecture separates ingest, estimation, rapid screening, high-fidelity confirmation, presentation, and audit storage. It permits the rapid model to refresh candidates at a frequent cadence while a reference pipeline verifies cases above defined thresholds. Updates should be versioned and released first in shadow mode. The evaluation plan should predeclare datasets, baselines, hardware, horizons, accuracy and covariance metrics, alert thresholds, end-to-end latency, and independent reviewers. It should then publish distributions across ordinary and stress conditions. Such a plan would make BQP's company-reported benchmark far more decision-relevant than a headline speed ratio and would show where the method helps rather than claiming universal replacement.

Cadence should be chosen from the decision cycle, not from the fastest possible refresh. Some objects may need immediate reassessment after a high-quality observation or maneuver notice; others can wait for a scheduled catalog batch. The system should prioritize by uncertainty growth, encounter sensitivity, sensor availability, and mission consequence, then retain enough compute capacity for surges. A measured pilot can compare the candidate pipeline with the incumbent across several weeks of real or replayed data, recording missed escalations, excess alerts, analyst time, and final dispositions. That operational comparison is the strongest way to establish value while preserving cautious use of a novel model.

Results should also be segmented by orbit class, prediction horizon, observation quality, and event severity. Aggregate averages can conceal the cases that matter most. A deployment decision should be based on the worst credible performance within the approved use envelope, clear escalation outside it, and demonstrated recovery when data or compute services fail. This approach makes latency useful without allowing it to override accuracy, accountability, or operator authority.

Independent exercises should include realistic handoffs between automated triage and experienced analysts.

1. Publish a locked benchmark specification before measuring speed.
2. Evaluate accuracy and covariance calibration across orbit classes and horizons.
3. Use held-out and stress scenarios, including maneuvers and degraded observations.
4. Compare end-to-end workflow latency, not only propagation time.
5. Retain an independent high-fidelity reference and documented fallback.
6. Audit model, data, configuration, and operator disposition for every consequential case.

The BlacKnight Take

BQP's reported result identifies a potentially valuable operational wedge: use physics AI to refresh a large candidate set quickly, then focus expensive computation and human attention where it matters. The reported 250-times figure is not yet the conclusion; it is the beginning of an evaluation plan. Its credibility depends on comparable configurations, calibrated uncertainty, and reproducible independent testing against the decisions users must make.

The winning product will not be the fastest propagator in isolation. It will be the system that shortens the decision loop without hiding uncertainty, supports a conservative fallback, and earns trust after edge cases. That is why BQP's technical benchmark and its enterprise integration strategy must be judged together.

Frequently Asked Questions