Define portability before converting
Decide what must remain equivalent: accepted input shapes, numeric outputs, latency, memory, batching, deterministic behavior or all of them. A model that loads but changes a downstream classification threshold is not portable for that product.
Document the source framework and checkpoint, preprocessing, postprocessing and expected tensor names. Many apparent runtime differences originate outside the graph because image normalization, tokenization or label mapping changed during the move.
Pin the graph contract
Record the exporter version and ONNX opset. Inspect the graph for operator domains, attributes, dynamic axes and custom functions. Check the destination runtime's documented support rather than assuming a familiar operator name guarantees the same version semantics.
Candle's August 23 Reshape patch illustrates the risk. The runtime supported the operator, yet one attribute and a dimension-inference edge case were incomplete. Compatibility matrices should therefore distinguish presence from tested semantic coverage.
Create a parity suite
Run identical inputs through the source and target systems and compare outputs with tolerances appropriate to precision. Include typical samples, worst-case sequence lengths, empty dimensions, batch-size changes, unusual characters and the smallest valid input.
Compare intermediate tensors when the final result diverges. The first mismatching node narrows investigation and prevents teams from compensating for a runtime bug by changing product thresholds downstream.
Test the selected execution provider
CPU, CUDA, Metal and other backends may use different kernels or support different operator subsets. Run the parity suite on the hardware and provider that will serve production traffic, including the intended quantization and optimization passes.
Capture warnings about graph rewrites, unsupported nodes and CPU fallback. A model can appear correct while silently moving expensive work off the accelerator, turning a compatibility issue into a latency or capacity incident.
Make portability continuously testable
Store a small, rights-cleared fixture set and expected outputs with the deployment code. Re-run it whenever the converter, runtime, driver, model or graph optimization changes. Add the runtime artifact hash to release records.
Define a rollback path and preserve the last validated package. ONNX reduces the cost of moving models, but disciplined parity testing is what converts an exchange format into a reliable production boundary.
Sources & further reading
Social-media activity is treated as a signal of attention, not proof. Product claims are attributed to the linked publisher or announcement.