> For the complete documentation index, see [llms.txt](https://docs.zama.org/concrete-ml/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zama.org/concrete-ml/0.4/deep-learning/onnx_support.md). # Using ONNX In addition to Concrete-ML models and to [custom models in torch](/concrete-ml/0.4/deep-learning/torch_support.md), it is also possible to directly compile [ONNX](https://onnx.ai/) models. This can be particularly appealing, notably to import models trained with Keras. ONNX models can be compiled by directly importing models that are already quantized with Quantization Aware Training (QAT). or by performing Post-Training Quantization (PTQ) with Concrete-ML. ## Simple example The following example shows how to compile an ONNX model using PTQ. The model was initially trained using Keras before being exported to ONNX. The training code is not shown here. {% hint style="warning" %} This example uses Post-Training Quantization, i.e. the quantization is not performed during training. Thus this model would not have good performance in FHE. Quantization Aware Training should be added by the model developer and importing QAT ONNX models can be done [as shown below](#quantization-aware-training). {% endhint %} ```python import numpy import onnx import tensorflow import tf2onnx from concrete.ml.torch.compile import compile_onnx_model from concrete.numpy.compilation import Configuration class FC(tensorflow.keras.Model): """A fully-connected model.""" def __init__(self): super().__init__() hidden_layer_size = 10 output_size = 5 self.dense1 = tensorflow.keras.layers.Dense( hidden_layer_size, activation=tensorflow.nn.relu, ) self.dense2 = tensorflow.keras.layers.Dense(output_size, activation=tensorflow.nn.relu6) self.flatten = tensorflow.keras.layers.Flatten() def call(self, inputs): """Forward function.""" x = self.flatten(inputs) x = self.dense1(x) x = self.dense2(x) return self.flatten(x) n_bits = 6 input_output_feature = 2 input_shape = (input_output_feature,) num_inputs = 1 n_examples = 5000 # Define the Keras model keras_model = FC() keras_model.build((None,) + input_shape) keras_model.compute_output_shape(input_shape=(None, input_output_feature)) # Create random input input_set = numpy.random.uniform(-100, 100, size=(n_examples, *input_shape)) # Convert to ONNX tf2onnx.convert.from_keras(keras_model, opset=14, output_path="tmp.model.onnx") onnx_model = onnx.load("tmp.model.onnx") onnx.checker.check_model(onnx_model) # Compile quantized_numpy_module = compile_onnx_model( onnx_model, input_set, n_bits=2 ) # Create test data from the same distribution and quantize using # learned quantization parameters during compilation x_test = tuple(numpy.random.uniform(-100, 100, size=(1, *input_shape)) for _ in range(num_inputs)) qtest = quantized_numpy_module.quantize_input(x_test) y_clear = quantized_numpy_module(*qtest) y_fhe = quantized_numpy_module.forward_fhe.encrypt_run_decrypt(*qtest) print("Execution in clear: ", y_clear) print("Execution in FHE: ", y_fhe) print("Equality: ", numpy.sum(y_clear == y_fhe), "over", numpy.size(y_fhe), "values") ``` {% hint style="warning" %} While Keras was used in this example, it is not officially supported as additional work is needed to test all of Keras' types of layer and models. {% endhint %} ## Quantization Aware Training QAT models contain quantizers in the ONNX graph. These quantizers ensure that the inputs to the Linear/Dense and Conv layers are quantized. Since these QAT models have quantizers that are configured during training to a specific number of bits, the ONNX graph will need to be imported using the same settings: ```python n_bits_qat = 3 # number of bits for weights and activations during training quantized_numpy_module = compile_onnx_model( onnx_model, input_set, import_qat=True, n_bits=n_bits_qat, ) ``` ## Supported operators The following operators are supported for evaluation and conversion to an equivalent FHE circuit. Other operators were not implemented either due to FHE constraints, or because they are rarely used in PyTorch activations or scikit-learn models. * Abs * Acos * Acosh * Add * Asin * Asinh * Atan * Atanh * AveragePool * BatchNormalization * Cast * Celu * Clip * Constant * Conv * Cos * Cosh * Div * Elu * Equal * Erf * Exp * Flatten * Gemm * Greater * GreaterOrEqual * HardSigmoid * HardSwish * Identity * LeakyRelu * Less * LessOrEqual * Log * MatMul * Mul * Not * Or * PRelu * Pad * Pow * ReduceSum * Relu * Reshape * Round * Selu * Sigmoid * Sin * Sinh * Softplus * Sub * Tan * Tanh * ThresholdedRelu * Transpose * Where * onnx.brevitas.Quant --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.zama.org/concrete-ml/0.4/deep-learning/onnx_support.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.