# Built-in Model Examples

These examples illustrate the basic usage of built-in Concrete ML models. For more examples showing how to train high-accuracy models on more complex data-sets, see the [Demos and Tutorials](https://docs.zama.org/concrete-ml/1.1/getting-started/showcase) section.

## FHE constraints considerations

In Concrete ML, built-in linear models are exact equivalents to their scikit-learn counterparts. As they do not apply any non-linearity during inference, these models are very fast (\~1ms FHE inference time) and can use high-precision integers (between 20-25 bits).

Tree-based models apply non-linear functions that enable comparisons of inputs and trained thresholds. Thus, they are limited with respect to the number of bits used to represent the inputs. But as these examples show, in practice 5-6 bits are sufficient to exactly reproduce the behavior of their scikit-learn counterpart models.

In the examples below, built-in neural networks can be configured to work with user-specified accumulator sizes, which allow the user to adjust the speed/accuracy trade-off.

{% hint style="info" %}
It is recommended to use [simulation](https://docs.zama.org/concrete-ml/1.1/advanced-topics/compilation#fhe-simulation) to configure the speed/accuracy trade-off for tree-based models and neural networks, using grid-search or your own heuristics.
{% endhint %}

## List of examples

### 1. Linear models

[![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   Linear Regression example](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/LinearRegression.ipynb) [![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   Logistic Regression example](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/LogisticRegression.ipynb) [![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   Linear Support Vector Regression example](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/LinearSVR.ipynb) [![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   Linear SVM classification](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/SVMClassifier.ipynb)

These examples show how to use the built-in linear models on synthetic data, which allows for easy visualization of the decision boundaries or trend lines. Executing these 1D and 2D models in FHE takes around 1 millisecond.

### 2. Generalized linear models

[![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   Poisson Regression example](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/PoissonRegression.ipynb) [![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   Generalized Linear Models comparison](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/GLMComparison.ipynb)

These two examples show generalized linear models (GLM) on the real-world [OpenML insurance](https://www.openml.org/d/41214) data-set. As the non-linear, inverse-link functions are computed, these models do not use [PBS](https://docs.zama.org/concrete-ml/1.1/getting-started/concepts#cryptography-concepts), and are, thus, very fast (\~1ms execution time).

### 3. Decision tree

[![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)    Decision Tree Classifier](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/DecisionTreeClassifier.ipynb)

Using the [OpenML spams](https://www.openml.org/d/44) data-set, this example shows how to train a classifier that detects spam, based on features extracted from email messages. A grid-search is performed over decision-tree hyper-parameters to find the best ones.

[![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)    Decision Tree Regressor](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/DecisionTreeRegressor.ipynb)

Using the [House Price prediction](https://www.openml.org/search?type=data\&sort=runs\&id=537) data-set, this example shows how to train regressor that predicts house prices.

### 4. XGBoost and Random Forest classifier

[![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   XGBoost/Random Forest example](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/XGBClassifier.ipynb)

This example shows how to train tree-ensemble models (either XGBoost or Random Forest), first on a synthetic data-set, and then on the [Diabetes](https://www.openml.org/d/37) data-set. Grid-search is used to find the best number of trees in the ensemble.

### 5. XGBoost regression

[![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   XGBoost Regression example](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/XGBRegressor.ipynb)

Privacy-preserving prediction of house prices is shown in this example, using the [House Prices](https://www.openml.org/d/43926) data-set. Using 50 trees in the ensemble, with 5 bits of precision for the input features, the FHE regressor obtains an $$R^2$$ score of 0.90 and an execution time of 7-8 seconds.

### 6. Fully connected neural network

[![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   NN Iris example](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/FullyConnectedNeuralNetwork.ipynb) [![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   NN MNIST example](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/FullyConnectedNeuralNetworkOnMNIST.ipynb)

Two different configurations of the built-in, fully-connected neural networks are shown. First, a small bit-width accumulator network is trained on [Iris](https://www.openml.org/d/61) and compared to a PyTorch floating point network. Second, a larger accumulator (>8 bits) is demonstrated on [MNIST](http://yann.lecun.com/exdb/mnist/).

### 7. Comparison of models

[![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   Classifier comparison](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/ClassifierComparison.ipynb) [![](https://1311909216-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FNEnktIWhxQbxRV4NTB6h%2Fuploads%2Fgit-blob-58623ebfbfd17f0ca8ded37018c740198f9e09f7%2Fjupyter_logo.png?alt=media)   Regressor comparison](https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/RegressorComparison.ipynb)

Based on three different synthetic data-sets, all the built-in classifiers are demonstrated in this notebook, showing accuracies, inference times, accumulator bit-widths, and decision boundaries.