> For the complete documentation index, see [llms.txt](https://docs.zama.org/concrete-ml/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zama.org/concrete-ml/1.5/built-in-models/neural-networks.md).

# Neural networks

Concrete ML provides simple built-in neural networks models with a scikit-learn interface through the `NeuralNetClassifier` and `NeuralNetRegressor` classes.

|                                                                        Concrete ML                                                                        | scikit-learn                                                                                                 |
| :-------------------------------------------------------------------------------------------------------------------------------------------------------: | ------------------------------------------------------------------------------------------------------------ |
| [NeuralNetClassifier](https://github.com/zama-ai/concrete-ml/blob/release/1.5.x/docs/references/api/concrete.ml.sklearn.qnn.md#class-neuralnetclassifier) | [MLPClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html) |
|  [NeuralNetRegressor](https://github.com/zama-ai/concrete-ml/blob/release/1.5.x/docs/references/api/concrete.ml.sklearn.qnn.md#class-neuralnetregressor)  | [MLPRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html)   |

The neural network models are implemented with [skorch](https://skorch.readthedocs.io/en/stable/index.html), which provides a scikit-learn-like interface to Torch models (more [here](/concrete-ml/1.5/explanations/inner-workings/external_libraries.md#skorch)).

Concrete ML models are multi-layer, fully-connected, networks with customizable activation functions and have a number of neurons in each layer. This approach is similar to what is available in scikit-learn when using the `MLPClassifier`/`MLPRegressor` classes. The built-in models train easily with a single call to `.fit()`, which will automatically quantize weights and activations. These models use Quantization Aware Training, allowing good performance for low precision (down to 2-3 bits) weights and activations.

While `NeuralNetClassifier` and `NeuralNetClassifier` provide scikit-learn-like models, their architecture is somewhat restricted to make training easy and robust. If you need more advanced models, you can convert custom neural networks as described in the [FHE-friendly models documentation](/concrete-ml/1.5/deep-learning/fhe_friendly_models.md).

{% hint style="warning" %}
Good quantization parameter values are critical to make models [respect FHE constraints](/concrete-ml/1.5/getting-started/concepts.md#model-accuracy-considerations-under-fhe-constraints). Weights and activations should be quantized to low precision (e.g., 2-4 bits). The sparsity of the network can be tuned [as described below](#overflow-errors) to avoid accumulator overflow.
{% endhint %}

{% hint style="warning" %}
Using `nn.ReLU` as the activation function benefits from an optimization where [quantization uses powers-of-two scales](/concrete-ml/1.5/explanations/quantization.md#quantization-special-cases). This results in much faster inference times in FHE, thanks to a TFHE primitive that performs fast division by powers of two.
{% endhint %}

## Example usage

To create an instance of a Fully Connected Neural Network (FCNN), you need to instantiate one of the `NeuralNetClassifier` and `NeuralNetRegressor` classes and configure a number of parameters that are passed to their constructor. Note that some parameters need to be prefixed by `module__`, while others don't. The parameters related to the model (i.e., the underlying `nn.Module`), must have the prefix. The parameters related to training options do not require the prefix.

```python
from concrete.ml.sklearn import NeuralNetClassifier
import torch.nn as nn

n_inputs = 10
n_outputs = 2
params = {
    "module__n_layers": 2,
    "max_epochs": 10,
}

concrete_classifier = NeuralNetClassifier(**params)
```

The [Classifier Comparison notebook](/concrete-ml/1.5/tutorials/ml_examples.md) shows the behavior of built-in neural networks on several synthetic data-sets.

![Comparison neural networks](https://2241057372-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FTeFWUaIKrKiFj6iYvz8W%2Fuploads%2Fgit-blob-f0e9b2c245b6039ab3c43ac5dc00e5f57a0bc0b5%2Fneural_nets_builtin.png?alt=media)

The figure above right shows the Concrete ML neural network, trained with Quantization Aware Training in an FHE-compatible configuration. The figure compares this network to the floating-point equivalent, trained with scikit-learn.

### Architecture parameters

* `module__n_layers`: number of layers in the FCNN, must be at least 1. Note that this is the total number of layers. For a single, hidden layer NN model, set `module__n_layers=2`
* `module__activation_function`: can be one of the Torch activations (e.g., nn.ReLU, see the full list [here](/concrete-ml/1.5/deep-learning/torch_support.md#activation-functions)). Neural networks with `nn.ReLU` activation benefit from specific optimizations that make them around 10x faster than networks with other activation functions.

### Quantization parameters

* `n_w_bits` (default 3): number of bits for weights
* `n_a_bits` (default 3): number of bits for activations and inputs
* `n_accum_bits`: maximum accumulator bit-width that is desired. By default, this is unbounded, which, for weight and activation bit-width settings, [may make the trained networks fail in compilation](#overflow-errors). When used, the implementation will attempt to keep accumulators under this bit-width through [pruning](/concrete-ml/1.5/explanations/pruning.md) (i.e., setting some weights to zero)
* `power_of_two_scaling` (default True): forces quantization scales to be powers-of-two, which, when coupled with the ReLU activation, benefits from strong FHE inference time optimization. See this [section](/concrete-ml/1.5/explanations/quantization.md#quantization-special-cases) in the quantization documentation for more details.

### Training parameters (from skorch)

* `max_epochs`: The number of epochs to train the network (default 10)
* `verbose`: Whether to log loss/metrics during training (default: False)
* `lr`: Learning rate (default 0.001)

Other parameters from skorch can be found in the [skorch documentation](https://skorch.readthedocs.io/en/stable/classifier.html).

### Advanced parameters

* `module__n_hidden_neurons_multiplier`: The number of hidden neurons will be automatically set proportional to the dimensionality of the input. This parameter controls the proportionality factor and is set to 4 by default. This value gives good accuracy while avoiding accumulator overflow. See the [pruning](/concrete-ml/1.5/explanations/pruning.md) and [quantization](/concrete-ml/1.5/explanations/quantization.md) sections for more info.

### Class weights

You can give weights to each class to use in training. Note that this must be supported by the underlying PyTorch loss function.

```python
    from sklearn.utils.class_weight import compute_class_weight
    params["criterion__weight"] = compute_class_weight("balanced", classes=classes, y=y_train)
```

### Overflow errors

The `n_accum_bits` parameter influences training accuracy as it controls the number of non-zero neurons that are allowed in each layer. Increasing `n_accum_bits` improves accuracy, but should take into account precision limitations to avoid an overflow in the accumulator. The default value is a good compromise that avoids an overflow in most cases, but you may want to change the value of this parameter to reduce the breadth of the network if you have overflow errors.

Furthermore, the number of neurons on intermediate layers is controlled through the `n_hidden_neurons_multiplier` parameter - a value of 1 will make intermediate layers have the same number of neurons as the number of dimensions of the input data.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.zama.org/concrete-ml/1.5/built-in-models/neural-networks.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.