> For the complete documentation index, see [llms.txt](https://docs.zama.org/concrete-ml/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zama.org/concrete-ml/0.3/advanced-topics/skorch_usage.md).

# Using Skorch

Concrete-ML uses [Skorch](https://skorch.readthedocs.io/en/stable/) to implement multi-layer, fully-connected torch neural networks in a way that is compatible with the Scikit-learn API.

This wrapper implements Torch training boilerplate code, alleviating the work that needs to be done by the user. It is possible to add hooks during the training phase, for example once an epoch is finished.

Skorch allows the user to easily create a classifier or regressor around a neural network (NN), implemented in Torch as a `nn.Module`, which is used by Concrete-ML to provide a fully-connected multi-layer NN with a configurable number of layers and optional pruning (see [pruning](/concrete-ml/0.3/advanced-topics/pruning.md) and the [neural network documentation](/concrete-ml/0.3/built-in-models/neural-networks.md) for more information).

Under the hood, Concrete-ML uses a Skorch wrapper around a single torch module, `SparseQuantNeuralNetImpl`. More information can be found [in the API guide](https://github.com/zama-ai/concrete-ml/blob/release/0.3.x/docs/_apidoc/concrete.ml.sklearn.html#concrete.ml.sklearn.qnn.SparseQuantNeuralNetImpl).

```
class SparseQuantNeuralNetImpl(nn.Module):
    """Sparse Quantized Neural Network classifier.
```

## Parameter choice

A linear or convolutional layer of an NN will compute a linear combination of weights and inputs (also called a *'multi-sum'*). For example, a linear layer will compute:

$$\mathsf{output}^k = \sum\_i^Nw\_{i}^kx\_i$$

where $$k$$ is the k-th neuron in the layer. In this case, the sum is taken on a single dimension. A convolutional layer will compute:

$$\mathsf{output}*{xy}^{k} = \sum\_c^{N}\sum\_j^{K\_h}\sum\_i^{K\_w}w*{cji}^kx\_{c,y+j,x+i}^k$$

where $$k$$ is the k-th filter of the convolutional layer and $$N$$, $$K\_h$$ and $$K\_w$$ are the number of input channels, the kernel height and the kernel width, respectively.

Following the formulas for the resulting bit width of quantized linear combinations described [here](/concrete-ml/0.3/getting-started/concrete_numpy.md#limitations-for-fhe-friendly-neural-networks), it can be seen that the maximum dimensionality of the input and weights that can make the result exceed 8 bits:

$$\Omega = \mathsf{floor} \left( \frac{2^{n\_{\mathsf{max}}} - 1}{(2^{n\_{\mathsf{weights}}} - 1)(2^{n\_{\mathsf{inputs}}} - 1)} \right)$$

Here, $$n\_{\mathsf{max}} = 8$$ is the maximum precision allowed.

For example, if $$n\_{\mathsf{weights}} = 2$$ and $$n\_{\mathsf{inputs}} = 2$$ with $$n\_{\mathsf{max}} = 8$$, the worst case is where all inputs and weights are equal to their maximal value $$2^2-1=3$$. In this case, there can be at most $$\Omega = 28$$ elements in the multi-sums.

In practice, the distribution of the weights of a neural network is Gaussian, with many weights either 0 or having a small value. This enables exceeding the worst-case number of active neurons without having to risk overflowing the bitwidth. The parameter `n_hidden_neurons_multiplier` is multiplied with $$\Omega$$ to determine the total number of non-zero weights that should be kept in a neuron.

The pruning mechanism is already implemented in `SparseQuantNeuralNetImpl`, and the user only needs to determine the parameters listed above. They can be chosen in a way that is convenient, e.g. maximizing accuracy.

Increasing `n_hidden_neurons_multiplier` can lead to improved performance, as long as the compiled NN does not exceed 8 bits of accumulator bitwidth.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.zama.org/concrete-ml/0.3/advanced-topics/skorch_usage.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.