> For the complete documentation index, see [llms.txt](https://docs.zama.org/concrete-ml/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zama.org/concrete-ml/0.3/advanced-topics/pruning.md).

# Pruning

Pruning is a method to reduce neural network complexity, usually applied in order reduce the computation cost or memory size. Pruning is used in Concrete-ML to control the size of accumulators in neural networks, thus making them FHE compatible. See [here](/concrete-ml/0.3/getting-started/concrete_numpy.md#limitations-for-fhe-friendly-neural-networks) for an explanation of the accumulator bitwidth constraints.

In neural networks, a neuron computes a linear combination of inputs and learned weights, then applies an activation function.

![Artificial Neuron (from: wikipedia)](https://3332742850-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVSEDLkZUW3FweoKsRQJ2%2Fuploads%2Fgit-blob-111dced4bbe9cad0a53c444da3e2e6956812bd1f%2FArtificial_neuron.png?alt=media)

The neuron computes:

$$y\_k = \phi\left(\sum\_i w\_ix\_i\right)$$

When building a full neural network, each layer will contain multiple neurons, which are connected to the neuron outputs of a previous layer or to the inputs.

![Fully Connected Neural Network](https://3332742850-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVSEDLkZUW3FweoKsRQJ2%2Fuploads%2Fgit-blob-54c57e4fba2ff1a7f7b3cb4e9f7daa524bdc6339%2Fnetwork.png?alt=media)

For every neuron shown in each layer of the figure above, the linear combinations of inputs and learned weights are computed. Depending on the values of the inputs and weights, the sum $$v\_k = \sum\_i w\_ix\_i$$ - which for Concrete-ML neural networks is computed with integers - can take a range of different values.

To respect the bit width constraint of the FHE [Table Lookup](https://docs.zama.ai/concrete-numpy/tutorials/table_lookup), the values of the accumulator $$v\_k$$ must remain small to be representable with only 8 bits. In other words, the values must be between 0 and 255.

Pruning a neural network entails fixing some of the weights $$w\_k$$ to be zero during training. This is advantageous to meet FHE constraints, as irrespective of the distribution of $$x\_i$$, multiplying these input values by 0 does not increase the accumulator value.

Fixing some of the weights to 0 makes the network graph look more similar to the following:

![Pruned Fully Connected Neural Network](https://3332742850-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVSEDLkZUW3FweoKsRQJ2%2Fuploads%2Fgit-blob-8a53e293e6ebe297f668f91bdb24a0994450bbcf%2Fprunednet.png?alt=media)

While pruning weights can reduce the prediction performance of the neural network, studies show that a high level of pruning (above 50%) can often be applied. See here how Concrete-ML uses pruning in [Fully Connected Neural Networks](https://github.com/zama-ai/concrete-ml/blob/release/0.3.x/docs/_apidoc/concrete.ml.sklearn.html#concrete.ml.sklearn.qnn.NeuralNetClassifier).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.zama.org/concrete-ml/0.3/advanced-topics/pruning.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.