# GPT4All

### How to configure and use it? <a href="#how-to-configure-and-use-it" id="how-to-configure-and-use-it"></a>

**Pre-Requisite(s)**

* `model` (optional) - Set which model you want to use. Defaults to `orca-mini-3b.ggmlv3.q4_0`

**Running in a Colab/Kaggle/Python scripts(s)**

```python
from genai_stack.model import Gpt4AllModel

llm = Gpt4AllModel.from_kwargs()
model_response = llm.predict("How many countries are there in the world?")
print(model_response["result"])
```

**Running the model in a webserver**

If you want to run the model in a webserver and interact with it with HTTP requests, the model provides a way to run it.

1. As a Python script

```python
from fastapi.responses import JSONResponse
from genai_stack.model import Gpt4AllModel

llm = Gpt4AllModel.from_kwargs()
llm.run_http_server(response_class=JSONResponse)
```

A server should start as below

```bash
INFO:     Started server process [137717]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8082 (Press CTRL+C to quit)
```

Make HTTP requests.\
URL - <http://localhost:8082/predict/>

```python
import requests
response = requests.post("http://localhost:8082/predict/", data="How many countries are there in the world?")
print(response.text)
```

2. As a CLI

Create a `model.json` file with the following contents:

```json
{
    "model": {
        "name": "gpt4all",
        "fields": {
            "model": "ggml-gpt4all-j-v1.3-groovy"
        }
    }
}
```

Run the below command:

```bash
genai-stack start --config_file model.json
```

```bash
 ██████╗ ███████╗███╗   ██╗ █████╗ ██╗    ███████╗████████╗ █████╗  ██████╗██╗  ██╗    
██╔════╝ ██╔════╝████╗  ██║██╔══██╗██║    ██╔════╝╚══██╔══╝██╔══██╗██╔════╝██║ ██╔╝    
██║  ███╗█████╗  ██╔██╗ ██║███████║██║    ███████╗   ██║   ███████║██║     █████╔╝     
██║   ██║██╔══╝  ██║╚██╗██║██╔══██║██║    ╚════██║   ██║   ██╔══██║██║     ██╔═██╗     
╚██████╔╝███████╗██║ ╚████║██║  ██║██║    ███████║   ██║   ██║  ██║╚██████╗██║  ██╗    
 ╚═════╝ ╚══════╝╚═╝  ╚═══╝╚═╝  ╚═╝╚═╝    ╚══════╝   ╚═╝   ╚═╝  ╚═╝ ╚═════╝╚═╝  ╚═╝
INFO:     Started server process [641734]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8082 (Press CTRL+C to quit)
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://genaistack.aiplanet.com/v0.1.0/components/models-llms/gpt4all.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.