Merge branch 'main' of https://github.com/continuedev/continue

1 files changed, 23 insertions, 1 deletions

diff --git a/docs/docs/customization.md b/docs/docs/customization.md
index 096b42b2..74149ff9 100644
--- a/docs/docs/customization.md
+++ b/docs/docs/customization.md
@@ -14,7 +14,7 @@ Local Models
 
 - [Ollama](#run-llama-2-locally-with-ollama) - If you have a Mac, Ollama is the simplest way to run open-source models like Code Llama.
 - [GGML](#local-models-with-ggml) - Use llama-cpp-python to run a local server with any open-source model.
-- [LlamaCpp](#llama-cpp) - Use llama.cpp directly instead of llama-cpp-python.
+- [LlamaCpp](#llamacpp) - Use llama.cpp directly instead of llama-cpp-python.
 
 Open-Source Models (not local)
 
@@ -117,6 +117,28 @@ config = ContinueConfig(
 )
 ```
 
+### Llama.cpp
+
+Run the llama.cpp server binary to start the API server. If running on a remote server, be sure to set host to 0.0.0.0:
+```shell
+.\server.exe -c 4096 --host 0.0.0.0 -t 16 --mlock -m models\meta\llama\codellama-7b-instruct.Q8_0.gguf
+```
+
+After it's up and running, change `~/.continue/config.py` to look like this:
+
+```python
+from continuedev.src.continuedev.libs.llm.ggml import GGML
+
+config = ContinueConfig(
+    ...
+    models=Models(
+        default=LlamaCpp(
+            max_context_length=4096,
+            server_url="http://localhost:8080")
+    )
+)
+```
+
 ### Together
 
 The Together API is a cloud platform for running large AI models. You can sign up [here](https://api.together.xyz/signup), copy your API key on the initial welcome screen, and then hit the play button on any model from the [Together Models list](https://docs.together.ai/docs/models-inference). Change `~/.continue/config.py` to look like this: