From 456a8239f18ac5159fffc2484050f94ac3f36317 Mon Sep 17 00:00:00 2001
From: Wanis Elabbar <70503629+elabbarw@users.noreply.github.com>
Date: Fri, 1 Sep 2023 18:43:44 +0100
Subject: Update codellama.md to provide instructions for FastChat API (#439)

* Update codellama.md to provide instructions for FastChat API

* Update codellama.md

---------

Co-authored-by: Nate Sesti <33237525+sestinj@users.noreply.github.com>
---
 docs/docs/walkthroughs/codellama.md | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/docs/docs/walkthroughs/codellama.md b/docs/docs/walkthroughs/codellama.md
index 68e99948..ff5b9cf3 100644
--- a/docs/docs/walkthroughs/codellama.md
+++ b/docs/docs/walkthroughs/codellama.md
@@ -67,3 +67,18 @@ config = ContinueConfig(
 ```
 
 3. Reload the VS Code window for changes to take effect
+
+## FastChat API
+1. Setup the FastChat API (https://github.com/lm-sys/FastChat) to use one of the Codellama models on Hugging Face (e.g: codellama/CodeLlama-7b-Instruct-hf).
+2. Start the OpenAI compatible API (ref: https://github.com/lm-sys/FastChat/blob/main/docs/openai_api.md).
+3. Change your Continue config file to look like this:
+
+```python
+config = ContinueConfig(
+    ...
+    models=Models(default=OpenAI(
+        model="CodeLlama-7b-Instruct-hf",
+        openai_server_info={'api_base': 'http://localhost:8000/v1'})
+
+```
+4. Reload the VS Code window for changes to take effect.
-- 
cgit v1.2.3-70-g09d2