diff options
| author | Wanis Elabbar <70503629+elabbarw@users.noreply.github.com> | 2023-09-01 18:43:44 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-09-01 10:43:44 -0700 |
| commit | 456a8239f18ac5159fffc2484050f94ac3f36317 (patch) | |
| tree | 353d10667a8c462931948f7bdf3f144761a414a8 /docs | |
| parent | 731577cc5dfb882fe2239dae70ac8091edaafd84 (diff) | |
| download | sncontinue-456a8239f18ac5159fffc2484050f94ac3f36317.tar.gz sncontinue-456a8239f18ac5159fffc2484050f94ac3f36317.tar.bz2 sncontinue-456a8239f18ac5159fffc2484050f94ac3f36317.zip | |
Update codellama.md to provide instructions for FastChat API (#439)
* Update codellama.md to provide instructions for FastChat API
* Update codellama.md
---------
Co-authored-by: Nate Sesti <33237525+sestinj@users.noreply.github.com>
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/docs/walkthroughs/codellama.md | 15 |
1 files changed, 15 insertions, 0 deletions
diff --git a/docs/docs/walkthroughs/codellama.md b/docs/docs/walkthroughs/codellama.md index 68e99948..ff5b9cf3 100644 --- a/docs/docs/walkthroughs/codellama.md +++ b/docs/docs/walkthroughs/codellama.md @@ -67,3 +67,18 @@ config = ContinueConfig( ``` 3. Reload the VS Code window for changes to take effect + +## FastChat API +1. Setup the FastChat API (https://github.com/lm-sys/FastChat) to use one of the Codellama models on Hugging Face (e.g: codellama/CodeLlama-7b-Instruct-hf). +2. Start the OpenAI compatible API (ref: https://github.com/lm-sys/FastChat/blob/main/docs/openai_api.md). +3. Change your Continue config file to look like this: + +```python +config = ContinueConfig( + ... + models=Models(default=OpenAI( + model="CodeLlama-7b-Instruct-hf", + openai_server_info={'api_base': 'http://localhost:8000/v1'}) + +``` +4. Reload the VS Code window for changes to take effect. |