diff options
| author | Nate Sesti <33237525+sestinj@users.noreply.github.com> | 2023-10-09 18:37:27 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-10-09 18:37:27 -0700 |
| commit | f09150617ed2454f3074bcf93f53aae5ae637d40 (patch) | |
| tree | 5cfe614a64d921dfe58b049f426d67a8b832c71f /docs | |
| parent | 985304a213f620cdff3f8f65f74ed7e3b79be29d (diff) | |
| download | sncontinue-f09150617ed2454f3074bcf93f53aae5ae637d40.tar.gz sncontinue-f09150617ed2454f3074bcf93f53aae5ae637d40.tar.bz2 sncontinue-f09150617ed2454f3074bcf93f53aae5ae637d40.zip | |
Preview (#541)
* Strong typing (#533)
* refactor: :recycle: get rid of continuedev.src.continuedev structure
* refactor: :recycle: switching back to server folder
* feat: :sparkles: make config.py imports shorter
* feat: :bookmark: publish as pre-release vscode extension
* refactor: :recycle: refactor and add more completion params to ui
* build: :building_construction: download from preview S3
* fix: :bug: fix paths
* fix: :green_heart: package:pre-release
* ci: :green_heart: more time for tests
* fix: :green_heart: fix build scripts
* fix: :bug: fix import in run.py
* fix: :bookmark: update version to try again
* ci: ๐ Update package.json version [skip ci]
* refactor: :fire: don't check for old extensions version
* fix: :bug: small bug fixes
* fix: :bug: fix config.py import paths
* ci: ๐ Update package.json version [skip ci]
* ci: :green_heart: platform-specific builds test #1
* feat: :green_heart: ship with binary
* fix: :green_heart: fix copy statement to include.exe for windows
* fix: :green_heart: cd extension before packaging
* chore: :loud_sound: count tokens generated
* fix: :green_heart: remove npm_config_arch
* fix: :green_heart: publish as pre-release!
* chore: :bookmark: update version
* perf: :green_heart: hardcode distro paths
* fix: :bug: fix yaml syntax error
* chore: :bookmark: update version
* fix: :green_heart: update permissions and version
* feat: :bug: kill old server if needed
* feat: :lipstick: update marketplace icon for pre-release
* ci: ๐ Update package.json version [skip ci]
* feat: :sparkles: auto-reload for config.py
* feat: :wrench: update default config.py imports
* feat: :sparkles: codelens in config.py
* feat: :sparkles: select model param count from UI
* ci: ๐ Update package.json version [skip ci]
* feat: :sparkles: more model options, ollama error handling
* perf: :zap: don't show server loading immediately
* fix: :bug: fixing small UI details
* ci: ๐ Update package.json version [skip ci]
* feat: :rocket: headers param on LLM class
* fix: :bug: fix headers for openai.;y
* feat: :sparkles: highlight code on cmd+shift+L
* ci: ๐ Update package.json version [skip ci]
* feat: :lipstick: sticky top bar in gui.tsx
* fix: :loud_sound: websocket logging and horizontal scrollbar
* ci: ๐ Update package.json version [skip ci]
* feat: :sparkles: allow AzureOpenAI Service through GGML
* ci: ๐ Update package.json version [skip ci]
* fix: :bug: fix automigration
* ci: ๐ Update package.json version [skip ci]
* ci: :green_heart: upload binaries in ci, download apple silicon
* chore: :fire: remove notes
* fix: :green_heart: use curl to download binary
* fix: :green_heart: set permissions on apple silicon binary
* fix: :green_heart: testing
* fix: :green_heart: cleanup file
* fix: :green_heart: fix preview.yaml
* fix: :green_heart: only upload once per binary
* fix: :green_heart: install rosetta
* ci: :green_heart: download binary after tests
* ci: ๐ Update package.json version [skip ci]
* ci: :green_heart: prepare ci for merge to main
---------
Co-authored-by: GitHub Action <action@github.com>
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/docs/customization/context-providers.md | 16 | ||||
| -rw-r--r-- | docs/docs/customization/models.md | 14 | ||||
| -rw-r--r-- | docs/docs/reference/Models/anthropicllm.md | 13 | ||||
| -rw-r--r-- | docs/docs/reference/Models/ggml.md | 17 | ||||
| -rw-r--r-- | docs/docs/reference/Models/googlepalmapi.md | 15 | ||||
| -rw-r--r-- | docs/docs/reference/Models/huggingfaceinferenceapi.md | 15 | ||||
| -rw-r--r-- | docs/docs/reference/Models/huggingfacetgi.md | 11 | ||||
| -rw-r--r-- | docs/docs/reference/Models/llamacpp.md | 13 | ||||
| -rw-r--r-- | docs/docs/reference/Models/ollama.md | 13 | ||||
| -rw-r--r-- | docs/docs/reference/Models/openai.md | 11 | ||||
| -rw-r--r-- | docs/docs/reference/Models/openaifreetrial.md | 9 | ||||
| -rw-r--r-- | docs/docs/reference/Models/queuedllm.md | 13 | ||||
| -rw-r--r-- | docs/docs/reference/Models/replicatellm.md | 15 | ||||
| -rw-r--r-- | docs/docs/reference/Models/textgenui.md | 13 | ||||
| -rw-r--r-- | docs/docs/reference/Models/togetherllm.md | 15 | ||||
| -rw-r--r-- | docs/docs/reference/config.md | 2 | ||||
| -rw-r--r-- | docs/docs/walkthroughs/codebase-embeddings.md | 2 | ||||
| -rw-r--r-- | docs/docs/walkthroughs/codellama.md | 10 |
18 files changed, 142 insertions, 75 deletions
diff --git a/docs/docs/customization/context-providers.md b/docs/docs/customization/context-providers.md index 3147f90e..e9f5e9cf 100644 --- a/docs/docs/customization/context-providers.md +++ b/docs/docs/customization/context-providers.md @@ -22,7 +22,7 @@ Setup instructions are below for each (showing the import and config object). Yo Type '@issue' to reference the title and contents of a GitHub issue. ```python -from continuedev.src.continuedev.plugins.context_providers.github import GitHubIssuesContextProvider +from continuedev.plugins.context_providers.github import GitHubIssuesContextProvider ``` ```python @@ -37,7 +37,7 @@ GitHubIssuesContextProvider( Type '@search' to reference the results of codebase search, just like the results you would get from VS Code search. ```python -from continuedev.src.continuedev.plugins.context_providers.search import SearchContextProvider +from continuedev.plugins.context_providers.search import SearchContextProvider ``` ```python @@ -49,7 +49,7 @@ SearchContextProvider() # No arguments necessary Type '@url' to reference the contents of a URL. You can either reference preset URLs, or reference one dynamically by typing '@url https://example.com'. The text contents of the page will be fetched and used as context. ```python -from continuedev.src.continuedev.plugins.context_providers.url import URLContextProvider +from continuedev.plugins.context_providers.url import URLContextProvider ``` ```python @@ -61,7 +61,7 @@ URLContextProvider(preset_urls=["https://continue.dev/docs/customization"]) Type '@diff' to reference all of the changes you've made to your current branch. This is useful if you want to summarize what you've done or ask for a general review of your work before committing. ```python -from continuedev.src.continuedev.plugins.context_providers.diff import DiffContextProvider +from continuedev.plugins.context_providers.diff import DiffContextProvider ``` ```python @@ -73,7 +73,7 @@ DiffContextProvider() Type '@tree' to reference the contents of your current workspace. The LLM will be able to see the nested directory structure of your project. ```python -from continuedev.src.continuedev.plugins.context_providers.filetree import FileTreeContextProvider +from continuedev.plugins.context_providers.filetree import FileTreeContextProvider ``` ```python @@ -85,7 +85,7 @@ FileTreeContextProvider() Type '@google' to reference the results of a Google search. For example, type "@google python tutorial" if you want to search and discuss ways of learning Python. ```python -from continuedev.src.continuedev.plugins.context_providers.google import GoogleContextProvider +from continuedev.plugins.context_providers.google import GoogleContextProvider ``` ```python @@ -101,7 +101,7 @@ Note: You can get an API key for free at [serper.dev](https://serper.dev). Type '@terminal' to reference the contents of your IDE's terminal. ```python -from continuedev.src.continuedev.plugins.context_providers.terminal import TerminalContextProvider +from continuedev.plugins.context_providers.terminal import TerminalContextProvider ``` ```python @@ -169,7 +169,7 @@ This example is a situation where you request all of the data (issues in this ca There are other scenarios where you might want to just get information on demand, for example by typing '@url https://continue.dev/docs/context-providers' and having the ContextProvider fetch the contents of that URL dynamically. For this case, you can implement the `DynamicContextProvider` class like this: ```python -from continuedev.src.continuedev.plugins.context_providers.dynamic import DynamicContextProvider +from continuedev.plugins.context_providers.dynamic import DynamicContextProvider class ExampleDynamicProvider(DynamicProvider): title = "example" diff --git a/docs/docs/customization/models.md b/docs/docs/customization/models.md index 8004130d..5bb8a860 100644 --- a/docs/docs/customization/models.md +++ b/docs/docs/customization/models.md @@ -27,7 +27,7 @@ Open-Source Models (not local) In `config.py`, you'll find the `models` property: ```python -from continuedev.src.continuedev.core.models import Models +from continuedev.core.models import Models config = ContinueConfig( ... @@ -53,7 +53,7 @@ If by chance the provider has the exact same API interface as OpenAI, the `OpenA If you'd like to use OpenAI models but are concerned about privacy, you can use the Azure OpenAI service, which is GDPR and HIPAA compliant. After applying for access [here](https://azure.microsoft.com/en-us/products/ai-services/openai-service), you will typically hear back within only a few days. Once you have access, instantiate the model like so: ```python -from continuedev.src.continuedev.libs.llm.openai import OpenAI +from continuedev.libs.llm.openai import OpenAI config = ContinueConfig( ... @@ -100,8 +100,8 @@ def template_alpaca_messages(msgs: List[Dict[str, str]]) -> str: It can then be used like this: ```python -from continuedev.src.continuedev.libs.llm.chat import template_alpaca_messages -from continuedev.src.continuedev.libs.llm.ollama import Ollama +from continuedev.libs.llm.chat import template_alpaca_messages +from continuedev.libs.llm.ollama import Ollama ... config=ContinueConfig( ... @@ -114,7 +114,7 @@ config=ContinueConfig( ) ``` -This exact function and a few other default implementations are available in [`continuedev.src.continuedev.libs.llm.prompts.chat`](https://github.com/continuedev/continue/blob/main/continuedev/src/continuedev/libs/llm/prompts/chat.py). +This exact function and a few other default implementations are available in [`continuedev.libs.llm.prompts.chat`](https://github.com/continuedev/continue/blob/main/continuedev/src/continuedev/libs/llm/prompts/chat.py). ## Customizing the /edit Prompt @@ -139,7 +139,7 @@ Output nothing except for the code. No code block, no English explanation, no st It can then be used like this: ```python -from continuedev.src.continuedev.libs.llm.ollama import Ollama +from continuedev.libs.llm.ollama import Ollama ... config=ContinueConfig( ... @@ -154,4 +154,4 @@ config=ContinueConfig( ) ``` -A few pre-made templates are available in [`continuedev.src.continuedev.libs.llm.prompts.edit`](https://github.com/continuedev/continue/blob/main/continuedev/src/continuedev/libs/llm/prompts/edit.py). +A few pre-made templates are available in [`continuedev.libs.llm.prompts.edit`](https://github.com/continuedev/continue/blob/main/continuedev/src/continuedev/libs/llm/prompts/edit.py). diff --git a/docs/docs/reference/Models/anthropicllm.md b/docs/docs/reference/Models/anthropicllm.md index b35761f0..68f8f9d3 100644 --- a/docs/docs/reference/Models/anthropicllm.md +++ b/docs/docs/reference/Models/anthropicllm.md @@ -5,7 +5,7 @@ import ClassPropertyRef from '@site/src/components/ClassPropertyRef.tsx'; Import the `AnthropicLLM` class and set it as the default model: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.libs.llm.anthropic import AnthropicLLM +from continuedev.libs.llm.anthropic import AnthropicLLM config = ContinueConfig( ... @@ -27,14 +27,19 @@ Claude 2 is not yet publicly released. You can request early access [here](https <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={true} default=""/> <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "claude-2", "type": "string"}' required={false} default="claude-2"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {}, "type": "object"}' required={false} default="{}"/> diff --git a/docs/docs/reference/Models/ggml.md b/docs/docs/reference/Models/ggml.md index 7fa2a3fc..6f214d27 100644 --- a/docs/docs/reference/Models/ggml.md +++ b/docs/docs/reference/Models/ggml.md @@ -7,7 +7,7 @@ See our [5 minute quickstart](https://github.com/continuedev/ggml-server-example Once the model is running on localhost:8000, change `~/.continue/config.py` to look like this: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.libs.llm.ggml import GGML +from continuedev.libs.llm.ggml import GGML config = ContinueConfig( ... @@ -24,20 +24,29 @@ config = ContinueConfig( ## Properties <ClassPropertyRef name='server_url' details='{"title": "Server Url", "description": "URL of the OpenAI-compatible server where the model is being served", "default": "http://localhost:8000", "type": "string"}' required={false} default="http://localhost:8000"/> +<ClassPropertyRef name='api_base' details='{"title": "Api Base", "description": "OpenAI API base URL.", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='api_type' details='{"title": "Api Type", "description": "OpenAI API type.", "enum": ["azure", "openai"], "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='api_version' details='{"title": "Api Version", "description": "OpenAI API version. For use with Azure OpenAI Service.", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='engine' details='{"title": "Engine", "description": "OpenAI engine. For use with Azure OpenAI Service.", "type": "string"}' required={false} default=""/> ### Inherited Properties <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to use (optional for the GGML class)", "default": "ggml", "type": "string"}' required={false} default="ggml"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {"edit": "Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags."}, "type": "object"}' required={false} default="{'edit': 'Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags.'}"/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={false} default=""/> diff --git a/docs/docs/reference/Models/googlepalmapi.md b/docs/docs/reference/Models/googlepalmapi.md index 4823dbd1..d9cb0cc2 100644 --- a/docs/docs/reference/Models/googlepalmapi.md +++ b/docs/docs/reference/Models/googlepalmapi.md @@ -5,8 +5,8 @@ import ClassPropertyRef from '@site/src/components/ClassPropertyRef.tsx'; The Google PaLM API is currently in public preview, so production applications are not supported yet. However, you can [create an API key in Google MakerSuite](https://makersuite.google.com/u/2/app/apikey) and begin trying out the `chat-bison-001` model. Change `~/.continue/config.py` to look like this: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.core.models import Models -from continuedev.src.continuedev.libs.llm.hf_inference_api import GooglePaLMAPI +from continuedev.core.models import Models +from continuedev.libs.llm.hf_inference_api import GooglePaLMAPI config = ContinueConfig( ... @@ -28,14 +28,19 @@ config = ContinueConfig( <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "Google PaLM API key", "type": "string"}' required={true} default=""/> <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "chat-bison-001", "type": "string"}' required={false} default="chat-bison-001"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {}, "type": "object"}' required={false} default="{}"/> diff --git a/docs/docs/reference/Models/huggingfaceinferenceapi.md b/docs/docs/reference/Models/huggingfaceinferenceapi.md index 9dbf23ed..fee892cc 100644 --- a/docs/docs/reference/Models/huggingfaceinferenceapi.md +++ b/docs/docs/reference/Models/huggingfaceinferenceapi.md @@ -5,8 +5,8 @@ import ClassPropertyRef from '@site/src/components/ClassPropertyRef.tsx'; Hugging Face Inference API is a great option for newly released language models. Sign up for an account and add billing [here](https://huggingface.co/settings/billing), access the Inference Endpoints [here](https://ui.endpoints.huggingface.co), click on โNew endpointโ, and fill out the form (e.g. select a model like [WizardCoder-Python-34B-V1.0](https://huggingface.co/WizardLM/WizardCoder-Python-34B-V1.0)), and then deploy your model by clicking โCreate Endpointโ. Change `~/.continue/config.py` to look like this: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.core.models import Models -from continuedev.src.continuedev.libs.llm.hf_inference_api import HuggingFaceInferenceAPI +from continuedev.core.models import Models +from continuedev.libs.llm.hf_inference_api import HuggingFaceInferenceAPI config = ContinueConfig( ... @@ -29,15 +29,20 @@ config = ContinueConfig( ### Inherited Properties <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to use (optional for the HuggingFaceInferenceAPI class)", "default": "Hugging Face Inference API", "type": "string"}' required={false} default="Hugging Face Inference API"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {"edit": "Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags."}, "type": "object"}' required={false} default="{'edit': 'Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags.'}"/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={false} default=""/> diff --git a/docs/docs/reference/Models/huggingfacetgi.md b/docs/docs/reference/Models/huggingfacetgi.md index 1275c13f..176b0539 100644 --- a/docs/docs/reference/Models/huggingfacetgi.md +++ b/docs/docs/reference/Models/huggingfacetgi.md @@ -14,15 +14,20 @@ import ClassPropertyRef from '@site/src/components/ClassPropertyRef.tsx'; ### Inherited Properties <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "huggingface-tgi", "type": "string"}' required={false} default="huggingface-tgi"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {"edit": "Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags."}, "type": "object"}' required={false} default="{'edit': 'Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags.'}"/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={false} default=""/> diff --git a/docs/docs/reference/Models/llamacpp.md b/docs/docs/reference/Models/llamacpp.md index 362914f8..b20c7675 100644 --- a/docs/docs/reference/Models/llamacpp.md +++ b/docs/docs/reference/Models/llamacpp.md @@ -11,7 +11,7 @@ Run the llama.cpp server binary to start the API server. If running on a remote After it's up and running, change `~/.continue/config.py` to look like this: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.libs.llm.llamacpp import LlamaCpp +from continuedev.libs.llm.llamacpp import LlamaCpp config = ContinueConfig( ... @@ -34,15 +34,20 @@ config = ContinueConfig( ### Inherited Properties <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "llamacpp", "type": "string"}' required={false} default="llamacpp"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {"edit": "Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags."}, "type": "object"}' required={false} default="{'edit': 'Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags.'}"/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={false} default=""/> diff --git a/docs/docs/reference/Models/ollama.md b/docs/docs/reference/Models/ollama.md index 64a326b7..b6d418ce 100644 --- a/docs/docs/reference/Models/ollama.md +++ b/docs/docs/reference/Models/ollama.md @@ -5,7 +5,7 @@ import ClassPropertyRef from '@site/src/components/ClassPropertyRef.tsx'; [Ollama](https://ollama.ai/) is an application for Mac and Linux that makes it easy to locally run open-source models, including Llama-2. Download the app from the website, and it will walk you through setup in a couple of minutes. You can also read more in their [README](https://github.com/jmorganca/ollama). Continue can then be configured to use the `Ollama` LLM class: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.libs.llm.ollama import Ollama +from continuedev.libs.llm.ollama import Ollama config = ContinueConfig( ... @@ -25,15 +25,20 @@ config = ContinueConfig( ### Inherited Properties <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "llama2", "type": "string"}' required={false} default="llama2"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {"edit": "Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags."}, "type": "object"}' required={false} default="{'edit': 'Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags.'}"/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={false} default=""/> diff --git a/docs/docs/reference/Models/openai.md b/docs/docs/reference/Models/openai.md index 039c1bf7..81af404d 100644 --- a/docs/docs/reference/Models/openai.md +++ b/docs/docs/reference/Models/openai.md @@ -7,7 +7,7 @@ The OpenAI class can be used to access OpenAI models like gpt-4 and gpt-3.5-turb If you are locally serving a model that uses an OpenAI-compatible server, you can simply change the `api_base` in the `OpenAI` class like this: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.libs.llm.openai import OpenAI +from continuedev.libs.llm.openai import OpenAI config = ContinueConfig( ... @@ -43,13 +43,18 @@ Options for serving models locally with an OpenAI-compatible server include: <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "type": "string"}' required={true} default=""/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "OpenAI API key", "type": "string"}' required={true} default=""/> <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> -<ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use for requests.", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {}, "type": "object"}' required={false} default="{}"/> diff --git a/docs/docs/reference/Models/openaifreetrial.md b/docs/docs/reference/Models/openaifreetrial.md index 8ebe92a7..1cf1154f 100644 --- a/docs/docs/reference/Models/openaifreetrial.md +++ b/docs/docs/reference/Models/openaifreetrial.md @@ -36,14 +36,19 @@ These classes support any models available through the OpenAI API, assuming your <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "type": "string"}' required={true} default=""/> <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> -<ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {}, "type": "object"}' required={false} default="{}"/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={false} default=""/> diff --git a/docs/docs/reference/Models/queuedllm.md b/docs/docs/reference/Models/queuedllm.md index c9a0b4b1..9ea28c58 100644 --- a/docs/docs/reference/Models/queuedllm.md +++ b/docs/docs/reference/Models/queuedllm.md @@ -7,7 +7,7 @@ QueuedLLM exists to make up for LLM servers that cannot handle multiple requests If you are already using another LLM class and are experiencing this problem, you can just wrap it with the QueuedLLM class like this: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.libs.llm.queued import QueuedLLM +from continuedev.libs.llm.queued import QueuedLLM config = ContinueConfig( ... @@ -27,15 +27,20 @@ config = ContinueConfig( ### Inherited Properties <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "queued", "type": "string"}' required={false} default="queued"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {}, "type": "object"}' required={false} default="{}"/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={false} default=""/> diff --git a/docs/docs/reference/Models/replicatellm.md b/docs/docs/reference/Models/replicatellm.md index 0dc5f838..167db379 100644 --- a/docs/docs/reference/Models/replicatellm.md +++ b/docs/docs/reference/Models/replicatellm.md @@ -5,8 +5,8 @@ import ClassPropertyRef from '@site/src/components/ClassPropertyRef.tsx'; Replicate is a great option for newly released language models or models that you've deployed through their platform. Sign up for an account [here](https://replicate.ai/), copy your API key, and then select any model from the [Replicate Streaming List](https://replicate.com/collections/streaming-language-models). Change `~/.continue/config.py` to look like this: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.core.models import Models -from continuedev.src.continuedev.libs.llm.replicate import ReplicateLLM +from continuedev.core.models import Models +from continuedev.libs.llm.replicate import ReplicateLLM config = ContinueConfig( ... @@ -30,14 +30,19 @@ If you don't specify the `model` parameter, it will default to `replicate/llama- <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "Replicate API key", "type": "string"}' required={true} default=""/> <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "replicate/llama-2-70b-chat:58d078176e02c219e11eb4da5a02a7830a283b14cf8f94537af893ccff5ee781", "type": "string"}' required={false} default="replicate/llama-2-70b-chat:58d078176e02c219e11eb4da5a02a7830a283b14cf8f94537af893ccff5ee781"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {"edit": "Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags."}, "type": "object"}' required={false} default="{'edit': 'Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags.'}"/> diff --git a/docs/docs/reference/Models/textgenui.md b/docs/docs/reference/Models/textgenui.md index e0d757e4..680ff6c4 100644 --- a/docs/docs/reference/Models/textgenui.md +++ b/docs/docs/reference/Models/textgenui.md @@ -5,7 +5,7 @@ import ClassPropertyRef from '@site/src/components/ClassPropertyRef.tsx'; TextGenUI is a comprehensive, open-source language model UI and local server. You can set it up with an OpenAI-compatible server plugin, but if for some reason that doesn't work, you can use this class like so: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.libs.llm.text_gen_interface import TextGenUI +from continuedev.libs.llm.text_gen_interface import TextGenUI config = ContinueConfig( ... @@ -28,15 +28,20 @@ config = ContinueConfig( ### Inherited Properties <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "text-gen-ui", "type": "string"}' required={false} default="text-gen-ui"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {"edit": "Here is the code before editing:\n```\n{{{code_to_edit}}}\n```\n\nHere is the edit requested:\n\"{{{user_input}}}\"\n\nHere is the code after editing:"}, "type": "object"}' required={false} default="{'edit': 'Here is the code before editing:\n```\n{{{code_to_edit}}}\n```\n\nHere is the edit requested:\n"{{{user_input}}}"\n\nHere is the code after editing:'}"/> <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "The API key for the LLM provider.", "type": "string"}' required={false} default=""/> diff --git a/docs/docs/reference/Models/togetherllm.md b/docs/docs/reference/Models/togetherllm.md index e0dc35de..a7eae025 100644 --- a/docs/docs/reference/Models/togetherllm.md +++ b/docs/docs/reference/Models/togetherllm.md @@ -5,8 +5,8 @@ import ClassPropertyRef from '@site/src/components/ClassPropertyRef.tsx'; The Together API is a cloud platform for running large AI models. You can sign up [here](https://api.together.xyz/signup), copy your API key on the initial welcome screen, and then hit the play button on any model from the [Together Models list](https://docs.together.ai/docs/models-inference). Change `~/.continue/config.py` to look like this: ```python title="~/.continue/config.py" -from continuedev.src.continuedev.core.models import Models -from continuedev.src.continuedev.libs.llm.together import TogetherLLM +from continuedev.core.models import Models +from continuedev.libs.llm.together import TogetherLLM config = ContinueConfig( ... @@ -30,14 +30,19 @@ config = ContinueConfig( <ClassPropertyRef name='api_key' details='{"title": "Api Key", "description": "Together API key", "type": "string"}' required={true} default=""/> <ClassPropertyRef name='title' details='{"title": "Title", "description": "A title that will identify this model in the model selection dropdown", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> -<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='unique_id' details='{"title": "Unique Id", "description": "The unique ID of the user.", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='model' details='{"title": "Model", "description": "The name of the model to be used (e.g. gpt-4, codellama)", "default": "togethercomputer/RedPajama-INCITE-7B-Instruct", "type": "string"}' required={false} default="togethercomputer/RedPajama-INCITE-7B-Instruct"/> -<ClassPropertyRef name='max_tokens' details='{"title": "Max Tokens", "description": "The maximum number of tokens to generate.", "default": 1024, "type": "integer"}' required={false} default="1024"/> +<ClassPropertyRef name='system_message' details='{"title": "System Message", "description": "A system message that will always be followed by the LLM", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='context_length' details='{"title": "Context Length", "description": "The maximum context length of the LLM in tokens, as counted by count_tokens.", "default": 2048, "type": "integer"}' required={false} default="2048"/> <ClassPropertyRef name='stop_tokens' details='{"title": "Stop Tokens", "description": "Tokens that will stop the completion.", "type": "array", "items": {"type": "string"}}' required={false} default=""/> +<ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_p' details='{"title": "Top P", "description": "The top_p of the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='top_k' details='{"title": "Top K", "description": "The top_k of the completion.", "type": "integer"}' required={false} default=""/> +<ClassPropertyRef name='presence_penalty' details='{"title": "Presence Penalty", "description": "The presence penalty Aof the completion.", "type": "number"}' required={false} default=""/> +<ClassPropertyRef name='frequency_penalty' details='{"title": "Frequency Penalty", "description": "The frequency penalty of the completion.", "type": "number"}' required={false} default=""/> <ClassPropertyRef name='timeout' details='{"title": "Timeout", "description": "Set the timeout for each request to the LLM. If you are running a local LLM that takes a while to respond, you might want to set this to avoid timeouts.", "default": 300, "type": "integer"}' required={false} default="300"/> <ClassPropertyRef name='verify_ssl' details='{"title": "Verify Ssl", "description": "Whether to verify SSL certificates for requests.", "type": "boolean"}' required={false} default=""/> <ClassPropertyRef name='ca_bundle_path' details='{"title": "Ca Bundle Path", "description": "Path to a custom CA bundle to use when making the HTTP request", "type": "string"}' required={false} default=""/> <ClassPropertyRef name='proxy' details='{"title": "Proxy", "description": "Proxy URL to use when making the HTTP request", "type": "string"}' required={false} default=""/> +<ClassPropertyRef name='headers' details='{"title": "Headers", "description": "Headers to use when making the HTTP request", "type": "object", "additionalProperties": {"type": "string"}}' required={false} default=""/> <ClassPropertyRef name='prompt_templates' details='{"title": "Prompt Templates", "description": "A dictionary of prompt templates that can be used to customize the behavior of the LLM in certain situations. For example, set the \"edit\" key in order to change the prompt that is used for the /edit slash command. Each value in the dictionary is a string templated in mustache syntax, and filled in at runtime with the variables specific to the situation. See the documentation for more information.", "default": {"edit": "Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags."}, "type": "object"}' required={false} default="{'edit': 'Consider the following code:\n```\n{{{code_to_edit}}}\n```\nEdit the code to perfectly satisfy the following user request:\n{{{user_input}}}\nOutput nothing except for the code. No code block, no English explanation, no start/end tags.'}"/> diff --git a/docs/docs/reference/config.md b/docs/docs/reference/config.md index a160a5c2..16d6bebe 100644 --- a/docs/docs/reference/config.md +++ b/docs/docs/reference/config.md @@ -11,7 +11,7 @@ Continue can be deeply customized by editing the `ContinueConfig` object in `~/. <ClassPropertyRef name='steps_on_startup' details='{"title": "Steps On Startup", "description": "Steps that will be automatically run at the beginning of a new session", "default": [], "type": "array", "items": {"$ref": "#/definitions/Step"}}' required={false} default="[]"/> <ClassPropertyRef name='disallowed_steps' details='{"title": "Disallowed Steps", "description": "Steps that are not allowed to be run, and will be skipped if attempted", "default": [], "type": "array", "items": {"type": "string"}}' required={false} default="[]"/> <ClassPropertyRef name='allow_anonymous_telemetry' details='{"title": "Allow Anonymous Telemetry", "description": "If this field is set to True, we will collect anonymous telemetry as described in the documentation page on telemetry. If set to False, we will not collect any data.", "default": true, "type": "boolean"}' required={false} default="True"/> -<ClassPropertyRef name='models' details='{"title": "Models", "description": "Configuration for the models used by Continue. Read more about how to configure models in the documentation.", "default": {"default": {"title": null, "system_message": null, "context_length": 2048, "model": "gpt-4", "max_tokens": 1024, "stop_tokens": null, "timeout": 300, "verify_ssl": null, "ca_bundle_path": null, "proxy": null, "prompt_templates": {}, "api_key": null, "llm": null, "class_name": "OpenAIFreeTrial"}, "summarize": {"title": null, "system_message": null, "context_length": 2048, "model": "gpt-3.5-turbo", "max_tokens": 1024, "stop_tokens": null, "timeout": 300, "verify_ssl": null, "ca_bundle_path": null, "proxy": null, "prompt_templates": {}, "api_key": null, "llm": null, "class_name": "OpenAIFreeTrial"}, "edit": null, "chat": null, "saved": []}, "allOf": [{"$ref": "#/definitions/Models"}]}' required={false} default="{'default': {'title': None, 'system_message': None, 'context_length': 2048, 'model': 'gpt-4', 'max_tokens': 1024, 'stop_tokens': None, 'timeout': 300, 'verify_ssl': None, 'ca_bundle_path': None, 'proxy': None, 'prompt_templates': {}, 'api_key': None, 'llm': None, 'class_name': 'OpenAIFreeTrial'}, 'summarize': {'title': None, 'system_message': None, 'context_length': 2048, 'model': 'gpt-3.5-turbo', 'max_tokens': 1024, 'stop_tokens': None, 'timeout': 300, 'verify_ssl': None, 'ca_bundle_path': None, 'proxy': None, 'prompt_templates': {}, 'api_key': None, 'llm': None, 'class_name': 'OpenAIFreeTrial'}, 'edit': None, 'chat': None, 'saved': []}"/> +<ClassPropertyRef name='models' details='{"title": "Models", "description": "Configuration for the models used by Continue. Read more about how to configure models in the documentation.", "default": {"default": {"title": null, "model": "gpt-4", "system_message": null, "context_length": 2048, "stop_tokens": null, "temperature": null, "top_p": null, "top_k": null, "presence_penalty": null, "frequency_penalty": null, "timeout": 300, "verify_ssl": null, "ca_bundle_path": null, "proxy": null, "headers": null, "prompt_templates": {}, "template_messages": null, "api_key": null, "llm": null, "class_name": "OpenAIFreeTrial"}, "summarize": {"title": null, "model": "gpt-3.5-turbo", "system_message": null, "context_length": 2048, "stop_tokens": null, "temperature": null, "top_p": null, "top_k": null, "presence_penalty": null, "frequency_penalty": null, "timeout": 300, "verify_ssl": null, "ca_bundle_path": null, "proxy": null, "headers": null, "prompt_templates": {}, "template_messages": null, "api_key": null, "llm": null, "class_name": "OpenAIFreeTrial"}, "edit": null, "chat": null, "saved": []}, "allOf": [{"$ref": "#/definitions/Models"}]}' required={false} default="{'default': {'title': None, 'model': 'gpt-4', 'system_message': None, 'context_length': 2048, 'stop_tokens': None, 'temperature': None, 'top_p': None, 'top_k': None, 'presence_penalty': None, 'frequency_penalty': None, 'timeout': 300, 'verify_ssl': None, 'ca_bundle_path': None, 'proxy': None, 'headers': None, 'prompt_templates': {}, 'template_messages': None, 'api_key': None, 'llm': None, 'class_name': 'OpenAIFreeTrial'}, 'summarize': {'title': None, 'model': 'gpt-3.5-turbo', 'system_message': None, 'context_length': 2048, 'stop_tokens': None, 'temperature': None, 'top_p': None, 'top_k': None, 'presence_penalty': None, 'frequency_penalty': None, 'timeout': 300, 'verify_ssl': None, 'ca_bundle_path': None, 'proxy': None, 'headers': None, 'prompt_templates': {}, 'template_messages': None, 'api_key': None, 'llm': None, 'class_name': 'OpenAIFreeTrial'}, 'edit': None, 'chat': None, 'saved': []}"/> <ClassPropertyRef name='temperature' details='{"title": "Temperature", "description": "The temperature parameter for sampling from the LLM. Higher temperatures will result in more random output, while lower temperatures will result in more predictable output. This value ranges from 0 to 1.", "default": 0.5, "type": "number"}' required={false} default="0.5"/> <ClassPropertyRef name='custom_commands' details='{"title": "Custom Commands", "description": "An array of custom commands that allow you to reuse prompts. Each has name, description, and prompt properties. When you enter /<name> in the text input, it will act as a shortcut to the prompt.", "default": [{"name": "test", "prompt": "Write a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.", "description": "This is an example custom command. Use /config to edit it and create more"}], "type": "array", "items": {"$ref": "#/definitions/CustomCommand"}}' required={false} default="[{'name': 'test', 'prompt': "Write a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.", 'description': 'This is an example custom command. Use /config to edit it and create more'}]"/> <ClassPropertyRef name='slash_commands' details='{"title": "Slash Commands", "description": "An array of slash commands that let you map custom Steps to a shortcut.", "default": [], "type": "array", "items": {"$ref": "#/definitions/SlashCommand"}}' required={false} default="[]"/> diff --git a/docs/docs/walkthroughs/codebase-embeddings.md b/docs/docs/walkthroughs/codebase-embeddings.md index 012395b6..66e7bc1b 100644 --- a/docs/docs/walkthroughs/codebase-embeddings.md +++ b/docs/docs/walkthroughs/codebase-embeddings.md @@ -46,8 +46,6 @@ config=ContinueConfig( 6. When you open a workspace, Continue will generate the embeddings. You can then enter '/codebase \<QUESTION\>' to ask a question with embeddings-based retrieval. 7. Please share feedback in [Discord](https://discord.gg/NWtdYexhMs)! -> Note: There is a known bug that requires different imports for the PyPI package. If you encounter "No module named 'continuedev.src', you should replace all instances of 'continuedev.src.continuedev' in `config.py` with just 'continuedev'. - ## Parameters After retrieving the top `n_retrieve` results from the vector database, an additional re-reranking step uses 2 LLM calls to select the top `n_final` results to use to answer the question. If you want to increase the speed of the query at the cost of relevancy, you can skip the re-ranking step by setting `use_reranking` to `False`. Then the top `n_final` results will just be directly calculated from the vector database. diff --git a/docs/docs/walkthroughs/codellama.md b/docs/docs/walkthroughs/codellama.md index 4bf18c57..298d996e 100644 --- a/docs/docs/walkthroughs/codellama.md +++ b/docs/docs/walkthroughs/codellama.md @@ -12,8 +12,8 @@ If you haven't already installed Continue, you can do that [here](https://market 4. Update your Continue config file to look like this: ```python -from continuedev.src.continuedev.core.models import Models -from continuedev.src.continuedev.libs.llm.together import TogetherLLM +from continuedev.core.models import Models +from continuedev.libs.llm.together import TogetherLLM config = ContinueConfig( ... @@ -33,7 +33,7 @@ config = ContinueConfig( 3. Change your Continue config file to look like this: ```python -from continuedev.src.continuedev.libs.llm.ollama import Ollama +from continuedev.libs.llm.ollama import Ollama config = ContinueConfig( ... @@ -53,8 +53,8 @@ config = ContinueConfig( 2. Change your Continue config file to look like this: ```python -from continuedev.src.continuedev.core.models import Models -from continuedev.src.continuedev.libs.llm.replicate import ReplicateLLM +from continuedev.core.models import Models +from continuedev.libs.llm.replicate import ReplicateLLM config = ContinueConfig( ... |