Preview (#541)

* Strong typing (#533)

* refactor: :recycle: get rid of continuedev.src.continuedev structure

* refactor: :recycle: switching back to server folder

* feat: :sparkles: make config.py imports shorter

* feat: :bookmark: publish as pre-release vscode extension

* refactor: :recycle: refactor and add more completion params to ui

* build: :building_construction: download from preview S3

* fix: :bug: fix paths

* fix: :green_heart: package:pre-release

* ci: :green_heart: more time for tests

* fix: :green_heart: fix build scripts

* fix: :bug: fix import in run.py

* fix: :bookmark: update version to try again

* ci: 💚 Update package.json version [skip ci]

* refactor: :fire: don't check for old extensions version

* fix: :bug: small bug fixes

* fix: :bug: fix config.py import paths

* ci: 💚 Update package.json version [skip ci]

* ci: :green_heart: platform-specific builds test #1

* feat: :green_heart: ship with binary

* fix: :green_heart: fix copy statement to include.exe for windows

* fix: :green_heart: cd extension before packaging

* chore: :loud_sound: count tokens generated

* fix: :green_heart: remove npm_config_arch

* fix: :green_heart: publish as pre-release!

* chore: :bookmark: update version

* perf: :green_heart: hardcode distro paths

* fix: :bug: fix yaml syntax error

* chore: :bookmark: update version

* fix: :green_heart: update permissions and version

* feat: :bug: kill old server if needed

* feat: :lipstick: update marketplace icon for pre-release

* ci: 💚 Update package.json version [skip ci]

* feat: :sparkles: auto-reload for config.py

* feat: :wrench: update default config.py imports

* feat: :sparkles: codelens in config.py

* feat: :sparkles: select model param count from UI

* ci: 💚 Update package.json version [skip ci]

* feat: :sparkles: more model options, ollama error handling

* perf: :zap: don't show server loading immediately

* fix: :bug: fixing small UI details

* ci: 💚 Update package.json version [skip ci]

* feat: :rocket: headers param on LLM class

* fix: :bug: fix headers for openai.;y

* feat: :sparkles: highlight code on cmd+shift+L

* ci: 💚 Update package.json version [skip ci]

* feat: :lipstick: sticky top bar in gui.tsx

* fix: :loud_sound: websocket logging and horizontal scrollbar

* ci: 💚 Update package.json version [skip ci]

* feat: :sparkles: allow AzureOpenAI Service through GGML

* ci: 💚 Update package.json version [skip ci]

* fix: :bug: fix automigration

* ci: 💚 Update package.json version [skip ci]

* ci: :green_heart: upload binaries in ci, download apple silicon

* chore: :fire: remove notes

* fix: :green_heart: use curl to download binary

* fix: :green_heart: set permissions on apple silicon binary

* fix: :green_heart: testing

* fix: :green_heart: cleanup file

* fix: :green_heart: fix preview.yaml

* fix: :green_heart: only upload once per binary

* fix: :green_heart: install rosetta

* ci: :green_heart: download binary after tests

* ci: 💚 Update package.json version [skip ci]

* ci: :green_heart: prepare ci for merge to main

---------

Co-authored-by: GitHub Action <action@github.com>

1 files changed, 64 insertions, 0 deletions

diff --git a/server/continuedev/libs/util/strings.py b/server/continuedev/libs/util/strings.py
new file mode 100644
index 00000000..f2b6035f
--- /dev/null
+++ b/server/continuedev/libs/util/strings.py
@@ -0,0 +1,64 @@
+from typing import Tuple
+
+
+def dedent_and_get_common_whitespace(s: str) -> Tuple[str, str]:
+    lines = s.splitlines()
+    if len(lines) == 0:
+        return "", ""
+
+    # Longest common whitespace prefix
+    lcp = lines[0].split(lines[0].strip())[0]
+    # Iterate through the lines
+    for i in range(1, len(lines)):
+        # Empty lines are wildcards
+        if lines[i].strip() == "":
+            continue  # hey that's us!
+        # Iterate through the leading whitespace characters of the current line
+        for j in range(0, len(lcp)):
+            # If it doesn't have the same whitespace as lcp, then update lcp
+            if j >= len(lines[i]) or lcp[j] != lines[i][j]:
+                lcp = lcp[:j]
+                if lcp == "":
+                    return s, ""
+                break
+
+    return "\n".join(map(lambda x: x.lstrip(lcp), lines)), lcp
+
+
+def strip_code_block(s: str) -> str:
+    """
+    Strips the code block from a string, if it has one.
+    """
+    if s.startswith("```\n") and s.endswith("\n```"):
+        return s[4:-4]
+    elif s.startswith("```") and s.endswith("```"):
+        return s[3:-3]
+    elif s.startswith("`") and s.endswith("`"):
+        return s[1:-1]
+    return s
+
+
+def remove_quotes_and_escapes(output: str) -> str:
+    """
+    Clean up the output of the completion API, removing unnecessary escapes and quotes
+    """
+    output = output.strip()
+
+    # Replace smart quotes
+    output = output.replace("“", '"')
+    output = output.replace("”", '"')
+    output = output.replace("‘", "'")
+    output = output.replace("’", "'")
+
+    # Remove escapes
+    output = output.replace('\\"', '"')
+    output = output.replace("\\'", "'")
+    output = output.replace("\\n", "\n")
+    output = output.replace("\\t", "\t")
+    output = output.replace("\\\\", "\\")
+    if (output.startswith('"') and output.endswith('"')) or (
+        output.startswith("'") and output.endswith("'")
+    ):
+        output = output[1:-1]
+
+    return output