141d2e0cc4
Description: This PR introduces support for counting tokens within the Azure code and updating these counts in Redis. The token counting logic is embedded directly in the Azure code rather than using a standardized point for all token counting logic. Reasoning: • Azure does not currently support obtaining token usage from their streaming endpoint, unlike OpenAI. • To enable immediate functionality, the token counting logic is placed within the Azure code itself. • The implementation supports GPT-4o. Future Considerations: • When Azure eventually adds support for token usage from the streaming endpoint, we will migrate to using Azure’s built-in capabilities. • This will ensure full utilization of Azure OpenAI features as they achieve parity with OpenAI. Changes: • Added token counting logic to the Azure code. • Updated Redis with the token counts. Testing: • Verified the implementation works with GPT-4o. Conclusion: This is a temporary solution to enable token counting in Azure. We will adapt our approach as Azure enhances its feature set to include token usage from their streaming endpoint. ## Test plan Tested locally with debugger <!-- All pull requests REQUIRE a test plan: https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> ## Changelog <!-- 1. Ensure your pull request title is formatted as: $type($domain): $what 2. Add bullet list items for each additional detail you want to cover (see example below) 3. You can edit this after the pull request was merged, as long as release shipping it hasn't been promoted to the public. 4. For more information, please see this how-to https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c? Audience: TS/CSE > Customers > Teammates (in that order). Cheat sheet: $type = chore|fix|feat $domain: source|search|ci|release|plg|cody|local|... --> <!-- Example: Title: fix(search): parse quotes with the appropriate context Changelog section: ## Changelog - When a quote is used with regexp pattern type, then ... - Refactored underlying code. --> |
||
|---|---|---|
| .. | ||
| branded | ||
| browser | ||
| build-config | ||
| client-api | ||
| codeintellify | ||
| cody-context-filters-test-dataset | ||
| cody-shared | ||
| cody-ui | ||
| common | ||
| eslint-plugin-wildcard | ||
| extension-api | ||
| extension-api-types | ||
| http-client | ||
| jetbrains | ||
| observability-client | ||
| observability-server | ||
| shared | ||
| storybook | ||
| template-parser | ||
| testing | ||
| vscode | ||
| web | ||
| web-sveltekit | ||
| wildcard | ||
| BUILD.bazel | ||
| README.md | ||
Frontend packages
List
- web: The web application deployed to http://sourcegraph.com/
- browser: The Sourcegraph browser extension adds tooltips to code on different code hosts.
- vscode: The Sourcegraph VS Code extension.
- extension-api: The Sourcegraph extension API types for the Sourcegraph extensions. Published as
sourcegraph. - extension-api-types: The Sourcegraph extension API types for client applications that embed Sourcegraph extensions and need to communicate with them. Published as
@sourcegraph/extension-api-types. - sandboxes: All demos-mvp (minimum viable product) for the Sourcegraph web application.
- shared: Contains common TypeScript/React/SCSS client code shared between the browser extension and the web app. Everything in this package is code-host agnostic.
- branded: Contains React components and implements the visual design language we use across our web app and e.g. in the options menu of the browser extension. Over time, components from
sharedandbrandedpackages should be moved into thewildcardpackage. - wildcard: Package that encapsulates storybook configuration and contains our Wildcard design system components. If we're using a component in two or more different areas (e.g.
web-appandbrowser-extension) then it should live in thewildcardpackage. Otherwise the components should be better colocated with the code where they're actually used. - search: Search-related code that may be shared between all clients, both branded (e.g. web, VS Code extension) and unbranded (e.g. browser extension)
- storybook: Storybook configuration.
Further migration plan
-
Fix circular dependency in TS project-references graph wildcard package should not rely on web and probably shared, branded too. Ideally it should be an independent self-contained package.
-
Decide on package naming and update existing package names. Especially it should be done for a shared package because we have multiple
sharedfolders inside of other packages. It's hard to understand from where dependency is coming from and it's not possible to refactor import paths using find-and-replace. -
Investigate if we can painlessly switch to
npmworkspaces. -
Content of packages shared and branded should be moved to wildcard and refactored using the latest FE rules and conventions. Having different packages clearly communicates the migration plan. Developers first should look for components in the wildcard package and then fall-back to legacy packages if wildcard doesn't have the solution to their problem yet.
-
shared contains utility functions, types, polyfills, etc which is not a part of the Wildcard component library. These modules should be moved into utils package and other new packages: e.g. api for GraphQL client and type generators, etc.
-
Packages should use package name (e.g.
@sourcegraph/wildcard) for imports instead of the relative paths (e.g.../../../../wildcard/src/components/Markdown) to avoid long relative-paths and make dependency graph between packages clear. (Typescript will warn if packages have circular dependencies). It's easy to refactor such isolated packages, extract functionality into new ones, or even into new repositories.