Skip to content

Commit

Permalink
fix(auto-edit): fix the false notification for auto-edit non eligibil…
Browse files Browse the repository at this point in the history
…ity (#6899)

## Context

This PR fixes an issue where users would receive a non-eligibility
notification for auto-edits even when they were in the experiment flag
and it is enabled for them. This happens because of the flaky network
issue, and although we get error while evaluating the feature flag, we
return `false` by default. This creates noisy alerts for the users.

<img width="1054" alt="image"
src="https://github.com/user-attachments/assets/fe14d5e3-1e00-4715-ab59-48693db2249f"
/>

To address this, I added a check that ensures the user is likely viewing
the settings editor (either the JSON settings file or the graphical
Settings UI) before displaying the non-eligibility notification.

## Test plan
1. Limit the networks bandwidth using charles proxy.
2. Switch to the non pro account or override the feature flag
`cody-autoedit-experiment-enabled-flag` to false.
3. You should only get notification if either `settings.json` or the
settings UI is open
  • Loading branch information
hitesh-1997 authored Jan 31, 2025
1 parent 128f24a commit 4ca3877
Show file tree
Hide file tree
Showing 9 changed files with 63 additions and 12 deletions.
4 changes: 2 additions & 2 deletions vscode/src/autoedits/adapters/cody-gateway.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ describe('CodyGatewayAdapter', () => {
expect.objectContaining({
stream: false,
model: options.model,
temperature: 0.001,
temperature: 0.1,
response_format: { type: 'text' },
prediction: {
type: 'content',
Expand Down Expand Up @@ -100,7 +100,7 @@ describe('CodyGatewayAdapter', () => {
expect.objectContaining({
stream: false,
model: options.model,
temperature: 0.001,
temperature: 0.1,
response_format: { type: 'text' },
prediction: {
type: 'content',
Expand Down
2 changes: 1 addition & 1 deletion vscode/src/autoedits/adapters/cody-gateway.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ export class CodyGatewayAdapter implements AutoeditsModelAdapter {
const body: FireworksCompatibleRequestParams = {
stream: false,
model: options.model,
temperature: 0.001,
temperature: 0.1,
max_tokens: maxTokens,
response_format: {
type: 'text',
Expand Down
4 changes: 2 additions & 2 deletions vscode/src/autoedits/adapters/fireworks.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ describe('FireworksAdapter', () => {
expect.objectContaining({
stream: false,
model: options.model,
temperature: 0.001,
temperature: 0.1,
max_tokens: expect.any(Number),
response_format: { type: 'text' },
prediction: {
Expand Down Expand Up @@ -92,7 +92,7 @@ describe('FireworksAdapter', () => {
expect.objectContaining({
stream: false,
model: options.model,
temperature: 0.001,
temperature: 0.1,
max_tokens: expect.any(Number),
response_format: { type: 'text' },
prediction: {
Expand Down
2 changes: 1 addition & 1 deletion vscode/src/autoedits/adapters/fireworks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ export class FireworksAdapter implements AutoeditsModelAdapter {
const body: FireworksCompatibleRequestParams = {
stream: false,
model: options.model,
temperature: 0.001,
temperature: 0.1,
max_tokens: maxTokens,
response_format: {
type: 'text',
Expand Down
2 changes: 1 addition & 1 deletion vscode/src/autoedits/adapters/sourcegraph-chat.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ describe('SourcegraphChatAdapter', () => {
expect(chatOptions).toMatchObject({
model: 'anthropic/claude-2',
maxTokensToSample: getMaxOutputTokensForAutoedits(options.codeToRewrite),
temperature: 0.001,
temperature: 0.1,
prediction: {
type: 'content',
content: 'const x = 1',
Expand Down
2 changes: 1 addition & 1 deletion vscode/src/autoedits/adapters/sourcegraph-chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ export class SourcegraphChatAdapter implements AutoeditsModelAdapter {
{
model: option.model,
maxTokensToSample: maxTokens,
temperature: 0.001,
temperature: 0.1,
prediction: {
type: 'content',
content: option.codeToRewrite,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ describe('SourcegraphCompletionsAdapter', () => {
expect(params).toMatchObject({
model: 'anthropic/claude-2',
maxTokensToSample: getMaxOutputTokensForAutoedits(options.codeToRewrite),
temperature: 0.001,
temperature: 0.1,
messages: [{ speaker: 'human', text: ps`user message` }],
prediction: {
type: 'content',
Expand Down
2 changes: 1 addition & 1 deletion vscode/src/autoedits/adapters/sourcegraph-completions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export class SourcegraphCompletionsAdapter implements AutoeditsModelAdapter {
model: option.model as ModelRefStr,
messages,
maxTokensToSample: maxTokens,
temperature: 0.001,
temperature: 0.1,
prediction: {
type: 'content',
content: option.codeToRewrite,
Expand Down
55 changes: 53 additions & 2 deletions vscode/src/autoedits/create-autoedits-provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,28 @@ export function createAutoEditsProvider({
)
}

/**
* Displays an error notification to the user about non-eligibility for auto edits,
* but only if the user is currently in the Settings view (to avoid spamming them).
*
* This is because because of the flaky network issues we could evaluate the default feature flag value to false
* and show the non eligibility notification to the user even if they have access to the feature.
* Generally the users should see the notification only when they manually change the vscode config which could be either
* through the settings UI or `settings.json` file.
*
* @param {string | undefined} nonEligibilityReason - The reason why the user is currently not eligible
* for auto edits. If not provided, no notification occurs.
*/
export async function handleAutoeditsNotificationForNonEligibleUser(
nonEligibilityReason?: string
): Promise<void> {
const switchToAutocompleteText = 'Switch to autocomplete'
if (!nonEligibilityReason || !isSettingsEditorOpen()) {
return
}

const switchToAutocompleteText = 'Switch to autocomplete'
const selection = await vscode.window.showErrorMessage(
`Error: ${nonEligibilityReason ?? AUTOEDITS_NON_ELIGIBILITY_MESSAGES.FEATURE_FLAG_NOT_ELIGIBLE}`,
`Error: ${nonEligibilityReason}`,
switchToAutocompleteText
)
if (selection === switchToAutocompleteText) {
Expand All @@ -122,6 +137,42 @@ export async function handleAutoeditsNotificationForNonEligibleUser(
}
}

/**
* Checks whether the current view in VS Code is the Settings editor (JSON or UI).
*
* This function performs two checks:
* 1. Detect if the active text editor points to a known settings file (e.g., settings.json, settings.jsonc).
* 2. If there's no text editor open, examine the "Tab" label to see if it's the built-in Settings UI.
*
* Note: Using the tab's label is locale-specific; if a user runs VS Code in a non-English locale,
* or if the label changes in future VS Code versions, this heuristic may fail.
*
* @returns {boolean} True if the user is most likely viewing the Settings editor (JSON or UI), false otherwise.
*/
function isSettingsEditorOpen(): boolean {
const activeEditor = vscode.window.activeTextEditor

// 1) If there's an active text editor, check if the file name matches typical settings files
if (activeEditor) {
const fsPath = activeEditor.document.uri.fsPath
if (fsPath.endsWith('settings.json') || fsPath.endsWith('settings.jsonc')) {
return true
}
return false
}

// 2) If there's no activeTextEditor, the user might be in the graphical Settings UI or have no editor at all
const activeTab = vscode.window.tabGroups.activeTabGroup?.activeTab
if (!activeTab) {
// No tab at all: definitely not a JSON settings file;
// could be just an empty Editor area, Start page, or something else
return false
}

// The built-in Settings UI tab typically has the label "Settings" (in English).
return activeTab.label === 'Settings'
}

export function isUserEligibleForAutoeditsFeature(
autoeditsFeatureFlagEnabled: boolean,
authStatus: AuthStatus,
Expand Down

0 comments on commit 4ca3877

Please sign in to comment.