feat(auto-edit): fix the temperature value regression with the auto-e… #26137
ci.yml
on: push
fast_tests_matrix_prep
0s
slow_tests_matrix_prep
0s
build
2m 25s
Matrix: test-integration
Matrix: test-unit
Matrix: test-e2e
Annotations
10 errors, 15 warnings, and 3 notices
test-e2e (macos)
Worker teardown timeout of 30000ms exceeded.
Failed worker ran 9 tests:
test/e2e/chat-atFile.test.ts:229:3 › editing a chat message with @-mention
test/e2e/chat-atFile.test.ts:271:3 › @-mention file range
test/e2e/chat-atFile.test.ts:310:3 › @-mention symbol in chat
test/e2e/chat-atFile.test.ts:367:3 › Add Selection to Cody Chat
test/e2e/chat-history.test.ts:17:3 › restore chat from sidebar history view
test/e2e/chat-history.test.ts:50:3 › delete chat from sidebar history view
test/e2e/chat-input.test.ts:191:3 › chat model selector
test/e2e/chat-input.test.ts:242:3 › editing follow-up messages in chat view
test/e2e/chat-messages.test.ts:17:5 › chat assistant response code buttons
|
test-e2e (macos)
Worker teardown timeout of 30000ms exceeded.
Failed worker ran 1 test:
test/e2e/chat-messages.test.ts:17:5 › chat assistant response code buttons
|
auto-edits.test.ts:182:3 › autoedits: triggers a multi-line diff view when edit affects existing lines:
vscode/test/e2e/auto-edits.test.ts#L160
1) auto-edits.test.ts:182:3 › autoedits: triggers a multi-line diff view when edit affects existing lines
Error: Screenshot comparison failed:
44076 pixels (ratio 0.03 of all image pixels) are different.
Expected: /Users/runner/work/cody/cody/vscode/test/e2e/__snapshots__/auto-edits.test.ts/macos/autoedits-suffix-decoration-70-suggested.png
Received: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-1c59f-edit-affects-existing-lines/macos/autoedits-suffix-decoration-70-suggested-actual.png
Diff: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-1c59f-edit-affects-existing-lines/macos/autoedits-suffix-decoration-70-suggested-diff.png
Call log:
- expect.toHaveScreenshot(macos/autoedits-suffix-decoration-70-suggested.png) with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 44077 pixels (ratio 0.03 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 44076 pixels (ratio 0.03 of all image pixels) are different.
158 | await page.waitForTimeout(500)
159 |
> 160 | await expect(page).toHaveScreenshot(
| ^
161 | [snapshotPlatform, `${snapshotName}-suggested.png`],
162 | SNAPSHOT_ASSERTIONS
163 | )
at autoeditsTestHelper (/Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:160:28)
at /Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:186:9
|
auto-edits.test.ts:182:3 › autoedits: triggers a multi-line diff view when edit affects existing lines:
vscode/test/e2e/auto-edits.test.ts#L160
1) auto-edits.test.ts:182:3 › autoedits: triggers a multi-line diff view when edit affects existing lines
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
44076 pixels (ratio 0.03 of all image pixels) are different.
Expected: /Users/runner/work/cody/cody/vscode/test/e2e/__snapshots__/auto-edits.test.ts/macos/autoedits-suffix-decoration-70-suggested.png
Received: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-1c59f-edit-affects-existing-lines-retry1/macos/autoedits-suffix-decoration-70-suggested-actual.png
Diff: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-1c59f-edit-affects-existing-lines-retry1/macos/autoedits-suffix-decoration-70-suggested-diff.png
Call log:
- expect.toHaveScreenshot(macos/autoedits-suffix-decoration-70-suggested.png) with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 44077 pixels (ratio 0.03 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 44076 pixels (ratio 0.03 of all image pixels) are different.
158 | await page.waitForTimeout(500)
159 |
> 160 | await expect(page).toHaveScreenshot(
| ^
161 | [snapshotPlatform, `${snapshotName}-suggested.png`],
162 | SNAPSHOT_ASSERTIONS
163 | )
at autoeditsTestHelper (/Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:160:28)
at /Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:186:9
|
auto-edits.test.ts:198:3 › autoedits: triggers an inline completion when edit is an insertion immediately after the cursor:
vscode/test/e2e/auto-edits.test.ts#L160
2) auto-edits.test.ts:198:3 › autoedits: triggers an inline completion when edit is an insertion immediately after the cursor
Error: Screenshot comparison failed:
20127 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/cody/cody/vscode/test/e2e/__snapshots__/auto-edits.test.ts/macos/autoedits-inline-completion-29-suggested.png
Received: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-912dc-mmediately-after-the-cursor/macos/autoedits-inline-completion-29-suggested-actual.png
Diff: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-912dc-mmediately-after-the-cursor/macos/autoedits-inline-completion-29-suggested-diff.png
Call log:
- expect.toHaveScreenshot(macos/autoedits-inline-completion-29-suggested.png) with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 28719 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 8628 pixels (ratio 0.01 of all image pixels) are different.
- waiting 250ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 20127 pixels (ratio 0.01 of all image pixels) are different.
158 | await page.waitForTimeout(500)
159 |
> 160 | await expect(page).toHaveScreenshot(
| ^
161 | [snapshotPlatform, `${snapshotName}-suggested.png`],
162 | SNAPSHOT_ASSERTIONS
163 | )
at autoeditsTestHelper (/Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:160:28)
at /Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:202:9
|
auto-edits.test.ts:198:3 › autoedits: triggers an inline completion when edit is an insertion immediately after the cursor:
vscode/test/e2e/auto-edits.test.ts#L160
2) auto-edits.test.ts:198:3 › autoedits: triggers an inline completion when edit is an insertion immediately after the cursor
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
20127 pixels (ratio 0.01 of all image pixels) are different.
Expected: /Users/runner/work/cody/cody/vscode/test/e2e/__snapshots__/auto-edits.test.ts/macos/autoedits-inline-completion-29-suggested.png
Received: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-912dc-mmediately-after-the-cursor-retry1/macos/autoedits-inline-completion-29-suggested-actual.png
Diff: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-912dc-mmediately-after-the-cursor-retry1/macos/autoedits-inline-completion-29-suggested-diff.png
Call log:
- expect.toHaveScreenshot(macos/autoedits-inline-completion-29-suggested.png) with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 28719 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 8628 pixels (ratio 0.01 of all image pixels) are different.
- waiting 250ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 20127 pixels (ratio 0.01 of all image pixels) are different.
158 | await page.waitForTimeout(500)
159 |
> 160 | await expect(page).toHaveScreenshot(
| ^
161 | [snapshotPlatform, `${snapshotName}-suggested.png`],
162 | SNAPSHOT_ASSERTIONS
163 | )
at autoeditsTestHelper (/Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:160:28)
at /Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:202:9
|
auto-edits.test.ts:214:3 › autoedits: triggers an inline decoration when an inline completion is desired:
vscode/test/e2e/auto-edits.test.ts#L160
3) auto-edits.test.ts:214:3 › autoedits: triggers an inline decoration when an inline completion is desired, but the insertion position is before the cursor position
Error: Screenshot comparison failed:
26568 pixels (ratio 0.02 of all image pixels) are different.
Expected: /Users/runner/work/cody/cody/vscode/test/e2e/__snapshots__/auto-edits.test.ts/macos/autoedits-inline-decoration-insertion-30-suggested.png
Received: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-c9082--before-the-cursor-position/macos/autoedits-inline-decoration-insertion-30-suggested-actual.png
Diff: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-c9082--before-the-cursor-position/macos/autoedits-inline-decoration-insertion-30-suggested-diff.png
Call log:
- expect.toHaveScreenshot(macos/autoedits-inline-decoration-insertion-30-suggested.png) with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 26550 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 26568 pixels (ratio 0.02 of all image pixels) are different.
158 | await page.waitForTimeout(500)
159 |
> 160 | await expect(page).toHaveScreenshot(
| ^
161 | [snapshotPlatform, `${snapshotName}-suggested.png`],
162 | SNAPSHOT_ASSERTIONS
163 | )
at autoeditsTestHelper (/Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:160:28)
at /Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:218:9
|
auto-edits.test.ts:214:3 › autoedits: triggers an inline decoration when an inline completion is desired:
vscode/test/e2e/auto-edits.test.ts#L160
3) auto-edits.test.ts:214:3 › autoedits: triggers an inline decoration when an inline completion is desired, but the insertion position is before the cursor position
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
26550 pixels (ratio 0.02 of all image pixels) are different.
Expected: /Users/runner/work/cody/cody/vscode/test/e2e/__snapshots__/auto-edits.test.ts/macos/autoedits-inline-decoration-insertion-30-suggested.png
Received: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-c9082--before-the-cursor-position-retry1/macos/autoedits-inline-decoration-insertion-30-suggested-actual.png
Diff: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-c9082--before-the-cursor-position-retry1/macos/autoedits-inline-decoration-insertion-30-suggested-diff.png
Call log:
- expect.toHaveScreenshot(macos/autoedits-inline-decoration-insertion-30-suggested.png) with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 26550 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 26550 pixels (ratio 0.02 of all image pixels) are different.
158 | await page.waitForTimeout(500)
159 |
> 160 | await expect(page).toHaveScreenshot(
| ^
161 | [snapshotPlatform, `${snapshotName}-suggested.png`],
162 | SNAPSHOT_ASSERTIONS
163 | )
at autoeditsTestHelper (/Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:160:28)
at /Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:218:9
|
auto-edits.test.ts:230:3 › autoedits: triggers inline decorations when multiple insertions are required on different lines:
vscode/test/e2e/auto-edits.test.ts#L160
4) auto-edits.test.ts:230:3 › autoedits: triggers inline decorations when multiple insertions are required on different lines
Error: Screenshot comparison failed:
23241 pixels (ratio 0.02 of all image pixels) are different.
Expected: /Users/runner/work/cody/cody/vscode/test/e2e/__snapshots__/auto-edits.test.ts/macos/autoedits-inline-decoration-multiple-insertions-different-lines-44-suggested.png
Received: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-d9853-required-on-different-lines/macos/autoedits-inline-decoration-multiple-insertions-different-lines-44-suggested-actual.png
Diff: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-d9853-required-on-different-lines/macos/autoedits-inline-decoration-multiple-insertions-different-lines-44-suggested-diff.png
Call log:
- expect.toHaveScreenshot(macos/autoedits-inline-decoration-multiple-insertions-different-lines-44-suggested.png) with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 23240 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 23241 pixels (ratio 0.02 of all image pixels) are different.
158 | await page.waitForTimeout(500)
159 |
> 160 | await expect(page).toHaveScreenshot(
| ^
161 | [snapshotPlatform, `${snapshotName}-suggested.png`],
162 | SNAPSHOT_ASSERTIONS
163 | )
at autoeditsTestHelper (/Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:160:28)
at /Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:234:9
|
auto-edits.test.ts:230:3 › autoedits: triggers inline decorations when multiple insertions are required on different lines:
vscode/test/e2e/auto-edits.test.ts#L160
4) auto-edits.test.ts:230:3 › autoedits: triggers inline decorations when multiple insertions are required on different lines
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
23240 pixels (ratio 0.02 of all image pixels) are different.
Expected: /Users/runner/work/cody/cody/vscode/test/e2e/__snapshots__/auto-edits.test.ts/macos/autoedits-inline-decoration-multiple-insertions-different-lines-44-suggested.png
Received: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-d9853-required-on-different-lines-retry1/macos/autoedits-inline-decoration-multiple-insertions-different-lines-44-suggested-actual.png
Diff: /Users/runner/work/cody/cody/vscode/test-results/auto-edits-autoedits-trigg-d9853-required-on-different-lines-retry1/macos/autoedits-inline-decoration-multiple-insertions-different-lines-44-suggested-diff.png
Call log:
- expect.toHaveScreenshot(macos/autoedits-inline-decoration-multiple-insertions-different-lines-44-suggested.png) with timeout 5000ms
- verifying given screenshot expectation
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- 23240 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- taking page screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- captured a stable screenshot
- 23240 pixels (ratio 0.02 of all image pixels) are different.
158 | await page.waitForTimeout(500)
159 |
> 160 | await expect(page).toHaveScreenshot(
| ^
161 | [snapshotPlatform, `${snapshotName}-suggested.png`],
162 | SNAPSHOT_ASSERTIONS
163 | )
at autoeditsTestHelper (/Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:160:28)
at /Users/runner/work/cody/cody/vscode/test/e2e/auto-edits.test.ts:234:9
|
Slow Test:
vscode/chat-atFile.test.ts#L1
vscode/chat-atFile.test.ts took 1.4m
|
Slow Test:
vscode/command-custom.test.ts#L1
vscode/command-custom.test.ts took 1.2m
|
Slow Test:
vscode/initial-context.test.ts#L1
vscode/initial-context.test.ts took 54.3s
|
Slow Test:
vscode/command-core.test.ts#L1
vscode/command-core.test.ts took 46.6s
|
Slow Test:
vscode/command-edit.test.ts#L1
vscode/command-edit.test.ts took 44.8s
|
Slow Test:
vscode/command-custom.test.ts#L1
vscode/command-custom.test.ts took 1.1m
|
Slow Test:
vscode/initial-context.test.ts#L1
vscode/initial-context.test.ts took 47.6s
|
Slow Test:
vscode/command-edit.test.ts#L1
vscode/command-edit.test.ts took 38.0s
|
Slow Test:
vscode/command-core.test.ts#L1
vscode/command-core.test.ts took 37.1s
|
Slow Test:
vscode/chat-rateLimit.test.ts#L1
vscode/chat-rateLimit.test.ts took 31.0s
|
Slow Test:
vscode\chat-atFile.test.ts#L1
vscode\chat-atFile.test.ts took 1.6m
|
Slow Test:
vscode\command-custom.test.ts#L1
vscode\command-custom.test.ts took 1.4m
|
Slow Test:
vscode\initial-context.test.ts#L1
vscode\initial-context.test.ts took 1.1m
|
Slow Test:
vscode\command-core.test.ts#L1
vscode\command-core.test.ts took 54.2s
|
Slow Test:
vscode\command-edit.test.ts#L1
vscode\command-edit.test.ts took 51.2s
|
🎭 Playwright Run Summary
17 skipped
41 passed (10.2m)
|
🎭 Playwright Run Summary
8 failed
auto-edits.test.ts:182:3 › autoedits: triggers a multi-line diff view when edit affects existing lines
auto-edits.test.ts:198:3 › autoedits: triggers an inline completion when edit is an insertion immediately after the cursor
auto-edits.test.ts:214:3 › autoedits: triggers an inline decoration when an inline completion is desired, but the insertion position is before the cursor position
auto-edits.test.ts:230:3 › autoedits: triggers inline decorations when multiple insertions are required on different lines
auto-edits.test.ts:246:3 › autoedits: triggers inline decorations when multiple separate insertions are required on the same line
auto-edits.test.ts:260:1 › autoedits: triggers a suffix decoration and renders correctly in files that use tab based indentation
auto-edits.test.ts:274:1 › autoedits: does not show any suggestion if the suffix decoration spans further than the end of the file
chat-messages.test.ts:17:5 › chat assistant response code buttons ──────────────────────────────
1 flaky
chat-atFile.test.ts:229:3 › editing a chat message with @-mention ──────────────────────────────
10 skipped
39 passed (15.1m)
|
🎭 Playwright Run Summary
17 skipped
41 passed (11.8m)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
playwright-recordings macos
|
13.2 MB |
|