Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logcap #668

Open
wants to merge 30 commits into
base: main
Choose a base branch
from
Open

Logcap #668

wants to merge 30 commits into from

Conversation

teocns
Copy link
Contributor

@teocns teocns commented Feb 4, 2025

Capture terminal output to be displayed in the agentops dashboard. Session-based capture functionality is currently under development. Captures will be displayed in the AgentOps dashboard under "Terminal".

At the moment we capture:

  • logging, i.e logging.info(), logging.error(), etc.
  • stdout/stderr, i.e print()

Key changes in this PR:

  1. HTTP Client Refactoring
  • Consolidated HTTP methods (GET, POST, PUT, DELETE) into a single _make_request method
  • Added PUT and DELETE methods in HttpClient
  • Made payload parameter optional
  1. New Logging & Instrumentation Features
  • Added instrumentation.py for OpenTelemetry setup and configuration
  • Added log_capture.py with classes for capturing stdout/stderr and logging
  • Added session-specific logging with ANSI color support
  • Added log export functionality to send logs to AgentOps backend
  1. Session Updates
  • Added logging components to Session class
  • Added log exporter for sending logs to backend
  • Updated session cleanup to handle logging components
  1. Testing

Copy link

codecov bot commented Feb 4, 2025

Codecov Report

Attention: Patch coverage is 71.04478% with 97 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
agentops/log_capture.py 70.96% 54 Missing ⚠️
agentops/session.py 57.95% 37 Missing ⚠️
agentops/instrumentation.py 93.02% 3 Missing ⚠️
agentops/http_client.py 86.66% 2 Missing ⚠️
agentops/helpers.py 66.66% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@teocns teocns requested a review from areibman February 6, 2025 01:57
teocns added 27 commits February 6, 2025 04:11
Signed-off-by: Teo <teocns@gmail.com>
Rich formatted text with various styles (color, bold, italic)
Direct ANSI codes with proper newlines
Mixed color text in both stdout and stderr
All ANSI codes are preserved in the logged output

Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Added a private _make_request method that handles all the common request
logicSimplified the public HTTP methods (GET/POST/PUT/DELETE) to use the
common handlerEach method only specifies its unique parameters and
passes them to _make_requestAdded DELETE method for
completenessMaintained all existing error handling and response
processingKept the connection pooling and header preparation logic
unchanged

Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
…it from `Session`

Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
…ed for this session

Signed-off-by: Teo <teocns@gmail.com>

logger -> agentops_logger

Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
…_sesssion"

Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
…tegers or slices, not str

Signed-off-by: Teo <teocns@gmail.com>
Signed-off-by: Teo <teocns@gmail.com>
@teocns
Copy link
Contributor Author

teocns commented Feb 6, 2025

The current implementation has a potential issue: it doesn't guarantee that logs from one session won't leak into another when running concurrently.

This needs to be addressed with callstack lookup or other methods. @areibman: otherwise, how does agentops actually know which session to assign llm events to? would the same mechanism work?

…uests)

Signed-off-by: Teo <teocns@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant