Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When the random seed is set, it causes duplicate traceId and spanId. #4376

Open
lyred193 opened this issue Jan 3, 2025 · 3 comments · May be fixed by #4377
Open

When the random seed is set, it causes duplicate traceId and spanId. #4376

lyred193 opened this issue Jan 3, 2025 · 3 comments · May be fixed by #4377
Labels
bug Something isn't working

Comments

@lyred193
Copy link

lyred193 commented Jan 3, 2025

Describe your environment

OS: ALL
Python version: Python 3.10.16
SDK version: 1.29.0

What happened?

After setting a random seed in the Python application, the traceId and spanId are duplicated after each restart of the application.

Steps to Reproduce

from opentelemetry.sdk.trace import RandomIdGenerator

if name == "main":
import random
random.seed(10)
id_generator = RandomIdGenerator()
trace_id = id_generator.generate_trace_id()
span_id = id_generator.generate_span_id()
# Every time the main function is run, the trace_id is always 164207228320579316746596838417247989971
print(trace_id)
# Every time the main function is run, the span_id is always 273610340023782072
print(span_id)

Expected Result

Each time the main function runs, the trace_id and span_id are different.

Actual Result

Every time the main function is run, the trace_id is always 164207228320579316746596838417247989971
Every time the main function is run, the span_id is always 273610340023782072

Additional context

No response

Would you like to implement a fix?

Yes

@lyred193 lyred193 added the bug Something isn't working label Jan 3, 2025
@aabmass
Copy link
Member

aabmass commented Jan 10, 2025

I'm not sure the current behavior is really a bug, this is what I would expect to happen.

Could you explain a little more the use case for changing the behavior?

@marcuslimdw
Copy link

marcuslimdw commented Jan 15, 2025

imagine you're running some scientific computing code that involves randomness and you want to persist the seed for reproducibility. if your teammates run the instrumented code with the same seed (to check your results), they'll generate traces with the same ID - is that a good thing? minimally, I can imagine it'd be inconvenient, because each run would no longer be uniquely identifiable from a tracing perspective (you'd need to know out-of-band information like the timestamp)

while I think it makes sense to be able to control the seed of the OTel random generator, I don't think it should use the global seed.

@lyred193
Copy link
Author

I'm not sure the current behavior is really a bug, this is what I would expect to happen.

Could you explain a little more the use case for changing the behavior?

In some scenarios, such as model performance evaluation, we prefer the split between training and testing data to be random. By setting a fixed random seed, we can ensure that each model uses the same dataset for training and validation, enabling fair comparisons.

In this context, integrating OpenTelemetry can lead to the duplication of traceId and spanId if the application is restarted.

Here’s a demo:

import uvicorn
from fastapi import FastAPI, HTTPException
from logging import getLogger

_logger = getLogger(name)

def init():
import random
random.seed(10)

app = FastAPI()

@app.get("/health")
async def health_check():
return {"data": "ok"}

@app.get("/evaluation")
async def evaluation():
# Randomly generate a list of test numbers
random_numbers = random.sample(range(1, 101), 10)
'''
Other operations
test model 1
test model 2
'''
return {"data": {"score": 0.6}}

if name == "main":
init()
uvicorn.run(app, host="0.0.0.0", port=8000)
In this demo, when accessing the /evaluation endpoint, the generated traceId is 164207228320579316746596838417247989971, and the spanId is 273610340023782072, with the span name being /evaluation.

After restarting the application (due to an update or a manual restart), accessing the /health endpoint generates the same traceId 164207228320579316746596838417247989971 and spanId 273610340023782072. However, the span name has changed to /health, which is clearly incorrect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
3 participants