Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Multipass Setup for user_events metrics testing on Ubuntu 24.04 VM #127

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions opentelemetry-user-events-metrics/local-setup/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# Local Setup for Testing `user_events` with *Multipass*:

This directory contains `cloud-init.yaml`, a configuration file to set up an Ubuntu 24.04 virtual machine environment with essential tools required for running and validating user_events example contained in this repository. The setup includes enabling `user_events` and installing the `perf` tool, along with utilities for decoding `user_events` data and processing `proto` data.

## Prerequisites

1. **Virtualization Support**: Ensure that your machine supports virtualization. Virtualization options vary by operating system:
- **Windows**: Requires Hyper-V, available on Windows 10 Pro or Enterprise and above.
- **macOS**: Requires Apple Silicon (M1, M2) or Intel hardware with virtualization enabled in BIOS.
- **Linux**: Ensure KVM or VirtualBox is available and enabled in your distribution. Run `lsmod | grep kvm` to verify if KVM is installed.

2. **Install Multipass**: Multipass is a tool for managing Ubuntu instances as lightweight virtual machines. Installation varies by OS, follow the instructions available in [official site](https://multipass.run/install).

After installation, verify by running:
```bash
PS> multipass --version
```

## Usage

1. **Launch a VM with cloud-init.yaml:**
Use the following command to start a VM with the specified configuration:
```bash
PS> multipass launch --name my-test-vm -m 6G -d 10G --cloud-init cloud-init.yaml
Launched: my-test-vm
```

This will take some time to create and configure the VM. Validate that the VM is created and running:
```bash
PS> multipass list
Name State IPv4 Image
my-test-vm Running 172.27.162.116 Ubuntu 24.04 LTS
```

2. **Login into the VM:** You should get the ubuntu bash shell after login:
```bash
PS> multipass shell my-test-vm
..
ubuntu@my-test-vm:~$
```

Verify that the user_events in available and enabled:
```bash
ubuntu@my-test-vm:~$ grep USER_EVENTS /boot/config-6.8.0-48-generic
CONFIG_USER_EVENTS=y
```

3. **Run perf tool:** Invoke the perf tool to capture the user-events. Keep it running:
```bash
ubuntu@my-test-vm:~$ sudo perf record -e user_events:otlp_metrics
```

4. **Run user_events example** Open another shell, and build and run the opentelemetry-user-events-metrics exporter example:
```bash
PS> multipass shell my-test-vm
ubuntu@my-test-vm:~$ cd opentelemetry-rust-contrib/opentelemetry-user-events-metrics/ && cargo build --example basic-metrics --all-features
ubuntu@my-test-vm:~$ $ sudo ~/opentelemetry-rust-contrib/target/debug/examples/basic-metrics
Tracepoint registered successfully.
```

5. Terminate perf capture (Ctrl+C) after some time. It should show something like
```bash
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.175 MB perf.data (5 samples) ]
```
6. Convert perf data to json:
```bash
ubuntu@my-test-vm:~$ sudo chmod uog+r ./perf.data
ubuntu@my-test-vm:~$ perf-decode ./perf.data > perf.json
```
7. Ensure that `perf.json` contains something like:
```bash
"./perf.data": [ { "n": "user_events:otlp_metrics", "protocol": 0, "version": "v0.19.00", "buffer": [ ... ], "meta": { "time": 816.790831600, "cpu": 0, "pid": 4957, "tid": 4958 } } ]
```
8. Convert perf json to OpenTelemetry format:
```bash
ubuntu@my-test-vm:~$ source userevents-env/bin/activate
(userevents-env) ubuntu@my-test-vm:~$ python3 decrypt_python.py perf.json
```

9. The output will look something like this:

```plaintext
resource_metrics {
resource {
attributes {
key: "service.name"
value {
string_value: "metric-demo"
}
}
}
scope_metrics {
scope {
name: "user-event-test"
}
metrics {
name: "counter_f64_test"
description: "test_decription"
unit: "test_unit"
sum {
data_points {
start_time_unix_nano: 1731569774055345718
time_unix_nano: 1731569834056820774
as_double: 60
attributes {
key: "mykey1"
value {
string_value: "myvalue1"
}
}
attributes {
key: "mykey2"
value {
string_value: "myvalue2"
}
}
}
aggregation_temporality: AGGREGATION_TEMPORALITY_DELTA
is_monotonic: true
}
}
metrics {
name: "counter_u64_test"
description: "test_decription"
unit: "test_unit"
sum {
data_points {
start_time_unix_nano: 1731569774055358318
time_unix_nano: 1731569834056835474
as_int: 60
attributes {
key: "mykey1"
value {
string_value: "myvalue1"
}
}
attributes {
key: "mykey2"
value {
string_value: "myvalue2"
}
}
}
aggregation_temporality: AGGREGATION_TEMPORALITY_DELTA
is_monotonic: true
}
}
...
}
}
```

10. **Cleanup the VM:** Once tests are done, if required the VM can be stopped and/or deleted with below steps:
```bash
PS> multipass stop my-test-vm
```

And to delete:
```bash
PS> multipass delete my-test-vm
PS> multipass purge
```
71 changes: 71 additions & 0 deletions opentelemetry-user-events-metrics/local-setup/cloud-init.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# cloud-init.yaml
#cloud-config
package_update: true
packages:
- git
- wget
- build-essential
- cmake
- flex
- bison
- linux-tools-generic
- python3.12-venv

runcmd:
- sudo apt-get update
- sudo apt-get install -y git wget build-essential cmake flex bison linux-tools-generic python3.12-venv
- sudo -u ubuntu bash -c "
cd /home/ubuntu &&
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y &&
source /home/ubuntu/.cargo/env &&
git clone https://github.com/microsoft/LinuxTracepoints &&
cd LinuxTracepoints && mkdir build && cd build && cmake .. && make &&
sudo cp bin/perf-decode /usr/local/bin &&
cd /home/ubuntu && rm -rf LinuxTracepoints &&
git clone https://github.com/open-telemetry/opentelemetry-rust-contrib.git /home/ubuntu/opentelemetry-rust-contrib &&
python3 -m venv /home/ubuntu/userevents-env &&
source /home/ubuntu/userevents-env/bin/activate &&
pip install opentelemetry-proto
"
- |
cat << 'EOF' > /home/ubuntu/decrypt_python.py
import sys
import json
from opentelemetry.proto.collector.metrics.v1.metrics_service_pb2 import ExportMetricsServiceRequest

# Check if a filename is provided
if len(sys.argv) < 2:
print("Usage: python decrypt_metric.py <filename>")
sys.exit(1)

try:
# Read the JSON file
with open(sys.argv[1], 'r', encoding='utf-8-sig') as file:
data = json.load(file)

# Traverse through each item in the JSON data
for key in data:
for item in data[key]:
result = ExportMetricsServiceRequest()
# Convert the buffer list to bytes and parse it
buffer_data = bytes(item['buffer'])
result.ParseFromString(buffer_data)
print(result)

except json.JSONDecodeError as e:
print(f"JSON decoding error: {e}")
print("Please ensure the JSON file is properly formatted.")
except Exception as e:
print(f"An error occurred: {e}")
EOF

users:
- name: ubuntu
sudo: ["ALL=(ALL) NOPASSWD:ALL"]
groups: sudo
home: /home/ubuntu
shell: /bin/bash
lock_passwd: false # Ensure the password is not locked
plain_text_passwd: "ubuntu" # Set a password for user 'ubuntu' (you can set this to any preferred password)

ssh_pwauth: true # Enable password authentication for SSH
Loading