Privacy Policy

Last updated: March 1, 2026

1. Who we are

MetriLLM is an open-source project that benchmarks local LLM inference performance. This website is operated by the MetriLLM team. For any privacy-related inquiry, contact us at contact@metrillm.com.

2. Data we collect

2a. Website analytics

We use PostHog (EU-hosted instance) for anonymous web analytics. We operate in cookie-less mode (persistence: 'memory'): no cookies are stored on your device, no cross-session tracking occurs, and no personally identifiable information (PII) is collected. We record page views, referrer, browser type, and country — nothing more.

2b. Benchmark results (uploads)

When you run the CLI with the --share flag, the following data is uploaded:

• Model name, family, parameter size, quantization
• Hardware info: CPU, core count, total RAM, OS, architecture
• Performance metrics: tokens/second, time-to-first-token, memory usage
• Quality scores per category
• Runtime info (e.g. Ollama version)

Uploads are always opt-in. No upload happens without the explicit --share flag. Uploaded results are public and appear on the leaderboard. No IP address or device identifier is stored alongside the result.

2c. CLI telemetry

The CLI includes optional, anonymous telemetry via PostHog (EU-hosted). It tracks aggregate usage events (bench_started, bench_completed) with a hashed, non-reversible device identifier (hash of CPU + OS + arch). Telemetry is opt-in and can be disabled at any time with --no-telemetry or by setting "telemetry": false in ~/.metrillm/config.json.

2d. Newsletter

If you subscribe to the newsletter, we collect your email address via Loops. You can unsubscribe at any time via the link in every email.

3. Data we do NOT collect

• No accounts or passwords
• No cookies (website is fully cookie-less)
• No prompt content or model outputs (only scores)
• No IP addresses stored with benchmark results
• No cross-site tracking or advertising

4. Where data is stored

Benchmark results are stored in Supabase (PostgreSQL, hosted in the EU). Analytics data is processed by PostHog EU (eu.i.posthog.com). Newsletter data is managed by Loops.

5. Data retention

Uploaded benchmark results are retained indefinitely as part of the public dataset. If you wish to have a specific result removed, contact us at contact@metrillm.com with the result ID.

6. Your rights

Under GDPR, you have the right to access, rectify, or delete your personal data. Since we collect minimal data and no accounts exist, most rights are satisfied by design. For any request, email contact@metrillm.com.

7. Changes

We may update this policy. Changes will be posted on this page with an updated date.