Privacy Policy

1. Who we are

This Privacy Policy describes how benchmarkfree.dev (“BenchmarkFree”, “we”, “us”, or “our”) collects, uses, and shares personal data when you use our website and services at https://benchmarkfree.dev (the “Service”).

Privacy and GDPR-related inquiries: [email protected] (subject line: “Privacy request” or “GDPR-related inquiry”). We act as the data controller for personal data described in this policy.

If you are in the EU/UK and mandatory law requires an EU representative at our scale of processing, we will publish those details here. Until then, the contact address above is your point of contact for data protection matters.

2. Scope

This policy applies to visitors, registered users, and anyone who interacts with BenchmarkFree, including quick evaluations, batch tasks, prompt libraries, model configuration, leaderboards, and related features.

By using the Service, you acknowledge that you have read this Privacy Policy. If you do not agree, please do not use the Service.

3. Information we collect

3.1 Account information

• Email address and password (stored as a secure hash, not in plain text)
• Display name (optional)
• Account identifiers, authentication timestamps, and session data

3.2 Evaluation and content data

• Prompts, follow-up questions, and prompt metadata you select or create
• Model outputs generated during evaluations
• Automated judge scores, timing metrics (e.g. TTFT, total latency), and token usage where available
• Your acceptance or rejection of automated scores
• Evaluation history, task configurations, and related session identifiers

3.3 Model credentials (persistent storage)

If you save custom or third-party model configurations, we persistently store credentials you provide — such as API keys, base URLs, deployment identifiers, and related connection settings — in our database so you do not have to re-enter them each session.

We use these credentials solely to make API calls to the model providers you select, on your behalf and at your direction, to validate connections and run evaluations you explicitly initiate. Your keys are therefore used to send requests to third-party providers (e.g. OpenAI, Anthropic, Google, or endpoints you supply), not only held on our systems.

3.4 Technical and usage data

• IP address, browser type, device information, and request logs
• Pages visited, features used, and error or performance diagnostics
• Cookies and similar technologies (see Section 13 and our Cookie Policy)

4. How we use your information

We use personal data to:

• Provide, operate, and maintain the Service
• Authenticate you and manage your account and sessions
• Run model evaluations, automated judge scoring, leaderboards, and related analytics you request
• Store your prompts, results, and configuration
• Analyse aggregated or anonymised evaluation data to improve the accuracy, reliability, and features of the Service (this does not include using your identifiable prompts or outputs to train AI models)
• Send service-related communications (e.g. security or policy updates)
• Detect abuse, fraud, and security incidents
• Comply with legal obligations and enforce our Terms of Service

Judge scoring: When you request automated scoring, evaluation outputs may be sent to a judge model configured by BenchmarkFree (which you select from available judge options in the product) for scoring purposes. This is separate from sending your prompts to candidate models you choose for generation.

We do not sell your personal data. We do not use your prompts or evaluation outputs to train, fine-tune, or develop AI models.

5. API keys, encryption, and access controls

Saved API keys and similar secrets are encrypted at rest using AES-based authenticated encryption (Fernet) before being written to our database. They are transmitted over TLS when used to call provider APIs. This is not reversible encoding (e.g. base64 alone); decryption requires our application encryption key.

Keys are used only as described in Section 3.3. They are not exposed to other users of the Service.

Access to stored credentials is restricted to authorised systems and personnel on a need-to-know basis for operating, securing, and supporting the Service (for example, automated decryption at runtime to perform an evaluation you initiated). We do not use your API keys for purposes unrelated to your requested evaluations.

You are responsible for the lawfulness of content you submit and for rotating or revoking keys in your provider accounts. No method of transmission or storage is completely secure.

6. Public leaderboards and your choices

When your evaluations meet published eligibility criteria, aggregated results (such as model identifiers, scores, and ranking metrics) may be included in public leaderboards. We do not publish your email address or account name on public leaderboards. Raw prompt text and full model outputs are not displayed publicly by default.

Leaderboard entries are generally not linked to your personal account identity in the public view. You can avoid contributing to public leaderboards by using private prompts, keeping batch tasks non-public, or not completing flows that qualify for public ranking.

If you believe your data has been included in error, contact [email protected]. See also our Terms of Service (Section 7).

7. Legal bases

7.1 EEA, UK, and Switzerland (GDPR / UK GDPR)

If you are in the European Economic Area, the United Kingdom, or Switzerland, we process personal data on the following bases:

• Contract — to provide the Service you request
• Legitimate interests — to secure, improve, and operate the Service (including aggregated analytics), subject to your rights
• Consent — where required (e.g. non-essential cookies, where applicable)
• Legal obligation — where we must retain or disclose data by law

7.2 Hong Kong (PDPO)

If you are in Hong Kong, we handle personal data in accordance with the Personal Data (Privacy) Ordinance (Cap. 486). We collect data for purposes that are directly related to providing the Service, keep data no longer than necessary for those purposes, and implement practical security measures. You may request access to or correction of your personal data by contacting [email protected].

7.3 Other regions

Where other mandatory local laws apply, we process personal data as needed to provide the Service and as permitted or required by those laws.

8. Sharing and processors

We share data only as needed to run the Service, including with:

• Infrastructure providers — hosting, databases, caching, and content delivery (including Supabase, Upstash, and Cloudflare), typically under data processing terms
• AI model providers you select — when you run evaluations, your prompts and inputs are sent to the model providers you choose to generate outputs
• Judge models for scoring — evaluation outputs may additionally be sent to a judge model (selected by you from available judge options, operated via providers such as OpenAI, Anthropic, Google, or Azure) for automated scoring you request
• Professional advisers or authorities — when required by law or to protect rights and safety

These parties process data under contractual obligations consistent with this policy. We do not authorise them to use your content for their own marketing. Use of third-party AI APIs is also subject to each provider's terms.

9. International transfers

Your data may be processed in countries other than your own, including where our infrastructure or model providers operate.

Where required by applicable law (including GDPR), we rely on appropriate safeguards, including:

• Standard Contractual Clauses (SCCs) approved by the European Commission (and UK International Data Transfer Addendum where applicable)
• Data processing agreements (DPAs) with subprocessors such as Supabase, Upstash, Cloudflare, and major AI API providers, where available
• Other lawful transfer mechanisms recognised under applicable data protection law

You may request more information about transfer safeguards by emailing [email protected].

10. Retention and deletion

We retain personal data for as long as your account is active or as needed to provide the Service, comply with law, resolve disputes, and enforce agreements.

After a verified deletion request, we delete or anonymise personal data within 30 days in most cases. Complex requests may take up to 90 days where permitted by applicable law (e.g. GDPR Article 12(3)).

We may retain limited data longer where required by law, for legitimate security purposes, or to resolve disputes — for example, audit logs or records we are legally obliged to keep. Such data is restricted to what is necessary and protected appropriately.

11. Your rights and account deletion

Depending on your location, you may have the right to:

• Access, correct, or delete your personal data
• Restrict or object to certain processing
• Data portability
• Withdraw consent where processing is consent-based (without affecting prior lawful processing)
• Lodge a complaint with a supervisory authority (e.g. in the EU/UK)

Account deletion: Email [email protected] from the address linked to your account with the subject line “Account deletion request”. We may verify ownership before processing.

We aim to respond to privacy and deletion requests within 30 days and to complete deletion within the timeframes in Section 10, or sooner where required by law.

Nothing in this section limits mandatory rights under data protection laws in your country of residence.

12. Cookies and similar technologies

We use cookies for essential functions (such as keeping you signed in) and preferences (such as language). We may use privacy-oriented analytics (including Cloudflare Web Analytics) to understand aggregate usage.

A detailed list of cookies, purposes, durations, and providers is in our Cookie Policy. Where non-essential cookies require consent under applicable law, we will obtain that consent before use.

13. Children

The Service is not directed to anyone under 16 years of age, and you must meet that minimum age (or the higher minimum required by the laws of your country or region) to create an account or use evaluation features.

If we become aware that a user is below the applicable minimum age, we will promptly suspend the account and delete associated personal data, except where a longer retention period is required by law.

14. Changes to this policy

We may update this Privacy Policy from time to time. We will post the revised version on this page and update the effective date.

For material changes, we will notify you at least 30 days before the changes take effect, by email (if we have your address) and/or through a prominent notice on the Service.

Where a change affects processing that relies on your consent, we will obtain fresh consent where required by law. Continued use after notice may constitute acceptance only for changes that do not require separate consent under applicable law.

15. Contact

Questions about this Privacy Policy or our data practices: [email protected]