Introduction & Getting Started

Developer Documentation Hub

Select a domain below to access ultra-detailed APIs, Evaluation Protocols, and Schemas.

1. Platform Architecture & Ecosystem Overview

The Acadify AI Lab ecosystem is designed as a highly distributed, zero-trust evaluation network tailored for frontier intelligence and foundation models.

The Execution Engine

At the core of Acadify is the Deterministic Execution Engine (DEE). Unlike traditional benchmarking platforms that rely on static regex matching or simple cross-entropy evaluations, the DEE spins up isolated, ephemeral Kubernetes pods for every single model generation. This allows us to safely execute potentially malicious or hallucinated code generated by an LLM during SWE-bench evaluations without compromising the host infrastructure.

The SME Verification Network

Standard crowd-sourcing introduces noise, data poisoning, and hallucinated consensus. Acadify mitigates this by maintaining a proprietary network of top 4% STEM PhDs, FAANG Principal Engineers, and Offensive Security Researchers. When you submit RLHF or SFT datasets via the API, the payloads are routed to specific Domain Nodes.

• Node Alpha (Mathematics & Logic): Verifies zero-shot reasoning traces and formal logic proofs.
• Node Beta (Software Engineering): Verifies complex SWE-bench patches and algorithmic optimizations.
• Node Gamma (Security & Alignment): Verifies adversarial robustness, jailbreak resistance, and PII leakage.

2. Authentication & Key Lifecycle Management

The Acadify API strictly enforces Bearer Token Authentication across all endpoints. To access any API, you must provision an Enterprise Secret Key via your Acadify Dashboard.

Generating API Keys

We support multiple environments to ensure development keys do not pollute production evaluation metrics.

Using Your Key

Include your secret key in the Authorization header of every HTTP request.

curl https://api.acadifysolution.com/v1/eval/status   -H "Authorization: Bearer aca_live_abc123def456"   -H "Content-Type: application/json"

Key Rotation & Compromise

If you suspect an aca_live_ key has been leaked (e.g., committed to a public GitHub repository), you must immediately trigger the Key Revocation endpoint. Our system scans public GitHub repositories and will automatically revoke your key if a leak is detected, emitting a key_compromised Webhook event.

3. Rate Limits, SLAs & Quotas

Acadify enforces strict rate limits to ensure absolute stability of the Deterministic Execution Engine. Enterprise SLAs override default limits.

Understanding Headers

Every API response includes specific headers detailing your current quota consumption:

• X-RateLimit-Limit: Your absolute maximum RPM (Requests Per Minute).
• X-RateLimit-Remaining: The number of requests remaining in the current minute window.
• X-RateLimit-Reset: The Unix timestamp indicating when the quota resets.

Handling 429 Too Many Requests

If you breach your quota, the API will return a 429 Too Many Requests HTTP status code. You must implement exponential backoff logic in your client SDKs. Do not implement aggressive polling, as sustained 429 breaches may result in an automated 24-hour IP ban.

import time
import requests

def make_request_with_backoff(url, headers, max_retries=5):
    retries = 0
    backoff_factor = 2
    while retries < max_retries:
        response = requests.get(url, headers=headers)
        if response.status_code == 200:
            return response.json()
        elif response.status_code == 429:
            sleep_time = backoff_factor ** retries
            print(f"Rate limited. Retrying in {sleep_time} seconds...")
            time.sleep(sleep_time)
            retries += 1
        else:
            raise Exception(f"API Error: {response.status_code}")
    raise Exception("Max retries exceeded")

4. Webhooks & Asynchronous Event Streaming

Because SWE-bench evaluations and Human-in-the-Loop SME validations can take anywhere from 10 minutes to 72 hours, synchronous API responses are impossible. Acadify utilizes a robust Webhook streaming architecture to notify your systems when an evaluation is complete.

Configuring Endpoint URLs

Register your receiving HTTPS endpoint in the Acadify Dashboard. Your endpoint must be capable of receiving POST requests and must return a 200 OK status code within 3 seconds. If your system takes longer than 3 seconds to process the payload, Acadify will assume a timeout and will retry the webhook up to 5 times using exponential backoff.

Signature Verification

To ensure webhook payloads actually originate from Acadify and have not been tampered with, every request includes an Acadify-Signature header. This is a HMAC-SHA256 signature generated using your Webhook Secret.

import hmac
import hashlib

def verify_signature(payload_body, secret_token, signature_header):
    expected_signature = hmac.new(
        key=secret_token.encode(),
        msg=payload_body,
        digestmod=hashlib.sha256
    ).hexdigest()
    
    return hmac.compare_digest(expected_signature, signature_header)

Common Event Types

5. Quickstart: Your First API Request

Now that you understand the architecture, authentication, and webhooks, let's initiate your first API request to check the health status of the execution engine.

Ping the Status Endpoint

curl https://api.acadifysolution.com/v1/system/health   -H "Authorization: Bearer aca_live_abc123def456"

Expected Response

{
  "status": "operational",
  "components": {
    "execution_engine": "online",
    "sme_routing_node": "online",
    "webhook_dispatcher": "online"
  },
  "latency_ms": 12
}

If you receive this response, your authentication is correct, your network is successfully pinging our API gateway, and you are ready to begin deploying models into the execution sandboxes. Navigate to the Evaluation Protocols page to begin your first SWE-bench run.