Assessment Lab documentation

The Assessment Lab gives you everything you need for CV Intake at Scale:

• an AI gateway with real models (OpenAI- and Anthropic-compatible APIs) and sim-1, a free simulated model that takes about 20 seconds per call;
• your own Kubernetes namespace on our cluster, with a public HTTPS hostname;
• about 150 real CVs (personal details replaced with fake ones) to develop against, about 20 of them with reference labels;
• runs that send CV jobs to your service and measure it: practice runs, then one Final Run.

Read this page end to end. The limits and the delivery guarantees matter.

Contents

1. Getting started
2. Authentication and errors
3. Models and prices
4. Synchronous model calls
5. Async jobs
6. Webhooks
7. sim-1, the simulated model
8. The dev dataset
9. The extraction schema
10. Runs: how we send you CVs
11. Posting results
12. Run metrics, the scale pass and the quality score
13. Budgets
14. Your namespace
15. API reference
16. Troubleshooting

All times are UTC. All request and response bodies are JSON (UTF-8) unless stated otherwise.

Getting started

1. Sign up at https://lab.uptal.com/signup with the access code from your invitation. You get:
   • an API key (lab_...). It is shown once: store it safely. You use it for every API call and to log in to the portal. You can generate a new one from the dashboard (the old one stops working at once).
   • a webhook signing secret (whsec_...), always visible on your dashboard. We sign every webhook and every run delivery we send you with it.
   • a slug such as c4k9x2mq: your Lab username. It is random and identifies your environment. Your namespace is cand-<slug> and your public hostname is https://<slug>.lab.uptal.com.
2. Download your kubeconfig from the dashboard (or GET /v1/me/kubeconfig) and check it works: kubectl --kubeconfig lab.kubeconfig get pods.
3. Deploy your service to your namespace and expose it on https://<slug>.lab.uptal.com with an Ingress.
4. Register your ingest URL, for example https://<slug>.lab.uptal.com/api/cv-jobs (dashboard, or PUT /v1/me/ingest-url).
5. Start a practice run from the dashboard, look at the metrics, improve, repeat.
6. When you are ready, start the Final Run. It is one shot.

Base URLs of the gateway:

Everything you can do on the dashboard is also available as a JSON API (see API reference).

Authentication and errors

Send your API key on every request:

Authorization: Bearer lab_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

x-api-key: lab_... is accepted too, so the official Anthropic SDKs work with only a base-URL change.

• OpenAI SDKs: base_url = "https://lab.uptal.com/v1", api_key = "lab_...".
• Anthropic SDKs: base_url = "https://lab.uptal.com", api_key = "lab_...".

Errors produced by the gateway have this shape:

{"error": "rate_limited", "message": "Too many job submissions (limit 30/s).", "request_id": "req_6f1c..."}

Every response carries an X-Request-Id header. Errors returned by a real model provider (for example an invalid request body) are passed through in the provider's own format with the provider's status code.

Models and prices

GET /v1/models returns the live list. Prices are in USD per 1 million tokens and are what your budget is charged.

astra, sol, terra, luna, mini and mini-4.1 are OpenAI models (several of them reasoning models: their reasoning tokens are billed as output). opus and fable are Anthropic Claude models. Requests with more than 272,000 input tokens on the OpenAI models are billed at 2x input and 1.5x output.

Throughput limits. Every real model has a throughput limit (tokens per minute) that is shared by all candidates using the Lab at the same time. When it is reached the gateway answers 429 with "error": "model_rate_limited" and a Retry-After header (an async job fails with error.code: "model_rate_limited" and error.retry_after, in seconds). It is not charged. Retry after the delay, with jitter, and spread your live calls instead of sending them all at once. sim-1 has no such limit.

Synchronous model calls

The request and response bodies are those of the original APIs; set model to one of the names above. The gateway holds the connection until the model answers (up to 300 seconds for real models).

Attribute every call to a CV. Put the CV's id in metadata.cv_id of the request body (or send the header X-Lab-Cv-Id: <cv_id>). We use it to attribute model calls and cost to CVs in your run metrics, to bill Final Run calls to your eval budget, and sim-1 needs it to know which CV you are asking about. The gateway removes metadata before forwarding (for /v1/messages it keeps only metadata.user_id).

Documents. There is no file upload API: send documents inline.

• Responses API, PDF: {"type": "input_file", "filename": "cv.pdf", "file_data": "data:application/pdf;base64,<...>"}
• Responses API, image: {"type": "input_image", "image_url": "data:image/jpeg;base64,<...>"}
• Chat Completions, image: {"type": "image_url", "image_url": {"url": "data:image/png;base64,<...>"}}
• Messages API: {"type": "document", "source": {"type": "base64", "media_type": "application/pdf", "data": "<...>"}} and {"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": "<...>"}}

DOCX is not accepted as a file input by any model: convert it to text (or images) yourself.

Restrictions.

• No streaming: stream: true is rejected.
• Only your own function tools; built-in provider tools (web search, code interpreter, file search, MCP, ...) are rejected. Responses API background mode is rejected (use async jobs); store is always false, so previous_response_id does not work.
• Output tokens are capped at 32,000 per request: max_output_tokens / max_completion_tokens / max_tokens above that are lowered, and requests without one get 32,000. Set a realistic value: it also decides how much budget the request needs (see Budgets).
• Requests larger than 1 MB (documents) are processed a limited number at a time across the Lab; when that is momentarily full you get 429 concurrency_limit with Retry-After.
• Concurrency per API key: at most 8 concurrent synchronous sim-1 requests and at most 8 concurrent synchronous requests to real models, of which at most 3 to Anthropic models (opus, fable; this limit counts sync calls and running async jobs together, and there is also a Lab-wide cap for these models). Beyond that you get 429 concurrency_limit with Retry-After.

Example:

curl https://lab.uptal.com/v1/responses \
  -H "Authorization: Bearer $LAB_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "luna", "input": "Say hello", "metadata": {"cv_id": "cv_0001"}}'

Async jobs

Submit any model request as a job and get the result by webhook, by polling, or both.

Submit: POST /v1/jobs

{
  "endpoint": "/v1/responses",
  "body": { "model": "sim-1", "input": "..." },
  "callback_url": "https://<slug>.lab.uptal.com/api/lab-webhooks",
  "metadata": { "cv_id": "sim-000123", "anything": "you like" }
}

• endpoint: /v1/responses, /v1/chat/completions or /v1/messages.
• body: exactly what you would send to that endpoint synchronously.
• callback_url (optional): where we POST the result. Allowed: https://<slug>.lab.uptal.com/... (your host), http://<service>.cand-<slug>.svc.cluster.local[:port]/... (a Service in your namespace), or another public https:// URL (handy for local development).
• metadata (optional, max 4 KB): returned to you unchanged. Put the CV id in metadata.cv_id.

Response 202:

{"id": "job_9b1f...", "object": "job", "status": "queued", "created_at": "2026-10-14T09:00:00.000000Z"}

Limits per API key:

• 30 submissions per second, then 429 rate_limited.
• 400 jobs queued + running at the same time (of which at most 50 for real models), then 429 too_many_open_jobs.
• sim-1: at most 30,000 calls per UTC day (sync + jobs together), then 429 daily_limit. A scale run needs about 6,300.
• Job requests for real models may be at most 15 MB.
• Real-model jobs are executed a few at a time per key (currently 6, of which at most 3 on Anthropic models); the rest wait in queued.
• A submission can fail with a 5xx. In that case no job was created and it is safe to submit again.

Poll: GET /v1/jobs/{id}

Rate limit: 20 requests per second per key.

{
  "id": "job_9b1f...",
  "object": "job",
  "status": "succeeded",
  "model": "sim-1",
  "endpoint": "/v1/responses",
  "result": { "...": "the response body the synchronous endpoint would have returned" },
  "error": null,
  "metadata": { "cv_id": "sim-000123" },
  "created_at": "2026-10-14T09:00:00.000000Z",
  "completed_at": "2026-10-14T09:00:21.400000Z"
}

status is queued, running, succeeded or failed. A finished job's result is kept for 24 hours; after that the job still exists but result is null. succeeded means the model answered: you still have to check the answer itself (finish reason, valid JSON). When failed, error is an object:

Webhooks

When a job finishes and it has a callback_url, we POST:

{"id": "job_9b1f...", "status": "succeeded", "result": {"...": "..."}, "error": null, "metadata": {"cv_id": "sim-000123"}}

with the header

X-Lab-Signature: t=1791975123,v1=5f2c...e9

where v1 = hex(HMAC_SHA256(webhook_secret, t + "." + raw_request_body)). Verify it against the raw body (before any JSON parsing), compare in constant time, and reject timestamps that are too old (we recommend 5 minutes). PHP:

[$t, $v1] = [null, null];
foreach (explode(',', $request->header('X-Lab-Signature', '')) as $part) {
    [$k, $v] = array_pad(explode('=', $part, 2), 2, null);
    if ($k === 't') { $t = $v; } elseif ($k === 'v1') { $v1 = $v; }
}
$expected = hash_hmac('sha256', $t . '.' . $request->getContent(), $webhookSecret);
$valid = $t !== null && $v1 !== null && hash_equals($expected, $v1) && abs(time() - (int) $t) <= 300;

Delivery guarantees. Design for all of these:

• At least once. If your endpoint does not answer 2xx within 5 seconds, we retry after 5 s, 30 s and 120 s. Answer quickly and do the work asynchronously.
• Duplicates. The same webhook can arrive more than once, sometimes seconds apart, sometimes concurrently.
• Out of order. Webhooks do not necessarily arrive in the order the jobs finished.
• Lost. A webhook can never arrive at all. The job is still available with GET /v1/jobs/{id}: poll the jobs you have not heard about.

The same secret and header format are used for the run deliveries we send to your ingest URL.

sim-1, the simulated model

sim-1 is free and behaves like a slow, imperfect production model. Use it for load and for everything you want to test without spending budget.

• Latency: typically about 20 seconds per call, with a long tail (some calls take a minute or more, never more than 90 seconds). The same holds for sync calls and jobs.
• Output: it ignores your prompt and your input. It answers with the extraction JSON (as the assistant's text, in the normal response shape of the endpoint you called) for the CV identified by metadata.cv_id:
  • for the cv_ids of a sim-mode run (sim-000001, sim-000002, ...) it returns the extraction of the CV that run job's file_url points to;
  • for the dev CVs that have public labels it returns their labels;
  • for anything else it returns plausible but meaningless JSON.
• Faults, like a real model and a real network:
  • a job submission can fail with a 500 (no job is created; submit again);
  • a job can fail with model_overloaded (transient; try again);
  • a successful answer can be malformed: invalid JSON, cut off in the middle (try again);
  • some documents are refused: chat finish_reason: "content_filter", responses status: "incomplete" with incomplete_details.reason: "content_filter", messages stop_reason: "refusal". This is decided per cv_id: retrying never helps, so report that CV as failed;
  • webhooks can be duplicated, delayed, reordered or lost (see Webhooks).
• Fault decisions are random but reproducible per job, so a fresh retry is a fresh draw.

The dev dataset

• GET /v1/dataset/dev lists the dev CVs: cv_id, mime_type, pages, file_url, and labels_url for the ~20 CVs that come with reference labels (gold extractions in the schema below).
• GET /v1/files/{cv_id} downloads a dev file (send your API key).
• GET /v1/dataset/dev/labels/{cv_id} returns the reference labels of a labelled dev CV.
• Download rate limit: 50 per second per key.

The files are real CVs from our production traffic with personal details replaced by fake ones: PDFs (text, designed, scanned), images, Word files, English, Arabic and bilingual. Some uploads are not CVs at all. The hidden holdout set used for scoring is drawn from the same population.

Treat as_of as 2026-10-01 for every computation that depends on "today" (the run deliveries tell you as_of too).

The extraction schema

This is exactly what you must return in data for each CV:

{
  "is_cv": true,
  "document_language": "en",            // "en" | "ar" | "mixed" | "other"
  "full_name": "string|null",           // as written (original script)
  "email": "string|null",
  "phone": "string|null",               // E.164 e.g. "+966500001234"; first/primary number
  "location": { "city": "string|null", "country_code": "string|null" },   // current residence; ISO 3166-1 alpha-2
  "nationality_code": "string|null",    // ISO alpha-2, ONLY if explicitly stated
  "current_title": "string|null",       // title of the ongoing role (is_current) — null if no role is ongoing
  "total_experience_months": "int|null",// professional experience up to `as_of` (2026-10-01 for gold), overlapping periods counted once; year-only start = January, year-only end = December
  "experience": [ { "title": "string", "company": "string|null", "start": "YYYY-MM|YYYY|null", "end": "YYYY-MM|YYYY|null", "is_current": false } ],
  "education": [ { "degree": "string|null", "field": "string|null", "institution": "string|null", "end_year": "int|null" } ],
  "skills": ["string"],
  "languages": ["string"]                // spoken languages, English names, e.g. "Arabic"
}

Rules: never guess (null when absent); values as written in the CV (don't translate) except normalised fields (codes, dates, phone, languages); Hijri dates → Gregorian; when is_cv is false every other field is null / empty (including document_language; the scorer counts a non-null value on a non-CV as a hallucination).

Runs: how we send you CVs

Your ingest URL

Register the URL where we deliver CV jobs: dashboard, or

curl -X PUT https://lab.uptal.com/v1/me/ingest-url -H "Authorization: Bearer $LAB_API_KEY" \
  -H "Content-Type: application/json" -d '{"url": "https://<slug>.lab.uptal.com/api/cv-jobs"}'

It must be https://<slug>.lab.uptal.com/<any path> (your own host, port 443).

Deliveries

For every CV in a run we send:

POST <your ingest URL>
Content-Type: application/json
X-Lab-Signature: t=<unix>,v1=<hex hmac>

{
  "run_id": "run_8c2e...",
  "cv_id": "sim-000123",
  "mode": "sim",
  "file_url": "https://lab.uptal.com/v1/files/sim-000123?run=run_8c2e...&exp=...&sig=...",
  "mime_type": "application/pdf",
  "as_of": "2026-10-01",
  "submitted_at": "2026-10-14T09:00:00.100000Z"
}

• Answer 2xx within 3 seconds. Accept the job and process it asynchronously.
• No 2xx within 3 s → we retry after 5 s, 15 s and 45 s. A CV that never gets a 2xx counts as rejected (and will almost certainly be lost).
• We sometimes deliver the same job again on purpose, even after you answered 2xx. (run_id, cv_id) identifies a job; the same pair must produce one result.
• file_url is a signed link that works without your API key until shortly after the run's deadline. In the Final Run's quality phase the link only works while that phase is running.
• mode is sim (use sim-1 with metadata.cv_id = the delivery's cv_id) or live (use real models).
• In sim mode several cv_ids can point to the same file. Treat every cv_id as a different CV: each one needs its own model call that carries its cv_id. We check this.

Run types

• Start runs from the dashboard or with POST /v1/runs {"type": "practice-sim" | "practice-live"}; the Final Run needs {"type": "final", "confirm": true}. The Final Run cannot be cancelled. Practice runs can be cancelled (POST /v1/runs/{id}/cancel).
• You can have one sim run and one live run active at a time; nothing else can start while your Final Run is active.
• Only one sim run runs at a time across all candidates. Others wait in a queue (Final Runs go ahead of practice runs); your dashboard and GET /v1/runs/{id} show queue_position. Do not leave your Final Run to the last hour.
• A practice-sim run that still has at least 5% of its CVs unanswered and receives no new result for 5 minutes after its last delivery ends early (end_reason: "stalled"), so a broken service does not block the queue.
• A run starts sending as soon as it leaves the queue. Item n is sent at start + n / rate.
• The quality phase of the Final Run starts automatically when the scale phase has finished.
• A run finishes when every CV has a result (and our deliberate re-sends are done), or at its deadline plus a 15-minute grace period (5 minutes for practice runs), whichever comes first.

Posting results

POST /v1/results
Authorization: Bearer lab_...

{"cv_id": "sim-000123", "run_id": "run_8c2e...", "status": "done", "data": { ...extraction... }, "error": null}

• status is done (with data = an object in the schema) or failed (data null, error a short string or object explaining why, for example "content_filter").
• Response 202 {"accepted": true, "duplicate": false}.
• The first result for a (run_id, cv_id) is the one that counts. Every later result for the same pair is stored and counted as a duplicate ("duplicate": true in the response). Duplicates fail the scale run.
• 404 unknown_cv_id / run_not_found if the pair does not belong to one of your runs; 409 run_not_started before the run starts and 409 not_delivered for a CV we have not delivered to you yet.
• Rate limit 100 per second per key, max 64 KB per result. Results are accepted after a run has finished, but they count as late.

Run metrics, the scale pass and the quality score

GET /v1/runs/{id} (and the dashboard) shows, per run:

• deliveries: CVs delivered, accepted, rejected, delivery attempts, retried CVs, deliberate re-sends;
• results: CVs with a result, results received, duplicates, failed results, results for unknown cv_ids;
• latency per CV (first delivery → first result): p50, p95, max;
• SLA: CVs whose first result arrived before the deadline;
• lost: CVs with no result by the deadline + grace;
• model calls per CV (from the gateway's log of calls carrying metadata.cv_id): average, max, distribution, results not backed by a completed model call, calls without a cv_id during the run;
• 429s you received during the run, by endpoint; cost attributed to the run's CVs.

Passing the scale run

The scale phase of the Final Run (and each practice-sim, with its own deadline) passes when all of these hold:

1. at least 99.5% of the CVs have their first result within 30 minutes of the run start, each backed by a model call that carried that cv_id and completed after we delivered the CV and before you posted its result;
2. zero duplicate results;
3. at most 1% of the results have status failed.

The quality score

Live runs (practice-live and the quality phase of the Final Run) are scored against gold labels, 0 to 100:

• Free-text fields (titles, companies, institutions, degrees, fields of study, skills, city) are compared after normalisation; when they still differ, an LLM judge decides whether they mean the same thing.
• A field where the gold value is null/empty scores 1 only if you also return null/empty.
• Hallucinations: returning a value where the CV has none (gold null/empty) is counted. The final score is the mean document score minus 15 × your hallucination rate.
• A CV without a result, or with a failed result, scores 0.
• Your dashboard shows the overall score and per-field accuracy. The quality phase of the Final Run uses the hidden holdout set.

Budgets

• sim-1 is free (calls are counted, not billed).
• Costs are computed from the token usage the provider reports, at the prices in Models and prices.
• Before a real-model request runs, its maximum cost is reserved: estimated input plus its maximum output tokens (capped at 32,000) at the model's prices. If what is left in the bucket (minus reservations of your requests in flight) cannot cover it, the request is refused with 402 budget_exhausted (async jobs: refused at submission, or failed with budget_exhausted). You are charged the actual usage afterwards. A low max_output_tokens keeps the reservation small.
• GET /v1/budget (or the dashboard) shows spend and what is left.

Your namespace

Your kubeconfig points at https://k8s.lab.uptal.com:6443 with namespace cand-<slug> and gives you a namespace-scoped identity.

What you can manage: Deployments, StatefulSets, Pods (incl. logs, exec, port-forward), Services (ClusterIP), ConfigMaps, Secrets, PersistentVolumeClaims, Ingresses, Jobs, CronJobs and HorizontalPodAutoscalers. You can read Events, your ResourceQuota, LimitRange and NetworkPolicies, and use kubectl top pods.

What you cannot do: change NetworkPolicies, quotas, RBAC or ServiceAccounts; create Endpoints, ExternalName Services, externalIPs, NodePort or LoadBalancer Services, or Traefik custom resources; use priority classes; run privileged pods or mount host paths (Pod Security Standard baseline is enforced; NET_RAW is dropped).

Quota (whole namespace). The cluster is shared, so the quota is small: design for it.

Per-container defaults and maximum (LimitRange). A container without explicit resources gets a request of 50m CPU / 96 MiB and a limit of 500m CPU / 512 MiB. No container may exceed 2 CPU / 3 GiB.

Set explicit requests and limits on every container. With the default limits only 4 containers fit in the namespace (4 x 500m = the 2-core CPU limit), and a pod that does not fit the quota is rejected at creation (you see it in kubectl get events / kubectl describe replicaset, not as a Pending pod). For example, give small workers requests: {cpu: 50m, memory: 128Mi} and limits: {cpu: 200m, memory: 256Mi} so that ten of them fit. The work is I/O-bound (waiting on the model), so keep worker counts and per-process concurrency sensible rather than adding pods. kubectl describe quota shows what you use.

Network:

• Inbound: from pods in your namespace, from the Lab gateway (webhooks, run deliveries) and from the public Ingress.
• Outbound: only to pods in your namespace, cluster DNS and the gateway (http://lab-gateway.assessment-system.svc.cluster.local or https://lab.uptal.com). Nothing else: no other Internet access, no external databases or APIs, no package downloads at runtime. Run what you need (database, queue, cache) inside your namespace and bake dependencies into your images.

Public HTTPS: one host per candidate. Expose your service on exactly <slug>.lab.uptal.com. A certificate is issued automatically by Let's Encrypt within a minute or two. Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: intake
  annotations:
    traefik.ingress.kubernetes.io/router.tls.certresolver: le
spec:
  rules:
    - host: <slug>.lab.uptal.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: intake
                port:
                  number: 80
  tls:
    - hosts: [<slug>.lab.uptal.com]

Every rule's host (and every tls.hosts entry) must be exactly your host. Do not set tls.secretName or a defaultBackend. Route different parts of your app (API, dashboard) by path on that one host.

Images: your pods always re-pull their image when they start (imagePullPolicy is forced to Always; the node only checks the tag with the registry and keeps already-downloaded layers cached). So every image must come from a registry the cluster can reach: a public image (Docker Hub, GitHub Container Registry, ...), or your own private registry with a pull secret you create in your namespace (kubectl create secret docker-registry regcred --docker-server=... --docker-username=... --docker-password=...) and reference with imagePullSecrets. Images that exist only on your laptop cannot be used. Image pulls are done by the node, so the outbound network rules above do not apply to them. The node is linux/amd64: build for that platform.

API reference

All endpoints are under https://lab.uptal.com and need Authorization: Bearer lab_... unless marked public.

The documentation is also available as Markdown at https://lab.uptal.com/docs.md.

Troubleshooting

• 401 unauthorized: check the Authorization: Bearer lab_... header. If you generated a new key, the old one no longer works.
• Lots of 429: you are over a rate or concurrency limit. Wait for Retry-After seconds, add jitter, and bound your concurrency instead of retrying in a loop.
• Many rejected deliveries in a run: your ingest endpoint must answer 2xx within 3 seconds. Accept, persist, answer, and do the work in the background. Check that your Ingress host and path match the registered URL.
• Webhooks never arrive: make sure your callback_url is reachable from the gateway (your host or a Service in your namespace) and answers 2xx within 5 seconds. Some webhooks are never delivered by design: poll jobs you have not heard back about.
• No TLS certificate on your host: the Ingress host must be exactly <slug>.lab.uptal.com with the router.tls.certresolver: le annotation; give it a minute and check kubectl describe ingress.
• Pods missing or stuck in Pending: a pod that exceeds the quota is never created. Check kubectl get events, kubectl describe quota and set explicit, smaller requests/limits.
• ImagePullBackOff: check the image name, tag and platform (linux/amd64), and your pull secret for private registries.
• Your pods cannot reach some service on the Internet: that is by design (see Network).
• Lab problems only. If the Lab itself is down and not your code (signup fails, your namespace or hostname is unreachable, or the gateway returns 5xx errors that don't match the documented sim faults for more than 10 minutes), email support@uptal.com with your Lab username and the request id. We don't answer questions about how to approach the task.