Embedding Studio

Documentation

Everything here is a command you can run or a number we measured.

Quickstart

Four minutes, using a file we prepared so you do not have to make one.

  1. Create a workspace. $5 of credit, which is 5,000 decisions. No card.
  2. New project, from any template.
  3. On the project screen, click the labelled example. It fits, calibrates, and shows the before and after on rows it never fitted on.
  4. Route: move the slider to the precision you need. It answers in review load and reviewer hours.
  5. Deploy: take the calibration file, or call /decide.

To see what it says before labelling anything, use the unlabelled example on the same screen. That reports the answer mix and the confidence spread and claims no accuracy, because nothing in that file says what the right answer was.

The five steps

A dataset is cut 60 / 20 / 20. The head learns from the first slice, the temperature and the band are fitted on the second, and the third is held out and never touched. Every number you are shown comes from that third slice.

Teach

One row per past case: the record in your text column, your team's disposition in a column named after each question. At least 50 usable rows, and honestly 300+ per question before a fitted head reliably beats the engine. Below that the head usually loses and is refused, which the screen says in those words.

Route

The band is the widest automatic share whose precision holds your target on a one-sided 95% lower bound. That is why twelve correct out of twelve certifies 0.816 and not 1.0: a narrow band pays for being narrow. A target nothing can hold sends everything to review and names the target your data could support.

Watch

Population stability index on the mix of answers, which needs no labels. Our own drift runs showed a covariate shift leaving a calibration intact and a prior shift breaking it on all five corpora, which is why the mix is what is watched.

Decide

Base URL https://embedding-studio-api.fly.dev, authenticated with a workspace key in X-API-Key.

curl -X POST $API/v1/projects/<slug>/decide \
  -H "X-API-Key: $STUDIO_KEY" \
  -H "content-type: application/json" \
  -d '{"text": "I was charged twice, please refund one.",
       "idempotency_key": "ticket-8814"}'
  • route is auto only when every question cleared its band. Anything else is review, because a record is what a person picks up.
  • Every response carries an audit block naming the engine checksum, the reader, the vegaml version and the calibration version that produced it.
  • Send the same idempotency_key again after a timeout and you get the first answer back rather than a second charge.
  • Up to 50 records at once with texts. A 402 means the balance is spent.

A dry run, and drift

A file with no dispositions in it does two things: it tells you what the sheet says about your real traffic before anyone labels a row, and it is the drift check, because the monitor reads the mix of answers and nothing else.

curl -X POST $API/v1/projects/<slug>/samples \
  -H "X-API-Key: $STUDIO_KEY" -F file=@traffic.txt

curl $API/v1/samples/<id> -H "X-API-Key: $STUDIO_KEY"

curl -X POST $API/v1/projects/<slug>/watch \
  -H "X-API-Key: $STUDIO_KEY" -H "content-type: application/json" \
  -d '{"sample_id": "<id>", "min_answers": 50}'

One record per line, or a CSV with your text column. At least 20 records, because population stability index on fewer than that is noise. It reports no accuracy: fitting is the step that needs labels most, taking 60% of them, and calibration needs them too because fitting a temperature means comparing confidence against truth.

Every endpoint

GET    /v1/templates                      the starter question sheets
POST   /v1/projects                       create one
GET    /v1/projects                       list them
GET    /v1/projects/<slug>                one, with its jobs and fits
DELETE /v1/projects/<slug>                purge the files, keep the trail
POST   /v1/projects/<slug>/datasets       a labelled CSV, queues a fit
POST   /v1/projects/<slug>/samples        records with no labels
GET    /v1/projects/<slug>/samples        list them
GET    /v1/samples/<id>                   the mix and the answers
GET    /v1/jobs/<id>                      progress
GET    /v1/projects/<slug>/fits           every fit
GET    /v1/fits/<id>                      before and after, both frontiers
GET    /v1/fits/<id>/route?target_precision=0.95
POST   /v1/fits/<id>/policy               save the band
GET    /v1/projects/<slug>/calibration    read it
GET    /v1/projects/<slug>/calibration/export   download it
POST   /v1/projects/<slug>/decide         answer a record
GET    /v1/projects/<slug>/decisions      what the deployment did
POST   /v1/projects/<slug>/watch          a drift verdict
GET    /v1/projects/<slug>/audit          the whole trail
GET    /v1/credits                        balance and ledger

Keys

A workspace key starts esk_ and is shown once. Only its SHA-256 is stored, with an eight-character prefix so you can name one in a list and revoke it. A key is scoped to one workspace and can do everything in it except administer: the admin endpoints accept a verified session only, so a leaked key is never an admin credential.

Credits

$0.001 a decision. A new workspace gets $5, which is 5,000 decisions, with no card. There is no charge for fitting, calibrating, dry runs or drift checks today.

Balances are integers in millionths of a dollar, every movement is a ledger row carrying the balance after it, and GET /v1/credits returns both. If the inference process fails to answer, the credits for the records it did not answer come back as their own ledger row rather than by editing the spend.

What it refuses to do

  • Serve a head that lost. A fitted head whose cross-validated accuracy does not beat the engine on the same rows is not served, and the Teach screen says so rather than reading as though teaching worked.
  • Approximate an unreachable target. Everything goes to review, and the screen names the target your data could support.
  • Load a calibration that has been edited. The checksum covers alpha, the per-question fit and the band, so a file changed by hand stops matching the version stamped on it and vegaml refuses it.
  • Claim accuracy for a file with no labels. You get the answer mix and the confidence spread, which is what there is.

Self-hosting

The engine is vegaml, Apache-2.0, and the default reader (Qwen3.5-0.8B) is Apache-2.0 and ungated, so the bundle redistributes without asking anyone. The weights and the calibration are yours.

pip install vegaml
vegaml-serve --calibration calibration.json --threads 1

# refit from your own labelled rows, no account needed
vegaml-calibrate fit rows.jsonl --out calibration.json

# check drift with no labels; exits 1 when the traffic has moved
vegaml-calibrate check answers.jsonl --calibration calibration.json

One torch thread is the measured setting for the shipped reader on CPU: 258 ms a decision against 1,268 ms at torch's default of four, with identical answers. The right count belongs to the reader, not the engine, so a plain-transformer reader prefers four.