Load Testing Max with the Max API

A guide for customers who host Max in their own environment and want to measure how many concurrent users it supports. It covers the API calls to use, a ready-to-run k6 script, the infrastructure to run it from, and how to read the results.

Before you start — please read

• You run and own the test. This guide is provided as-is. Your team builds, runs and maintains the test tooling and the infrastructure it runs on. AnswerRocket does not run or support customer load-test tooling, but will review your results with you.
• Never test production. Use a non-production environment that mirrors production (same server size, same data, same agents). Expect it to become slow or unavailable when it reaches its limit — that is what the test is for.
• Every simulated question is real. Each one runs the full answer pipeline: your LLM endpoint, your data warehouse queries, and Max's skills. A 30-minute run at 100 users is roughly 1,000–2,000 questions. Confirm LLM and warehouse cost and rate limits with their owners first.
• Results are specific to your environment. Capacity depends on server size, data volume, agent configuration and the questions asked. Numbers from one environment do not transfer to another.
• Tell AnswerRocket before you run. Share the test window through your support channel so we can help interpret anything unexpected.

1. How a load test works

Each simulated user does what a real user does in the web app:

1. Create a chat thread against an agent (copilot).
2. Queue a question. The call returns immediately; the answer is produced in the background. Start the timer here.
3. Poll for the answer every 5 seconds until it reports finished. Stop the timer.
4. Pause 60–120 seconds (reading time), then ask the next question. Start a new thread every 3–5 questions.
   Answer time = queue call → poll that reports the answer finished.

2. Prerequisites

3. API reference

Every call is an HTTPS POST of a GraphQL document to one endpoint.

3.1 Authentication

API keys belong to a user and start with arc.

1. Create a dedicated test user (for example [email protected]) and give it the Admin or Create API Key right, plus access to the agent you will test.
2. Sign in as that user, open Account Settings, and generate an API key in the API key section.
3. Store the key as a secret. Revoke it when testing is finished.

Each user has one API key; generating a new one replaces the old one. Do not generate a key on an account that other integrations already use — it will break them.

3.2 Find the agent ID (once, before the test)

query { getCopilots { copilotId name } }

3.3 Create a thread (once per simulated session)

mutation CreateThread($copilotId: UUID!) {
  createChatThread(copilotId: $copilotId) { id }
}

Variables: {"copilotId": "<agent ID>"} → thread ID at data.createChatThread.id.

3.4 Queue a question (start the timer)

mutation Ask($threadId: UUID!, $question: String!) {
  queueChatQuestion(threadId: $threadId, question: $question) { id threadId }
}

Returns immediately. The entry ID is at data.queueChatQuestion.id.

3.5 Poll for the answer (every 5 seconds)

query Poll($id: UUID!) {
  chatEntry(id: $id) { id answer { hasFinished error answeredAt } }
}

3.6 Two rules that keep the test realistic

• Use queueChatQuestion, not askChatQuestion. askChatQuestion holds the HTTP connection open until the answer is complete, often for minutes. That measures connection limits rather than the application, and is not how the web app behaves.
• Poll only the small fields shown above. Full answers (message, reportResults, diagnostics) can be several MB each. Requesting them every 5 seconds adds load that real users never generate. If you need the answer text, fetch it once after it finishes.

3.7 Quick connectivity check

curl -s "https://<your-max-hostname>/api/sdk/graphql" \
  -H "Authorization: Bearer $MAX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"query { getCopilots { copilotId name } }"}'

A JSON list of agents means the network path, hostname and key all work.

4. Recommended tool: k6

We recommend k6 (open source, by Grafana Labs). Any tool that can send HTTP requests in a loop (JMeter, Locust, Gatling, Azure Load Testing) works with the API above; the script below is a ready starting point.

4.1 Load-generator infrastructure

4.2 The script

Save as max-load.js, with a questions.txt beside it (one question per line).

import http from 'k6/http';
import { sleep, check } from 'k6';
import { Trend, Rate, Counter } from 'k6/metrics';
import { SharedArray } from 'k6/data';
// ---- Configuration (pass with -e NAME=value) ----
const URL = `${__ENV.MAX_URL}/api/sdk/graphql`;          // e.g. https://max-test.example.com
const COPILOT_ID = __ENV.COPILOT_ID;                      // from getCopilots
const PARAMS = {
  headers: { Authorization: `Bearer ${__ENV.MAX_API_KEY}`, 'Content-Type': 'application/json' },
  timeout: '60s',
};
const POLL_SECONDS = 5;
const ANSWER_TIMEOUT_SECONDS = 600;
const QUESTIONS_PER_THREAD = 4;
const THINK_MIN = 60, THINK_MAX = 120;                     // seconds between questions
const questions = new SharedArray('questions', () =>
  open(__ENV.QUESTIONS_FILE || './questions.txt').split('\n').map(s => s.trim()).filter(Boolean));
// ---- Metrics ----
const answerTime = new Trend('answer_time', true);         // queue -> finished, successful answers
const answerFailed = new Rate('answer_failed');            // failed or timed out
const answerTimeouts = new Counter('answer_timeouts');
// ---- Load profile: ramp, then hold each step 10 minutes ----
const step = (users) => [{ duration: '2m', target: users }, { duration: '10m', target: users }];
export const options = {
  scenarios: {
    ramp: {
      executor: 'ramping-vus',
      startVUs: 0,
      stages: [...step(5), ...step(10), ...step(20), ...step(30), ...step(50), ...step(75), ...step(100)],
      gracefulRampDown: '10m',
    },
  },
  thresholds: {
    // Stop the whole test once more than 5% of answers fail (evaluated after 5 minutes).
    answer_failed: [{ threshold: 'rate<0.05', abortOnFail: true, delayAbortEval: '5m' }],
  },
};
function gql(query, variables, op) {
  const res = http.post(URL, JSON.stringify({ query, variables }), { ...PARAMS, tags: { op } });
  const ok = check(res, {
    [`${op}: HTTP 200`]: (r) => r.status === 200,
    [`${op}: no GraphQL errors`]: (r) => { try { return !r.json('errors'); } catch (e) { return false; } },
  });
  return ok ? res.json('data') : null;
}
export default function () {
  const thread = gql(
    'mutation($c: UUID!) { createChatThread(copilotId: $c) { id } }',
    { c: COPILOT_ID }, 'createChatThread');
  if (!thread) { sleep(30); return; }
  const threadId = thread.createChatThread.id;
  for (let i = 0; i < QUESTIONS_PER_THREAD; i++) {
    const question = questions[Math.floor(Math.random() * questions.length)];
    const start = Date.now();
    const queued = gql(
      'mutation($t: UUID!, $q: String!) { queueChatQuestion(threadId: $t, question: $q) { id } }',
      { t: threadId, q: question }, 'queueChatQuestion');
    if (!queued) { answerFailed.add(true); sleep(30); continue; }
    const entryId = queued.queueChatQuestion.id;
    let finished = false, failed = true;
    while ((Date.now() - start) / 1000 < ANSWER_TIMEOUT_SECONDS) {
      sleep(POLL_SECONDS);
      const polled = gql(
        'query($id: UUID!) { chatEntry(id: $id) { answer { hasFinished error } } }',
        { id: entryId }, 'chatEntry');
      const answer = polled && polled.chatEntry && polled.chatEntry.answer;
      if (answer && answer.hasFinished) { finished = true; failed = !!answer.error; break; }
    }
    if (finished && !failed) answerTime.add(Date.now() - start);
    if (!finished) answerTimeouts.add(1);
    answerFailed.add(failed);
    sleep(THINK_MIN + Math.random() * (THINK_MAX - THINK_MIN));
  }
}

4.3 Running it

Check connectivity first with a 1-user smoke run: add -u 1 -i 1 to the command below. It overrides the load profile and runs one user through one thread of 4 questions, which takes about 5–10 minutes with the pauses. Then run the full test:

docker run --rm -i -v "$PWD:/scripts" -w /scripts grafana/k6 run \
  -e MAX_URL="https://<your-max-hostname>" \
  -e MAX_API_KEY="$MAX_API_KEY" \
  -e COPILOT_ID="<agent ID>" \
  --out csv=results.csv \
  --summary-export=summary.json \
  max-load.js

The default profile steps through 5 → 10 → 20 → 30 → 50 → 75 → 100 simulated users, holding each for 10 minutes (about 85 minutes in total). Edit the stages line to change it.

5. Measuring a run

5.1 When to stop

Stop at the first step where any of these holds; the previous step is the environment's practical ceiling. Agree the exact thresholds with your stakeholders before the run — these are suggested starting points.

• More than 5% of questions fail or time out (the script aborts automatically).
• 95th-percentile answer time exceeds 5 minutes.
• The web app stops responding for more than 2 minutes.

5.2 Monitoring the Max server

Use your cloud's built-in VM metrics (Azure Monitor, AWS CloudWatch, Google Cloud Monitoring) for CPU, available memory and disk I/O. For Docker Compose installs, also record per-container usage on the Max server for the whole run:

while true; do
  date -u +%FT%TZ
  docker stats --no-stream --format '{{.Name}} cpu={{.CPUPerc}} mem={{.MemUsage}}'
  vmstat 1 2 | tail -1
  sleep 30
done >> max-host-metrics.log

5.3 What to record per step

• Concurrent simulated users and questions started per minute
• Answer time: median and 95th percentile (answer_time)
• Failed and timed-out questions, count and % (answer_failed, answer_timeouts)
• Max server CPU %, available memory and disk I/O
• Step start and end times in UTC
  results.csv holds every sample with a timestamp, so per-step figures can be computed after the run.

6. Safety and clean-up

• Run in the agreed window only, against a non-production environment.
• The environment normally recovers by itself within minutes of the load stopping. If it is still unresponsive 5 minutes after the test ends, contact AnswerRocket support before restarting anything — logs from that moment help find the cause.
• Avoid large exports and data refreshes during the run; they add load the test does not measure.
• Every simulated question creates a real chat thread under the test user. They do not affect other users and can be left in place.
• Revoke the test API key when you are done.

7. Sharing results with AnswerRocket

Send the following through your support channel and we will correlate it with the application logs to confirm the bottleneck:

• The test as run: tool, load profile, think time, question list
• Step start and end times (UTC)
• summary.json and results.csv (or your tool's equivalent)
• Max server CPU, memory and disk graphs for the whole window, plus max-host-metrics.log if collected
• Server size(s) and Max version during the test