Testing
Test your monitors without network calls or an API key, using a fake Jev client.
jev-events/testing exports mockJev(), a fake client that answers from a function you write. Pass it to monitor() as client, feed items with from(), and run() judges them all and returns the stats. Nothing reaches TypeSafe, so tests need no key and cost nothing.
import { from, monitor, noul } from "jev-events";
import { mockJev } from "jev-events/testing";
import { expect, test } from "vitest";
const questions = {
outage: noul("Does this log line describe a failure a human should look at?"),
};
test("pages someone for outages only", async () => {
// Answers per question id; a number is the probability of yes.
const jev = mockJev(({ state }) => ({
outage: JSON.stringify(state).includes("ECONNREFUSED") ? 0.96 : 0.03,
}));
const paged: string[] = [];
const logs = monitor({
source: from(["GET /health 200", "db: connect ECONNREFUSED 10.0.0.5:5432"]),
questions,
client: jev,
}).on("outage", { min: 0.8 }, (e) => {
paged.push(e.item.text);
});
// A finite source: judge everything, then stop.
const stats = await logs.run();
expect(paged).toEqual(["db: connect ECONNREFUSED 10.0.0.5:5432"]);
expect(stats.judged).toBe(2);
// Exactly what Jev saw for the second line:
expect(jev.calls[1]?.state).toEqual({
item: { text: "db: connect ECONNREFUSED 10.0.0.5:5432" },
});
});The examples on this page run in the project's own test suite, so they keep working as the library changes.
Answers
mockJev takes a function from the request to an answer per question id. Shorthands keep tests short:
| Question | Shorthand | Means |
|---|---|---|
| noul | 0.93 | The probability of yes |
| choice | "spam" | This label, with probability 1 |
| choice | { spam: 0.7, other: 0.3 } | These probabilities; the highest wins |
| score | 2.4 | The expected score |
Any full SDK response works too. A missing answer, or a label the question doesn't have, throws, so typos in tests fail loudly.
Actions
Monitors start in dry-run, so a test can check what an action would do without it running. Each action reports an action event with its status: dry-run, done, skipped with a reason, or failed. stats.actions counts them.
import { defineAction, from, monitor, noul } from "jev-events";
import { mockJev } from "jev-events/testing";
import { expect, test } from "vitest";
const refunded: string[] = [];
// Your own action. Like a built-in one, it only runs once dryRun is false.
const refund = defineAction({
platform: "*",
name: "billing.refund",
describe: (e) => `refund ticket ${e.item.id}`,
run: async (e) => {
refunded.push(e.item.id);
},
});
function tickets(dryRun: boolean) {
return monitor({
source: from([
{ id: "t1", text: "I was charged twice this month. Please refund one of them." },
{ id: "t2", text: "How do I export my data?" },
]),
questions: { refund: noul("Is the customer asking for their money back?") },
client: mockJev(({ state }) => ({ refund: JSON.stringify(state).includes("refund") ? 0.95 : 0.02 })),
dryRun,
}).on("refund", { min: 0.9 }, refund);
}
test("in dry-run, says what it would do and does nothing", async () => {
const wouldDo: string[] = [];
const stats = await tickets(true)
.on("action", (e) => wouldDo.push(`${e.status}: ${e.description}`))
.run();
expect(wouldDo).toEqual(["dry-run: refund ticket t1"]);
expect(stats.actions.done).toBe(0);
});
test("armed, refunds the right ticket once", async () => {
const stats = await tickets(false).run();
expect(stats.actions.done).toBe(1);
expect(refunded).toEqual(["t1"]);
});Built-in actions behave the same, but they need their platform's source, so test those in dry-run against your own account: run the monitor with npx tsx, and read the [dry-run] would … lines or the audit log.
Options
mockJev(respond, { latencyMs: 120, inputTokens: 150, model: "jev-mock" });latencyMs delays each answer, which is useful for testing maxLagMs and queues. inputTokens sets the usage reported for each request, for testing budgets.
Inspect requests
jev.calls records every request in order, so you can check exactly what Jev was shown, as the last line of the first example does. It's the quickest way to debug a question that answers differently than you expected.