behackl/pi-unigpt
Watch
1
0
Fork
You've already forked pi-unigpt
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

UniGPT provider for pi

Browse source 
Native /login with a verified API key, automatic model discovery from
UniGPT's /models endpoint, per-deployment deployment-id headers, and
model metadata from pi's bundled catalog.
This commit is contained in:
Benjamin Hackl 2026-11-03 20:01:28 +01:00
commit 4e854ba8c2
12 changed files with 2416 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

27
.github/workflows/ci.yml vendored Normal file
View file

@ -0,0 +1,27 @@
name: CI
on:
push:
branches: [main]
pull_request:
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node: [22, 24]
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
with:
version: 11
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm run ci

6
.gitignore vendored Normal file
View file

@ -0,0 +1,6 @@
node_modules/
.env
.env.*
!.env.example
*.tsbuildinfo
.DS_Store

21
LICENSE Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Benjamin Hackl
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

126
README.md Normal file
View file

@ -0,0 +1,126 @@
# pi-unigpt
Use [UniGPT](https://unigpt.uni-graz.at), the University of Graz's MyGPT
platform, as a model provider in [pi](https://pi.dev).
- Log in with **`/login unigpt`**; your key is checked before pi saves it.
- Models are discovered automatically. No model list to maintain.
- Context size, output limit, and image support come from pi's own model
catalog, so they stay current as pi updates.
- Tool calling, streaming, and image input work through pi's built-in
OpenAI Responses support.
Requires pi **1.5.x** and Node.js **22.19+**.
## Install
```sh
pi install git:github.com/behackl/pi-unigpt
```
Then start pi (or run `/reload` in a running session) and log in:
```text
/login unigpt
```
Paste your UniGPT API key into the masked dialog, **not into the chat**. Pi
confirms how many models your key can use. Pick one with `/model`; UniGPT
models are listed under `unigpt`.
To try the extension without installing it:
```sh
pi -e git:github.com/behackl/pi-unigpt
```
## Commands
| Command | What it does |
| --- | --- |
| `/login unigpt` | Save and verify your API key |
| `/logout` | Remove the saved key from pi (it stays valid at UniGPT) |
| `/model` | Pick a model; also reloads the model list |
| `/unigpt-refresh` | Reload the UniGPT model list now and show how many models you have |
## How models are described
UniGPT's API only lists deployment IDs such as `gpt-5.4`. Those IDs are
OpenAI model names, so the extension looks each one up in pi's bundled model
catalog and uses its name, context window, output limit, and image support.
Where OpenAI and Azure list different limits, the smaller one is used.
IDs that pi does not know get pi's standard defaults for custom models:
128k context, 16k output, text only. Update pi to pick up newer models.
Two things are deliberately not taken from the catalog:
- **Thinking levels are off for every model.** UniGPT ignores the requested
reasoning effort and uses a fixed one per deployment, so pi's thinking
levels would have no effect. Reasoning models still reason.
- **Costs are shown as zero.** That means "not tracked", not "free".
You can override any value in `~/.pi/agent/models.json`:
```json
{
"providers": {
"unigpt": {
"modelOverrides": {
"gpt-5.4": { "contextWindow": 200000 }
}
}
}
}
```
## Your API key
Pi stores the key under `unigpt` in `~/.pi/agent/auth.json`. That file is
private to your user account but **not encrypted**. To keep the key out of
it, replace the stored value with an environment variable or a password
manager command; pi resolves either when it needs the key:
```json
{
"unigpt": { "type": "api_key", "key": "${UNIGPT_API_KEY}" }
}
```
```json
{
"unigpt": { "type": "api_key", "key": "!op read op://Private/UniGPT/credential" }
}
```
## Upgrading from a models.json setup
If you configured UniGPT by hand before, delete the `unigpt` entry under
`providers` in `~/.pi/agent/models.json` (overrides as shown above are
fine). Otherwise your old model list and key take precedence over the
extension. Then run `/reload` and `/login unigpt`.
## Troubleshooting
- **No UniGPT models in `/model`:** run `/unigpt-refresh`; it reports the
exact problem (rejected key, unreachable server, empty model list).
- **`pi -p` says a UniGPT model was not found:** print mode only reads the
saved model list and never fetches it. Start pi interactively once after
logging in; later print runs reuse that list.
- **A specific model fails with HTTP 500:** some deployments are
misconfigured on the server side; try another model and tell the UniGPT
team.
## Development
```sh
pnpm install
pnpm run ci # type check + tests
```
Tests use fake keys and a local HTTP server; they never contact UniGPT or
read your pi configuration.
## License
MIT

53
package.json Normal file
View file

@ -0,0 +1,53 @@
{
"name": "pi-unigpt",
"version": "0.1.0",
"description": "UniGPT (University of Graz) provider for pi: /login with an API key, automatic model discovery.",
"license": "MIT",
"author": "Benjamin Hackl",
"repository": {
"type": "git",
"url": "git+https://github.com/behackl/pi-unigpt.git"
},
"homepage": "https://github.com/behackl/pi-unigpt#readme",
"bugs": {
"url": "https://github.com/behackl/pi-unigpt/issues"
},
"type": "module",
"keywords": [
"pi-package",
"pi",
"pi-extension",
"unigpt",
"mygpt",
"uni-graz"
],
"files": [
"src",
"README.md",
"LICENSE"
],
"engines": {
"node": ">=22.19.0"
},
"pi": {
"extensions": [
"./src/index.ts"
]
},
"scripts": {
"check": "tsc --noEmit",
"test": "node --import tsx --test test/*.test.ts",
"ci": "pnpm run check && pnpm test"
},
"peerDependencies": {
"@earendil-works/pi-ai": ">=1.5.0 <2",
"@earendil-works/pi-coding-agent": ">=1.5.0 <2"
},
"devDependencies": {
"@earendil-works/pi-ai": "1.5.0",
"@earendil-works/pi-coding-agent": "1.5.0",
"@types/node": "^22.19.0",
"tsx": "^4.20.0",
"typescript": "^5.9.0"
}
}

1445
pnpm-lock.yaml generated Normal file
View file

File diff suppressed because it is too large Load diff

6
pnpm-workspace.yaml Normal file
View file

@ -0,0 +1,6 @@
# No dependency needs install scripts: tsx ships esbuild's binary as an
# optional dependency, and the others are transitive pi-ai dependencies.
allowBuilds:
'@google/genai': false
esbuild: false
protobufjs: false

47
src/index.ts Normal file
View file

@ -0,0 +1,47 @@
import type { ExtensionAPI, ExtensionCommandContext } from "@earendil-works/pi-coding-agent";
import { createUnigptProvider, PROVIDER_ID } from "./provider.ts";
const REFRESH_TIMEOUT_MS = 20_000;
type Level = "info" | "warning" | "error";
function report(ctx: ExtensionCommandContext, message: string, level: Level) {
if (ctx.hasUI) ctx.ui.notify(message, level);
// Without a UI (print/JSON modes), keep stdout clean for the agent's output.
else console.error(`[unigpt] ${message}`);
}
export async function refreshCatalog(ctx: ExtensionCommandContext): Promise<void> {
// A status check, unlike resolving auth, never runs a configured key command.
if (!ctx.modelRegistry.getProviderAuthStatus(PROVIDER_ID).configured) {
report(ctx, "UniGPT is not configured. Run /login unigpt first.", "warning");
return;
}
const result = await ctx.modelRegistry.refresh({
providers: [PROVIDER_ID],
force: true,
signal: AbortSignal.timeout(REFRESH_TIMEOUT_MS),
});
const error = result.errors.get(PROVIDER_ID);
if (result.aborted || error) {
const reason = error?.message ?? `no response within ${REFRESH_TIMEOUT_MS / 1000}s`;
report(ctx, `UniGPT refresh failed (${reason}); keeping the cached model list.`, "warning");
return;
}
const count = ctx.modelRegistry.getProvider(PROVIDER_ID)?.getModels().length ?? 0;
report(
ctx,
count > 0
? `UniGPT: ${count} models available. Select one with /model.`
: "UniGPT lists no models for this API key.",
count > 0 ? "info" : "warning",
);
}
export default function unigpt(pi: ExtensionAPI) {
pi.registerProvider(createUnigptProvider());
pi.registerCommand("unigpt-refresh", {
description: "Reload the UniGPT model list now",
handler: (_args, ctx) => refreshCatalog(ctx),
});
}

228
src/provider.ts Normal file
View file

@ -0,0 +1,228 @@
import {
createProvider,
type Model,
type Provider,
} from "@earendil-works/pi-ai";
import { openAIResponsesApi } from "@earendil-works/pi-ai/api/openai-responses.lazy";
import { getBuiltinModels } from "@earendil-works/pi-ai/providers/all";
export const PROVIDER_ID = "unigpt";
export const BASE_URL = "https://api.mygpt.uni-graz.at/api/v1/llms/openai";
export const WEB_URL = "https://unigpt.uni-graz.at";
/** pi's own defaults for custom models.json entries, used when an ID is not in pi's catalog. */
export const FALLBACK_CONTEXT_WINDOW = 128_000;
export const FALLBACK_MAX_TOKENS = 16_384;
export const REQUEST_TIMEOUT_MS = 15_000;
type Fetch = typeof globalThis.fetch;
/** Capabilities of a deployment, as far as they can be known locally. */
export interface ModelMetadata {
name: string;
input: ("text" | "image")[];
contextWindow: number;
maxTokens: number;
}
export type MetadataLookup = (id: string) => ModelMetadata | undefined;
/**
* UniGPT's catalog returns bare deployment IDs, and those IDs are OpenAI model IDs.
* Look them up in pi's bundled catalog, which pi keeps current, instead of keeping a
* table here. OpenAI and Azure list different limits for some models; take the
* conservative intersection so pi never plans for more than either host allows.
*/
export const lookupCatalogMetadata: MetadataLookup = (id) => {
const matches = (["openai", "azure"] as const).flatMap((provider) =>
getBuiltinModels(provider).filter((model) => model.id === id),
);
if (matches.length === 0) return undefined;
return {
name: matches[0].name,
input: matches.every((model) => model.input.includes("image")) ? ["text", "image"] : ["text"],
contextWindow: Math.min(...matches.map((model) => model.contextWindow)),
maxTokens: Math.min(...matches.map((model) => model.maxTokens)),
};
};
function isRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function isValidId(id: unknown): id is string {
// IDs become an HTTP header value, so reject padding and control characters.
return typeof id === "string" && id.length > 0 && id.trim() === id && !/[\u0000-\u001f\u007f]/.test(id);
}
export interface ParsedCatalog {
ids: string[];
/** Entries that were present but unusable, e.g. missing or malformed IDs. */
skipped: number;
}
/**
* Skip individual bad entries so one odd deployment cannot hide all others, but reject
* a malformed or entirely unusable catalog rather than replacing a good cache with it.
*/
export function parseCatalog(payload: unknown): ParsedCatalog {
if (!isRecord(payload) || !Array.isArray(payload.data)) {
throw new Error("UniGPT returned an invalid model catalog: expected a data array.");
}
const ids = payload.data
.map((entry: unknown) => (isRecord(entry) ? entry.id : undefined))
.filter(isValidId);
if (payload.data.length > 0 && ids.length === 0) {
throw new Error("UniGPT returned a model catalog without any valid model IDs.");
}
return { ids: [...new Set(ids)], skipped: payload.data.length - ids.length };
}
export function makeModel(
id: string,
baseUrl: string,
lookup: MetadataLookup = lookupCatalogMetadata,
): Model<"openai-responses"> {
const known = lookup(id);
return {
id,
name: known?.name ?? id,
provider: PROVIDER_ID,
api: "openai-responses",
baseUrl,
headers: { "deployment-id": id },
input: known?.input ?? ["text"],
// UniGPT ignores reasoning.effort and applies a fixed effort per deployment, so
// exposing pi's thinking levels would offer controls that have no effect.
reasoning: false,
contextWindow: known?.contextWindow ?? FALLBACK_CONTEXT_WINDOW,
maxTokens: known?.maxTokens ?? FALLBACK_MAX_TOKENS,
// University users are not billed per token; zero disables cost estimates.
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
};
}
function statusError(response: Response): Error {
const status = response.status;
if (status === 401 || status === 403) {
return new Error(`UniGPT rejected the API key (HTTP ${status}). Check your key and API access.`);
}
if (status === 429) {
const retryAfter = Number(response.headers.get("retry-after"));
const wait = Number.isFinite(retryAfter) && retryAfter > 0 ? ` Retry in ${Math.ceil(retryAfter)}s.` : " Try again shortly.";
return new Error(`UniGPT is rate-limiting requests (HTTP 429).${wait}`);
}
return new Error(`UniGPT model discovery failed (HTTP ${status}). Try again later.`);
}
export async function fetchCatalog(
baseUrl: string,
apiKey: string,
signal: AbortSignal,
fetchImpl: Fetch = globalThis.fetch,
timeoutMs = REQUEST_TIMEOUT_MS,
): Promise<ParsedCatalog> {
signal.throwIfAborted();
const timeout = AbortSignal.timeout(timeoutMs);
const requestSignal = AbortSignal.any([signal, timeout]);
// Caller cancellation propagates as-is; our own deadline becomes a readable error.
const checkAborted = () => {
signal.throwIfAborted();
if (timeout.aborted) {
throw new Error(`UniGPT did not respond within ${Math.round(timeoutMs / 1000)}s. Try again later.`);
}
};
let response: Response;
try {
response = await fetchImpl(`${baseUrl.replace(/\/+$/, "")}/models`, {
headers: { Authorization: `Bearer ${apiKey}`, Accept: "application/json" },
signal: requestSignal,
// Do not forward a credential to a redirect target.
redirect: "error",
});
} catch {
checkAborted();
throw new Error("Could not reach the UniGPT model catalog. Check the endpoint and network connection.");
}
if (!response.ok) {
// Do not echo server bodies: they could contain credentials or other private data.
void response.body?.cancel().catch(() => {});
throw statusError(response);
}
let payload: unknown;
try {
payload = await response.json();
} catch {
checkAborted();
throw new Error("UniGPT returned a model catalog that is not valid JSON.");
}
checkAborted();
return parseCatalog(payload);
}
export interface UnigptProviderOptions {
/** Primarily useful for tests; users can override baseUrl through models.json. */
baseUrl?: string;
fetch?: Fetch;
lookup?: MetadataLookup;
}
export function createUnigptProvider(options: UnigptProviderOptions = {}): Provider<"openai-responses"> {
const baseUrl = options.baseUrl ?? BASE_URL;
const fetchImpl = options.fetch ?? globalThis.fetch;
const lookup = options.lookup ?? lookupCatalogMetadata;
return createProvider<"openai-responses">({
id: PROVIDER_ID,
name: "UniGPT",
baseUrl,
models: [],
api: openAIResponsesApi(),
auth: {
apiKey: {
name: "UniGPT API key",
async login(interaction) {
interaction.signal.throwIfAborted();
interaction.notify({
type: "info",
message: "Paste your UniGPT API key. Pi stores it in its private auth.json.",
links: [{ url: WEB_URL, label: "UniGPT (University of Graz)" }],
});
const key = (await interaction.prompt({
type: "secret",
message: "UniGPT API key",
})).trim();
interaction.signal.throwIfAborted();
if (!key) throw new Error("An API key is required.");
interaction.notify({ type: "progress", message: "Checking API access…" });
const { ids, skipped } = await fetchCatalog(baseUrl, key, interaction.signal, fetchImpl);
const skippedNote = skipped > 0 ? ` (${skipped} malformed catalog entries ignored)` : "";
interaction.notify({
type: "info",
message: ids.length > 0
? `API key accepted: ${ids.length} UniGPT models available${skippedNote}. Select one with /model.`
: "API key accepted, but UniGPT lists no models for it yet. Ask your UniGPT administrator, then run /unigpt-refresh.",
});
return { type: "api_key", key };
},
async resolve({ credential, signal }) {
signal.throwIfAborted();
if (!credential?.key) return undefined;
return {
auth: { apiKey: credential.key },
env: credential.env,
source: "stored credential",
};
},
},
},
async fetchModels(context) {
context.signal.throwIfAborted();
if (context.credential?.type !== "api_key" || !context.credential.key) {
throw new Error("Run /login unigpt before discovering models.");
}
const endpoint = context.baseUrl ?? baseUrl;
const { ids } = await fetchCatalog(endpoint, context.credential.key, context.signal, fetchImpl);
return ids.map((id) => makeModel(id, endpoint, lookup));
},
});
}

67
test/command.test.ts Normal file
View file

@ -0,0 +1,67 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import type { ExtensionCommandContext } from "@earendil-works/pi-coding-agent";
import { refreshCatalog } from "../src/index.ts";
interface Fake {
configured?: boolean;
hasUI?: boolean;
result?: { aborted: boolean; errors: Map<string, Error> };
count?: number;
}
function fakeContext(fake: Fake) {
const notices: { message: string; level: string }[] = [];
const refreshes: unknown[] = [];
const ctx = {
hasUI: fake.hasUI ?? true,
ui: { notify: (message: string, level: string) => notices.push({ message, level }) },
modelRegistry: {
getProviderAuthStatus: () => ({ configured: fake.configured ?? true }),
getProviderAuth: () => { throw new Error("must not resolve credentials"); },
refresh: async (options: unknown) => {
refreshes.push(options);
return fake.result ?? { aborted: false, errors: new Map() };
},
getProvider: () => ({ getModels: () => Array.from({ length: fake.count ?? 0 }) }),
},
} as unknown as ExtensionCommandContext;
return { ctx, notices, refreshes };
}
test("refresh asks to log in without resolving credentials or calling the network", async () => {
const { ctx, notices, refreshes } = fakeContext({ configured: false });
await refreshCatalog(ctx);
assert.equal(refreshes.length, 0);
assert.deepEqual(notices, [{ message: "UniGPT is not configured. Run /login unigpt first.", level: "warning" }]);
});
test("refresh targets only UniGPT, bounded by a deadline, and reports the model count", async () => {
const { ctx, notices, refreshes } = fakeContext({ count: 3 });
await refreshCatalog(ctx);
const [options] = refreshes as { providers: string[]; signal: AbortSignal }[];
assert.deepEqual(options.providers, ["unigpt"]);
assert.ok(options.signal instanceof AbortSignal);
assert.deepEqual(notices, [{ message: "UniGPT: 3 models available. Select one with /model.", level: "info" }]);
});
test("refresh reports errors, timeouts, and empty catalogs as warnings", async () => {
const failed = fakeContext({ result: { aborted: false, errors: new Map([["unigpt", new Error("HTTP 503")]]) } });
await refreshCatalog(failed.ctx);
assert.match(failed.notices[0].message, /refresh failed \(HTTP 503\); keeping the cached/);
const timedOut = fakeContext({ result: { aborted: true, errors: new Map() } });
await refreshCatalog(timedOut.ctx);
assert.match(timedOut.notices[0].message, /no response within 20s/);
const empty = fakeContext({ count: 0 });
await refreshCatalog(empty.ctx);
assert.deepEqual(empty.notices, [{ message: "UniGPT lists no models for this API key.", level: "warning" }]);
});
test("without a UI, refresh reports on stderr instead of staying silent", async (t) => {
const lines: string[] = [];
t.mock.method(console, "error", (line: string) => lines.push(line));
const { ctx, notices } = fakeContext({ hasUI: false, count: 2 });
await refreshCatalog(ctx);
assert.equal(notices.length, 0);
assert.deepEqual(lines, ["[unigpt] UniGPT: 2 models available. Select one with /model."]);
});

372
test/provider.test.ts Normal file
View file

@ -0,0 +1,372 @@
import assert from "node:assert/strict";
import { once } from "node:events";
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
import { test, type TestContext } from "node:test";
import {
createModels,
InMemoryCredentialStore,
InMemoryModelsStore,
type AuthEvent,
type AuthInteraction,
} from "@earendil-works/pi-ai";
import {
BASE_URL,
createUnigptProvider,
FALLBACK_CONTEXT_WINDOW,
FALLBACK_MAX_TOKENS,
fetchCatalog,
lookupCatalogMetadata,
makeModel,
parseCatalog,
type ModelMetadata,
} from "../src/provider.ts";
const KEY = "test-key-not-a-real-secret";
const signal = () => new AbortController().signal;
const catalog = (...ids: string[]) => Response.json({ data: ids.map((id) => ({ id })) });
function interaction(key = KEY, events: AuthEvent[] = []): AuthInteraction {
return { prompt: async () => key, notify: (event) => events.push(event) };
}
const ids = async (...args: Parameters<typeof fetchCatalog>) => (await fetchCatalog(...args)).ids;
async function runtime(fetchImpl: typeof fetch, withCredential = true) {
const credentials = new InMemoryCredentialStore();
if (withCredential) {
await credentials.modify("unigpt", async () => ({ type: "api_key", key: KEY }));
}
const modelsStore = new InMemoryModelsStore();
const models = createModels({ credentials, modelsStore });
const provider = createUnigptProvider({ fetch: fetchImpl });
models.setProvider(provider);
return { models, provider, credentials, modelsStore };
}
test("parses IDs, deduplicates in catalog order, and permits an empty catalog", () => {
assert.deepEqual(parseCatalog({ data: [{ id: "second" }, { id: "first" }, { id: "second" }] }), { ids: ["second", "first"], skipped: 0 });
assert.deepEqual(parseCatalog({ data: [] }), { ids: [], skipped: 0 });
});
test("skips unusable entries but rejects malformed or entirely unusable catalogs", () => {
assert.deepEqual(
parseCatalog({ data: [{ id: "good" }, null, {}, { id: 42 }, { id: "" }, { id: " padded " }, { id: "header\ninjection" }] }),
{ ids: ["good"], skipped: 6 },
);
for (const payload of [null, [], {}, { data: {} }, { data: [null] }, { data: [{ id: "a\r\nb" }] }]) {
assert.throws(() => parseCatalog(payload), /invalid|without any valid/);
}
});
const known: Record<string, ModelMetadata> = {
"model-a": { name: "Model A", input: ["text", "image"], contextWindow: 272_000, maxTokens: 128_000 },
};
const lookup = (id: string) => known[id];
test("known deployments take pi's catalog metadata; each gets its own header", () => {
const a = makeModel("model-a", BASE_URL, lookup);
const b = makeModel("model-b", BASE_URL, lookup);
assert.deepEqual(a.headers, { "deployment-id": "model-a" });
assert.deepEqual(b.headers, { "deployment-id": "model-b" });
assert.equal(a.provider, "unigpt");
assert.equal(a.api, "openai-responses");
assert.equal(a.name, "Model A");
assert.deepEqual(a.input, ["text", "image"]);
assert.equal(a.contextWindow, 272_000);
assert.equal(a.maxTokens, 128_000);
// UniGPT ignores reasoning.effort, so no deployment advertises thinking controls.
assert.equal(a.reasoning, false);
assert.deepEqual(a.cost, { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 });
assert.equal(a.promptCache, undefined);
});
test("unknown deployments fall back to pi's custom-model defaults", () => {
const b = makeModel("model-b", BASE_URL, lookup);
assert.equal(b.name, "model-b");
assert.deepEqual(b.input, ["text"]);
assert.equal(b.contextWindow, FALLBACK_CONTEXT_WINDOW);
assert.equal(b.maxTokens, FALLBACK_MAX_TOKENS);
assert.equal(b.reasoning, false);
});
test("catalog lookup uses pi's bundled OpenAI/Azure entries and takes the conservative limit", () => {
const gpt4o = lookupCatalogMetadata("gpt-4o");
assert.ok(gpt4o, "pi's catalog should know gpt-4o");
assert.deepEqual(gpt4o.input, ["text", "image"]);
assert.ok(gpt4o.contextWindow >= 100_000 && gpt4o.maxTokens >= 4_096);
assert.equal(lookupCatalogMetadata("definitely-not-a-model"), undefined);
});
test("discovery uses bearer auth, no deployment header, a timeout signal, and no redirects", async () => {
const fakeFetch: typeof fetch = async (url, options) => {
assert.equal(url, BASE_URL + "/models");
const headers = new Headers(options?.headers);
assert.equal(headers.get("Authorization"), `Bearer ${KEY}`);
assert.equal(headers.get("deployment-id"), null);
assert.equal(headers.get("Accept"), "application/json");
assert.equal(options?.redirect, "error");
assert.ok(options?.signal instanceof AbortSignal);
return catalog("model-a");
};
assert.deepEqual(await ids(BASE_URL + "///", KEY, signal(), fakeFetch), ["model-a"]);
});
test("unconfigured providers do not fetch or become available", async () => {
let calls = 0;
const { models } = await runtime(async () => { calls++; return catalog("a"); }, false);
assert.deepEqual(await models.getAvailable("unigpt"), []);
assert.equal(await models.getAuth("unigpt"), undefined);
assert.equal((await models.refresh()).errors.size, 0);
assert.equal(calls, 0);
});
test("native login masks input, validates access, and lets pi persist the credential", async () => {
let calls = 0;
const events: AuthEvent[] = [];
const { models, credentials } = await runtime(async () => { calls++; return catalog("a", "b"); }, false);
await models.login("unigpt", "api_key", {
prompt: async (prompt) => {
assert.equal(prompt.type, "secret");
return ` ${KEY} `;
},
notify: (event) => events.push(event),
});
assert.equal(calls, 1);
assert.ok(!JSON.stringify(events).includes(KEY));
assert.ok(events.some((e) => e.type === "info" && e.links?.some((link) => link.url.startsWith("https://unigpt.uni-graz.at"))));
assert.ok(events.some((e) => e.type === "info" && /2 UniGPT models available/.test(e.message)));
assert.deepEqual(await credentials.read("unigpt"), { type: "api_key", key: KEY });
assert.equal((await models.refresh()).errors.size, 0);
assert.deepEqual(models.getModels("unigpt").map((m) => m.id), ["a", "b"]);
assert.equal((await models.getAvailable("unigpt")).length, 2);
await models.logout("unigpt");
assert.equal(await credentials.read("unigpt"), undefined);
assert.deepEqual(await models.getAvailable("unigpt"), []);
});
test("empty and rejected API keys never overwrite a saved credential", async () => {
let calls = 0;
const { models, credentials } = await runtime(async () => { calls++; return new Response("private upstream body", { status: 401 }); });
await assert.rejects(models.login("unigpt", "api_key", interaction(" ")), /required/);
assert.equal(calls, 0);
await assert.rejects(models.login("unigpt", "api_key", interaction("invalid-new-key")), /401/);
assert.deepEqual(await credentials.read("unigpt"), { type: "api_key", key: KEY });
});
test("login with an empty catalog still saves the key but explains the empty model list", async () => {
const events: AuthEvent[] = [];
const { models, credentials } = await runtime(async () => catalog(), false);
await models.login("unigpt", "api_key", interaction(KEY, events));
assert.deepEqual(await credentials.read("unigpt"), { type: "api_key", key: KEY });
assert.ok(events.some((e) => e.type === "info" && /lists no models/.test(e.message)));
});
test("cancelled login never saves a key or sends validation traffic", async () => {
const controller = new AbortController();
let calls = 0;
const { models, credentials } = await runtime(async () => { calls++; return catalog("a"); }, false);
await assert.rejects(models.login("unigpt", "api_key", {
signal: controller.signal,
prompt: async () => { controller.abort(); return KEY; },
notify: () => {},
}), { name: "AbortError" });
assert.equal(calls, 0);
assert.equal(await credentials.read("unigpt"), undefined);
});
test("discovery errors are useful but never echo secrets or server error bodies", async () => {
for (const status of [401, 403, 404, 429, 500]) {
await assert.rejects(fetchCatalog(BASE_URL, KEY, signal(), async () => new Response(KEY, { status })), (error: Error) => {
assert.match(error.message, new RegExp(String(status)));
assert.ok(!error.message.includes(KEY));
return true;
});
}
await assert.rejects(fetchCatalog(BASE_URL, KEY, signal(), async () => { throw new Error(KEY); }), (error: Error) => {
assert.match(error.message, /Could not reach/);
assert.ok(!error.message.includes(KEY));
return true;
});
await assert.rejects(fetchCatalog(BASE_URL, KEY, signal(), async () => new Response("not-json")), /not valid JSON/);
});
test("rate limits surface Retry-After", async () => {
const limited = async () => new Response(null, { status: 429, headers: { "Retry-After": "30" } });
await assert.rejects(fetchCatalog(BASE_URL, KEY, signal(), limited), /rate-limiting.*Retry in 30s/);
});
test("a stalled endpoint produces a readable timeout, not an AbortError", async () => {
const stalled: typeof fetch = (_url, options) => new Promise<Response>((_resolve, reject) => {
options!.signal!.addEventListener("abort", () => reject(options!.signal!.reason), { once: true });
});
await assert.rejects(fetchCatalog(BASE_URL, KEY, signal(), stalled, 20), (error: Error) => {
assert.equal(error.name, "Error");
assert.match(error.message, /did not respond within/);
return true;
});
});
test("discovery propagates cancellation to fetch", async () => {
const controller = new AbortController();
let started!: () => void;
const ready = new Promise<void>((resolve) => { started = resolve; });
const promise = fetchCatalog(BASE_URL, KEY, controller.signal, async (_url: unknown, options?: RequestInit) => {
started();
return new Promise<Response>((_resolve, reject) => {
options!.signal!.addEventListener("abort", () => reject(options!.signal!.reason), { once: true });
});
});
await ready;
controller.abort();
await assert.rejects(promise, { name: "AbortError" });
});
test("refresh persists catalog metadata only, restores offline, and keeps cache on failure", async () => {
let fail = false;
const { models, provider, credentials, modelsStore } = await runtime(async () => {
if (fail) return new Response("unavailable", { status: 503 });
return catalog("a", "b");
});
assert.equal((await models.refresh()).errors.size, 0);
const stored = await modelsStore.read("unigpt");
assert.ok(stored);
assert.ok(!JSON.stringify(stored).includes(KEY));
assert.deepEqual(stored.models.map((m) => m.id), ["a", "b"]);
fail = true;
assert.equal((await models.refresh()).errors.size, 1);
assert.deepEqual(provider.getModels().map((m) => m.id), ["a", "b"]);
let calls = 0;
const offline = createModels({ credentials, modelsStore });
offline.setProvider(createUnigptProvider({ fetch: async () => { calls++; throw new Error("no network"); } }));
assert.equal((await offline.refresh({ allowNetwork: false })).errors.size, 0);
assert.equal(calls, 0);
assert.deepEqual(offline.getModels("unigpt").map((m) => m.id), ["a", "b"]);
});
test("malformed refresh preserves the previous catalog; partial and valid refreshes replace it", async () => {
let response = () => catalog("a", "b");
const { models } = await runtime(async () => response());
await models.refresh();
response = () => Response.json({ data: [{}, null] });
assert.equal((await models.refresh()).errors.size, 1);
assert.deepEqual(models.getModels("unigpt").map((m) => m.id), ["a", "b"]);
response = () => Response.json({ data: [{ id: "a" }, {}] });
assert.equal((await models.refresh()).errors.size, 0);
assert.deepEqual(models.getModels("unigpt").map((m) => m.id), ["a"]);
response = () => catalog("b", "c");
await models.refresh();
assert.deepEqual(models.getModels("unigpt").map((m) => m.id), ["b", "c"]);
response = () => catalog();
await models.refresh();
assert.deepEqual(models.getModels("unigpt"), []);
});
test("discovery honors the effective base URL supplied by pi", async () => {
const alternate = "https://gateway.example.test/v1";
const provider = createUnigptProvider({ fetch: async (url) => {
assert.equal(url, alternate + "/models");
return catalog("a");
} });
await provider.refreshModels!({
credential: { type: "api_key", key: KEY },
baseUrl: alternate,
signal: signal(),
allowNetwork: true,
publish: async (publication) => { publication.update?.(); return true; },
});
assert.equal(provider.getModels()[0].baseUrl, alternate);
});
/** A local Responses endpoint plus a configured runtime whose catalog lists `ids`. */
async function localGateway(t: TestContext, ids: string[], handler: (req: IncomingMessage, res: ServerResponse) => void) {
const server = createServer(handler);
server.listen(0, "127.0.0.1");
await once(server, "listening");
t.after(() => new Promise<void>((resolve) => { server.closeAllConnections(); server.close(() => resolve()); }));
const address = server.address();
assert.ok(address && typeof address !== "string");
const credentials = new InMemoryCredentialStore();
await credentials.modify("unigpt", async () => ({ type: "api_key", key: KEY }));
const models = createModels({ credentials });
models.setProvider(createUnigptProvider({
baseUrl: `http://127.0.0.1:${address.port}/v1`,
fetch: async () => catalog(...ids),
lookup: () => undefined,
}));
await models.refresh();
return models;
}
function sendEvents(res: ServerResponse, events: unknown[]) {
res.writeHead(200, { "Content-Type": "text/event-stream" });
for (const event of events) res.write(`data: ${JSON.stringify(event)}\n\n`);
res.end("data: [DONE]\n\n");
}
test("real Responses transport sends deployment/auth headers, streams text, and records usage", async (t) => {
const requests: { url?: string; headers: IncomingMessage["headers"]; body: Record<string, unknown> }[] = [];
const models = await localGateway(t, ["deployment-a", "deployment-b"], (req, res) => {
let body = "";
req.on("data", (chunk) => { body += chunk; });
req.on("end", () => {
requests.push({ url: req.url, headers: req.headers, body: JSON.parse(body) });
const item = { id: "msg_1", type: "message", role: "assistant", status: "completed", content: [{ type: "output_text", text: "Hello!", annotations: [] }] };
sendEvents(res, [
{ type: "response.created", response: { id: "resp_1", status: "in_progress" } },
{ type: "response.output_item.added", output_index: 0, item: { ...item, content: [] } },
{ type: "response.output_text.delta", item_id: "msg_1", output_index: 0, content_index: 0, delta: "Hello!" },
{ type: "response.output_item.done", output_index: 0, item },
{ type: "response.completed", response: { id: "resp_1", status: "completed", output: [item], usage: { input_tokens: 12, output_tokens: 3, total_tokens: 15 } } },
]);
});
});
for (const id of ["deployment-a", "deployment-b"]) {
const model = models.getModel("unigpt", id)!;
const stream = models.streamSimple(model, { messages: [{ role: "user", content: "Hello", timestamp: Date.now() }] });
const eventTypes: string[] = [];
for await (const event of stream) eventTypes.push(event.type);
const result = await stream.result();
assert.equal(result.stopReason, "stop", result.errorMessage);
assert.ok(eventTypes.includes("text_delta"));
assert.equal(result.content[0].type, "text");
assert.equal(result.usage.input, 12);
assert.equal(result.usage.output, 3);
}
assert.equal(requests.length, 2);
for (const [index, request] of requests.entries()) {
const id = ["deployment-a", "deployment-b"][index];
assert.equal(request.url, "/v1/responses");
assert.equal(request.headers.authorization, `Bearer ${KEY}`);
assert.equal(request.headers["deployment-id"], id);
assert.equal(request.body.model, id);
assert.equal(request.body.stream, true);
assert.equal(request.body.reasoning, undefined);
assert.equal(request.body.max_output_tokens, FALLBACK_MAX_TOKENS);
}
});
test("Responses transport preserves structured tool calls", async (t) => {
const models = await localGateway(t, ["a"], (req, res) => {
req.resume();
req.on("end", () => {
const item = { type: "function_call", id: "fc_1", call_id: "call_1", name: "ping", arguments: '{"target":"api"}', status: "completed" };
sendEvents(res, [
{ type: "response.created", response: { id: "resp_2", status: "in_progress" } },
{ type: "response.output_item.added", output_index: 0, item: { ...item, arguments: "" } },
{ type: "response.function_call_arguments.delta", item_id: "fc_1", output_index: 0, delta: item.arguments },
{ type: "response.output_item.done", output_index: 0, item },
{ type: "response.completed", response: { id: "resp_2", status: "completed", output: [item], usage: { input_tokens: 5, output_tokens: 4, total_tokens: 9 } } },
]);
});
});
const result = await models.completeSimple(models.getModel("unigpt", "a")!, {
messages: [{ role: "user", content: "Ping the API", timestamp: Date.now() }],
tools: [{ name: "ping", description: "Ping a target", parameters: { type: "object", properties: { target: { type: "string" } }, required: ["target"] } }],
});
assert.equal(result.stopReason, "toolUse", result.errorMessage);
const call = result.content.find((part) => part.type === "toolCall");
assert.ok(call && call.type === "toolCall");
assert.equal(call.name, "ping");
assert.deepEqual(call.arguments, { target: "api" });
});

18
tsconfig.json Normal file
View file

@ -0,0 +1,18 @@
{
"compilerOptions": {
"target": "ES2023",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"noEmit": true,
"allowImportingTsExtensions": true,
"skipLibCheck": true,
"types": [
"node"
]
},
"include": [
"src/**/*.ts",
"test/**/*.ts"
]
}