LmScript Client

Dependency-Free Typescript client for LmScript.

The Complete API documentation is available at the project's JSR page.

Installation

Installing from JSR

Follow the instructions on the project's JSR page.

Usage

There are more complete examples in the client's examples folder.

Adding text to context

Use .push to add text to the context.

Use .system, .user and .assistant to create the messages with the required formatting for them to be assigned to their roles.

The role functions can receive a single string that will be applied to .push, or it receives a callback that passes the client object where you can call any of the supported functions.

const { captured } = await client
  .system("You are a helpful assistant.")
  .user(question1)
  .assistant((m) => m.gen("answer1", { maxTokens: 256 }))
  .run();

Generation

Generates the text and captures it with a name.

const {
  captured: { language },
} = await client.push("The best programming language is ").gen("language").run();

console.log(language);

Selection

Selects one of the choices.

const {
  captured: { language },
} = await client
  .push("The best programming language is ")
  .select("language", { choices: ["javascript", "typescript"] })
  .run();

console.log(language);

Repeat

Repeats a previous capture.

const {
  captured: { language },
} = await client
  .push("The best programming language is")
  .gen("language")
  .push(". I think that ")
  .repeat("language")
  .push(" is the best language because")
  .gen("explanation", { maxTokens: 256 })
  .run();

console.log(language);

Backends

SGLang Backend

Use it with a regular SGLang server.

import { LmScript } from "@lmscript/client";
import { SGLangBackend } from "@lmscript/client/backends/sglang";

const backend = new SGLangBackend(`http://localhost:30004`);
const client = new LmScript(backend, {
  template: "mistral",
  temperature: 0.1,
});

Runpod Serverless Backend

Use it LmScript's Runpod Serverless Docker Image.

import { LmScript } from "@lmscript/client";
import { RunpodServerlessBackend } from "@lmscript/client/backends/runpod-serverless-sglang";

const backend = new RunpodServerlessBackend(
  getEnvVarOrThrow("RUNPOD_URL"),
  getEnvVarOrThrow("RUNPOD_TOKEN"),
);
const client = new LmScript(backend);

License

MIT

Contributing

Running all examples tests all features of the client.

The examples run in Deno and no dependency setup is required.

• Executing the SGLang Runpod tests, local server

  RUNPOD_URL="http://localhost:8000" RUNPOD_TOKEN="anything" deno run --allow-net --allow-env examples/runpod-serverless-sglang.ts

• Executing the SGLang tests, local server

  deno run --allow-net --allow-env examples/sglang.ts