Skip to main content

A simple generic string tokenizer written in TypeScript

This package works with Bun, Cloudflare Workers, Node.js, Deno, Browsers
JSR Score
100%
Published
a month ago (1.0.3)
type alias TokenMatcherFunction

A function that returns the matched string that the text parameter starts with. Should return null if there is no match.

Definition

(text: string) => string | null

Add Package

deno add @santerijps/tokenizer

Import symbol

import { type TokenMatcherFunction } from "@santerijps/tokenizer";

Add Package

npx jsr add @santerijps/tokenizer

Import symbol

import { type TokenMatcherFunction } from "@santerijps/tokenizer";

Add Package

yarn dlx jsr add @santerijps/tokenizer

Import symbol

import { type TokenMatcherFunction } from "@santerijps/tokenizer";

Add Package

pnpm dlx jsr add @santerijps/tokenizer

Import symbol

import { type TokenMatcherFunction } from "@santerijps/tokenizer";

Add Package

bunx jsr add @santerijps/tokenizer

Import symbol

import { type TokenMatcherFunction } from "@santerijps/tokenizer";