Skip to main content
Home
Browse packagesDocsSign in

@seally/simple-lexer@0.1.2
Built and signed on GitHub Actions

latest
Seally/jsr-simple-lexer

An extremely simple JavaScript RegExp-based lexer.

This package works with DenoIt is unknown whether this package works with Cloudflare Workers, Node.js, Bun, Browsers
It is unknown whether this package works with Cloudflare Workers
It is unknown whether this package works with Node.js
This package works with Deno
It is unknown whether this package works with Bun
It is unknown whether this package works with Browsers
JSR Score
100%
Published
a week ago (0.1.2)
class SimpleLexer

A very basic lexer that simply just takes string and RegExp patterns and spits out the associated tokens with no advanced bells and whistles.

Examples

▶

Example 1

 
const lexer = new SimpleLexer({
    tokens: {
        inlineWhitespace: /[ \t]+/u,
        lineEnding: /\r\n?|[\n\f\v\u2028\u2029]/u,
        word: /[^ \t\r\n\f\v\u2028\u2029]+/u,
    }
});

const tokens = lexer.tokenize(input);

for (const token of tokens) {
    // ...
}

Constructors

new
SimpleLexer(definition: D)

Type Parameters

Methods

match(
input: string,
offset: number,
): SimpleLexerToken<_ExtractType<D["tokens"]>> | null

Matches a token at the given offset in input.

tokenize(
input: string,
offset?: number,
): Generator<SimpleLexerToken<_ExtractType<D["tokens"]>>, void>

Tokenizes the input starting at the given offset.

Add Package

deno add @seally/simple-lexer

Import symbol

import { SimpleLexer } from "@seally/simple-lexer";

---- OR ----

Import directly with a jsr specifier

import { SimpleLexer } from "jsr:@seally/simple-lexer";

Add Package

npx jsr add @seally/simple-lexer

Import symbol

import { SimpleLexer } from "@seally/simple-lexer";

Add Package

yarn dlx jsr add @seally/simple-lexer

Import symbol

import { SimpleLexer } from "@seally/simple-lexer";

Add Package

pnpm dlx jsr add @seally/simple-lexer

Import symbol

import { SimpleLexer } from "@seally/simple-lexer";

Add Package

bunx jsr add @seally/simple-lexer

Import symbol

import { SimpleLexer } from "@seally/simple-lexer";