@tokenizer/token provides TypeScript definitions for an strtok3 token. It is useful when you are using strtok3-based parsing in TypeScript and want type information for tokenizer tokens.
Project status
- The source appears mostly quiet rather than actively maintained, with the latest published TypeScript-definition update shown as v0.3.0 on 2021-07-12, and only an upstream push dated 2025-02-10 (no newer update details provided).
- Update cadence looks sporadic, with long gaps between documented updates (2020-01-13, 2021-02-28, 2021-07-12) and no evidence of recent releases despite the 2025 upstream activity.
AI summary generated
Recent updates
v0.3.0
Release v0.3.0 updates the TypeScript type definitions for @tokenizer/token to support using Uint8Array subtypes (for example, Buffer) instead of only Uint8Array. The change is implemented by expanding generics on IGetToken and IToken and updating the get and put method signatures to accept a configurable Array type.
Featuresv0.2.0
v0.2.0 updates the TypeScript type definitions for @tokenizer/token to improve cross-platform compatibility. The main change replaces Buffer-based method parameters with Uint8Array in the IGetToken and IToken interfaces.
Breakingv0.1.1
v0.1.1 primarily updates the package metadata. The release notes mention adding the missing MIT license field, and the diff also includes changes to npm packaging configuration and README content.
v0.1.0
Release v0.1.0 of @tokenizer/token has been published to npm. The provided release notes contain no specific details about features, fixes, or breaking changes beyond the version reference, so developers should review the npm package changelog and diff for actual code impact.