Back to Explore

Borewit/tokenizer-inflate

GitHub
4 updates · last 90 daysOpen source

Last release:

@tokenizer/inflate is a Node.js package for efficiently parsing and extracting data from ZIP files using a tokenizer-based approach, optimized to minimize memory usage. It supports streaming and random-access style workflows, integrates with strtok3 for partial extraction, and provides callbacks to handle specific files or stop extraction conditionally.

Project status

  • Actively maintained: upstream was pushed on 2026-10-25, and several updates landed in October 2026.
  • Cadence is uneven, with a long gap after November 2025 followed by a burst of updates in October 2026.

AI summary generated

AI-generated from public sources. May be inaccurate. Report

Recent updates

  • v0.6.0

    v0.6.0 adds ZIP traversal budgets and processes random-access central-directory entries incrementally, reducing the need to materialize the full directory before extraction. The code also adds ZIP traversal options and changes data-descriptor scanning and limit-error message behavior that are not described in the release notes.

    BreakingFeatures
  • v0.5.2

    v0.5.2 documents ZIP handler usage and changes central-directory parsing to use a bounded tokenizer view. The implementation also updates and promotes `strtok3` to a runtime dependency, and changes how the tokenizer position is restored when directory parsing fails.

    Breaking
  • v0.5.1

    v0.5.1 ships a targeted bug fix related to bounded inflation when ZIP entries include trailing ZIP descriptor bytes after the deflate stream. The code now explicitly recovers from Node.js error ERR_TRAILING_JUNK_AFTER_STREAM_END under those bounded conditions, and adds regression tests to cover both recoverable and non-recoverable cases.

  • v0.5.0

    v0.5.0 introduces opt-in per-entry ZIP processing limits intended to prevent unbounded scanning, buffering, and decompression. The public surface also expands to export the new limit configuration type and the limit-exceeded error type. A small dependency bump (token-types) is included.

    Features
  • v0.4.1

    Release v0.4.1 notes claim an implementation change (replacing fflate with DecompressionStream), a gzip handler bug fix, and two dependency bumps. However, the provided code diff for v0.4.0 to v0.4.1 shows only a package.json version bump and the addition of a prepublishOnly script, with no visible code changes to decompression or gzip handling.

    Features
  • v0.4.0

    v0.4.0 includes a major internal implementation change for gzip and deflate decompression, switching from the fflate library to the built-in DecompressionStream APIs. The provided release notes are effectively empty (they say this was a bad release and to use v0.4.1), so most impactful behavior changes are undocumented.

  • v0.3.0

    Release v0.3.0 adds gzip inflation support (via a new handler) and bumps the runtime dependency debug from 4.4.0 to 4.4.1. The codebase also appears to have been refactored to split ZIP handling into dedicated modules and to add tests/fixtures for additional archive formats.

    Features
  • v0.2.7

    v0.2.7 primarily updates the project to require Node.js 18 or newer and bumps the runtime dependency debug from 4.3.7 to 4.4.0. The release notes are narrowly scoped, but the actual diff also updates CI configuration and a large set of development dependency versions (and their transitive lockfile entries).

    Breaking
  • v0.2.6

    Release v0.2.6 makes a small package change by removing the unused npm dependency named "link". The diff also updates README documentation and examples, but there are no source code changes shown beyond dependency removal.

  • v0.2.5

    Release v0.2.5 focuses on ZIP parsing correctness, specifically around iterating central directory entries and properly reading until the end of the central directory. The code changes are mostly in ZipHandler’s ZIP traversal logic, with additional test coverage added for S3-backed tokenizers.

  • v0.2.4

    v0.2.4 primarily adjusts how the zip parser detects and stops when it encounters the central directory instead of continuing to parse local file headers. The change also refactors signature handling by introducing a shared Signature constant and using tokenized peeks rather than manual byte prefix checks.

  • v0.2.3

    v0.2.3 is primarily described as a bug fix for reading certain ZIP files. The code diff shows a larger internal refactor of ZIP header parsing, including changes to how ZIP data descriptors and record tokens are decoded, plus a minor dependency bump (strtok3).

  • v0.2.2

    v0.2.2 is a small patch release focused on enabling ZIP inflation when the underlying tokenizer does not support random access. The release notes mention a bug fix for reading without random-read support, but the code shows additional behavioral changes in how central directory parsing is gated.

  • v0.2.1

    The release notes for v0.2.1 claim there are no changes. However, the code diff shows a full rename/move from the @tokenizer/deflate package to @tokenizer/inflate, including package metadata, documentation, and debug namespace.

    Breaking