With default options (compression = kSnappyCompression), a data block whose raw
size reaches 2^32 bytes is handed to snappy, which cannot represent it. LevelDB
ignores that and writes the block as "snappy-compressed" anyway. Every record in
that block — including small, unrelated records that happened to share it — becomes
permanently unreadable once the memtable is flushed. Put returned OK, the flush
reports nothing, and the WAL holding the only good copy is deleted after the flush.
A block can only get this big from one large value (block_size is clipped to
4 MiB and the size check in TableBuilder::Add runs after the entry is added),
so the trigger is a single value just under 4 GiB. Values of 4 GiB and above
already fail earlier, in WriteBatch (#718), so this is the band of values that
LevelDB accepts and acknowledges, and then loses.
Expected behavior
A Put that returns OK stays readable after the memtable is flushed and after a
clean close/reopen. If a block cannot be compressed, it is stored uncompressed —
which TableBuilder::WriteBlock already does when compression "fails".
Actual behavior
db->Put("before", "BEFORE-VALUE") -> OK
db->Put("k", <4294967243 x 'a'>) -> OK (raw block = exactly 2^32 bytes)
Get before flush -> both OK
CompactRange(nullptr, nullptr) -> (void; nothing logged)
Get "k" -> Corruption: corrupted snappy compressed block length
Get "before" -> Corruption: corrupted snappy compressed block length <-- 12-byte neighbour
close + reopen -> same, permanently
One byte smaller (raw block 2^32 − 1) round-trips. The identical value with
compression = kNoCompression round-trips.
How the block is destroyed depends on the snappy version linked, but the data is
lost with both:
snappy
what snappy does with a ≥ 2^32 input
what LevelDB writes
read error
≤ 1.2.2 (what distributions ship)
writes a stream whose varint32 length header is N mod 2^32
a 201 MB block that decodes to the wrong length
corrupted snappy compressed block contents
≥ 1.3.0 (google/snappy@1a09131, Sep 2026)
refuses: writes nothing, *compressed_length = 0
a 0-byte "compressed" block; the whole .ldb is 93 bytes
corrupted snappy compressed block length
With snappy 1.3.x the 4 GiB value never reaches disk at all.
Reproduction
Self-contained; needs ~14 GB RAM and ~5 GB disk per run, ~1 minute.
The snappy stream format stores the uncompressed length as a varint32 ("up to a
maximum of 2^32 - 1", format_description.txt). For length >= 2^32, snappy ≤ 1.2.2
encodes length mod 2^32, and snappy ≥ 1.3.0 writes nothing and sets outlen = 0.
Either way the wrapper returns true.
table/table_builder.cc:158-170 then takes the compressed branch, because a wrapped
or empty output is always "smaller than 7/8 of the input":
case kSnappyCompression: {
std::string* compressed = &r->compressed_output;
if (port::Snappy_Compress(raw.data(), raw.size(), compressed) &&
compressed->size() < raw.size() - (raw.size() / 8u)) {
block_contents = *compressed;
} else {
// Snappy not supported, or compressed less than 12.5%, so just
// store uncompressed form
block_contents = raw;
type = kNoCompression;
}
The else branch already does the right thing. It is just never reached.
Why one block can get this big — table/table_builder.cc:117-122, the size check
runs after the entry is appended, so a single large value becomes a single huge
block regardless of block_size:
Make the wrapper say no when snappy cannot represent the input, so the existing
uncompressed path is taken:
--- a/port/port_stdcxx.h
+++ b/port/port_stdcxx.h
@@ -37,6 +37,7 @@
#include <cstdint>
+#include <limits>
@@ -90,6 +91,8 @@ inline bool Snappy_Compress(const char* input, size_t length,
std::string* output) {
#if HAVE_SNAPPY
+ // The snappy stream format stores the uncompressed length as a varint32.
+ if (length > std::numeric_limits<uint32_t>::max()) return false;
output->resize(snappy::MaxCompressedLength(length));
(Treating outlen == 0 as failure would also cover snappy ≥ 1.3.0, but not the
1.2.x builds most distributions ship, so the explicit length check is the one
that works with every snappy.)
With this patch applied to main, the failing case above prints => intact
with both snappy 1.3.1 and 1.2.2, and the 2^32 − 1 control is unchanged.
Scope of the fix. This makes a 2^32-byte block survive by storing it raw. On
the read path, Block::restart_offset_ is a uint32_t (table/block.h:38,
computed at table/block.cc:37). So an uncompressed block whose restart array
starts at or beyond 2^32 (block ≥ 2^32 + 8 bytes) has a separate 32-bit problem;
for example, a lone 4294967295-byte value with kNoCompression reads back
NotFound after a flush on main. That one is independent of compression and is
not part of this report. Rejecting oversized values at write time would close both.
Affected versions
main @ 7ee830d (2026-03-10): reproduced, with snappy 1.3.1 and 1.2.2.
1.23: reproduced, in earlier testing with the same probe (message there is
corrupted compressed block contents, since 1.23 used one message for both read
failures).
Between 1.23 and main, port/port_stdcxx.h and table/table_builder.cc changed
only to add zstd. The snappy wrapper and the kSnappyCompression case are
identical, so older snappy-enabled releases are presumably affected too
(not tested).