www.it-connect.fr
HollowByte: Silent OpenSSL DoS Flaw Patched Without a CVE
Eleven bytes. That’s all it takes for a remote, unauthenticated attacker to exhaust a server’s memory through OpenSSL. This flaw has a name: HollowByte, but the problem is that it was fixed in silence, with no CVE to flag it.
Eleven Bytes to Freeze a Server’s Memory
It all starts with the way OpenSSL reads the very beginning of a TLS handshake. Each handshake message begins with a 4-byte header, three of which are used to declare the size of the message body that follows. Vulnerable versions blindly trusted that value: they reserved a receive buffer matching the announced size before even receiving a single byte of data.
As a result, a malicious packet of just 11 bytes is enough to trigger an unchecked memory allocation of up to 131 KB. Meanwhile, the execution thread waits for data that will never arrive. On paper, this technique resembles the Slowloris vulnerability. But HollowByte adds an accumulation effect tied to the way the glibc library works. When the attacker’s connection ends, OpenSSL does free the buffer, but glibc does not immediately return small and medium-sized allocations to the operating system: the library keeps them for potential reuse.
By launching waves of connections with random sizes, the attacker prevents the allocator from recycling those freed blocks. The memory heap becomes fragmented, and the server’s resident memory keeps climbing. “Even after the attacker disconnects, the server remains bloated for a long time. The only way to recover that memory is to terminate the process,” notes Okta’s Red Team, which uncovered the issue.
In their report, Okta’s researchers describe tests carried out on OpenSSL instances paired with NGINX:
- On a machine with 1 GB of RAM, a denial of service was triggered by the OOM Killer (Out Of Memory) mechanism after 547 MB of frozen and fragmented memory.
- On a system with 16 GB of RAM, the attack consumed 25% of the total memory while staying below the connection cap, making traditional connection-limit protections ineffective.
This security issue affects many solutions, since OpenSSL is embedded almost everywhere. The flaw could potentially impact web servers (Apache, NGINX), runtimes (Node.js, Python, Ruby, PHP), and databases (MySQL, PostgreSQL).
A Patch Without a CVE: Why That’s a Problem
The OpenSSL team made the necessary code changes to fix this security issue. From now on, the buffer is expanded only as the bytes actually arrive on the execution thread. The problem is that this fix was quietly merged into OpenSSL 4.0.1, following on from version 4.0 presented in this article. The patch was then backported to versions 3.6.3, 3.5.7, 3.4.6, and 3.0.21. These versions were released on June 9, 2026.
The issue is that no CVE reference was assigned to HollowByte. As a result, this security flaw is largely invisible to scanners and will therefore go undetected. How can that be explained? In practice, OpenSSL treated the issue as a simple hardening fix rather than a security vulnerability. The result: no associated CVE.
That said, release 4.0.1, published on June 9, 2026, is a significant security update. It fixes 18 CVEs, the most severe being a use-after-free in PKCS7_verify() tracked as CVE-2026-45447, along with two regressions, but none of those vulnerabilities correspond to HollowByte. This security issue was patched in silence, even though there is a discreet trace in the source code commits. Okta points to the following PRs: 30792, 30793, and 30794.
