https://appcoding.com/tags/algorithms/ Recent content in Algorithms on App Coding Hugo en-us Mon, 05 Oct 2026 00:00:00 +0000 https://appcoding.com/an-embedded-sketch-database-billions-of-events-megabytes-of-storage-error-bars-on-every-answer/ Mon, 05 Oct 2026 00:00:00 +0000 https://appcoding.com/an-embedded-sketch-database-billions-of-events-megabytes-of-storage-error-bars-on-every-answer/ <p>Counting unique visitors exactly means remembering every visitor. Say 6 million distinct IPv4 addresses hit a checkout service today. Packed at 4 bytes each, that set is 24 MB before any hash table overhead, and you need one per service, per region, per day you want to compare. Redis answers the same question with a HyperLogLog in about 12 KB per key, at a standard error of about 0.81%.</p> <p>The thin part is the database around it: a SQLite-shaped file of sketches with time buckets, rollups and a SQL front door (check before you start; this space moves). The version worth building stores nothing but mergeable sketches, grouped by dimension and time bucket, and every aggregate returns its error bound next to the value. You never keep the events. It&rsquo;s the extreme case of <a href="https://apicoding.com/the-best-small-infrastructure-tools-reduce-data-near-the-source-instead-of-storing-more/">reducing data near the source</a>: keep the answer, drop the rows.</p>