20 Primitive Project Ideas: Tiny Go Engines That Do the Essential 10% of Redis, Kafka and Etcd
Given the direction of AltSql, BareProxy, Precomputing and the VPN primitives work, there is still a surprisingly large design space for projects that are smaller than platforms but more fundamental than ordinary applications. The best candidates have a very simple sentence behind them: “X currently requires a framework, service or stack; this does the essential 10% in one tiny binary, library or file.”
Look for primitives that can begin as a credible 2,000–10,000-line Go project, have an immediately understandable demo, and later sit underneath commercial products rather than needing to become a SaaS business immediately. Here are twenty directions that fit that model.
Twenty primitives
1. Statefile (state primitive). A tiny durable state engine sitting somewhere between a JSON file and a database. Applications often need to remember a few thousand pieces of state but end up introducing SQLite, Redis or a cloud database. Statefile could provide atomic get/set/delete/compare-and-set, TTL, snapshots and crash recovery through one file. The interesting positioning isn’t “another KV database”. It’s “persistent application state without a database.” A CLI demo could literally be state set build.status running, kill the process halfway through writes, restart it and show that the state remains consistent.
2. Delta (change primitive). A tiny engine whose fundamental object isn’t a record but a change. Give it two states and it produces a deterministic, compact change representation; feed that representation elsewhere and reconstruct the new state. Git does something vastly more sophisticated, databases have WALs and CDC systems have enormous infrastructure around this concept, but there’s room for a deliberately tiny embeddable change primitive. It could become particularly useful for configuration sync, edge devices and the AltSql device-to-gateway architecture.
3. Once (event primitive). One of the deceptively hard infrastructure problems is making sure something happens once despite crashes, retries and duplicate messages. Imagine a tiny embeddable idempotency engine: once.Do("payment:123", fn). It records execution durably and stops the operation from being performed twice by accident. The first version could be extraordinarily small while solving a real distributed-systems problem. Commercial descendants could provide distributed coordination, API idempotency and job execution.
4. Lease (lease primitive). Applications constantly reinvent distributed locks, ownership leases and leader election using Redis, PostgreSQL, etcd or cloud services. Strip this down hard: acquire a named lease for N seconds, renew it, release it, find its owner. A local version could start with SQLite or its own tiny durable file; a network version could follow. The conceptual API is almost ridiculously small. That’s the appeal.
5. BareQueue (queue primitive). Not another Kafka competitor. Quite the opposite: a durable append/read/ack queue in one executable and maybe one data file. No clusters, partitions, schemas, dashboards or ecosystem. put, take, ack. Start locally, then optionally expose the same semantics over HTTP. Countless applications need more than an in-memory Go channel but far less than RabbitMQ or Kafka.
6. RetryDB (retry primitive). Failed operations are normally buried inside application logic. Make retries a durable primitive instead. An application submits an operation ID, payload, retry policy and endpoint or function; RetryDB guarantees scheduling, exponential backoff, persistence through restart and eventual dead-lettering. It fits the narrow space between a library and a job queue. The attractive commercial evolution is a reliable execution service, not just a retry library.
7. TemporalKV (time primitive). A key-value store where every assignment automatically has history. put("price", 10) followed later by put("price", 12), then get("price") gives 12 while get("price", yesterday) gives 10. It isn’t meant to compete with temporal databases. The idea is to make “what was the value at time T?” almost as cheap conceptually as ordinary KV. Useful for configuration, feature flags, monitoring, audit trails and edge systems.
8. TTL (expiration primitive). An engine whose sole specialty is remembering things until they expire. remember(key, value, 30m), exists(key), remaining(key). Rate limiting, sessions, temporary credentials, dedup windows, caches and abuse controls all keep reimplementing this mechanism. It could be both an embeddable Go package and a tiny server. The narrowness is the differentiation.
9. Count (counter primitive). A durable, very fast counter library and service: increment, decrement, read, reset, time windows. Sounds almost too trivial until you count the systems that need API quotas, page counters, event totals, rate limits and telemetry counts. Add approximate counters and cardinality later, but keep v0.1 almost absurdly simple. “Millions of durable counters without Redis” is a proposition anyone gets.
10. Gate (limit primitive). One tiny engine for deciding whether an operation may proceed. Input: identity, rule, timestamp. Output: allow or deny, plus a reason. Start with rate limits, concurrency limits and quotas. It could grow into a policy primitive later, but resist turning v0.1 into an authorization framework. gate.Allow("customer-42", "api", 100/minute) is almost the entire conceptual product.
11. Pulse (signal primitive). Applications expose health endpoints, heartbeats and watchdogs in dozens of different ways. Pulse reduces this to “I am alive” signals with deadlines. A service registers pulse("worker7", 30s) and another process can ask which expected components have stopped pulsing. A tiny foundation for process monitoring, cron monitoring, IoT supervision and edge infrastructure, without trying to become Prometheus.
12. ConfigLog (configuration primitive). Instead of storing configuration as mutable files, every change becomes an append-only event. The application always sees the current materialized configuration, while rollback, history and auditing come essentially free. Git’s philosophy, reduced to runtime configuration. Most compelling if the whole thing stays one binary and one file.
13. Ephemeral (secret-delivery primitive). Not a password manager and not another Vault. Its only job is delivering a secret exactly once, or for a very short period. Create token, retrieve payload, payload disappears. Useful for CI/CD, bootstrap credentials, temporary links and machine provisioning. Security makes this harder than most ideas here. It also makes a working one more valuable.
14. Distill (stream-reduction primitive). A narrow take on the data-distiller idea. Feed an endless stream into it; instead of storing everything, it continuously maintains a compact representation: first, last, minimum, maximum, changes, anomalies, representative samples and distributions. Storage stays bounded while input is effectively unlimited. The engine’s job isn’t compression. It deliberately decides what information deserves to survive.
15. Provenance (evidence primitive). Give the engine an object and its origin; every transformation creates another immutable provenance node. Later you can ask, “where did this value come from?” Obvious applications in OSINT, AI pipelines, journalism, scientific data and data engineering. A very small first version could simply create content hashes and parent-child transformation records. Eventually it could underpin evidence chains and auditing of AI-generated data.
16. Observe (observation primitive). Feed it observations such as host=A, latency=32. It stores not the whole stream but knowledge about what’s been observed: ranges, frequency, first and last occurrence, changes. It sits between logging, metrics and Distill. Instead of “store my telemetry,” the proposition becomes “remember what the system has learned about its environment.”
17. TinyGraph (relationship primitive). A deliberately minimal persistent graph engine with three operations: connect A to B, disconnect A from B, find connections. Graph databases have become large platforms; many programs just need durable relationships. The OSINT angle is strong, since entities, companies, people, vessels, domains, IP addresses and events naturally become nodes and edges. Don’t implement Cypher. Don’t build a graph database ecosystem. Make relationships themselves the primitive.
18. Remember (cache primitive). A persistent cache that survives restart and needs essentially no administration. Libraries exist, of course, but the proposition here is stronger: one embeddable engine with TTL, size bounds, eviction and disk persistence, and zero external services. Memory between RAM and a database.
19. Freeze (snapshot primitive). Give Freeze a directory, a database file or application state and it creates cheap immutable snapshots addressable by content hash. The real challenge is incremental snapshots and dedup while keeping the API microscopic. Backup products could be built around it later, but Freeze itself stays an infrastructure component.
20. Invariant (assertion primitive). Applications declare things that must remain true: disk_free > 10%, orders_pending < 500, gateway_seen < 60s, database_version == expected. The engine continuously evaluates those assertions and emits state transitions rather than endless metrics. Monitoring reduced to Boolean truth. Instead of asking developers to build dashboards, ask them to declare what “correct” means.
Composition beats platforms
The more interesting category is primitives that connect several of these concepts without becoming platforms. Change plus Once plus State gives you the beginnings of a tiny reliable workflow engine. Distill plus TemporalKV gives you an unusual telemetry database. Provenance plus TinyGraph plus Distill becomes a potentially powerful OSINT evidence engine. AltSql plus Delta gives you native device synchronization. BareProxy plus Gate gives you a tiny policy-aware edge proxy. Precomputing plus Invariant could continuously maintain answers and emit an event only when an important answer changes. The projects stay separate. Composition creates the larger product.
The five worth building first
Pick five to investigate seriously: Once, Distill, Provenance, TemporalKV and Invariant. Once has the clearest immediate developer utility. TemporalKV is straightforward enough to produce a convincing prototype fast. Provenance sits right in the emerging AI and data-trust problem. Invariant has a beautifully simple conceptual model. And Distill has the best chance of becoming something genuinely unusual, not just a smaller version of an established category.
Of those, Distill plus Precomputing is the most intriguing pairing. Instead of databases that mainly store what happened, you move toward engines that continuously decide what’s worth remembering and which answers are worth maintaining.
That’s a family of primitives, not a pile of tiny GitHub experiments.