Hazelcast is an open‑source in‑memory computing platform that provides a distributed data grid, a stream processing engine, and a suite of related services for building highly scalable, low‑latency applications. The core product, Hazelcast IMDG (In‑Memory Data Grid), implements a set of distributed data structures (maps, queues, topics, etc.) that are accessible through Java, C++, .NET, Python, and other client APIs. Hazelcast is developed by Hazelcast, Inc., a privately held company founded in 2008, and is released under the Apache License 2.0 for the community edition, with additional enterprise‑grade features available under commercial licenses.
History and development
Hazelcast was created by Tal Kira and David Abraham, two former employees of the Israeli startup Netezza, who sought to simplify the construction of distributed, fault‑tolerant applications. The first public release (0.1) appeared in 2008, and the project quickly gained traction in the Java community as a lightweight alternative to heavyweight middleware such as JBoss Cache and Oracle Coherence. In 2013, Hazelcast introduced Hazelcast Jet, a stream‑processing engine that could run on top of the data grid, and in 2018 the company rebranded its offering as the Hazelcast Platform, integrating IMDG, Jet, and management tools under a unified architecture. Subsequent releases added support for Kubernetes, cloud‑native deployments, and a range of language clients. As of 2024, the platform is used by more than 2 000 organizations worldwide, spanning sectors such as finance, telecommunications, gaming, and e‑commerce.
Architecture and core components
Hazelcast follows a peer‑to‑peer (P2P) architecture in which each node runs the same Hazelcast instance and participates in the cluster’s data distribution and management. The cluster employs a partitioned data model: the logical key space is divided into a configurable number of partitions (default 271), and each partition is owned by a primary member and optionally one or more backup members. This scheme enables linear scalability—adding nodes increases both storage capacity and processing throughput—while preserving data redundancy.
Key architectural elements include:
- Distributed data structures – Maps, Multi‑Maps, Sets, Queues, Lists, and atomic primitives (e.g.,
AtomicLong). These structures are implemented on top of the partitioned storage and expose familiar Java collection interfaces. - CP and AP consistency – Hazelcast implements both the CP (consistent, partition‑tolerant) and AP (available, partition‑tolerant) consistency models defined by the CAP theorem. CP data structures (e.g.,
CPSubsystemobjects likeFencedLock) use the Raft consensus algorithm to guarantee linearizability, while AP structures (e.g.,IMap) favor availability and eventual consistency. - Cluster discovery – Nodes locate each other via multicast, TCP/IP, AWS/ECS/EKS cloud discovery plugins, or Kubernetes DNS services. The discovery mechanism is configurable, allowing deployment in isolated data‑center networks or public clouds.
- Serialization – Hazelcast provides a pluggable serialization framework. By default, it uses Optimized Binary Serialization (OBJS), which stores object fields in a compact binary format. Users can also supply custom serializers, or employ third‑party formats such as Kryo, Protobuf, or JSON.
- Management Center – A web‑based console that monitors cluster health, visualizes partition distribution, and allows runtime configuration changes. The console connects to the cluster via an embedded JMX bridge and can be run as a standalone Docker image.
Features and APIs
Hazelcast exposes a rich set of APIs that cover data storage, computation, and messaging:
| Feature | Description |
|---|---|
Distributed map (IMap) | A key‑value store with optional indexing, query predicates, entry processors, and near‑cache support for read‑through latency reduction. |
| SQL engine | Since version 5.0, Hazelcast includes a ANSI‑SQL‑compliant query layer allowing ad‑hoc queries, joins, and aggregations over distributed maps. |
| Entry Processor | Server‑side code that runs atomically on a map entry, enabling bulk updates without data movement. |
| Event listeners | Asynchronous callbacks for entry, map, and cluster events, facilitating reactive programming patterns. |
| Jet stream processing | A DAG‑based API for defining pipelines that ingest data from maps, external sources (Kafka, JMS), or custom producers, and produce results to sinks or other maps. |
| Distributed executor service | A IExecutorService that schedules Callable or Runnable tasks on specific members or partitions, supporting both fire‑and‑forget and result‑returning semantics. |
| Messaging (topics & queues) | Publish‑subscribe (ITopic) and reliable queue (IQueue) abstractions that leverage the same partitioned infrastructure for low‑latency messaging. |
| Locking and coordination | Distributed locks (ILock), semaphores (ISemaphore), and countdown latches (ICountDownLatch) that operate under the CP subsystem for strong consistency. |
| WAN replication | Configurable replication links that synchronize data between geographically separated clusters, supporting active‑active or active‑passive topologies. |
| Security | TLS encryption for inter‑node and client‑node communication, role‑based access control (RBAC), LDAP integration, and token‑based authentication in the enterprise edition. |
The platform also provides language bindings for Java, C++, .NET, Python, Node.js, and Go, each exposing a subset of the core APIs while preserving the semantics of the underlying distributed structures.
Deployment and ecosystem
Hazelcast can be deployed in a variety of environments:
- On‑premises – A traditional installation on bare metal or virtual machines, managed via systemd scripts or Docker Compose.
- Cloud‑native – Official Helm charts for Kubernetes enable automated scaling, rolling upgrades, and integration with service meshes (e.g., Istio). Hazelcast also offers a managed cloud service (Hazelcast Cloud) that provisions clusters on AWS, Azure, or GCP with pay‑as‑you‑go pricing.
- Hybrid – Clusters can span on‑premises and cloud nodes, leveraging WAN replication and split‑brain protection mechanisms.
- Container orchestration – Hazelcast supports dynamic member discovery through Kubernetes DNS, AWS EC2 tags, and Azure VM scale sets, allowing seamless addition or removal of nodes without manual reconfiguration.
The ecosystem includes auxiliary projects such as Hazelcast Jet (now part of the platform), Hazelcast Go Client, and Hazelcast Operator for Kubernetes, which automates lifecycle management (creation, scaling, health checks). Integration connectors are available for popular data sources and sinks, including Apache Kafka, RabbitMQ, JDBC databases, and Elasticsearch, facilitating the construction of end‑to‑end pipelines.
Use cases and adoption
Hazelcast’s design targets workloads that demand sub‑millisecond data access and elastic scalability. Common use cases include:
- Caching layer – As a distributed cache, Hazelcast replaces traditional single‑node caches (e.g., Ehcache) to eliminate cache‑miss latency in high‑traffic web applications.
- Session replication – In stateless microservice architectures, Hazelcast stores HTTP session data, enabling seamless failover across instances.
- Real‑time analytics – Jet pipelines process streaming events (clickstreams, IoT telemetry) in memory, delivering low‑latency aggregations and alerts.
- Financial services – Low‑latency order books and risk calculations benefit from the deterministic latency and CP consistency guarantees offered by Hazelcast’s distributed locks and atomic structures.
- Gaming – Leaderboards, matchmaking queues, and in‑game state synchronization are implemented using distributed maps and topics with near‑cache acceleration.
Industry reports and case studies document deployments handling billions of entries and tens of thousands of operations per second. Performance benchmarks routinely show latency under 1 ms for read/write operations on a 10‑node cluster with SSD storage, and linear throughput scaling up to 1 M operations per second for map updates.
Licensing, community, and governance
The Hazelcast Community Edition is released under the Apache License 2.0, allowing free use, modification, and redistribution. The source code is hosted on GitHub, where contributors can submit pull requests, file issues, and participate in discussions. The project follows an open governance model: a core committers group maintains the codebase, while a technical steering committee (including representatives from Hazelcast, Inc. and major enterprise users) defines the roadmap.
Commercial offerings—Hazelcast Enterprise and Hazelcast Cloud—add features such as advanced security (TLS mutual authentication, LDAP/AD integration), hot‑restart persistence, management center clustering, and priority support. Licensing for these editions is subscription‑based, with tiered pricing based on cluster size and support level.
The community maintains an extensive documentation portal, a public mailing list, and a Slack workspace for real‑time assistance. Annual conferences (Hazelcast Summit) and webinars provide opportunities for knowledge sharing and feature