Multiple APIs and Protocols – a Unified Data Model
The iguazio Enterprise Data Cloud platform supports simultaneous, consistent and high-performance access through multiple industry standard and Amazon-compatible streaming, as well as table, object, and file APIs. Users can ingest any type of data using a variety of protocols and concurrently read the data using a different method. For example, one application can ingest an event stream while another application reads that data as a table or file.
The data store is decoupled from the stateless API layer and is based on a unique modeling concept where any type of data is stored in a normalized and search-optimized form just once, and transformed on the fly, eliminating the need for copies, complex pipelines, and long ETL processes.
Among the supported APIs, a set of RESTful APIs enables applications to send data directly from any client without installing additional components on the client side or developing API services, significantly simplifying the data ingestion process.
Currently supported APIs by Categories:
Application and Analytics Offload
The iguazio Enterprise Data Cloud platform’s parallel data processing engine indexes and encodes data in real-time, while supporting a variety of advanced procedures to offload common transactions, analytics queries, real-time streaming, time-series and machine-learning logic. By leveraging the mechanisms below, it seamlessly accelerates the performance of tools such as Hadoop and Spark by up to 100x without requiring any changes or endless tuning.
Query offload and smart scans – The API engines accept data and metadata queries, distribute them effectively across all CPUs and leverage the unique data encoding and indexing schemes to eliminate IOs, providing magnitudes faster analytics and eliminating network chatter. iguazio’s Spark DataFrame plug-in uses that feature and is seamless to applications.
Data and Vector processing – built-in procedures improve application concurrency, avoid locks and offload critical tasks to the data layer; they can operate on one or more data items and properties simultaneously and atomically with ACID guarantees.
• Real-time counters, time-series aggregations and queries
• Atomic, idempotent (exactly once) and conditional transactions
• Parallel updates and map reduce operations
• Data manipulation and filtering procedures
Breakthrough Performance at the Lowest Cost
iguazio built its solution from the ground-up to maximize CPU utilization and leverage the benefits of non-volatile memory, 100GbE RDMA, flash and dense storage, achieving extreme performance with consistency at the lowest cost. It shifts the balance from underutilized systems and inefficient code to extremely parallel, real-time and resource optimized implementation requiring fewer servers. To drive maximum capacity at the lowest storage cost, servers incorporate 8-24 high-density NVMe devices and connect to one or more external high-density disk/flash enclosures (JBOD/F).
Fine Grained Security Policies and Data Governance
iguazio data engines classify data transactions at 100Gb/s per server and provide fine-grained policies to control access, service levels, multi-tenancy, retention and data lifecycles.
A set of multi-layered security policies can be defined based on network, identity, tags, metadata or content elements allowing data sharing and governance across users, apps, data and business units. Strict security is achieved without copying data or degrading performance. The system integrates with the organization’s identity management systems and has extensive auditing mechanisms that provide insights on users’ activities at all levels.
Data lifecycle is managed automatically through predefined policies to handle tiering, movement, backups, retention, import and export. It eliminates the need for manually configured and expensive external systems, while leveraging strong metadata management and search capabilities within the platform.
Cloud Agility and Scalability for On-Prem and Hybrid clouds
iguazio delivers a fully integrated and elastically scalable cloud-native service, redefining the category with enterprise-grade capabilities and an application-centric, role-based, self-service operation.
Users and developers can create applications, consume, provision, search and share data through portals or simple RESTful APIs, similar to the public cloud; no IT involvement is needed. The platform is elastic and fault-tolerant. More computation and storage can be added seamlessly as needed, with built in projections and capacity planning to enable proactive operations.
The user and administrator management views provide abstractions and aggregations to simplify data management, and allow for holistic control of the data.
The platform can be deployed on-prem or in a cloud-hosted facility. Applications access data from anywhere using HTTP or other standard APIs, and data can be archived in cloud storage. The unique asynchronous API design mitigates dependency on latency and cuts network bandwidth, allowing fast access to data across distances.
Enterprise Grade with Simplified Operation
iguazio’s Enterprise Data Cloud platform is delivered as an integrated offering with enterprise resiliency and functionality in mind. IT operators do not need to create automation scripts and have tight management throughout the day. Instead, they set up the system through wizards, configure administration policies and register for system notifications.
The platform delivers end-to-end data protection without any single point of failure. It comes with built-in monitoring at all layers, self-diagnostics, logs, alarms and triggers with automated escalation. Telemetry data can be sent to the cloud, ensuring proactive support and 24x7 operations.
Through the multi-layered access and lifecycle policies, administrators can control data access, implement backups, import or export content and guarantee the system complies with business regulations.