KOR ​​​​limits period of downtime with Kannika Armory as a fast backup and restore solution for Kafka

KOR is a technology company that provides regulatory reporting solutions for the financial services industry. Handling immense amounts of data, they rely heavily on Kafka for data streaming. In the search for a high-performance backup and restore solution for fast recovery, KOR implemented Kannika Armory.

‍

In 2008, the world witnessed the most devastating financial crisis in nearly a century. It led the G7 countries to introduce a system to monitor market activity. With limited visibility at the time, regulators mandated trade reporting. Companies were required to report every trade to ensure transparency. In this context, KOR was established some years ago.

‍

“We obtained licenses​ to​ ​operate​ trade ​repositories ​– massive databases where companies report their trades, allowing regulators to extract insights from the data”, says Andreas Evers, Chief Technology Officer at KOR. “Additionally, we’ve built services that help customers report in a safer, more efficient way through features like transformations, controls, reconciliations, and data enhancement with open data.”

‍

Downtime could cost millions

As an intensive Kafka user, KOR depends on fast backup and restore capabilities. “If we experience an outage, we must quickly restore the data to minimize downtime”, Andreas explains. “Any interruption would prevent our customers from meeting regulatory requirements, leading to potential fines in the millions of dollars. Our reputation is on the line as well, as our customers place trust in us. Essentially, we’re a vital link in the regulatory reporting pipeline.”

‍

Before adopting Kannika Armory, KOR used an internal backup solution that could not meet the speed requirements in the event of a potential disaster. “We handle enormous volumes of data – currently in the terabytes, but we expect to manage petabytes in the near future. While backing up these clusters should be manageable, restoring them is more challenging. It would take us too long.  Speed was the key feature we sought in a solution,” Andreas adds.

‍

While collaborating with Cymo for ​other services​, KOR discovered Kannika Armory, a new product that was under development. “There is nothing comparable on the market. Kannika Armory allows us to restore large clusters quickly, exactly what we needed.”

‍

Testing and fine tuning Kannika Armory

Performance was the main reason why KOR chose Kannika Armory, but other features are convenient as well. For instance, Kannika Armory provides schema mapping, ensuring the correct schema is used when transferring between clusters. Compatibility with AWS S3 was also essential – Kannika Armory offers straightforward offloading of event data to a multitude of storage systems.

‍

As an early adopter of Kannika Armory, KOR’s feedback was instrumental in refining the tool. “We had regular discussions and could easily communicate our needs”, says Adil Houmadi, a Senior Backend Data Engineer who supported the implementation of Kannika Armory at KOR. “The product we use today is completely different from the one we initially tested. We worked closely with Kannika’s development team to ensure it met our requirements. Some of the features we requested have already been rolled out, while others are in development.”

‍

For example, KOR assisted in validating the user interface of Kannika Armory. They also requested an update to enable schema registry backups, a feature currently in construction.

‍

A future-proof solution

With Kannika Armory, KOR ensures continuity for its clients and is well-prepared for future challenges, such as evolving compliance requirements. For instance, data older than two years will rarely be requested by customers. But if they store this data as a backup, they can easily retrieve it when needed.

‍

Furthermore, as KOR scales to handle petabytes of data, Kannika Armory’s compression capabilities (reducing data storage to 15-20%) help keep backup storage needs manageable.

‍

“It’s great to work with a team that values our feedback”, Andreas concludes. “The people at Kannika are capable and have the right experience with Kafka. And since we are both startups, we also share an entrepreneurial spirit that resonates.”

‍

Read more about Kannika Armory here Kafka Backup and Recovery Solution | Secure Event Hub Data Protection

‍

Don’t hesitate to contact us for a demo of request your free trial.  

‍