Derive Scales Low-Latency Decentralized Trading Platform with AWS Graviton Amazon EKS and Amazon Aurora
#Derive #scaled #lowlatency #decentralized #trading #platform #AWS #Graviton #Amazon #EKS #Amazon #Aurora
The world of decentralized finance, or DeFi, is rapidly evolving, and at the forefront of this revolution is Derive, a pioneering protocol that creates unique and programmable on-chain derivatives. With the derivatives market poised for significant growth in the next decade, driven by increasing institutional participation, technological innovation, and a shift towards on-chain adoption, tokenization, and decentralized finance, Derive is well-positioned to play a leading role in this new landscape.
As a DeFi protocol, Derive operates a hybrid model, combining the efficiency of centralized order matching with the security and transparency of decentralized settlements and margin on its Derive chain, built with the OP Stack, an open-source Ethereum Layer 2 blockchain. This approach allows Derive to offer a seamless trading experience similar to centralized exchanges while maintaining user control over assets. But what really sets Derive apart is its ability to scale, and that’s where AWS comes in.
To achieve billions of dollars in trading volume and low-latency execution, Derive needed a robust compute and database infrastructure. That’s why they turned to AWS Graviton on Amazon Elastic Kubernetes Service (Amazon EKS) and Amazon Aurora. With AWS, Derive was able to build a highly efficient, high-performance compute running on Graviton nodes with a microservices architecture in Amazon EKS. This provided low latency and scalability, enabling rapid execution of trades.
But how exactly did Derive achieve this level of scalability? Let’s take a closer look at their architecture. The high-level architecture is composed of several key components, including:
- High-performance compute: Derive’s architecture includes a highly efficient, high-performance compute running on Graviton nodes with a microservices architecture in Amazon EKS. This provides low latency and scalability, enabling rapid execution of trades. Microservices are modular and horizontally scalable, allowing the solution to handle high volumes efficiently, with specialized services for core functions:
- Core matching engine: Built on Rust for low-latency, the core matching engine processes orders, matches trades, and manages state transitions.
- Risk management: Dedicated microservices handle standard and portfolio risk calculations, providing compliance and protecting both the platform and its users.
- Emitters: Services such as the Orderbook Emitter and Ticker Emitter broadcast state changes and price updates to external systems and users.
- Additional microservices: The architecture is extensible, supporting the addition of new microservices as needed for future growth.
- Database: Microservices transact data on PostgreSQL on Aurora, benefiting from its scalability and reliability.
During their launch, Derive had more than 50,000 users in a single day, making over 50 million API requests while trading. Amazon Aurora was instrumental in helping them quickly scale their database to service significant volume of read requests.
In addition to their high-performance compute and database, Derive’s hybrid exchange model requires a component that manages the aggregation rollup and settlement of trades, interfacing with its microservices to finalize transactions. For settlements, Derive built a Layer 2 blockchain as a scaling solution for the Ethereum blockchain, based on the open-source, OP Stack. This provides high throughput, low cost, and inherits Ethereum security, balancing speed and security.
User requests are facilitated through both WebSocket and REST API endpoints, providing secure and efficient handling of orders and market data. A Valkey-based queue system orchestrates the sequencing and processing of incoming orders, and a configurable rate limiter enforces fair usage policies and safeguards the exchange against abuse or overload. To deliver real-time market data and order updates with minimal latency, a Valkey-based publish/subscribe mechanism efficiently broadcasts these events to all connected clients.
The scalability of the architecture also includes event streaming, which tracks and captures state changes for communication to downstream microservices. This event-driven approach uses NATS, deployed on Amazon EKS, providing scalability and reliability across the platform. Beyond building a performant Rust-based engine, Derive built several other internal components that required domain-specific technical knowledge—one of them being a custom message queueing system that routes orders with a very tight latency range.
The team at AWS gave Derive amazing support through many in-person technical brainstorms with AWS resident experts. This really improved their execution speed. Shortly after launch, the exchange handled $3.7B in volumes and settled almost 1 million trades on-chain in one month.
Security is also a top priority for Derive. In addition to the inherent security of a decentralized settlement layer, Derive uses AWS security services such as AWS Control Tower for centralized cloud governance, AWS IAM Identity Center for streamlined identity management, AWS Key Management Service (AWS KMS) for secure encryption, and Amazon GuardDuty for intelligent threat detection. These measures enable Derive to align with stringent security requirements and high standards expected by institutional investors.
Derive collaborated with AWS since the inception of its journey, participating in the AWS Activate program for startups. They were able to access AWS credits after securing investment from leading venture capital firms, Coinbase Ventures and Framework Ventures, who are AWS Activate Providers. The team at AWS Startups has been instrumental in Derive’s journey, providing them with the funding, tools, and expertise to innovate in the DeFi space.
By harnessing the power of AWS infrastructure, Derive has not only optimized their costs but also ensured that their platform can scale seamlessly to meet growing demand, while remaining secure. This has enabled them to focus on what matters most: building a high-performing trading platform for their institutional and retail customers. As they continue to pioneer new frontiers in decentralized finance, AWS’s scalable and reliable infrastructure will be fundamental for their success.
To address their unique technical challenges and accelerate development, Derive used specialist experts across subject matter areas, including compute, databases, and Web3. It’s incredibly reassuring to know they can hop on a call and get deep, expert opinions whenever they need them.
Looking ahead, Derive is committed to continuously enhancing its capacity and reducing latency. In addition to ongoing improvements, Derive is excited about several key initiatives in its roadmap. First, a comprehensive replay and audit capability that will empower users to precisely replay orders and trades. This capability is designed to meet the needs of institutional clients who require granular historical data to debug algorithms and maintain fair execution on the exchange. Second, a deeper integration with the Ethereum Virtual Machine (EVM), positioning it as a natural extension of an EVM node.
This blockchain-based approach is expected to deliver several advantages, including reducing discrepancies between exchange balances and on-chain state; simplifying and streamlining the infrastructure for market, account, and trade data; and in the future, empowering users to seamlessly integrate Derive exchange components with other protocols. Derive believes these developments will position their exchange as a leader in blockchain-native trading and pave the way for broader innovation in the industry.
To learn more about decentralized derivatives and how AWS and Derive can support your DeFi initiatives, explore the following resources:
About the authors:
Joshua Kim is the Head of Engineering at Derive, where he leads the development of all off-chain and on-chain derivatives infrastructure. Under his leadership, Derive has processed over 2.5 million trades and more than $15 billion in notional volume, setting a new standard for speed and scalability in decentralized derivatives. Before Derive, Josh spent four years at Apple as lead engineer in the iPhone speaker team and built the Vision Pro anthropometry simulation team, with patents in speaker technology and computational anthropometry.
Dillon Lin is the Head of Growth at Derive, leading distribution across retail and institutional users. He started as an engineer, helped build the product, and now drives go-to-market in an industry with no set playbook. Derive has since executed the largest on-chain options trades in history multiple times. He’s fascinated by the second-order effects of new technologies meeting real users.
In conclusion, Derive’s success in scaling their low-latency, decentralized trading platform using AWS Graviton, Amazon EKS, and Amazon Aurora is a testament to the power of innovative technologies and strategic partnerships. As the DeFi space continues to evolve, it’s clear that Derive is well-positioned to play a leading role in shaping the future of decentralized finance. Whether you’re an institutional investor or a retail trader, Derive’s platform offers a secure, efficient, and scalable solution for navigating the complex world of derivatives. So why not join the revolution and explore the possibilities of decentralized finance with Derive and AWS? The future of trading is here, and it’s decentralized.
