Integrating your application with StarkEx

Search…
StarkEx Playground
Before you begin integrating your own StarkEx instance, get started with the StarkEx Playground. The Playground provides hands-on experience, shows you how StarkEx works, and introduces you to the StarkEx interface.
The StarkEx playground is an open StarkEx instance deployed on testnet, so anyone can try out StarkEx. You can do the following:
(Basic) Send a pre-prepared list of transactions to StarkEx and view the resulting on-chain data.
(Advanced) Build an application and send your own transactions through StarkEx.
Integrating your own StarkEx instance
When you begin working with StarkEx, you receive your own instance of StarkEx on testnet, and your own L1 StarkEx contract.
A real StarkEx instance compared with the Playground
A real StarkEx instance
StarkEx Playground
You are the only Operator. You have full control of your StarkEx instance.
There are multiple Operators.
You manage the vaultId allocation as part of your application.
The vaultId value that you define for your transaction can be taken by someone else.
You are the only one sending transactions, so you don't need to call get_first_unused_tx_id.

StarkEx processes transactions sequentially, based on the value of each transaction's id, so the ids in your list of transactions must include all integers in the range of values.

For example, for a list of three transactions, the list of transaction id values cannot be 0,1,3, it must be 0,1,2. It can also be non-sequential, such as 0,2,1. If the list doesn't include an integer, StarkEx stops processing when it reaches the missing id and waits for it to be submitted.
In order to get the next transaction id, you must query the StarkEx gateway by calling get_first_unused_tx_id, because other Playground users need to sync the next tx_id.

Be aware that StarkEx does not process two transactions with the same id and does not process a transaction if the transaction with the previous id has not been processed.
Prerequisites
(Recommended) You have used the StarkEx Playground. For more information, see the StarkEx Playground tutorial.
You have a dedicated instance of StarkEx for your application. For information on getting your own instance of StarkEx, contact us at [email protected]
High level steps
1.For ZK-Rollup and Validium, set up a service to query StarkEx's availability gateway, compute the new state, and approve it.
2.For Validium only, set up a Data Availability Committee (DAC).
3.​Enable alternative transactions to replace invalid transactions in a batch. One invalid transaction can potentially invalidate the entire batch.
4.​Add an event handler to your application to monitor and handle on-chain full withdrawal requests and serve them in a timely manner.
5.​Set up mutual authentication to enable encrypted communication between your application and StarkEx.
Data availability
In order to provide redundancy and trustlessness, StarkEx-powered applications must make their users' data always available.
Data availability ensures that the state you store in your users' vaults is perfectly synchronized with the state that StarkEx stores.
ZK-Rollup provides built-in trustlessness, and data is available on-chain as calldata. With Validium, the Data Availability Committee (DAC) provides a trustless solution to make your users’ data available off-chain.
Batch state approval (ZK-Rollup and Validium)
StarkEx does not submit a new state to the blockchain without your signed approval of the state. So every time that a new batch of transactions is ready, you must query StarkEx's availability gateway, compute the new state, and approve it.
In order to help you integrate this step into your process, StarkWare provides a reference implementation of code that queries and computes the state, known as the Committee Service. You can use this code as is or modify it. Alternatively, you can write your own code. The code repository also includes a hook you can use to add your own validation code to check additional conditions on the state.
You can run this code natively, or in a container, as described in the repository documentation.
About The Data Availability Committee (Validium only)
A DAC is a group of individuals or organizations. The members of the DAC commit to providing StarkEx-powered applications with redundancy and trustlessness in storing data and ensuring that data is always available.
StarkEx does not submit a new on-chain state without the DAC's signed approval.
DAC members install and run an implementation of the Committee Service, which does the following:
validates the data availability of a batch in a StarkEx instance
sends signed approval of the batch to StarkEx.
This service ensures that the state you store in your database, such as the state of your users’ vaults, is perfectly synchronized with the state that StarkEx stores.
Members keep copies of the off-chain data, and return that data to the public domain if StarkEx Operators do not service their users’ withdrawal requests. In such a case, the Application Smart Contract (ASC) stops accepting new state updates. The only action it allows is direct withdrawals of funds by users who are able to provide a Merkle proof for the latest state.
If you do not set up a DAC for an application using Validium, your application cannot be decentralized.
For information on StarkEx’s default Data Availability Committee members and more information on Data Availability, see StarkWare's website.
To be a DAC member, you need to install the Committee service and connect your server.
To set up a new DAC, you need to contact prospective members of your DAC.
Additional resources
​Data availability​
​Availability Gateway​
Setting up the Committee Service
Data availability committee (DAC) members need to install the Committee service and connect your server.
Prerequisites
Docker is installed and enabled
Procedure
For information on installing and executing the Committee Service, see the Committee Service Github repository.
Additional resources
​Data Availability. The Self-Custody Series, Part IV | by StarkWare​
About handling invalid transactions
The StarkEx gateway performs syntactic checks and context-free validation on each transaction it receives, and returns an error code if the transaction is not valid. In general, StarkEx detects errors as early as possible in the pipeline, but some errors are detected later, when StarkEx performs system checks that depend on the current state.
Whenever an invalid transaction occurs in a batch, it prevents StarkEx from processing the batch. Additionally, any subsequent transactions that depend upon the first become invalid as well.
For example, an invalid deposit to a vault might invalidate a subsequent withdrawal that depends upon the funds in that vault.
Other possible causes of invalid transactions are:
Bugs
Reorganization of the blockchain
An order id is not registered on-chain
A fact is not registered on-chain for a conditional transfer
The expiration time of a transaction does not allow enough time for the proof to take place
Alternative transactions
When StarkEx identifies an invalid transaction, StarkEx marks the transaction as invalid and sends a request to the application’s endpoint for a list of alternative transactions.
The alternative transactions within the list are processed in order. These transactions do not have ids.
If StarkEx reaches the end of your list of transactions but it hasn’t received enough total transactions to send to SHARP, StarkEx pauses and restarts processing the batch.
Built-in safety
As a safety measure, in order to maintain synchronization between your application’s state and the StarkEx state, if after restarting batch processing, the id of the transaction that was previously marked as invalid is processed again, StarkEx again automatically identifies that transaction as invalid and sends a REPLACED_BEFORE error, even if the transaction is now valid.
Most cases require manual intervention to continue processing the batch. In some cases, such as in the case of an on-chain reorg, you can prepare an automatic response to the REPLACED_BEFORE error. One possible response is to resend the same list of alternative transactions.
You can set up your system’s architecture to deal with REPLACED_BEFORE requests automatically, or insert any level of manual intervention you want for any reason, such as for a safety measure.
If the list of alternative transactions includes an invalid transaction, StarkEx discards the batch, pauses, and restarts processing the batch. Upon encountering the same transaction id, StarkEx again automatically identifies that transaction as invalid and sends a REPLACED_BEFORE error. In this case, you must investigate the actual cause of the error.
﻿Example: Invalid transaction and an alternative transaction list
The following example shows an error message for an invalid transaction. In response to this error, you need to send an list of one or more alternative transactions.
In this example the balance is known to be valid, but it does not yet appear on-chain.
{
    "reason_code": "INSUFFICIENT_ONCHAIN_BALANCE",
    "tx_id": 7,
    "reason_msg": "Not enough onchain balance to complete deposit. balance_used:10 onchain_balance:0 deposit: DepositRequest(vault_id=1, stark_key=2, token_id=3, amount=10)",
    "tx": {
        "stark_key": "0x2",
        "token_id": "0x3",
        "amount": "10",
        "vault_id": 1,
        "type": "DepositRequest"
    }
}
The following example shows a response with a list of one alternative transaction. In this case, it is the same transaction sent initially, assuming that the on-chain balance has been updated since receiving the error message:
{
    "alt_txs": [
        {
            "stark_key": "0x2",
            "token_id": "0x3",
            "amount": "10",
            "vault_id": 1,
            "type": "DepositRequest"
        }
    ]
}
Handling alternative transactions
When StarkEx identifies an invalid transaction, StarkEx marks the transaction as invalid and sends a request to the application’s endpoint for a list of alternative transactions.
Prerequisites
A server with an http endpoint to accept and respond to requests for your application. This endpoint serves the following purposes:
StarkEx sends the endpoint requests for alternative transactions when a transaction is invalid.
The endpoint responds to these requests with a list of alternative transactions.
A certificate or a barrier token that you provide to StarkEx to enable authenticating the endpoint.
Procedure
1.When StarkEx encounters an invalid transaction, StarkEx sends a request for a list of alternative transactions to your server.
2.Send a list of zero or more alternative transactions. The list can be empty if you simply want to skip the invalid transaction.
3.If you receive a REPLACED_BEFORE error, StarkEx identified the same transaction id. If the list of alternative transactions is valid, you can resend the same list.
Handling a Full Withdrawal request
StarkEx is a self-custodial platform and as such, it has a censorship resistance mechanism for withdrawal requests and perpetual trade requests. Be sure to monitor the on-chain Full Withdrawal requests and serve them in a timely manner.
If the application censors these requests, your user can freeze your application.
During the process of integrating your application with StarkEx, some testing flows check for a mechanism to handle these requests.
Additional resources
​Forced Operations​
​Spot trading - Full Withdrawal​
Setting up certificate-based mutual authentication
StarkEx communicates with your application via https, using SSL, to enable mutual authentication. StarkEx uses self-signed certificates, with StarkEx as the certificate authority (CA). You send StarkEx a certificate signing request (CSR), we sign it, and return two files: user.crt and our server.crt.
Procedure
1. Create a configuration file for the availability gateway, with any name, such as user.conf, with the following content:
[req]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
​
[ dn ]
CN = <public_key>
Where <public_key> is the Ethereum public key of the committee member, in checksum format.
2. Generate the user key for the availability gateway, where <availability_gw_user.key> is the name of the output file with the key:
$ openssl genrsa -out <availability_gw_user.key> 4096
IMPORTANT This is a private key. Do not share it.
3. Generate the certificate request for the availability gateway, where <user.csr> is the name of the output file with the certificate request:
$ openssl req -new -key <availability_gw_user.key> -out <availability_gw_user.csr> -config user.conf
4. Generate the user key for the StarkEx gateway, where <StarkEx_gw_user.key> is the name of the output file with the key:
$ openssl genrsa -out <StarkEx_gw_user.key> 4096
IMPORTANT This is a private key. Do not share it.
5. Generate the certificate request for the StarkEx gateway, where <StarkEx_gw_user.csr> is the name of the output file with the certificate request:
$ openssl req -new -key <StarkEx_gw_user.key> -out <StarkEx_gw_user.csr>
6. Send the CSRs you generated to StarkEx via any communication channel, such as Slack, Telegram, or email.
7. StarkWare prepares and sends you two sets of signed certificates named user.crt and server.crt: One set for the availability gateway and one set for the StarkEx gateway.
8. Install the certificate files in your enviroment, along with the user key.
Mutual authentication is now enabled.
Verifying certificate-based mutual authentication
Enter the following command: curl --cert user.crt --cacert server.crt --key user.key <StarkEx GW URL>/v2/gateway/is_alive
NOTE The StarkEx gateway uses the secure http protocol, so the URL begins with https.
You should see a result similar to the following:
GatewayServiceVersion2 is alive!