Technology stack expansion: Overview of the principles and potential application scenarios of zkTLS

Author : @Web3 _Mario

Abstract : I have been looking for new project directions recently. When I was doing product design, I encountered a technology stack that I had never been exposed to before. So I did some research and sorted out my learning experience and shared it with you. In general, zkTLS is a new technology that combines zero-knowledge proof (ZKP) and TLS (Transport Layer Security Protocol). It is mainly used in on-chain virtual machine environments in the Web3 track, and can be used without trusting third parties. Understand the circumstances, verify the authenticity of the off-chain HTTPS data it provides. The authenticity here includes three aspects. The data source does come from a certain HTTPS resource, the returned data has not been tampered with, and the effectiveness of the data can be guaranteed. Through this cryptographic implementation mechanism, on-chain smart contracts can obtain trusted access to off-chain Web2 HTTPS resources, breaking the data silos.

What is the TLS protocol

In order to have a deeper understanding of the value of zkTLS technology, it is necessary to give a brief overview of the TLS protocol. First of all, TLS (Transport Layer Security Protocol) is used to provide encryption, authentication and data integrity in network communications, ensuring the secure transmission of data between clients (such as browsers) and servers (such as websites). For non-network development friends, you may find that when visiting a website, some domain names are prefixed with https, while others are prefixed with http. When accessing the latter, mainstream browsers will prompt that it is not safe. The former is prone to the prompts of "your link is not a private link" or HTTPS certificate errors. The reason for this prompt is the availability of the TLS protocol.

Specifically, the so-called HTTPS protocol is to use the TLS protocol to ensure the privacy and integrity of information transmission based on the HTTP protocol, and make the authenticity of the server-side verifiable. We know that the HTTP protocol is a network protocol for plain text transmission, and this protocol cannot verify the authenticity of the server side, which creates several security issues:

1. The information you and the server may be monitored by third parties, causing privacy leakage;

2. You cannot verify the authenticity of the server side, that is, whether your request is hijacked by other malicious nodes and returns malicious information;

3. You cannot verify the integrity of the returned information, that is, whether it is possible to lose data due to network reasons;

And the TLS protocol is designed to solve these problems. Let me explain here. Some friends may know about the SSL protocol. In fact, the TLS protocol was developed based on SSL version 3.1. It was just a different name due to some commercial related issues, but it is actually the same lineage. So sometimes in some contexts, the two words can be interchangeable.

The main idea of ​​the TLS protocol to solve the above problems is:

1. Encrypted communication: Use symmetric encryption (AES, ChaCha20) to protect data and prevent eavesdropping.

2. Identity authentication: Verify the identity of the server through a digital certificate issued by a third party to a designated agency (such as the X.509 certificate) to prevent man-in-the-middle attacks (MITM).

3. Data integrity: Use HMAC (hash message authentication code) or AEAD (authentication encryption) to ensure that the data has not been tampered with.

Let's briefly explain the technical details of the HTTPS protocol based on the TLS protocol in the data interaction process. The whole process is divided into two stages. The first is the handshake stage, that is, the client negotiates security parameters with the server and establishes an encrypted session . The second is the data transmission stage, that is, encrypted communication using a session key. The specific process is divided into four steps:

1. The client sends ClientHello:

The client (such as a browser) sends a ClientHello message to the server, including:

• Supported TLS versions (such as TLS 1.3)
• Supported encryption algorithms (Cipher Suites, such as AES-GCM, ChaCha20)
• Client Random (for key generation)
• Key Sharing Parameters (such as ECDHE Public Key)
• SNI (Server Name Indicator) (optional, used to support multi-domain HTTPS)

The purpose is to let the server know the encryption capabilities of the client and prepare security parameters.

2. The server sends ServerHello:

The server responds to ServerHello messages, including:

• Selected encryption algorithm
• Server Random
• Server's certificate (X.509 certificate)
• Server's key sharing parameters (such as ECDHE public key)
• Finished message (used to confirm that handshake is completed)

The purpose is to let the client know the server's identity and confirm security parameters.

3. Client verification server:

The client performs the following:

• Verify server certificate: Make sure the certificate is issued by a trusted CA (Certificate Authority), while verifying whether the certificate has expired or has been revoked;
• Compute the shared key: Use the ECDHE public key of yourself and the server to calculate the Session Key, which is used for symmetric encryption of subsequent communications (such as AES-GCM).
• Send Finished message: Prove handshake data integrity and prevent man-in-the-middle attacks (MITM).

The purpose is to ensure that the server is trustworthy and generate a session key.

4. Start encrypting communication:

Clients and servers now use negotiated session keys for encrypted communication.

• Encrypt data using symmetric encryption (such as AES-GCM, ChaCha20) to improve speed and security.
• Data integrity protection: Use AEAD (such as AES-GCM) to prevent tampering.

Therefore, after these four steps, the HTTP protocol problem can be effectively solved. However, this basic technology widely used in Web2 networks has caused trouble for Web3 application development, especially when on-chain smart contracts want to access certain off-chain data, due to data availability issues, on-chain virtual machines will not Open the calling capacity of external data to ensure the traceability of all data, thereby ensuring the security of the consensus mechanism.

However, after a series of iterations, developers found that DApp still has a demand for off-chain data, so a series of oracle Oracle projects appeared, such as Chainlink and Python. They break this phenomenon of data islands by acting as a relay bridge between on-chain data and off-chain data. At the same time, in order to ensure the availability of relay data, these Oracles are generally implemented through the PoS consensus mechanism, that is, the cost of doing evil by relay nodes is higher than the benefits, so that they will not provide error information to the chain in terms of economic benefits. For example, if we want to access the weighted price of BTC in a smart contract on centralized exchanges such as Binance and Coinbase, we need to use these Oracle to add up the data off-chain access and transfer it to the on-chain smart contract and store it before we can use it .

What problems does zkTLS solve

However, people have found that this Oracle-based data acquisition solution has two problems:

1. The cost is too high: We know that in order to ensure that the data transmitted to the chain of Oracle is real data and has not been tampered with, it needs to be guaranteed by the PoS consensus mechanism. However, the security of the PoS consensus mechanism is based on the amount of pledged funds. This brings costs to maintenance. And usually, there is a large amount of data interaction redundancy in the PoS consensus mechanism, because when the data set needs to be repeatedly transmitted, calculated, and summarized in the network, it can pass consensus, which also increases the cost of data usage. So usually, in order to acquire customers, Oracle projects will only maintain some of the most mainstream data for free, such as the prices of mainstream assets such as BTC. For exclusive needs, you need to pay for it. This hinders application innovation, especially some long-tail and customized needs.

2. Too low efficiency: Usually, the consensus of the PoS mechanism takes a certain amount of time, which causes the lag of on-chain data, which is not conducive to some high-frequency access usage scenarios, because the data obtained on-chain is There is a large delay in real off-chain data.

In order to solve the above problems, zkTLS technology came into being. Its main idea is to introduce the ZKP zero-knowledge proof algorithm, so that the on-chain smart contracts can directly verify the data provided by a certain node, and it is indeed accessing a certain node. The data returned after HTTPS resources are not tampered with, which can avoid the high usage costs caused by traditional Oracle due to consensus algorithms.

Some friends may ask why not directly provide the ability to inline Web2 API calls in the on-chain VM environment. The answer is no, because the reason why an on-chain environment needs to maintain a closed data is to ensure the traceability of all data, that is, in the consensus process, all nodes have the accuracy of a certain data or a certain execution result. There is a unified evaluation logic, or an objective verification logic. This ensures that in an environment of complete trust, most well-behaved nodes can rely on their own redundant data to determine the authenticity of the direct result. However, due to Web2 data, it is difficult for you to build this unified evaluation logic, because due to some network delay reasons, the results obtained by different nodes accessing Web2 HTTPS resources are different, which adds difficulties to consensus. It is aimed at some high-frequency data fields. In addition, another key issue is that the security of the TLS protocol that the HTTPS protocol relies on, depends on the random number generated by the client (for key generation) and key sharing parameters, and implement it with the server side. Negotiation for encryption keys, but we know that the on-chain environment is open and transparent. If the smart contract maintains random numbers and key sharing parameters, the key data will be exposed, thus impairing data privacy.

Then zkTLS adopts another method, and its concept is to replace the high cost of bringing data availability through cryptographic protection. Similar to the optimization of OP-Rollup in L2. Specifically, by introducing ZKP zero-knowledge proof, and computing the resources obtained by requesting a certain HTTPS by off-chain relay node, relevant CA certificate verification information, timing proof, and data integrity proof based on HMAC or AEAD to generate proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof of proof Maintain necessary verification information and verification algorithms on the chain, so that smart contracts can verify the authenticity, effectiveness, and reliability of data sources without exposing key information. The specific algorithm details will not be discussed here. Interested friends can conduct in-depth research on them.

The biggest advantage of this technical solution is that it reduces the cost of achieving availability of Web2 HTTPS resources. This has stimulated many new demands, especially in reducing the on-chain price acquisition of long-tail assets, using authoritative websites in the Web2 world to do on-chain KYC, thereby optimizing DID and Web3 Game's technical architecture design. Of course, we can find that the impact of zkTLS on existing Web3 companies also exists, especially for the current mainstream oracle projects. Therefore, in order to cope with this impact, such as Chainlink, Python and other industry giants actively follow up on research in related directions, trying to still dominate the process of technology iteration, and at the same time it will also give birth to new business models, such as the original time-based charge Convert to charge by usage, Compute as a service, etc. Of course, the difficulty here is the same as most ZK projects, but it still lies in how to reduce the calculation cost and make it commercially valuable.

To sum up, when designing products, friends can also pay attention to the development trends of zkTLS and integrate this technology stack in appropriate aspects. Perhaps they can find some new directions in business innovation and technical architecture.