Storage of latest block number

The Oracle is getting the latest action from FIO history every 5 seconds. But when we restart the server, we read the all latest actions and calling wrap function from the start. Of course, it doesn't mint again but I think wrapping time can be long in this case. To prevent this problem, we need to save the latest block number to database or any external storage.what do you think about my suggest?

Should we track block number in the oracleledger table?

• Suggest that Oracles store a log file of all the transactions locally. If they then go down they can grab a time stamp or transaction ID from the log file to know which transactions have not yet been processed.

• If an oracle server crashes and the logs are lost, then the oracle would have to re-process all transactions (and rebuild the log file). This would be a one time process.

• Decision: We will start with using log files, but will ask BPs their opinion on this solution.

https://fioprotocol.atlassian.net/browse/BD-2563

Admin front-end UI

Is there a need for a front-end UI to review different transactions, or can we just rely on table lookups, etc.?

• An admin front-end will be useful for testing and we will likely want one to help troubleshoot failed wrap/unwrap transactions. But, it is not a MVP requirement.

https://fioprotocol.atlassian.net/browse/BD-2564

Process a single transaction at a time

Given the complexity of validating wrap and unwrap transactions all the way through to finality, both Todd and Alex have suggested that we limit oracles to only process a single transaction at a time

• Given that we are relying on manual monitoring of wrap/unwrap transactions to detect failures, limiting the oracle to a single transaction no longer makes sense. In other words, the oracle is now only doing the simple action of calling wrap or unwraptokens and is not doing any failover. So, a “single transaction” is already very small and contained and it would be difficult to NOT process only a single transaction at a time.

How often should get_actions be polled?

Need to determine how often to call get_actions on the FIO chain.

• Decision: Because usage of wrapping will be very light and there is a need to wait for finality of wrap/unwrap actions, polling once every 60 seconds is adequate.

  • TBD: Can we make this value configurable?

• https://fioprotocol.atlassian.net/browse/BD-2565

Transaction Retry

Because we are putting limited validation logic in the oracle, it may be necessary for Oracles to take action on failed transactions. It would be helpful to have a “transaction retry” function that can be called that retries specific wrap/unwrap transactions.

https://fioprotocol.atlassian.net/browse/BD-2561

Log events and exceptions

All events and errors should be logged. Because there will be limited validation, these logs will be the primary way for oracle node operators to troubleshoot issues.

https://fioprotocol.atlassian.net/browse/BD-2562

V1 History

The initial implementation calls get_actions against V1 history. Should we look at supporting Hyperion or other history solutions?

Discussed the V1 history implementation with the Oracle BPs and because it is a relatively simple call, it is not worth designing additional support for different history solutions. Once we deploy they will see how well it fits into their environments.

Setting gasPrice and gasLimit

What is the best way to enable Oracle BPs to set the Eth gas prices and limit?

Pawel: Recommends manually setting in an environment variable (or something similar).

Oracle Initialization

• Set the initial value of actionIndex = last_irreversible_block number (from FIO Chain) when server starts.

• What is the best way to store the actionIndex to persist when an Oracle restarts?

  • See “Storage of latest block number” above

Alice (via dApp) calls wraptokens inside the fio.oracle contract on FIO chain

Example: Calling wraptokens using Cryptonym:

fio.oracle contract actions

fio.oracle contract actions:

• Parameter Validation ( ensuring amount, token codes, pubkey and fees are all properly set )

• Search oracle registration table (contains all registered oracles) and tally up the total number of registered oracles

• Collect Oracle fees, sort, and find the median.

• Send fee to all oracles.

• Emplace wrapping details inside the oracleldgrs table.

  

• Send the wrapped amount from Alice to fio.oracle contract.

• Collect FIO/BP fees

• Increase Alices RAM by 512.

• Send successful response to Alice

Oracle monitors get_actions API on V1 History node

Every 5 seconds Oracle polls the get_actions API on History node Plugin to detect activity on the wrapping account (fio.oracle)

curl -s -XPOST http://44.242.158.42:8080/v1/history/get_actions -d '{
  "account_name": "fio.oracle",
  "pos": -1
}'

1. Retrieve all actions

2. For each action, if block_num > actionIndex then validate and process the action (call ERC-20 wrap)

3. Set actionIndex = block_num of most recent wraptokens action

const actionIdx = config.oracleCache.get("actionIndex");
          const dataLen = Object.keys(data.data.actions).length;
          var array = Array();
          for (var i = 0; i<dataLen;i++){
            if (data.data.actions[i].block_num > actionIdx) {
                array.push(data.data.actions[i]);
            }
            config.oracleCache.set("actionIndex", data.data.actions[dataLen-1].block_num)
          }
          return array;

• Since there eventually may be very many transactions, it may make sense to walk backward through the table using "pos" and "offset". Maybe you grab the most recent 5 actions and see if any of them are new. If ALL of them are new, then you need to grab the next 5 actions, etc.

  • https://fioprotocol.atlassian.net/browse/BD-2566

wrap transaction finality monitoring on FIO Chain

• Should the oracle monitor FIO chain for finality by ensuring block number is after the last_irreversible_block?

  • Decision: Yes, we need to wait for finality. But, it looks like the above code already does that since it is comparing the action block_num > actionIdx (and actionIdx is set to last_irreversible_block on startup)

Oracle validates the FIO chain wraptokens transaction.

• See “Exception handling” below

• Check the actions to confirm it is a wrapping action:

if (wrapData[i].action_trace.act.data.memo == "Token Wrapping")

Responding to invalid wraptokens transaction

• If exceptions are found, what actions does the Oracle take unwind wraptokens action?

  • Decision: Exceptions and other events should be logged by the oracle, but the oracle should NOT execute any retry or other recovery actions.

Oracle executes wrap on fio.erc20 contract on Ethereum chain

• wFIO recipient eth address, wFIO amount to mint (must match what was wrapped on FIO chain exactly), and the obtid of the FIO transaction are provided as parameters to wrap action

wrap(ethaddress, FIO (SUF) amount, obtid);ex. wrap(“0x00000000000”, 10000000, “0x123456789”);

fio.erc20 contract actions for pre-consensus calls to wrap function

fio.erc20 contract actions:

• See: https://fioprotocol.atlassian.net/wiki/spaces/FD/pages/91062320/fio.erc20+-+wFIO+Contract+Specification#Exception-handling

• When the initial oracles call wrap:

  • Contract sends an event.

  • The oracle receives the Event and the Ethereum Transaction ID (if successful)

  • The oracle alerts the user about wrapping result

  • Example wrap transaction: https://ropsten.etherscan.io/tx/0x3a90be65ad89fc5572890eb24d7790bed7d5ebf1797f2755278afbab3e9c6830

• There will be three different transaction IDs by the erc20 contract (two wrap transactions and one wrap/mint transaction) generated from three different Oracles. How does this information make it back to Alice who called wraptokens on the FIO chain? Should the oracle surface an Event? If so, what is listening to that event? Do we assume that the user is using an app that is able to listen to the erc20 events? All of this kind of assumes there is an App that the user is using to wrap tokens… Pawel Mastalerz

  • Same thing for unwrap

  • Decision: Log all events and errors. No further action required. The expectation is that Alice will NOT be monitoring the status of the various Ethereum transaction IDs. Alice will only be monitoring her account to see if WFIO appears. If not, she will raise a support issue.

fio.erc20 contract actions for final consensus call to wrap function

Example of three oracles calling wrap:

Using these oracle, we call the wrap function on FIO token contract using web3.js like below.
instance.wrap[accounts[5], 10000, "99e20de9bb9f178f3ff1328c089925daff1d5dcb1da7dceaad7fc5126a08eaf5". {from: accounts[1]}); 
instance.wrap[accounts[5], 10000, "99e20de9bb9f178f3ff1328c089925daff1d5dcb1da7dceaad7fc5126a08eaf5". {from: accounts[2]}); 
instance.wrap[accounts[5], 10000, "99e20de9bb9f178f3ff1328c089925daff1d5dcb1da7dceaad7fc5126a08eaf5". {from: accounts[3]});

• If an Oracle does not have enough ETH to cover the transaction, how do we notify the oracle operator that there is a problem?

  • Decision: Log all events and errors. No further action required.

• When the last oracle calls wrap, the transaction is executed

  • The mint function inside wrap is automatically executed

  • The Ethereum address provided receives wrapped FIO assets

  • An Event is emitted by the ERC-20 contract: wrapped(address ethaddress, uint256 amount, uint256 obtid);

  • The oracle receives the Event and the Ethereum Transaction ID (if successful) and logs the event

  • This final transaction will include a mint action.

  • Example wrap transaction: https://ropsten.etherscan.io/tx/0x3a90be65ad89fc5572890eb24d7790bed7d5ebf1797f2755278afbab3e9c6830

ERC-20 wrap validation

TBD: Adam to document how the contract handles invalid failed transactions. Put link to content here. Adam Androulidakis

Responding to invalid ERC-20 wrap transaction

• Any kind of recovery? Just send a failure message to the user and note that their FIO is stuck in the FIO wrapping account?

  • Decision: Log all events and errors. No further action required.

wrap transaction finality monitoring on Ethereum

• Alice receives the Ethereum Transaction ID and is responsible for monitoring the status.

  • Unless Alice is using some kind of custom application, she only has access to her Ethereum address and has to sit there and wait to see if WFIO appears. Is this the expectation?

    • Decision: Yes, this is the process. Alice will monitor her account and file a support ticket if she does not get her WFIO.

  • If Alice sees that it has failed, what does she do?

    • Decision: Alice files a support ticket and the problem is manually resolved.

• Does the oracle do any kind of monitoring or validation of the overall transaction (beyond responding to the ERC-20 Event)

  • Decision: Based on 5/13 discussion with Luke/Pawel/Dev team, no additional monitoring of the transaction on the Ethereum chain is required.

  • Decision: Based on 5/13 discussion with Luke/Pawel/Dev team, only limited monitoring of the transaction on the Ethereum chain is required.

  • All of the following do NOT need to be monitored or tracked by the Oracle:

    • A transaction gets stuck in a pre-consensus state.

    • A transaction disappears (e.g., ending up on a fork, etc.)

      • It is common to have transactions go into a mempool, and then transaction ends up in an uncle (orphan) block. If the transaction in the uncle block has not been validated elsewhere, then it should be returned to the mempool. But, there are situations where it can disappear from the mempool.

    • Waiting to make sure the block has reached finality.

Invalid chain

Chain passed to wraptokens action is not Ethereum (Note: this restriction is not enforced in the FIO Contract to allow for wrapping chain expansion without deployment of code)

Oracle logs error and does no further processing of the transaction.

Invalid Ethereum address

Public address passed to wraptokens action is not a valid Ethereum address (Note: this restriction is not enforced in the FIO Contract to allow for wrapping chain expansion without deployment of code)

Oracle logs error and does no further processing of the transaction.

Oracle Initialization

• lastBlockNumber is initialized with ETH latest blocknumber when the server is started

• What is the best way to store the lastBlockNumber to persist when an Oracle restarts?

  • See “Storage of latest block number” above

Alice (dApp) executes unwrap on Ethereum chain

unwrap(fio address, amount);
ex. unwrap(hard@edge, 100000000);

ERC-20 unwrap validation

TBD: Adam to document how the contract handles invalid failed transactions. Put link to content here. Adam Androulidakis

fio.erc20 contract actions for valid unwrap transaction

fio.erc20 contract actions

• Alice provides amount to unwrap and the FIO Address for the oracle to send the FIO to as parameters to unwrap action

• Alice is paying gas fee for the unwrap

• Transaction executed:

  • wFIO amount is burned

  • unwrap event emitted:
    unwrapped(string fioaddress, uint256 amount);

Oracle monitors unwrap event for transfers

• Oracle subscribes to fio.erc20 events and reacts to the unwrap event

fioContract.getPastEvents('unwrapped',{
            // filter: {id: 1},  
            fromBlock: lastBlockNumber,
            toBlock: 'latest'
        }

• Example unwrap transaction: https://ropsten.etherscan.io/tx/0x2bfa8b0219153ab3e6c7c67d6c1c81bfd6a556ea9dbde6c7e00212f548247ae7

Oracle validates unwrap transaction

• See “Exception handling” below

If exceptions are found, Oracle takes action to unwind transaction

• See Exception handling below

unwrap transaction finality monitoring on Ethereum chain

• Need process for assessing when it is okay to call unwraptokens. 6 blocks? 12 blocks?

  • Decision: for now use 12 blocks. Can adjust later.

  • https://fioprotocol.atlassian.net/browse/BD-2567

Oracle executes upwraptokens on fio.oracle contract on FIO chain

• Call the FIO unwrap action with ETH transaction ID and amount.

• On FIO side, we call the unwrap action using push_transcation function using fio.js

  const unwrapTokens = async (obt_id, fioAmount) => {
      let contract = 'fio.oracle',
      action = 'unwraptokens',
      oraclePrivateKey = process.env.PRIVATE_KEY,
      oraclePublicKey = process.env.PUBLIC_KEY,
      oracleAccount = 'qbxn5zhw2ypw',
      amount = fioAmount,
      obtId = obt_id,
      fioAddress = 'bp1@dapixdev';
      const info = await (await fetch(httpEndpoint + 'v1/chain/get_info')).json();
      const blockInfo = await (await fetch(httpEndpoint + 'v1/chain/get_block', { body: `{"block_num_or_id": ${info.last_irreversible_block_num}}`, method: 'POST' })).json()
      const chainId = info.chain_id;
      const currentDate = new Date();
      const timePlusTen = currentDate.getTime() + 10000;
      const timeInISOString = (new Date(timePlusTen)).toISOString();
      const expiration = timeInISOString.substr(0, timeInISOString.length - 1);
  
      const transaction = {
          expiration,
          ref_block_num: blockInfo.block_num & 0xffff,
          ref_block_prefix: blockInfo.ref_block_prefix,
          actions: [{
              account: contract,
              name: action,
              authorization: [{
                  actor: oracleAccount,
                  permission: 'active',
              }],
              data: {
                  fio_address: fioAddress,
                  amount: amount,
                  obt_id: obtId,
                  actor: oracleAccount
              },
          }]
      };
      var abiMap = new Map()
      var tokenRawAbi = await (await fetch(httpEndpoint + 'v1/chain/get_raw_abi', { body: `{"account_name": "fio.oracle"}`, method: 'POST' })).json()
      abiMap.set('fio.oracle', tokenRawAbi)
  
      var privateKeys = [oraclePrivateKey];
  
      const tx = await Fio.prepareTransaction({
          transaction,
          chainId,
          privateKeys,
          abiMap,
          textDecoder: new TextDecoder(),
          textEncoder: new TextEncoder()
      });
  
      const pushResult = await fetch(httpEndpoint + 'v1/chain/push_transaction', {
          body: JSON.stringify(tx),
          method: 'POST',
      });
  
      const json = await pushResult.json()
  
      if (json.type) {
      console.log('Error: ', json);
      } else if (json.error) {
      console.log('Error: ', json)
      } else {
      console.log('Result: ', json)
      }
  }
  

fio.oracle contract actions for calls to upwraptokens function

fio.oracle contract Actions:

• Parameter Validation ( min/max amount, fio address check )

• Verify the actor is a registered oracle

• Find the fio.address inside the fionames table

• Search for previous votes of the same obt_id

  • If found

    • Search and verify actor has not voted prior

    • copy vector and push account name to list of voted oracles to the vector of votes

    • modify voters table with new vector

  • If not found

    • add actor to new vector

    • emplace new record with voters information

      

• Compare number of votes with number of registered oracles

  • if number of votes equal the number of registered oracles, transfer amount from fio.oracle contract to the fio.address provided.

• Send success/fail response to the oracle

Responding to invalid upwraptokens transaction

• Any kind of recovery? Just send a failure message to the user and note that their WFIO is burned but the transfer of FIO failed?

  • Decision: Log all events and errors. No further action required.

• Retry?

  • Decision: No

• What if 2 upwraptokens transaction succeed, but 1 oracle fails?

  • Decision: We will likely want an admin UI to monitor the status of transactions, but this should NOT be built into the Oracle.

Oracle validates unwraptokens transaction

• The oracle does not need to validate unwraptokens transactions. Simply log any responses and errors to the unwraptokens call.

Ongoing monitoring of unwraptokens transactions

• Should Oracles monitor the status unwraptokens transactions to ensure none are stuck in the approval queue?

  • Decision: We will likely want an admin UI to monitor the status of transactions, but this should NOT be built into the Oracle.

Invalid FIO Address

FIO Address passed in with ERC-20 is not valid or does not exist

Oracle logs error and does no further processing of the transaction.