Page Comparison

...

...

...

...

...

...

...

...

...

...

Overview

Problem: Update architecture for FIP-1 to include the ability to page and cache data on wallets

One of the underlying issues in the FIO workflow is the lack of searching and querying of data based on time. This lack of functionality directly affects how the wallet partners show that data to their users. This proposal will attempt to address this issue with the use of a new index table.

The initial 1.2.0 Table migration is still recommended with this approach. It will significantly reduce the need for future migrations due to its member additions, and it also allows old requests to be bound into the newly proposed index table.

New table structure proposed (fio.request.obt):

struct ledgerItems {
   vector<uint64_t> fio_request_ids;
   vector<uint64_t> recordobt_ids;
}

struct [[eosio::action]] reqledger {
   uint64_t account;
   ledgerItems transactions;
   uint64_t primary_key() const { return account; }
}

...

During a new transaction, we’re able to remove records after reaching a designated limit. ( force use of history node ). If removal route is desired, we need to define limits. 50? 100? 1000? ( not right now ) Recommend to do during this release if history node is the recommended system for retrieving request data. Otherwise a new action call will be required to remove old request data for each account. The above example shows how removed data will still keep time integrity based on new available primary keys.

Proposed changes to chain_plugin:

Instead of iterating through ALL the records for paging, we’re able to utilize the new index table to assist in the offset and limit parameters. This allows us to now show the results from an earliest -> latest outline or vise versa, depending on the implementation.

...

Important to note that data can be retrieved from an API method by either “earliest to latest“ or “latest to earliest”, whichever will make for most effortless development. Pawel?

MAS-1642 -( Modify search logic for get_obt_data, get_pending_fio_requests, get_sent_fio_requests )

The current table structure for FIO requests does not support public key searching. This results in improper requests returned after transferring an FIO address. To achieve this, a new table needs to be created, prior data needs to be migrated, and the old table contents need to be removed.

...

This new contract update will be proposed via msig to the protocol. This will finalize all necessary changes to complete the transfer of FIO domains and addresses while maintaining request integrity. 

New Structure Notes 11/24/20 -

During the development of the above structure, some limitations arose that would hinder the scaling of account with large amounts of requests and obt actions. The following changes were made to better allow for searching payer and payee by status and account name.

• Removal of the Ledger

  • The purpose of this ledger was to quickly search ids but this limited CPU processing time for high volume users when needing to append or modify the vector container due to serialization.

• Additional searchable indexes inside the fiotrxts table.

  • 

    These index searches allow for multiple queriable conditions inside a single, bit shifted operation. This also allows future development to search by status(es).

  • New global data member, STATUS_MULTIPLIER is used for shifting operations.

• New time index search inside fiotrxts table.

  • 

    This will allow time-based state cleanup in the future.

Functional Testing

Design

• Overview of test plan with high-level bulleted test cases

Results

• Link to JS tests and description of which test sections were updated/added

• Paste results of tests

Performance Testing

Design

• Overview of test plan with high-level bulleted test cases

Results

• Summary of performance tests that were run and results

Fork Testing Plan

This is not a forking change since users on current chain code and new chain code will maintain the same state. What changes in the new chain code is how the FIO Request getters work. In the current code, they call into the current tables. In the new chain code, they will call into the new tables. So, wallets that point to an API that has not upgraded will eventually get null data once we fully transfer over to the new tables and delete the old ones. We will maintain both tables for a time, but at some point we will do a contact msig that deletes the old table.

...

So, we do not need a date in the chain code, the break will come when we do the msig.

Rollout/Release Plan

Release 1 - fio.contracts (Bravo-c1)

After this update and the initial call to migrtrx, new Request and OBT data coming in will be entered into both the old and new tables.

Step 1 introduces the new table structure and adds a new “temporary” action call called migrtrx. This action enables the top 21 block producers to start migrating existing data over to the new formatted index tables.

...

[FIP-1.b] Step 1: Add continuous migration of request and obt data (FIO #182)

The complete migration of data has to happen before fio chain is released. An off chain validation should be done to confirm this.

Release 2 - fio chain (Bravo-chain)

Step 2 updates the getters (e.g., get_xyz_fio_requests)

Users pointing to non-upgraded nodes will be retrieving data from the old tables and users pointing to upgraded nodes will be retrieving data from the new tables.

Release 3 - fio.contracts (Bravo-c2)

All fio chain nodes should be upgraded prior to rolling out Contract Step 2.

Step 2 does two things:

• Modifies the Request code to stop updating the old tables. After this release, only the new tables will get updated.

• Modifies the  migrtrx call. It will now remove data from the old tables.

...

• New OBT and Requests will be put into the new tables

• Get requests and OBT getters will point to the old tables so they will get “no request found” or the OBT data will not be found results.

Release 4 - fio.contracts (Bravo-c3)

Step 3 includes:

• The removal of migrtrx temp action

• Removes old table references (fioreqctxts, recordobts, fioreqstss)

...

• New OBT and Requests will be put into the new tables.

• Get requests and OBT getters will get a 500 “Error 3060003: contract table query exception” error.

Release Verification Plan

Casey Gardiner Need this. Add info on your scripts that will verify data, etc.