FIO Request/OBT Table Migration

Owned by Eric Butz
Last updated: Jun 02, 2021
4 min read

(Modified to include the “nuke” option on Testnet needed for the content field fix)

The following summarizes the table migration tasks detailed in: https://fioprotocol.atlassian.net/wiki/spaces/FD/pages/19529766

Overview

There are four phases to the migration:

  1. fio.contracts release that creates the new FIO Request table, adds an action that migrates data, and adds a temporary “ledger” table for tracking migration status.

  2. fio chain release that updates the getter code to point to the new FIO Request table

  3. fio.contracts release that modifies Request code to only update the new request table, changes the functionality of migration action to now remove data from the old tables.

  4. fio.contracts release that removes the temporary migration action and removes references to the old tables

Summary of the releases:

Release

Updates

New FIO Requests and Record OBTs…

Getters (get_sent_fio_requests) for nodes that have upgraded:

Getters for nodes that have NOT upgraded:

Release

Updates

New FIO Requests and Record OBTs…

Getters (get_sent_fio_requests) for nodes that have upgraded:

Getters for nodes that have NOT upgraded:

Step 1.

fio.contracts v2.3.0

New tables:

  • fiotrxts (new table to hold all requests and OBT records)

  • migrledgers (temporary table to track migration)

New actions:

  • migrtrx (temporary action for migration. In Step 1 it migrates records)

Get added to the old tables (fioreqctxts, recordobts, fioreqstss) and the new table (fiotrxts)

Access data from the old tables (fioreqctxts, recordobts, fioreqstss)

All nodes access data from the old tables.

Step 1.b

fio.contracts v2.3.1

Update action:

  • migrtrx - This call now nukes any data present in the new fiotrxts table.

 

 

 

Step 1.c

fio.contracts v2.3.2

New table:

  • fiotrxtss - (new renamed table to hold all Request and OBT records)

Update action:

  • migrtrx - This call is set back to the original migration functionality and now updates the renamed fiotrxtss table

 

 

 

Step 2. fio chain

fio bahamas

Update to getters to point to new table:

  • /get_obt_data

  • /get_pending_fio_requests

  • /get_sent_fio_requests

  • /get_cancelled_fio_requests

Get added to the old tables (fioreqctxts, recordobts, fioreqstss) and the new table (fiotrxts)

Access data from the new table (fiotrxts)

No impact for nodes that have not upgraded since new records are still being added into both new and old tables.

Step 3. fio.contracts #2

Modified actions:

  • migrtrx (In step 2 this action now removes data from the old Request and OBT tables)

Only get added to the new table (fiotrxts)

Access data from the new table (fiotrxts)

  • New requests will be entered into the new table.

  • Getter actions will return “no request found” since there will no longer be data in the old tables.

Step 4. fio.contracts #3

Removed actions:

  • migrtrx

  • Remove tables from ABI

  • Remove from header files

Only get added to the new table (fiotrxts)

Access data from the new table (fiotrxts)

  • New requests will be entered into the new table.

  • Getter actions will return “Error 3060003: contract table query exception”since there will no longer be references to the old tables.

Step 1 - Add new tables and migrate (fio.contracts release #1)

Step 1 introduces the new table structure and adds a new “temporary” action call called migrtrx. This action enables a block producer to start migrating existing data over to the new formatted index tables.

This will migrate the three current tables (fioreqctxts, recordobts, fioreqstss) into one new table (fiotrxts). It also creates and uses a temporary table (migrledgers) that is used during the migration process.

The follow example would migrate 5 records from a current table to the new table:

./clio -u http://localhost:8889 push action -j fio.reqobt migrtrx '{"amount":"5","actor":"qbxn5zhw2ypw"}' -p qbxn5zhw2ypw@active

(The maximum amount of records able to be transferred during a single transaction has been set to 10. Any input over this value will be set to max value. After all records have been migrated, the action will return back a success response.)

Notes:

  • It is preferable if a single BP takes this on and calls migrtrx until all data is migrated.

  • The complete migration of data has to happen before fio chain is released. An off chain validation should be done to confirm this.

  • All new FIO Request and Record OBT records will be entered into both the old and new tables.

Step 2 - Update to FIO Request getters (fio chain release)

Step 2 updates the getters (e.g., get_sent_fio_requests) to retrieve data from the new fiotrxts table

Notes:

  • Users pointing to non-upgraded nodes will be retrieving data from the old tables and users pointing to upgraded nodes will be retrieving data from the new tables.

Step 3 - Clean up data from old tables (fio.contracts release #2)

All fio chain nodes should be upgraded prior to rolling out Contract Step 2.

Step 3 does two things:

  • Modifies the Request code to stop updating the old tables. After this release, only the new tables will get updated.

  • Modifies the  migrtrx call. It will now remove data from the old tables.

Block producers are able to call migrtrx and remove up to 25 records during a single transaction. Added functionality also allows for the API endpoints to use the new data structure introduced in step 1.

Once the data has been deleted, wallets pointing to nodes that have not upgraded:

  • New OBT and Requests will be put into the new tables

  • Get requests and OBT getters will point to the old tables so they will get “no request found” or the OBT data will not be found results.

Step 4 - Removal of temporary actions and removal of old table references

Step 4 includes:

  • The removal of migrtrx temp action

  • Removes old table references (fioreqctxts, recordobts, fioreqstss)

Wallets pointing to nodes that have not upgraded:

  • New OBT and Requests will be put into the new tables.

  • Get requests and OBT getters will get a 500 “Error 3060003: contract table query exception” error.