FIO requests read performance analysis

purpose:

The purpose of this document is to capture findings regarding aspects of FIO request read performance and scaling analysis.

indexing summary of fiotrxtss

typedef multi_index<"fiotrxtss"_n, fiotrxt_info,        
indexed_by<"byrequestid"_n, const_mem_fun < fiotrxt_info, uint64_t, &fiotrxt_info::by_requestid>>,
indexed_by<"byreceiver"_n, const_mem_fun<fiotrxt_info, uint128_t, &fiotrxt_info::by_receiver>>,
indexed_by<"byoriginator"_n, const_mem_fun<fiotrxt_info, uint128_t, &fiotrxt_info::by_originator>>,
indexed_by<"bypayeracct"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_payeracct>>,
indexed_by<"bypayeeacct"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_payeeacct>>,
indexed_by<"byobttime"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_obttime>>,
indexed_by<"byreqtime"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_reqtime>>,
indexed_by<"bypayerstat"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_payerstat>>,
indexed_by<"bypayeestat"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_payeestat>>,
indexed_by<"bypayerobt"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_payerobt>>,
indexed_by<"bypayeeobt"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_payeeobt>>,
indexed_by<"bypayerreq"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_payerreq>>,
indexed_by<"bypayeereq"_n, const_mem_fun<fiotrxt_info, uint64_t, &fiotrxt_info::by_payeereq>
>>
fiotrxts_contexts_table;

get_pending_fio_requests

This getter finds fio requests for which the specified public key is the Payee. in other words it finds all the requests for which the specified account is yet to be paid.

this getter hashes the specified pub key then uses the resulting account to perform a secondary index lookup on the fiotrxtss table using the following query.

get_table_rows_params fio_table_row_params2 = 
get_table_rows_params{
        .json           = true,        
        .code           = fio_reqobt_code,        
        .scope          = fio_reqobt_scope,        
        .table          = fio_trx_lookup_table,        
        .lower_bound    = boost::lexical_cast<string>(hexstat),        
        .upper_bound    = boost::lexical_cast<string>(hexstat),        
        .key_type       = "i64",        
        .index_position = "9"};

it appears that there is a limit to the number of pending fio requests that a single account might be able to have. Specifically it seems that if there are more than around 1500 pending fio requests associated with an account then readability issues will begin to be seen due to the limitations of the API node processing power.

a test will be made to generate 3k pending fio requests, and then we will see what performance the get_pending_fio_requests exhibits.

after loading the table with 2500 requests, the results of the get_pending_fio_requests becomes incorrect. it returns 100 rows, and indicates there are 900 more remaining, and also sets the more to be 25.

When offset is set 2000 and limit is set 10 no requests found is the result, this is incorrect.

testing shows that the following calls also exhibit the same behavior as get_pending_fio_requests.

get_cancelled_fio_requests

get_received_fio_requests

get_sent_fio_request