get_fio_domains -- state read performance analysis
CLEAN UP AND HEADERS TOO
Purpose:
The purpose of this document is to capture the important details relating to the state read performance of the get_fio_domains endpoint in the Fio Protocol.
Details:
The get_fio_domains looks up all domains owned by the target account.
get table rows is called with the following information
get_table_rows_params domain_row_params = get_table_rows_params{.json=true, .code=fio_system_code, .scope=fio_system_scope, .table=fio_domains_table, .lower_bound=boost::lexical_cast<string>(::eosio::string_to_name(account_name.c_str())), .upper_bound=boost::lexical_cast<string>(::eosio::string_to_name(account_name.c_str())), .key_type = "i64", .index_position = "2"};get_table_rows_result domain_result = get_table_rows_by_seckey<index64_index, uint64_t>(domain_row_params, abi, [](uint64_t v) -> uint64_t {
return v; });
ADD links to table definition and indexing!!!!!!!
it seems that since we can have many many domains that are owned by the same account, we will want to make some tests to find limits of the readability of domains by account.
we will make 10-20k domains for an account on a local developer box then try to read through these.
I made a test branch to load a local chain with 21k domains for a single account.
branch name
feature/BD-4108-fiotest-develop-10122022
these tests and setup are in the file register-domains-one-account-max-load.js
it looks to me as if the indexing design of the domains is inherently limited to some number N
where N = the number of rows that can be processed by the node being queried.
if there are more rows than this then those additional rows can only be accessed using get table using --limit and -L and paging through results looking for the target account….
outlook for the domains table
lookups by primary key will remain effective using get table rows and brute for searches on the domains table.
lookups by account will remain effective somewhere up to 1k domains per account (the 1k is an unverified estimate).
beyond 1k domains per account it is possible for the get_fio_domains to return incorrect information regarding the number of domains remaining and other info in the result (the returned domains are correct in spite of this behavior).
mitigation for beyond 1k domains per account.
a strategy for reading many domains per account must be agreed on at the community level.
This will involve defining some sensible grouping of domains that effectively reduces the number of domains being indexed for each searchable value (combined indexes can be conceived to provide somewhat complex indexing for these records as user needs may required)
once the strategy is agreed then it is possible for any or all of the following solutions to be implemented and deployed.
a new lookup table can be considered to be added into state for these new lookups.
a new getter can be made using the new search criteria identified in item 1 above.
or
a table migration of the existing domains table can be strategized which will provide the necessary indexing for domains using a new index which is defined as necessary.
or
An off chain database can be created and used to organize domains as needed.