Overview
There are several areas of focus for QA. The following lists those areas including the targeted, actual, and gap resources. Risks to the FIO Protocol are noted.
Project | Notes | Target QA | Actual QA | Gap | Risks |
|---|---|---|---|---|---|
fio and fio.contracts automated testing | Target a 1:2 ratio of QA:Dev | 1.5 | 1 | 0.5 | QA is lagging development when writing QA automated tests which impacts the release. Impact to schedule goes from 2-6 weeks. |
Devnet/Testnet/Mainnet release testing | Involves running regression tests across the various test platforms, logging bugs, and following up on issues | 0.2 | 0 | 0.2 | Eric does all of the release testing. This is sub-optimal and results lighter testing resulting in more bugs getting through and increasing the risk to the chain experiencing unusable features or chain down time. |
FE Projects - Manual testing
| Manual QA testing of dashboard. | 0.3 | 0 | 0.3 | Pawel, GP, and Eric do manual testing. Very little regression testing is done. This is sub-optimal and results lighter testing resulting in more bugs getting through and increasing the risk to the dashboard being offline. Estimated impact to up time: 1-2% |
FE Projects - Automated testing
| Involves either outsourcing of automated test writing or the use of automated test packages. | 0.2 - If we use something like Proof’d automated test software. 1.0 - If we write our own automated tests | 0 | 0.2 - 1.0 | No automated regression tests for the dashboard increasing the risk to bugs getting through and causing the dashboard to be offline. Estimated impact to up time: 2-5% |
Total | 2.2 - 3.0 | 1 | 1.2 - 2.0 |
* A QA outsourcing group quoted $40K over 3-5 months to do manual testing of the Dashboard and write the initial automated tests. There would also be an ongoing maintenance cost.
** Estimated cost of outsourcing the registration site is approximately 1.5 FTE (~ $5-$10K depending on location). This includes project management, development, and QA. Does not include adding new profiles or working with integrating entities.
Centralized
1 x QA Manager
Test harness architecture
Oversee test development
2.5 FTE
fio.test new and backlog automated tests
Proof’d tests
0.5 FTE release testing
Community / Bounty
1 x QA Manager
Test harness architecture
Oversee test development
Create bounties for:
fio.test new and backlog automated tests
0.5 FTE
release testing
Develop/Maintain Proof’d
Outsource
1 x QA Manager
Test harness architecture
Oversee test development with outsource group
Release testing
Outsource
fio.test new and backlog automated tests
Develop/Maintain Proof’d test scripts
Add Comment