Overview
There are several areas of focus for QA. The following lists those areas including the targeted, actual, and gap resources. Risks to the FIO Protocol are noted.
Project | Notes | Target QA | Actual QA | Gap | Risks |
---|---|---|---|---|---|
fio and fio.contracts automated testing | Target a 1:2 ratio of QA:Dev | 1.5 | 1 | 0.5 | QA is lagging development when writing QA automated tests which impacts the release. Impact to schedule goes from 2-6 weeks. |
Devnet/Testnet/Mainnet release testing | Involves running regression tests across the various test platforms, logging bugs, and following up on issues | 0.2 | 0 | 0.2 | Eric does all of the release testing. This is sub-optimal and results lighter testing resulting in more bugs getting through and increasing the risk to the chain. |
FE Projects - Manual testing
| Manual QA testing of dashboard. | 0.3 | 0 | 0.3 | Pawel, GP, and Eric do manual testing. Very little regression testing is done. This is sub-optimal and results lighter testing resulting in more bugs getting through and increasing the risk to the dashboard being offline. Estimated impact to up time: 1-2% |
FE Projects - Automated testing
| Involves either outsourcing of automated test writing or the use of automated test packages. | 0.2 - If we use something like Proof’d automated test software. 1.0 - If we write our own automated tests | 0 | 0.2 - 1.0 | No automated regression tests for the dashboard increasing the risk to bugs getting through and causing the dashboard to be offline. Estimated impact to up time: 2-5% |
Centralized
1 x QA Manager
Test harness architecture
Oversee test development
2.5 FTE
fio.test new and backlog automated tests
Proof’d tests
0.5 FTE release testing
Community / Bounty
1 x QA Manager
Test harness architecture
Oversee test development
Create bounties for:
fio.test new and backlog automated tests
0.5 FTE
release testing
Develop/Maintain Proof’d
Outsource
1 x QA Manager
Test harness architecture
Oversee test development with outsource group
Release testing
Outsource
fio.test new and backlog automated tests
Develop/Maintain Proof’d test scripts
Add Comment