[Release n.n] Release Checklist
- Eric Butz
- Michael Leet
- Pawel Mastalerz
- Ed Rotthoff
This document is intended to assist release managers in tracking the progress of required work items for a given release. Lead developers can use this checklist to understand how they might assist the release manager in achieving a release of the FIO protocol.
Release Stakeholders
Stakeholder | Description | Channel |
|---|---|---|
Product Manager (PM) | Core Product Manager for release | product discord channel |
Release Manager | Primary tech resource for release | devs-internal channel |
FIP Developer | Developer responsible for coding change | devs-internal channel |
QA | QA or developer responsible for coding change | devs-internal channel |
BPs | Block Producers | Telegram Mainnet and Testnet |
FIO Developer Community | Broader FIO community of developers | Dev Hub |
Account Management | Foundation group responsible for partner communication | partner-success channel |
Marketing | Foundation group responsible for broad communication | marketing channel |
Please refer to the field descriptions below the checklist for more information on each item.
Subject | Owner | Description | Jira Links and Notes |
|---|---|---|---|
Release planning |
|
|
|
Update Master Release Schedule | PM | Update master release schedule with all release info |
|
Create project wiki | PM | Create a project wiki created for release under Releases This will be used to store all relevant release documents. Example. |
|
Release stories and subtasks | PM | Create and epic, stories, and subtasks for release. There should be a story for:
Subtasks should be created for each item in this checklist. |
|
Release kickoff meeting | PM | Set up a kickoff meeting with team to review release |
|
Release plan | Release Manager | Create rollout/release plan for this effort (added actions, removed actions, table migrations, order dependent detailed instructions on how to roll out these changes successfully. include all necessary MSIGs). This should include detailed instructions on how to verify that the rollout was performed successfully. Once complete, review the rollout plan and verification with architecture and QA/Integration teams. Release script template: [Release n.n] Release Script Master release plan: Master Release Plan See example: [fio 3.3 fio.contracts 2.7] Release script |
|
Security |
|
|
|
Security - Code audit | PM | A security focused audit of contract modifications should be strongly considered for any new actions, or any modifications to actions that involve changes in authorizations, or error logic. Code audits can be performed by core team members, or they can be performed by third parties specializing in EOSIO security issues. Findings for security audits are sensitive and may be published to the public after careful consideration of the consequences of publishing this material. Once vulnerabilities have been adequately addressed all findings and remediations should be posted to the wiki for the project in question. Provide a set of links to completed items, or provide a point of contact for your project if items are in work. Security documentation: https://fioprotocol.atlassian.net/wiki/spaces/FO/pages/141262894 |
|
Vulnerability testing | PM | The feature may benefit from security focused stress testing. This testing may be performed by core team members, or by contracted parties hired for the project. The testing might take an EOSIO specific focus, with the addition of penetration, fuzz, gaming the system, and other vulnerability testing is strongly advised. | This is still TBD on how to do this on a regular basis. |
Bug bounties | PM | Some features may benefit from placing a bounty on the contract modifications. Your feature should consider offering a reward if any community member is able to discover security flaws, or vulnerabilities in new features. These bounties may be long lived, and they may require the use of private test instances of the fio test net to provide hackers ample opportunity to “break the chain”. | This is still TBD on how to execute |
LocalNet |
|
|
|
Local - Release rollout testing | Release Manager | Validate release script, updated by developers for each feature, detailing all addaction, createfee, and new and existing contract updates Perform local (single machine) test to validate release script See: https://developers.fioprotocol.io/docs/developers/devnet#local-testing |
|
LocalNet - Performance and Scaling | Release Manager | Verify performance and scalability tests run on LocalNet (and results added to Dev Spec) Identify performance and scalability tests to run on DevNet |
|
DevNet |
|
|
|
DevNet Setup | Release Manager | Environment: https://fioprotocol.atlassian.net/wiki/spaces/FO/pages/114131133 Repo: https://github.com/dapixio/fio-devnet Testing Overview: Devnet Testing MultiSig Process: Multisig Create msig: actions capturing release items for injest into mSig tool. |
|
DevNet - Contract Testing | Release Manager | Baseline Smoke test using fio.devtools/fio.test framework See: DevNet Contract Test | This step sets the baseline for future DevNet tests
|
DevNet - Fork testing | Release Manager | Complete fork testing per DevNet Fork Test. May include baseline smoke test per above This task has two flavors:
| Use of fio-devnet scripts one can verify update of 3 or 6 BPs at a time against the remaining BPs as blocks are processed Direct execution of release script items, especially setcontract, is helpful to see BPs block production continues normally. |
DevNet - Regression testing | Release Manager | Full fio.test regression tests completed and run cleanly on Devnet |
|
DevNet - mSig Release rollout testing | Release Manager | Validate release using mSigs. See FIO mSig Adminer. Note that this is the FIO mSig Admin tool (fork of cryptolions tool). See Devnet Testing for test and validation info. | This is TestNet simulaton test where the release script items are rolled up into multiSigs and executed on the DevNet chain. This step will exercise mSig propose, mSig 2/3 + 1 approvals, as well as the mSig execution step. Example mSig review and validation;
|
DevNet - Contract Testing | Release Manager | Same as above, but performed on msig version of Devnet |
|
DevNet - Regression testing | Release Manager |
| |
DevNet - Performance/Scalability testing | Release Manager | Perform DevNet performance and scalability tests Performance and Scalability testing is a responsibility of the Developer and QA, however, it may be necessary to identify release level performance and scaling requirements |
|
SDK Pre Release |
| Changes to Typescript SDK may be necessary for each release of the FIO protocol. SDK changes should go through a QA process. The release manager must integrate the SDK changes required for a release, then perform acceptance tests on these changes. |
|
SDK pre-release - release story | PM | Create story (if not already done) to track SDK Release |
|
SDK pre-release - dev complete | PM | Confirm that SDK development is complete for all features in release |
|
SDK pre-release - release candidate | Release Manager | Create release/n.n.x branch (where n=contract release number) Create pre-release (Release Candidate - FIO Typescript SDK vx.x.x-rc1) |
|
Testnet Setup |
|
|
|
Testnet setup - fio.test pre-release | Release Manager |
|
|
Testnet setup - SDK Update fio.test to point to SDK release branch | Release Manager | Update fio.test |
|
Testnet setup - fio.devtools pre-release | Release Manager | Create release/n.n.x_m.m.x branch (where n=contract release number and m=chain release number) |
|
Testnet setup - fio.contracts pre-release | Release Manager |
|
|
Testnet setup - fio pre-release | Release Manager |
|
|
Testnet Release |
|
|
|
Testnet release - Facilitate the creation and proposal of Release mSigs | Release Manager |
| Use of FIO Block Producer, i.e. nyvrxkxhiyql, and TestNet node, 52.35.164.8:8888, is recommended for proposal. |
Testnet release - BP mSig Approval | BP TestNet community |
|
|
Testnet release - BP mSig Execution | BP TestNet community |
| Use of FIO Block Producer, i.e. nyvrxkxhiyql, and TestNet node, 52.35.164.8:8888, is recommended for execution. Recommend mSig execution one at a time to properly validate. mSig execution frequency every 5 minutes recommended for chain block production. |
Testnet release - Release validation | Release Manager |
|
|
Testnet release - Smoketest | Release Manager | Update fio.test smoketest with new actions and getters and run against Testnet |
|
Testnet release - Manual Feature validation | Release Manager | Perform manual validation for all actions and getters listed in the Master Release Plan. Also work with developer to do any release validation testing noted in the Deployment and Rollout guides. |
|
Testnet release - Foundation Testnet API Node* | Release Manager | Update FIO Testnet API node See: https://developers.fioprotocol.io/docs/chain/node-build |
|
Testnet release - Communication | Release Manager |
|
|
Mainnet Setup |
|
|
|
Mainnet setup - Mainnet prep | Release |
|
|
Mainnet setup - fio.test | Release |
|
|
Mainnet setup - fio.devtools | Release |
|
|
Mainnet setup - fio.contract | Release |
|
|
Mainnet setup - fio | Release |
|
|
Mainnet Release |
|
|
|
Mainnet release - addaction and createfee msigs | Release |
Example:
|
|
Mainnet release - Contract msigs | Release |
|
|
Mainnet release - Release validation | Release |
|
|
Mainnet release - - Manual QA validation | QA | Perform manual validation for all actions and getters listed in the Master Release Plan. Also work with developer to do any release validation testing noted in the Deployment and Rollout guides. |
|
Post-deployment |
|
|
|
Post-deployment - Mainnet validation - fio chain | Release |
|
|
Post-deployment - Update FIO Hosted API Nodes | Release |
|
|
Post-deployment - Update FIP status | PM | Move FIP status to Final and include release links; update master README index |
|
Post-deployment - Update Devhub | PM |
|
|
Post-deployment - Update fiosdk_typescript-examples | PM | Add examples for new actions to fiosdk_typescript-examples repo | Not required |
Post-deployment - Partner upgrades | PM |
|
|
SDK Production Release |
|
|
|
Typescript SDK Release | Release | Tracks the release of the latest SDK to production.
|
|
Process Description --
Each item that is to be addressed in the checklist should have a story associate with it under the release epic in jira. also there should be a release section of the wiki which contains all of the artifacts generated for a given release, and under this wiki should also be contained all the details of the release planning and execution for future reference of the community.
Devnet testing
Background – In order for a release of the FIO protocol to be rolled out on test net and main net, the release manager must first assemble the set of commands that will upgrade the current main net version to become the new release. The commands may include adding actions to FIOs list of allowed contract actions, adding/modifying the fees associated with FIO actions, creating new FIO system accounts and contracts, and updating all of the required FIO contracts with the changes for the new release. The release manager must assemble a complete list of the required commands, and execute them on a private test network, then perform validations of the resulting release. The commands, and test results must be posted on the wiki for this release.
Objective – The objective of dev net testing is to assemble the list of commands required to upgrade FIO for this release, and to complete acceptance testing of a local private FIO network upgraded using these commands.
Tactics – on dev net, build the release version of the FIO Core code and the FIO contracts, copy these off to the side. Then build the dev net using the Main net versions of the contracts and core, and spin up the dev net. Perform all of the necessary clio commands to upgrade the contracts. once this is completed run a regression test to ensure all chain operations function as expected. if there are significant changes to the FIO Core code which result in special logic and timing regarding the core and contracts rollout (this is called a forking change), then special test plans must be made and executed in accordance with the needs of the releases unique needs. If no forking logic is being released then acceptance and QA tests and some performance tests will be performed and published onto the wiki
Deliverables – The release manager will post on the wiki the commands for this release, and they will also post on the wiki the test results for the tests performed.
Contract rollout commands – The release commands for each project included in the release should be a part of the rollout plan for each project included in the release, assemble the commands for the release into one list of commands, and fill any missing gaps by cooperating with the necessary leads if commands have not yet been provided. the following actions are commonly required --
Add action – add an action to the permitted list of actions in the FIO protocol
Add Fee – add a new fee for a new action in the FIO protocol.
Add new system account – When new contracts are added to FIO an owning account must be created.
Set system account authorizations – set permissions and authorizations on new system accounts
Set privileged.
update authorizations
Set Contract – set a new contract on the FIO protocol, or update an existing contract.
Set ABI – set a new ABI on the FIO protocol, or update an existing ABI.
See also
Contract testing – After contracts have been upgraded, acceptance testing must be completed, and findings must be published to the wiki.
Fork testing – If there are changes to the protocol which require backwards compatibility considerations then forking tests must be performed, these test require code changes to be made before the fork testing to ensure that the forking logic works equivalently in the dev net environment. Results and findings must be published to the wiki upon completion.
Performance testing – Some projects have dev net level performance test considerations, when a release includes the need for performance testing then tests must be run and findings published to the wiki for the release.
QA testing – QA regression tests should be completed on each release before dev net testing, but full regression on dev net only increases our confidence in a release. findings should be published to the wiki.
NOTE – other descriptions may be added if this is seen as helpful for the document