How do I test bug fixes and new features in Mautic?

Mautic has a regular release cycle, which means that new bug fixes are published every month, features once a quarter, and major changes which break backwards compatibility are released approximately annually.

Did you know that every single bug fix and feature has to be tested by two people, people like you?  The more people we have testing, the quicker we can get things released to the community.

Testing bug fixes and new features can be done in the web browser - no need to know how to install Mautic, apply patches or anything.

Here's a video walk-through and some steps below which explain how to go about testing.

 

  1. Sign up for an account at github.com - you will need this to leave comments and report your findings.
  2. Install the Gitpod extension for your browser - https://www.gitpod.io/docs/configure/user-settings/browser-extension
  3. Go to https://github.com/mautic/mautic/issues and look at the pinned posts - there will be an Open Source Friday pinned issue with a list of good PRs to test.
  4. Open the PR you want to test. If it's a bug fix, we need to first replicate the bug, before we test the fix. To do that:
    1. Go to https://github.com/mautic/mautic and make sure you are on the right branch - usually the default will be the correct one, but if you're testing something in an olrder branch you will find the click on the green 'Gitpod' button in the top right corner. 
    2. Log in with your GitHub credentials
    3. In the terminal window, once it's finished loading and you see the green text, type ddev exec bin/console d:f:l to load sample data for testing. NOTE: this purges the database, never use in production instances!
    4. Once all the fixtures are loaded, use the icon in the top right corner of the browser preview window to open Mautic in a new tab, and log in with admin/mautic
    5. Test the bug following the instructions in the pull request or the issue that is associated with it
  5. Once you've confirmed the bug, where relevant, go to the pull request with the fix (or if it's a feature, you can start here) and click the green Gitpod button to open a Mautic instance exactly as above, but with the bug fix applied.
  6. Wait for the instance to finish building then type ddev exec bin/console d:f:l to load sample data for testing. NOTE: this purges the database, never use in production instances!
  7. Once the fixtures have finished loading, open the Mautic instance in a new tab using the icon at the top right of the preview browser
  8. Test the fix - make sure it does what it should, and test anything else that might be impacted by the changes
  9. Go to the pull request and click the 'files changed' tab
  10. Click the green button to leave a review - if you're a developer, please also do a code review and leave a comment to say you've also reviewed the code. Make sure you are clear about:
    1. What you've tested
    2. What you found
    3. What you've not tested
    4. Anything you didn't understand, or couldn't reproduce
    5. Anything else you think might be relevant
    6. Don't forget to thank the person who created the pull request ;) 
  11. If you've tested and found it does what it should, and there are no issues, select the Approve radio button. If you found issues, use the 'Request changes' button. Even if it's a really small issue, please use this as it prevents us merging the pull request until those concerns are addressed. If you just want to leave a comment because you couldn't approve or request changes, for example if you couldn't test the PR at all, use the 'leave comment' option.
  12. Press submit review 
  13. Congratulations! You reviewed your first pull request! And thank you, from the whole community! cool

Article Number: 50
Author: Mar 17, 2023
Last Updated: Mar 17, 2023
Author: Ruth Cheesley [[email protected]]

Online URL: https://kb.mautic.org/article/how-do-i-test-bug-fixes-and-new-features-in-mautic.html