QA Starter Kit
Do you use different apps every day and notice things that bother you when you’re using them? Are you passionate about providing users with the best experience when using apps but don’t know where to start?
Great! This Starter Kit is for someone like you. Here, you’ll find all the information you need to become a good Software Test Engineer candidate. A Software Test Engineer has multiple career paths like Test Analyst, Test Automation Engineer, Test Manager, Test Architect, etc. We won’t overwhelm you with the theoretical aspects; we’ll focus on the practical.
But first, a Software Test Engineer needs the powers of acute observation and critical thinking. One piece of advice from our side: “Expect the unexpected!” Watch this video and tell us how many differences you notice.
Did you find more than 10? Awesome! If not, don’t worry, you’ll get there.
As a Software Test Engineer, your task is to ensure that bugs don’t reach end-users. The path from requirement analysis to test closure and release isn’t easy; this path is where Test Engineers learn crucial information about the product so that end-users won’t have applications full of bugs. If you want to read more about the basics of software test engineering, check this out!
This website doesn’t capture everything a Test Engineer does — there’s more to it. Creating test plans, test cases, and test reports is equally essential as finding bugs. Let’s start with test cases.
When writing test cases, we need to be very detailed. Someone who isn’t familiar with an app’s functionality should be able to run tests just by reading the test case. Any good test case needs to have the following:
Test case ID – This is usually auto-generated
Test case description – Title/short summary
Preconditions – If any are necessary for test case execution (eg. set your device to “Zoomed” view)
Test steps – Write as detailed as possible, e.g.,
- Login with a user…
- Click on “Settings”…
- Click on “Logout” …
Test data – Required accounts (admin, user, etc.)
Expected results – Make sure you have a reference for your “source of truth”
Attachments – Video/screenshots/links to designs/documentation
When you add test cases to a test plan, add the following:
Date – Time and date of execution (could be important for app versions)
Tester – Name of the tester
Environments – The environments where the test case will be executed (production/develop/QA/etc.)
App version – The app version used for the test case execution
Status – Pass/fail – Add this after you run the test case
Also, it’s good to organize your test cases in (at least three) categories:
Smoke test – The happy path with the most important features
Regression test – Extensive test cases covering main functionality
Edge case test – Edge cases
Try this: take any app (mobile/web/desktop) that has a login/registration flow and write as many test cases as you can. Write your tests cases in a spreadsheet, or use test management tools like Testrail, Zephyr, Xray, etc.
If you still don’t know what to test, look here for a bunch of helpful test scenarios. You can also find lots of good challenges at Testing Challenges if you want to take your game to the next level.
Once you’ve created test cases, you can start executing them. Some testers prefer to explore an application before making test cases and execution, and that’s fine. It’s called exploratory testing, and you can learn more about it from Atlassian.
Test execution is the phase where we verify that all parts of our app are working as expected. If we find bugs in our tests, we need to create bug reports or bug tickets. A bug report should also be detailed to help your teammates quickly reproduce the bug on their devices and help them understand the bug’s severity. Your bug report needs the following:
Title/Summary – This should be descriptive enough for your teammates to understand the issue
Preconditions – If any are necessary for reproduction (eg. set your device to “Zoomed” view, throttle your internet speed connection, etc.)
Test data – Accounts required (admin, user, etc.)
Steps to reproduce – Be detailed! Someone who’s never used the app should be able to reproduce the bug by following the steps
Actual result – Describe what happens after you executed the reproduction steps
Expected result – Describe what should have happened after you executed reproduction steps. Make sure to add “source of truth” (AC of the ticket, link to the project documentation, link to design, etc.)
Attachments – Videos/screenshots/logs and similar that should visually represent the bug. Having attachments is good proof that you can reproduce the bug. With attachments, you can save your teammates a lot of time when they’re reproducing the issue
Environment – The environment in which the was bug reproduced (production/develop/QA/etc.)
Build version – The app version is super useful to devs. They can easily track down changes from previous versions and locate which change caused the bug
Device – On which device was the bug reproduced? (Add technical specification)
OS version – The operating system used to reproduce the bug (e.g., macOS Big Sur 11.5, Windows 10, etc.)
Severity – Blocker, critical, major, minor, or trivial. If you don’t know how to assign the severity of a bug, read this
Test data – Accounts required (admin, user, etc.)
Level of reproducibility – Sometimes we run the same steps to reproduce a bug and we get different results. That’s why this part of the bug report is important: it can help you define the priority of the bug
Workaround – Sometimes you can get an app’s desired behavior by avoiding the bug. This is helpful when defining bug priority
Learn more about buggy websites and check out Abstracta.
Pick one that you find there and start discovering and reporting bugs. Use a spreadsheet or a bug reporting tool like JIRA, Asana, or Azure DevOps to document your findings. Be as descriptive and detailed as possible. Also, don’t forget to have fun when you’re testing! You’ll see that your Test Engineer spark will ignite after you find your first few bugs.
You’ve created and executed test cases? You’ve found and reported many bugs? Great! Now it’s time to wrap it all up in a test report. The test report is the document where all your teammates can learn about the work you’ve done. Here’s what your test report needs to have to bring your teammates up to speed and facilitate the product release:
Dates – Build dates, version dates, target release date, actual release date
Release blockers – Tickets that need to be done before release
Stability – Previous/current stability (for mobile), check for blockers in production environments, user complaints, etc.
Software Test Engineer summary section:
- All tickets included in the release
- Major features
- Tech debt tickets
- Regression risk – Work with devs. Check where code was changed and what changes can cause regression risks
- Test coverage:
- Manual – Show your test plans and test results (how many tests passed/failed)
- Automated – Show your test plans and test results (how many tests passed/failed)
- Device coverage – List all the devices that you executed test cases on. Include the device name, OS version, browser name, browser version, screen size, and device type — where applicable — along with whether it’s mobile, web, or desktop
- Bug analysis:
- Bugs found before code freeze
- Bugs found during code freeze
- Introduced bugs
- Bugs triaged for the next sprint
- Stakeholder sign off – Say multilingual support was introduced with the latest release. The app needs to go through linguistic testing before release. Or, a new analytics tool was introduced in the app and PMs/BAs are going to use those analytics for metrics: have them double-check if data is showing correctly
- Hidden features – Some features are incomplete but their code is merged into the release build. Make sure to point this out!
Release checklist section:
- Software testing and results – Ask if everybody is comfortable with the level of software testing performed on the release and if the team is comfortable with any known issues
- Dependencies – Check if your release is dependant on different teams (content, marketing, etc.)
- Store updates (if it’s a mobile app) – Check all metadata fields required for release in the App Store or Google Play Store
- Post-release checks – Smoke test plan (sanity check), update documentation, information about app health (stability, reviews, adoption rate), have a release history page!
- Team decision – Ask everyone on the project if they want to release the version to the production
Present your findings to your teammates if you can. Everyone appreciates thorough test coverage! This way, everyone will have more confidence in the app’s stability.
For practice, using your previous tasks, create your own test report based on the format above. If something isn’t applicable, skip it.
That’s it! If you read the starter kit and completed all three tasks, awesome! Send us your task and we’ll review it. You’ll definitely get constructive feedback, and maybe even a job offer.
Do your best, and happy testing!
Something to leave you with
Finally, here’s the list of useful articles we use again and again about software test engineering:
For more links, leave us a comment below or contact us via our social media!