The automated tests pass, and the release still breaks for someone. Usually the tests only ever ran in one environment, one browser and one screen size, and everything else depended on someone checking by hand before the deploy. For whoever leads the team, that shows up as bugs customers find first and releases that wait on manual testing.
Build a test config in 5 steps
We fix it with what we call a wide configuration, one end-to-end test suite that your build pipeline can run against any environment, browser or screen size just by changing a setting. Wider test coverage then costs a pipeline run rather than a tester's afternoon.
We build that configuration step by step, with Cypress for the tests and GitHub Actions for the pipeline. You can apply it to an existing project or start from our finished version on GitHub.
So, imagine you've decided to buy a car. Do you want to add any options? Perhaps automatic transmission, winter tires, climate control, or something else?
A testing framework without a well-thought-out configuration is like a car without options. It can get you to your destination, but the journey is unlikely to be comfortable.
Under the 'wide configuration,' we understand the ability to pass the most important options and/or arguments to your testing framework.
Explaining in detail we must be able to easily run the tests in any %put the item from the list below% we want.
- ENV [Environment, like DEV, UAT, PROD, etc]
- Browser [Chrome, Edge, Firefox, etc]
- Resolution [laptop, mobile, tablet]
The following article won't contain examples of Safari usage because its configuration will be defined in a separate article.
In our example, we will build out the configuration using a default Cypress Project. You can apply this config to your project or check out the final configuration in github.
Environment configuration
Let's start with a test example, look at this simple test below. The URL used in the test can be easily replaced with your project environment's URL.
To accomplish this, Cypress uses environment variables, described in its guide to environment variables.
example.cy.js file:
describe('spec', () => {
it('test example', () => {
cy.visit('https://example.cypress.io/');
});
});
to easily provide environment-related data as we desire, we need to start with adding dotenv and cross-env package dependencies. Then add a script to open cypress with a specified environment.
package.json file:
{
"name": "techfabric.cls",
"version": "1.0.0",
"scripts": {
"cypress:open": "cross-env E2E_ENV=.dev cypress open"
},
"dependencies": {
"cross-env": "^7.0.3",
"cypress": "^13.6.4",
"dotenv": "^16.3.1",
"typescript": "4.6.3"
}
}
Now create .env.dev file with url env variable:
.env.dev file with url env variable:
Then change cypress.config.ts configuration:
import dotenv from 'dotenv';
dotenv.config();
require('dotenv').config({ path: `.env${process.env.E2E_ENV}` });
const { defineConfig } = require('cypress');
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
config.env.url = process.env.CYPRESS_ENV_URL;
console.log(config.env);
return Object.assign(config);
},
baseUrl: process.env.CYPRESS_ENV_URL,
},
});
The final part is to update the test itself:
test example.cy.js file
From this moment, we simplify variables management for cases when we jump from one environment to another, everything we need is just to pass the correct .env file (or set of environment variables) to Cypress.
pass variables to cypress
Browser Configuration
Up until this moment, we have been using the default or selected browser.
Google Browser
The next step is to be able to use the chosen browser with preferred options. To make this happen:
- Update package.json scripts with chosen browsers:
package.json file
- Create browserLauncher.ts file in 'support' folder and provide desired arguments
browserLauncher.ts file
- Update setupNodeEvents method:
setupNodeEvents(on, config) {
on('before:browser:launch', (browser, launchOptions) => {
return launchBrowser(browser, launchOptions);
});
config.env.url = process.env.CYPRESS_ENV_URL;
console.log(config.env);
return Object.assign(config);
}
In result we can run any pre-defined browser and pass custom arguments.
run starting
fun finished
Resolution Preferences
Defining resolutions is a bit more complex… So, before we begin with the code let's agree on some criteria:
- **Framework must support desktop / mobile / tablet resolutions.**Desktop resolutions: 1920x1080, 1536x864.
- Mobile resolutions: Samsung S10 (360x760), Samsung Note S9 (414x846).
- Tablet resolutions: Samsung Galaxy Tab A (810x1080), Google Nexus 9 (800x1280).
- Framework must take resolution as a single and clear argument, so we don't need to pass many details via cmd.
- Browser object must be as close to the real one, as it could be.
The task seems to be quite simple since we can just pass viewport options:
viewportWidth: width,
viewportHeight: height,
But in this case, we won't handle such important things as:
- userAgent
- deviceScaleFactor
- mobile type
- orientation
- etc
To handle all advanced options we start with resolution objects creation.
- Create browser config file with resolution specification and objects.
// Tablet devices configurations
export const tabletBrowsers = {
'tablet_810x1080': {
name: 'Samsung Galaxy Tab A',
width: 810,
height: 1080,
deviceScaleFactor: 2,
mobile: false,
orientation: 'portrait',
userAgent: 'Mozilla/5.0 (Linux; Android 11; SAMSUNG SM-T870) Chrome/121.0.6167.160'
},
'tablet_800x1280': {
name: 'Google Nexus 9, Samsung Galaxy Tab S2',
width: 800,
height: 1280,
deviceScaleFactor: 2,
mobile: false,
orientation: 'portrait',
userAgent: 'Mozilla/5.0 (Linux; Android 11; Nexus 9) Chrome/121.0.6167.160'
}
};
browserConfig.ts file
- Update browserLauncher, now use resolution and browser type from browserConfig file:
browserLauncher.ts file
- Update cypress.config.ts, now pass resolution and set viewport options
cypress.config.ts file
- Update package.json scripts:
package.json file
As a result of the previous setup, we can run the tests in the chosen environment, using the preferred browser with the desired resolution and capabilities. Now time to integrate it to CI/CD so everyone benefits from the tests.
CI/CD Integration
name: Run Cypress using Wide Configuration
on:
workflow_dispatch: # allow for manual execution
inputs:
choice_browser:
type: choice
description: Select browser
options:
- chrome
- firefox
- edge
choice_environment:
type: choice
description: Select environment
options:
- dev
- uat
- prod
choice_platform:
type: choice
description: Select platform type
options:
- desktop
- tablet
- mobile
choice_resolution:
type: choice
description: Select screen resolution
options:
- 1536x864
- 1920x1080
- 360x760
- 414x846
- 810x1080
- 800x1280
jobs:
cypress-tests:
name: Run Cypress
runs-on: ubuntu-latest
environment: dev
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Install dependencies
run: npm install
- name: Run tests in ${{ github.event.inputs.choice_browser }}-${{ github.event.inputs.choice_resolution }}
run: |
npx cross-env E2E_ENV=.${{ github.event.inputs.choice_environment }} BROWSER_TYPE=${{ github.event.inputs.choice_platform }} RESOLUTION=${{ github.event.inputs.choice_resolution }} cypress run --browser ${{ github.event.inputs.choice_browser }}
After configuration is applied, we can start the test run, just select your options from the list in boxes:
select your configuration
And check the result:
Summary
Today, we've reviewed how to build a testing framework that is flexible enough to accommodate client needs such as custom environments, browsers, options, and resolutions.
We've also integrated everything into CI, so the development or automation QA team can benefit from our solution and everyone on the team can run the test suites with just a few clicks.
Check out this test project in our GitHub repository and apply useful configuration parts to your project. Good luck and see you later!
Frequently asked questions
Do I have to run every browser and resolution combination before each release?
No. The workflow takes one browser, one environment, one platform and one resolution per run, so a run covers a single combination. That makes the choice yours: a full sweep is as many runs as combinations you pick, and a lighter check can be a single run on the combination your customers use most. The workflow offers three browsers, three environments and six resolutions, so the full grid grows quickly.
Can I add another environment, like staging, to this setup?
Yes. Add a .env.staging file with its CYPRESS_ENV_URL, and add staging to the workflow's environment options. The workflow passes E2E_ENV=.staging, and cypress.config.ts loads .env plus that value, so the tests themselves don't change.
Why define a resolution object when Cypress can just set the viewport width and height?
Width and height only set the viewport. They leave out the user agent, device scale factor, mobile type and orientation, so a phone or tablet test can behave differently from what a real device shows. That's why each resolution in browserConfig.ts carries those values, and why the workflow can pass a single resolution argument instead of many details.