Developing a Playwright-Firebase Plugin to Enable Rapid Test Suite Authentication

01 Dec 2023

How we developed a plugin that allows developers to log in/out of their Playwright test suit with a single line of code

Developers who want to run E2E tests within their apps may be frustrated at the lack of choice available when considering frameworks for their app, and resolve to use Cypress. It’s a well-founded testing framework, that has plenty of plugins available. For Firebase authentication my go-to is Prescott Prue’s: cypress-firebase . It seamlessly attaches login commands to Cypress for fast Firebase authentication.

Playwright is the new hotshot on the testing scene — it’s fast and has VSCode integration. However, since it’s a relatively new framework, it lacks the open source contributions that have helped Cypress to flourish.

Here we outline how we developed a Playwright-Firebase Authentication plugin to enable developers to authenticate their test suite quickly in the Playwright space.

Want to jump straight into the code? Check out our plugin here !

Why Playwright?

For private repositories, every minute in the GitHub actions counts, including those long ones spent running your E2E tests. We prefer to set specific timeouts on our GitHub actions to prevent runaway processes from causing huge billing issues, but this has caused us some headaches. The Cypress E2E tests might pass, but because of their fluctuating duration, you might find that, despite passing within 4 minutes in the morning, it exceeds the 6-minute timer at 3pm. For small test suites, this is not really a concern, but it can become a significant blocker for larger suites.

There’s been a good deal of work done showing the performances of different testing suites, my favourite being the Checkly work done by Giovanni Rago, which measures the execution time of each testing framework for a variety of different scenarios. In each one, Playwright outperforms Cypress significantly.

The downside with Playwright, when it comes to GitHub actions, is downloading the dependencies. This is expensive for the run time and is probably the last thing ever considered — especially after you’ve already built half a suite of E2E tests. The real consideration then is how you want to parallelise your tests, or if you even can. The fortunate side is that Playwright, unlike Cypress, allows you to parallelise your tests for free.

Playwright, Cypress and Firebase

There is a significant difference between the way Cypress and Playwright run under the hood. For Cypress, the whole browser is embedded within the Cypress environment. This means that Node.js modules can be easily integrated within the test suite to enable you to use them within the browser to make authentication requests that can modify the user credentials in the response.

Playwright on the other hand is all about automating the browser. This is handy and is probably why Playwright is so much faster than Cypress. However, it means that the Playwright environment and the Browser environment are entirely isolated, save for the few commands Playwright enables for you to communicate with it.

For Firebase this matters. Commands enabling the signing-in procedure are accompanied by modifications to the window session, all of which you can observe. Try logging in to a website that uses SSO with Google. Click login, a pop-up appears briefly, and hey-presto, you’re in. But how? Take a look at the session storage, or the local storage. There’s a good chance that you’ll find something there relating to your authentication provider, something along the forms of this:

JavaScript

{
	uid: ...
	email:...
	emailVerified:...
	apiKey:...
	lastLoginAt:...
	photoURL: ...
	providerData:...
	stsTokenManager: {
		accessToken:...
		expirationTime:...
		refreshToken:...
	}

}

On closer inspection, you’ll find all sorts of interesting elements within this storage. Firstly, your display name, then an email, an access token, and a refresh token. How did this happen? Essentially, on the server side, the command signInWithPopUp was called and waits until the provider sends back confirmation to your Firebase instance that you truly are an authenticated user (high five).

Browser storage is one facet of this communication. We found that we were able to copy and paste this key/value pair into the session storage, without logging in normally, and had access to an internal site that used browser session persistence in their Firebase setup.

So, how can we automate this process on the Playwright browser?

Authentication

There are several avenues one can take when trying to authenticate a test suite in either Cypress or Playwright. Perhaps your application has a button that says Login , and once clicked it’ll open up a pop-up that redirects you to a Google authentication provider. After putting in your username, password, and clicking login, the whole world is merry that you truly are who you say you are.

We want to test our application. To log in we might go through the lacklustre steps delivered above:

JavaScript

test('Lets log in!', async ({page})=> {
	await page.goto('/login')
	await page.getByRole('login-button').click()
	const popupPromise = page.waitForEvent('popup');
	const popup = await popupPromise
	await popup.waitForLoadState();
	await popup.getByRole('username-field').fill('my-username')
	await popup.getByRole('password-field').fill('my-password')
	await popup.getByRole('login-button').click()
	//here, the authentication provider will redirect you back to the site
	await page.getByText("Hey! You're logged in!")
	...
})

These steps are probably outside the scope of what you’re testing where being logged in is a pre-requisite, and slows down the test suite. Furthermore, it’s generally not a recommendation to store a username and password in a working directory.

Another avenue that perhaps takes a better angle is to use signed custom tokens —for example, a JWT . With a test user ID, we can get a Firebase admin instance to sign it and send back a JWT access token. This is much better for creating a general plugin, as it’s provider-agnostic.

How do we go about this? Well, firstly, we’ll need a few ingredients:

A Firebase Service Account
Firebase Project Configurations
A Test User ID

These three components are all that’s needed, and generating the JWT can all be done on Playwright’s Node environment. We use the service account to create an admin Firebase instance, sign the test user’s ID, and receive back a token!

After receiving a JWT we have to try and get the attention of the website to tell us that we truly are who we say we are. In more technical terms, we’d like the automate the signInWithCustomToken process on the browser, such that we can receive back the necessary modifications to our session, and the Firebase instance server-side is updated when a user logs in.

A somewhat successful scheme might involve ignoring using the browser for this, and finding out what exactly gets modified in the window session and, with your user credentials, make those alterations programmatically. Firebase allows you to do this within Node, and returns all the relevant information that you would need to mimic what it does on the clients browser. Plug it into the session storage, refresh the page and you’re in.

In fact, until recently, I thought that was the best way of going about it. It had success for a specific type of Firebase persistence, and there was nothing inherently wrong with it. We were just “signing in” in Node, and passing along the information that Firebase would have done.

However, it dawned on me that there are three types of Firebase persistence. None , Session , and Local storage. If there was an application that used None then the entire scheme might fall flat. None means that when you refresh the page, you’re logged out. We were relying on refreshing the page so that the server-side would recognise a user is logged in from the session storage.

It’s also more future-proof, as we don’t have to forge the session storage ourselves, and instead allow Firebase to do all the heavy lifting for us.

The browser and Node

The reason I shied away from using the Browser as an environment to initialise the app is because I truly did not know how to import modules into the browser. From brief research, it looks like you can do so using the Browserify package. This allows you to bundle your code, alongside any dependencies it has inside one single JavaScript file to be used in a <script> tag, and might be a possible alternative to the current build.

At present, we use the command page.addScriptTag twice, allowing us to add scripts that contain the source to a gstatic website that hosts the Firebase app and auth JavaScript files. A third addScriptTag is included to actually import the relevant modules from these two massive JavaScript files which is attached to the window object. After that, we can finally automate the process in our page.evaluate like so:

Plain Text

// inside a class method that's used as a Fixture

if (this.user) {
            console.log('User already authenticated')
            return
  }
//getToken initialises an admin instance, and generates a new token based on 
//the uid. 
this.token = await getToken(this.serviceAccount, this.options, this.UID)

await this.addFirebaseScript(page) 

 //in the line above we import the JS modules app and auth 
// as firebase and Auth respectively. Typescript hates us for this,
//as it doesn't know we've imported those modules on the lines below.

await page.evaluate(
        async ({ token, config }) => {
          const apps = window.firebase.getApps() 
          const app = apps.length
            ? apps[0]
            : window.firebase.initializeApp(config)
          const auth = window.Auth.getAuth(app)
          await window.Auth.signInWithCustomToken(auth, token)
        },
        { token, config: this.options }
      )
      this.userSet = true

Now that we’ve initialised an app with our specific config, we can sign in with the custom token our admin instance generated within the Playwright Node environment. By signing in within the browser, we guarantee that the response body, and its modifications to the browser, are consistent with how our Firebase app sets its persistence on the server side.

Weaknesses

We import the Firebase app and auth modules from gstatic , which is exactly that — static. It means that when Firebase releases a new copy, we need to engineer a way to change the import from say version 9.6.10 to 9.7.0. As Firebase states , this is an acceptable alternative to installing with npm . However, it begs the question: how can we bump up this value to the most up-to-date Firebase version? In our plugin, we give the user the option to alter this, but currently it defaults to 10.5.0.

Testing our plugin

I created a simple React app for testing purposes, that connects to a Firebase project, with a sign-in-with-popup:

JavaScript

//App.js 
import { initializeApp } from 'firebase/app';
import { signInWithPopup, GoogleAuthProvider, getAuth, signOut } from 'firebase/auth';
import { useState, useEffect } from 'react'

initializeApp(JSON.parse(process.env.REACT_APP_FIREBASE_CONFIG))

const provider = new GoogleAuthProvider()

function App() {
  const auth = getAuth()
  const [user, setUser] = useState(auth.currentUser) //
 
  const handleSignIn = () => {
    signInWithPopup(auth, provider).then((result) => {
      setUser(result.user)
    }) //the regular way - click a button, sign in with a pop-up
  }
  const handleSignOut = () => {
    signOut(auth)
    setUser(null)

  }
  useEffect( //check at the beginning whether we're already authenticated
    () =>
      auth.onAuthStateChanged(async user => { //sets an event handler to setUser once we've authenticated
        if (user) {
          setUser(user) //this will re-render the site! 
        } else {
          setUser(null)
        }
      }),
    []
  )
  
return (
    <div className="App">
      <button onClick={handleSignIn}>Sign in here</button>
      <button onClick={handleSignOut}>Sign out here</button>
      {user ? <h1 >Welcome! You are now logged in</h1> : <h1>You are logged out</h1>}
			//if a user is registered, we should see this text pop up! 
    </div >
  );
}

export default App;

Simply put, our user is a state dependent on the auth instance we have. We set up a useEffect that instantiates an event listener called onAuthStateChanged which will use setUser to reload the page with a ‘You are logged in’ message.

And now that we’ve got a simple website down, let's test our authentication plugin to make sure it works!

JavaScript

test('has title', async ({ page, auth }: { page: Page, auth: any }) => {
  await page.goto('/', { waitUntil: 'networkidle' });
  await auth.login(page) //this connects to a fixture with the code above

  const txt = page.getByText('Welcome! You are now logged in')
  await expect(txt).toBeVisible()
  await auth.logout(page) 
  await expect(txt).not.toBeVisible()
});
//passes!

Lower that playback speed because that was fast! Within around 3 seconds, we’ve logged in, and logged out just by using the added fixture method auth.login ????️ . This video was recorded using the Playwright setting to record the test above. Going into the Playwright report, we can see our login process took just under 0.7s :

Wrapping up

Here we’ve demonstrated how the playwright-firebase plugin can enable developers to log in/out with just one line of code on their Playwright test suite. On top of that, we’ve shown how we can use the modules from Firebase within the browser environment in order to utilise their methods, which is the most appropriate way of authenticating.

Insight, imagination and expertly engineered solutions to accelerate and sustain progress.

Contact