Improve Automated Test Coverage Using GitHub Copilot

If you’re aiming to bolster the test coverage of your application quickly, then GitHub’s AI Assistant, Copilot, can help you bridge the gap between where you are and where you want to be.

Considering that GitHub’s research has shown that engineers using GitHub Copilot can complete their tasks 55% faster and engineers can merge code twice as quickly, it makes sense that Copilot is the most adopted AI developer tool!

Watch the video to learn how to make the most of GitHub’s AI assistant, Copilot.

In this post, we'll elaborate on the above video and explore how to harness Copilot's capabilities for backfilling tests, refactoring existing code, and crafting new features with ease. I will use React’s Tic-Tac-Toe Tutorial as the example codebase because it’s well-known, relatively straightforward, and–most importantly–has ZERO test coverage.

You can follow along yourself by downloading the Github repo using this link to our GitHub repository. However, if you choose to follow along, you might get different results given the unpredictable and open-ended nature of GenAI (generative artificial intelligence).

Getting Started

We’ll start at the “Completing the game” section of the Tic-Tac-Toe Tutorial, where a Square and Board component are both written and the app responds to clicks in the browser.

Another important part of getting set up for this article is installing GitHub Copilot in your IDE (integrated development environment). If you’re not subscribed to Copilot yet, as of writing this article, there is a 30-day free trial that you can use. After subscribing to Copilot, you can install it into your IDE using the instructions linked here. I’ll use JetBrains WebStorm, but there is also support for Azure Data Studio, Vim, Visual Studio, and Visual Studio Code.

Writing Tests with Inline Code Suggestions

Let’s start this experiment by creating a test file for the Board component called Board.spec.js that we’ll fill with a blend of our code and code suggested by Copilot.

The first step is giving Copilot some context as to what we’re doing, which we’ll do by adding the following comment at the top of the file.

Board.test.js

// Test file for the Board component

After that, hit Enter a few times and you’ll start to see Copilot’s inline suggestions (if you haven’t seen them already).

In my case, Copilot suggested a few imports that it thinks would be useful. I’ve also added my imports (and labeled them as such in the code block below) so that Copilot uses the tools that I want to work with.

Board.spec.js

// Test file for the Board component

// Imports that Copilot suggested
import React from 'react';
import { render, screen } from '@testing-library/react';
import Board from './Board';

// Imports that I added myself
import { describe, expect, it } from "@jest/globals";

Using just that context, Copilot suggested the following tests:

Board.spec.js

describe('Board', () => {
    it('renders a 3x3 grid of squares', () => {
        render(<Board />);
        const squares = screen.getAllByRole('button');
        expect(squares).toHaveLength(9);
    });
    it('renders nine squares with the correct values', () => {
        render(<Board />);
        const squares = screen.getAllByRole('button');
        squares.forEach((square) => {
           expect(square).toHaveTextContent('');
       });
    it('renders three groups of three Squares', () => {  // This test name was manually inserted
        render(<Board />);
        const groups = screen.getAllByRole('group');
        expect(groups).toHaveLength(3);
        groups.forEach((group) => {
            const squares = group.querySelectorAll('.square');
            expect(squares).toHaveLength(3);
        });
    })
}

While the code Copilot suggested is rife with errors and misstatements (most glaringly, the missing ) at the end of the file), it provides a great starting point for us to layer on our expertise. After some refining, here is the first iteration of the test file:

Board.spec.js

describe('Board', () => {
  it('renders nine squares', () => {
     render(<Board/>);
     const squares = screen.getAllByRole('button');
     expect(squares).toHaveLength(9);
     squares.forEach((square) => {
         expect(square).toHaveTextContent('');
     });
  });
  it('renders three groups of three Squares', () => {
     render(<Board/>);
     const groups = screen.getAllByRole('group');
     expect(groups).toHaveLength(3);
     groups.forEach((group) => {
         expect(group.children).toHaveLength(3);
     });
  });
});

After finessing the file and narrating my changes, I was able to generate an initial test file for the Board component in 2 minutes and 30 seconds, which is much faster than I could have written by myself.

Writing Tests with Copilot Chat

Let’s take a different approach for the other test file, Square.spec.js. This time we’ll use Copilot Chat to generate the code.

The interface of Copilot Chat should feel very familiar since it resembles almost every other messaging app. I’m going to send it a message prompting it to write me tests for the Square component. Here is a screenshot of our conversation:

After only 30 more seconds, we’ve doubled the number of working test files in our project! There are a few aspects of this conversation that are worth highlighting. First, like other LLMs (large language models), the prompt was in plain English; there is no additional overhead in learning specific ways to speak to the chatbot to extract what you want. Second, Copilot Chat cites its references at the end of its message. Copilot Chat referenced the other test file that we wrote earlier and the two component-files in our application. Judging from the results, Copilot did the work of finding relevant files, extracting the relevant styles and patterns, and applying those forward to the new test suite that it generated.

Those tests passed my review and they ran successfully, so I’ll accept them and add them to the codebase. So far, we’ve tried both of Copilot’s interfaces and generated a test file with each. In less than four minutes, our application code is completely covered by tests.

Refactoring Tests

Now that our application code is fully covered by tests, let’s move forward in the tutorial. The next task is to “Lift Up” state from the Square component to the Board component, which will allow us to build game logic and continue with the tutorial.

While the tutorial offers one approach to lifting state, I’d like to move forward using Test-Driven Development (TDD), which is the preferred approach at Crafted. Let’s use Copilot Chat to see if it will refactor our test suite for us. 

Since we’ve moved on to a new task, I started a new conversation with Copilot Chat to clear its context, prevent it from using old code that I hope to pivot away from, and reduce the likelihood of Copilot hallucinating.

I gave Copilot Chat the following prompt:

I want to refactor my application to lift up state from the Square component to the Board component. I'd like to start this process by updating the test files first. Please refactor the two test files Square.spec.js and Board.spec.js accordingly.

Copilot’s suggestion for Square.spec.js and Board.spec.js were, respectively:

Square.spec.js

// Test file for Square component

import React from 'react';
import { render, fireEvent, screen } from '@testing-library/react';
import { Square } from './Square';
import { describe, expect, it } from "@jest/globals";

describe('Square', () => {
   it('renders a square with no text', () => {
       render(<Square value={null} onClick={() => {}} />);
       const square = screen.getByRole('button');
       expect(square).toBeInTheDocument();
       expect(square).toHaveTextContent('');
   });

   it('calls onClick when clicked', () => {
       const handleClick = jest.fn();
       render(<Square value={null} onClick={handleClick} />);
       const square = screen.getByRole('button');
       fireEvent.click(square);
       expect(handleClick).toHaveBeenCalledTimes(1);
   });
});

Board.spec.js

// Test file for Board component

import React from 'react';
import { render, fireEvent, screen } from '@testing-library/react';
import Board from './Board';
import { describe, expect, it } from "@jest/globals";

describe('Board', () => {
   it('renders nine squares', () => {
       render(<Board />);
       const squares = screen.getAllByRole('button');
       expect(squares).toHaveLength(9);
       squares.forEach((square) => {
           expect(square).toHaveTextContent('');
       });
   });

   it('updates the square value to "X" when a square is clicked', () => {
       render(<Board />);
       const squares = screen.getAllByRole('button');
       fireEvent.click(squares[0]);
       expect(squares[0]).toHaveTextContent('X');
   });
});

The tests for the Square component look great, especially considering that Copilot updated the implementation of Square from taking no props to taking both a value and an onClick callback function.

The tests in Board.spec.js also look good, but interestingly, the Copilot Chat suggestion removed our test asserting that it “renders three groups of three Squares”. I’m going to take the new suggested tests and that single old test and move on.

Refactoring Code

When I run the new test suite, we have four passing tests and one failing test! Following the "Red, Green, Refactor" pattern defined by TDD, let’s look at the failing test and get that back to green. Of course, we’ll use Copilot to do just that. 

Since only one test in Square.spec.js is failing, we’ll start there. In Copilot Chat, I’ll give it the prompt, “Update Square.js so that it passes all of the tests in Square.spec.js.” 

I don’t mean to "yada, yada, yada," the beauty and complexity of AI, but it is starting to feel a bit more familiar. Here is what Copilot suggested for the implementation of Square.js:

Square.js

export function Square({ value, onClick }) {
   return (
       <button
           className="square"
           onClick={onClick}
       >
           {value}
       </button>
   );
}

Short, clean, to the point. I like that code for a component that just had state lifted out of it. 

Now, let’s tackle the Board component. The prompt that I gave Copilot Chat was nearly identical to the last one I wrote: Update Board.js so that it passes all of the tests in Board.spec.js. That prompt yielded the following code:

Board.js

import React, { useState } from 'react';
import { Square } from './Square';

export default function Board() {
   const initialSquares = Array(9).fill(null);
   const [squares, setSquares] = useState(initialSquares);

   const handleClick = (i) => {
       const newSquares = squares.slice();
       newSquares[i] = 'X';
       setSquares(newSquares);
   };

   return (
       <>
           {[0, 1, 2].map((row) => (
               <div className="board-row" role="group" key={row}>
                   {[0, 1, 2].map((col) => {
                       const index = row * 3 + col;
                       return (
                           <Square
                               value={squares[index]}
                               onClick={() => handleClick(index)}
                               key={index}
                           >/Square>
                       );
                   })
               </div>
           )})
       </>
   );
}

While this code presumably gets all of the code to pass, I’m not thrilled with the suggestion. Copilot took a very readable implementation of a Tic-Tac-Toe board and replaced it with two for loops. Readability matters more than efficiency for this project, so I prefer to keep editing this code. Luckily, I’m in the middle of a conversation with a very helpful resource. I responded with the message: Can you change the return statement in Board so that it looks more similar to the current version.

A few seconds later, I was given the following code block:

Board.js

import React, { useState } from 'react';
import { Square } from './Square';

export default function Board() {
    const initialSquares = Array(9).fill(null);
    const [squares, setSquares] = useState(initialSquares);

    const handleClick = (i) => {
        const newSquares = squares.slice();
        newSquares[i] = 'X';
        setSquares(newSquares);
    };

    return (
        <>
            <div className="board-row" role="group">
                <Square value={squares[0] onClick={() => handleClick(0)} />
                <Square value={squares[1] onClick={() => handleClick(1)} />
                <Square value={squares[2] onClick={() => handleClick(2)} />
            </div>
            <div className="board-row" role="group">
                <Square value={squares[3] onClick={() => handleClick(3)} />
                <Square value={squares[4] onClick={() => handleClick(4)} />
                <Square value={squares[5] onClick={() => handleClick(5)} />
            </div>
            <div className="board-row" role="group">
                <Square value={squares[6] onClick={() => handleClick(6)} />
                <Square value={squares[7] onClick={() => handleClick(7)} />
                <Square value={squares[8] onClick={() => handleClick(8)} />
            </div>
        </>
    );
}

Copilot adhered to my suggestion, but I still felt like we could do better. The code is more immediately digestible than the last iteration, but it doesn’t adhere to the principle of Don’t Repeat Yourself (DRY), which might cause us headaches in the future. I asked Copilot Chat to DRY up the code with this message: Can you refactor the call to Square into a helper function to reduce the amount of repetition in the file?

The next suggested iteration of Board’s implementation is:

Board.js

import React, { useState } from 'react';
import { Square } from './Square';

export default function Board() {
   const initialSquares = Array(9).fill(null);
   const [squares, setSquares] = useState(initialSquares);

   const handleClick = (i) => {
       const newSquares = squares.slice();
       newSquares[i] = 'X';
       setSquares(newSquares);
   };

   const renderSquare = (i) => {
       return <Square value={squares[i]] onClick={() => handleClick(i)} />
   };

   return (
       <>
           <div className="board-row" role="group">
               {renderSquare(0)}
               {renderSquare(1)}
               {renderSquare(2)}
           </div>
           <div className="board-row" role="group">
               {renderSquare(3)}
               {renderSquare(4)}
               {renderSquare(5)}
           </div>
           <div className="board-row" role="group">
               {renderSquare(6)}
               {renderSquare(7)}
               {renderSquare(8)}
           </div>
       </>
   );
}

After three prompts, I’ve arrived at code that I’m happy with. While the code is not completely DRY, it is well implemented, immediately readable, and leaves me in a good place to keep moving forward with my project.

Conclusion

Copilot is a fantastic tool that has the potential to drastically increase your productivity. Copilot will suggest lines of code in your IDE, but the Copilot Chat allows you to leverage the AI assistant for more complex changes like refactoring. While Copilot never seems to run out of suggestions, the quality of the result is still up to the engineer. 

GitHub Copilot is just one of the many tools we use at Crafted to write effective, performant, and maintainable code that helps you move quickly both today and in the future. Interested in partnering with our expert engineers to improve your own code and best practices? Reach out today!