An encounter with a mechanical developer

I’m using GitHub copilot for quite some time now, and it was my faithful companion during all the coding sessions, firefighting, bug hunting …

Initially I was very curious about this tool, and I read many articles written about it. To be honest, I was on the defensive whenever we discussed about it, as if it posed some existential threat.

Before trying it, I was under the impression that it is some sort of magic autocompletion engine that can spit large chunks of code to implement a feature from a comment ! Now after using it in all my projects, I can say that the reality couldn’t be further from the truth.

The way I think about it now is that it is a mediocre developer but with superhuman pattern recognition skills.

What It Is and What It Is Not

At first I was trying to insert lengthy comments that describe my intent hoping that copilot will generate a class or service that could implement the my description. Those first attempts proved unsuccessful and the generated code was either incorrect or even not compiling.

It keeps making the same trivial errors, like using a non existing class attribute or method, etc. Bellow is an example of a comment that caused copilot to fail:

// Test whether the graph is cyclic or not.
// This is an iterative function.
@Override
public boolean isCyclicIterative() {
  int i;
  boolean[] visited = new boolean[adjacencyList.size()];

  for (i = 0; i < adjacencyList.size(); i++)
     visited[i] = false;

  for (i = 0; i < adjacencyList.size(); i++) {
     if (isCyclicIterative_(i, i, visited))
        return true;
  }

  return false;
}

Copilot copied my recursive function code, and it take care of renaming all the variables, but it still a recursive one !

A formidable pattern-matching machine

After my first attempts, I forgot about Copilot and returned to my previous tasks. To my surprise, this proved to be the best strategy to work with Copilot. For example, when I was working on refactoring my code, I was able to use Copilot to automate tedious repetitive pattern replacements. Here an example of what I was able to do:

CREATE TABLE sf_transaction (
    LIKE public.sf_transaction_old INCLUDING ALL EXCLUDING INDEXES
) PARTITION BY RANGE (operation_date);

CREATE TABLE sf_transaction_y2017 PARTITION OF sf_transaction FOR
VALUES
FROM
    ('2017-01-01') TO ('2018-01-01');

CREATE TABLE sf_transaction_y2018 PARTITION OF sf_transaction FOR
VALUES
FROM
    ('2018-01-01') TO ('2019-01-01');

The following code was generated by Copilot:

CREATE TABLE sf_transaction_y2019 PARTITION OF sf_transaction FOR
VALUES
FROM
    ('2019-01-01') TO ('2020-01-01');

What is missing

Copilot is still missing some features that I would like to have. For example, some of the tedious tasks that I would like to automate are noy just a matter of generating the code, but also of creating a sub-project, a class or service that can implement the task.