Do not underestimate the time a candidate will need to complete that offline tech test you gave them

Test estimation is usually not performed by the same person that is submitting its text to the candidate

Tech tests are usually presented to the candidate by HR people, who most of the times don’t have the skills to estimate the duration of the task the candidate is being asked to perform.

A good HR professional will collect that info from a developer already working in the company, possibly the author of the test. Based on that, the HR person will then clear out with the candidate if they are fine to commit to work on the test for X hours.

The problem is that as everyone in the IT industry well knows, we as developers have a natural inclination to underestimate things. There are many personal reasons that can lead to underestimate every-day tasks: peer pressure, arrogance, not thinking things thoroughly enough, external factors like bad or wrong requirements.

In the case of offline assignments there are usually other things that are not taken into consideration because they are too far from the daily experience of a developer working full time for months on a solution that will probably resemble on a much larger scale the project given to the candidate: we could summarize them all under the big hat of “environment setup”.

Install IDE, languages and tools on a “safe” computer

Candidates are asked to work on their tech tests at home.

Of course no one really cares about the exact location where the coding technically takes place. At the same time we like to think candidates will not work on their assignment during their working hours.

We like to imagine our candidates being physically at home, with access to their own computer which is also the hardware they will use to work on the assignment because using their work laptop would not be a very smart idea.

Problem is that very likely on that computer they won’t have any development tool installed or at least not an updated one. Licenses can be another problem.

All their experiments, studies and non-profit pet projects are usually carried over on their work laptop, because that’s something safe to keep there and because on that laptop they will likely have access to licensed tools they couldn’t afford otherwise.

A simple setup of a typical Java web app will require to install >100Mb of JDK, 2-300Mb of IDE, 300Mb for a DBMS like MySQL, 50Mb for a SQL client.

How long do you think it can take to download 1Gb of data, run 4 executables, configure 4 applications?

Choose the right version of one or more libraries and handle incompatibilities

Many projects require to use one or more specific libraries. Sometimes there are even requirements on the version of the library/language to use.

The implementation of the project may then benefit from the addition of other libraries that, for example, may improve the way unit, integration and end-to-end tests are written.

The problem is that quite often a given version of LibraryA turns out to be incompatible with another specific version of LibraryB.

Troubleshooting this kind of issues is usually quite time consuming and involves some fun quality time hoping StackOverflow has the right post about it.

But even if everything runs smoothly, you should still factor in the amount of time required to pick the best library to perform a task.

This time is usually something a developer doesn’t consider because unless they are working on building a new project from scratch, in their “professional” project they will already find a bunch of configurations that possibly someone else spent time dealing with and that now just work.

And it’s not as simple as saying “let’s use exactly what I use at work” because there you may be working with something so old you would feel ashamed to show in your assignment. Or you could be asked to work on something that covers areas not covered by your daily job (e.g. you are asked to make some SOAP calls but at work you only deal with JSON Rest calls).

Deal with versioning, packaging and shipping

Another thing a typical Dev takes for granted is to have a properly configured access to a VCS system, like for example a Bitbutcket account requiring an SSH key properly set. A candidate at home may want to create a new account or to remove embarrassing repos like solutions developed for previous interviews.

Of course history of commits has to be crystal clear. So no branch merged without being stashed before, no commit without a proper message, no commit including 2 groups of changes only slightly related with each other.

A server to perform continuous integration is another thing usually taken for granted. At work you know that whenever you create a new project, it will be enough for you to clone and tweak a bit an existing job to package your solution with all external dependencies solved for you. Assuming you are ever asked to create a new job.

That’s usually not the case at home. Sure, there are tools to automate locally the builds of a project, but again those require to pick the right library, the right version, deal with incompatibilities. On top of that you will also have to make sure all the dependencies are declared correctly, with no redundancy or cycle left, that the build script can run seamlessly on any OS, that the shippable version of the build won’t contain certain artifacts generated during intermediate steps of the build (e.g. test reports).

Then there’s a decent list of tasks usually considered trivial but that can be challenging for someone without the right skills or at least quite time consuming. I’m thinking to the collection of all those basic assets (like the icon for a mobile app or a favicon on a website) or the template to apply to your website that will be usually chosen for you by your Product Owner or designer.

They are tiny but they all add up. There’s very little perceived value in adding them. But not adding them will make you lose big points because of a demonstrated lack of attention for the detail.

To document or not to document?

I know, I know. Code should be self-documenting. It should, but we all know in real life it’s not.

Let’s assume for a second that the assignment is easy enough not to require any documentation at all the be understood.

Would you really accept a code-only project with no readme.MD or any sort .txt explaining the design choices? If coded in a language like Java, would you really be happy to find that all public methods on an interface are not documented? 

Exaggerated test coverage

I hope we are all in agreement saying that a 100% code coverage with automated tests is not only useless but even dangerous.

In our day-to-day real world project we are often open to accept compromises, leaving entire areas untested. We can do that because adding a test for that unlikely corner case would cost too much or because we have a critical deadline to meet.

For some reason, things and metrics adopted change when dealing with assignment. How much more strict would you be in evaluating the path and variable combinations covered when reviewing an offline coding test? Would you require the same level of coverage and precision from your mate during a code review?

Feeling tired

When estimating the duration of an offline test there’s another variable that we should factor in and that I feel is often ignored but that can have a huge impact on the total time spent: it’s very very likely that the candidates will be tired when working on the assignment.

Add to that they will probably work on the assignment when the family is meant to gather for dinner or a sport event of the kids.

Also, not everyone has an appropriate space at home to work in peace on whatever they have. Most will improvise a working station and will learn in that very moment how hard is it to keep focus when immersed in your home environment.

Based on my personal experience, usually candidates will be told “we expect it will take you a couple of hours to complete this coding test”. I’ve never ever heard of an estimate bigger than 4 hours. And that number was shared with a tone of voice suggesting “feel ashamed of yourself if it takes you so long to submit”.

Reconsidering all the points highlighted in this post, how reasonable does an estimate of 2 hours now sound to you?