To complement their industry-leading peer review code challenges, the team at Geektastic have been testing a new product with their software developer community over the past nine months: Express Challenges.
The team set about creating a new type of challenge that was shorter than the 2-3 hour peer review take-home challenges and more cost-effective to use at scale.
Not every software developer has two hours to spend on a code challenge, and hiring teams don’t want to go to the expense of having every candidate thoroughly assessed with a take-home challenge, so the team tested a number of shorter, automated mechanics and felt that short multiple-choice challenges were a good fit.
The problem with automated testing is it is rarely rewarding for the candidates, especially if you don’t get many of the questions right. So the team focussed heavily on user experience, building in explanations. These explanations are surfaced alongside the answers so users come away from each question they don’t get right feeling they have learned something.
Because the Geektastic software developer community create and share express challenges via the platform, the team felt that it would be important to have a feedback loop for the creator to let them know whether they had created a good challenge. Each time a software developer takes a challenge they can vote on whether they found it educational and whether they would recommend it. The results have been overwhelmingly positive. Question creators aren’t rewarded for creating difficult or easy questions, but for creating educational and rewarding challenges.
To allow software developers to see how they compare to other software developers taking the same challenges the dev team at Geektastic introduced an Elo Ranking system. Elo is used predominantly in chess but also as a ranking system for video games competitions, major league baseball, American football and basketball.
Of course, the real acid test is whether Express Challenges are an indicator of someone’s coding ability. Rick Brownlow, co-founder of Geektastic has seen a strong correlation between the software developers that have performed well in peer review and those that have high Elo scores in Express. Whilst the true test of a software software developer’s skills should be determined by a peer review challenge, these express challenges allow software developers to take multiple challenges over time, and as their skills improve so does their Elo rating.
If you are a software developer looking to test your skills and see how you compare to software developers from top companies like Amazon and Google, register at Geektastic.
If you are from a hiring team looking for a solution to help you find the best candidates please drop the team at Geektastic an email and they’ll be in touch.