Skip to content
The AI-First Web Talking point

When AI passes the algorithm test, hiring has to test judgment of AI's work

HackerRank's CEO says interviews must now score how people frame, steer and judge AI output.

W
WebPulse Newsroom
AI-assisted · 2 min read
Share on X LinkedIn
When AI passes the algorithm test, hiring has to test judgment of AI's work

Photo: Sora Shimazaki / Pexels

In brief
  • HackerRank's CEO argued that AI made algorithm puzzles useless in interviews, so hiring must test how well people frame, steer and judge AI output.
  • He said a model that reviews its own output piled on more changes, so a human has to stay in the loop.

Hiring should now test whether a candidate can judge AI's work, not only produce code. That was the argument of Vivek Ravisankar, HackerRank's CEO, on the Software Engineering Daily podcast. He said AI made classic algorithm puzzles useless in interviews. In his view, the skill being hired moves from making to judging.

What was said

Ravisankar told host Josh Goldberg that AI "was able to one shot all of these different algorithm questions and made them pretty much useless in an interview setting". Every company, his own included, had to rethink hiring.

HackerRank now sets real tasks on code repositories. It cares less about whether the code is correct. It cares more about critical thinking, judgment and AI fluency.

Its AI interviewer scores three things. Did the candidate do the work? How well did they use AI? Do they understand what they built? For the second, the system splits the session into episodes. Each episode gets a grade for how well the person framed the problem, judged the output or steered the AI toward an answer. Those grades roll into one overall grade. A follow-up of about 15 minutes then asks about choices and new constraints. All three results merge into one score.

Why it matters

Ravisankar gave a warning from daily work. An engineer told him he had a flood of pull requests (proposed code changes), some touching hundreds of files. When the engineer asked the same model to review them, it returned about 50 more changes. Ravisankar linked this to a Shopify figure's term, slop grenade, and said at some level a human needs to be in the loop. He added: "I think we've changed the way that we build software, but I don't think we've changed the way that we review software."

Our reading: in that example, the reviewer was the same model that made the changes. It produced more changes, not a verdict. Review still needs a person who can frame the problem, judge the result and steer. A few questions follow for hiring managers and training leads. Can a candidate walk you through their AI session and say what they rejected? Can they explain why one option beat another? Do your juniors get practice reviewing AI output, or only writing code?

The other side

Ravisankar sells interview software, so this is a vendor's view. He said most companies still test code correctness today, so the shift he describes is not yet the norm.

He did not say fundamentals stop mattering. Using a pilot analogy, he said you still want a pilot who knows aeronautical engineering even if AI flies most of the trip. His first round for that case is a no-AI test of basics.

Goldberg asked how an AI interviewer can be trusted, given its biases and uneven results. Ravisankar pointed to candidate ratings of 4.6 to 4.7 out of 5, higher than for take-homes. Those ratings show candidate satisfaction, not hiring quality. He also said hiring is a regulated area. The excerpts we have do not show the rest of his answer.

Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.

The conversation this talking point comes from