Dr John Sullivan Talent Management Thought Leadership

After hearing brilliant interview answer after answer, you might be shocked to learn that AI wrote those answers. 

Article Descriptors| Recruiting /Interviews – AI generated answers – How to – 6 min read

Yes, it’s true that today, the great interview answers that you are hearing may not be genuine at all. But instead, they may have been provided to this “AI-boosted candidate.” In advance for memorization or in the case of live remote video interviews, the AI software may provide the boosted candidate with their answers live, out of camera view, but in real-time. Yes, the day when interviewers automatically get genuine interview answers without some work may soon be history. Four important steps exist in determining whether an interview answer is real or AI-generated.

Step I – Decide If AI Use By A Candidate Is A Good Or Bad Thing

Begin by realizing that most organizations now automatically disdain a candidate’s use of AI during your job search process. Unfortunately, most organizations fail to tell their candidates about their no-use expectations. Those involved in recruiting should also realize that there is a contrasting perspective about using AI. That contrasting viewpoint suggests that AI use by the candidate should be rated as a neutral or even a positive factor during hiring evaluations.

Most of us are getting to the point where we now expect most new employees to be capable of using AI technology as part of their job. So, one option is to ignore its use and consider it just another part of the candidate’s preparation (just like using spell check on their resumes).

If the decision is based on the recruiter and hiring manager, then it is crucial to recognize that the choice is dependent on the job (or information security). The job may require critical, narrow expertise beyond the capabilities of AI (e.g., math, science, and various edge cases). Another problem is that the hire may overly rely on AI and become complacent. Therefore, experts may be warranted to opt out of AI as well. Recruiters must discern the most effective approach to the use of AI in the work and make that decision with the hiring team. A simple approach is to ask the team if and how AI currently fails them in the job.

Additionally, if your leadership decides against a candidate’s use of AI during hiring. Here are four other AI identification steps that don’t require the use of technology.

Step II – Realize That AI Detection Software Can’t Identify Verbal AI Interview Answers

If you want genuine interview answers, I estimate up to 25% of tech job candidates are already cheating with AI. You should also know that even though candidates are using AI to generate their interview answers. Recruiters can’t respond in kind by using software to identify AI answers. Because interview answers are not in text form, they can’t be scanned with common AI detection software like OriganityAI, unlike cover letters and resumes. So, detecting fake interview answers requires the interviewer to take some proven actions that don’t involve technology. 

Step III – Action Steps Prior To The Interview

Before the interview process even begins, there are six foundation steps that you should execute. They include:

• Be Transparent And Let Applicants Know About Your Allow/Ban Policy – make sure that all applicants /candidates are fully aware of your organization’s policy on AI use during its hiring process. Because proactively informing all applicants (before they apply) about your restrictions. Maybe one of the most powerful actions for discouraging your candidates from even considering its use for getting interview answers. Further, make it clear that you are actively checking for any use of AI throughout the interview part of your process. Using it will be sufficient grounds for dismissal from either the hiring process or the job.
• Learn about how AI can distort your hiring results – although it might sound scary, using AI to generate interview answers is not complicated. Begin by educating yourself about the various ways that AI technology can provide an unfair advantage because it will help create a false image of the candidate who is using it. Those candidate advantages include that they will look more confident, they will answer interview questions faster, and their answers will be more complete. The next learning step is to become familiar with the software your candidates will likely use to generate their interview answers. The most popular ones often include Interview copilot, Huru, and the latest version of ChatGPT. 
• Save some time and determine if they have an AI-generated cover letter – if a candidate will try to bolster their chances using AI-generated material. You can almost guarantee they will use it first to write their cover letter. So, prior to the interview, use standard AI detection software (i.e., GPTZero, OriganityAI, Winstonai) to determine what percentage of a candidate’s cover letter (or resume) has been written with AI. Unfortunately, this AI-hiding software has never been infallible. Now, cloaking (hiding) software like Jasperai can guide candidates in rewriting their AI-generated text. 
• Generate your own “answer bank” for comparison purposes – for each standard interview question you will use. Generate an answer from at least three separate AI content generators. Use your answer bank to become familiar with these most common “canned AI answers.” It will make it easier for you to spot them on the fly whenever a candidate uses them. Also, note that you might need only generate these sample answers for the first 2 to 3 interview questions that you will use. Because if the candidate isn’t using AI for these first couple of questions, they won’t likely be using it at all
• Make a list of common “canned” AI-generated words/phrases – scan through your “answer bank” of canned AI answers. Look for unusual but common words frequently appearing in AI-generated interview answers. Over time, add to your list of AI identifiers as you discover new ones during your interviewing. These unusual but frequently used AI words might include journey, embark, community, and realm. Of course, hearing these words doesn’t guarantee that you are listening to an AI-generated answer. But it should cause you to look deeper into the candidate’s answers.
• Recording your interviews to keep the possibility of a later discovery open – recording all of your interviews not only protects you from most discrimination claims. Keeping a recorded interview record lets the candidate think the company can return to the video later. Using some unknown tool, we can determine if AI at least partially generated the candidate’s answers. Keeping this possibility open in their minds will help to minimize the use of AI-generated interview answers. And if you have the capability and the time to transcribe your video interviews. You can use AI identification software to spot the answers that AI generates.

Step IV – During The Interview, Take These AI Detection Actions

Because there is no standard “one-size-fits-all” approach to detecting AI-generated answers during the actual interview, feel free to pick and choose from among the following 10 “during the interview actions” until you settle on your final set of actions that you will use in every interview.

• Save some time by asking them directly, “Did you use AI?” – as the interview begins. Even though you have banned its use, you should check on this candidate’s compliance. By asking each candidate directly, “Have you used AI in any way in preparing your interview answers?”. Obviously, you should reject those candidates that answer yes.
• A major AI indicator is when interview answers are not customized/personalized – because AI software doesn’t know the context in which its provided answers are needed. A top sign of an AI-generated answer is its lack of customization for this candidate, job, company, and industry. Be on the alert when a candidate’s provided answers would apply to almost anyone at any company. Common personalization areas are usually omitted in AI-generated answers. Include the names of individuals, dates, numbers, and locations. So, if you suspect you are receiving AI answers. You should ask the next few interviewees to include incident-specific factors like numbers, names, or dates in their answers.
• Another major AI indicator is the candidate’s inability to answer follow-up questions – because AI-supplied interview answers are relatively shallow and generic. It makes sense to periodically ask the candidate a follow-up question that requires more details. Alternatively, ask them to reword their answer (because that’s nearly impossible when a candidate relies heavily on AI-generated answers). Or, after hearing an answer that appeared to be AI-influenced, ask them to provide a detailed example.
• Asking nontraditional interview questions will reveal an uneven preparedness – although AI can answer almost any interview question. It is unlikely that any candidate will ask AI to answer any nontraditional interview questions that aren’t even likely to come up. You can identify an AI-aided candidate because they will come across as super prepared for traditional questions. However, they will come across as unprepared for nontraditional interview questions. So, there is a major variation in performance (between standard and nontraditional interview questions). It should be taken as an indicator of AI use.
• Increase your identification effort for jobs that are likely to be filled by candidates who are familiar with AI – in my experience, applicants for jobs whose profession involves familiarity with AI. They are more likely to use AI to generate their interview answers. So, increase your efforts to identify AI answers during interviews for jobs in tech, communications, marketing, and, of course, AI jobs.
• Take it as a major AI indicator when a candidate’s answers to standard questions appear to be memorized – for most interviews (except some remote interviews). AI-generated answers to standard interview questions can only have maximum impact when the candidate memorizes them. So look for indications that a candidate has memorized their answers to standard interview questions. Or be suspicious when the candidate’s answers appear to be overly rehearsed.
• For remote interviews, look for signs that they are reading their answers – and the unique case of the remote interview. Candidates are able to read their interview answers on an off-camera device. So, during these types of interviews, you must watch closely if it appears that the candidate is reading their answers or if they are consistently looking at something off-camera. 
• The ultimate AI revealer question is to give them a real problem to solve – last but certainly not least. Everyone involved in recruiting should realize that you can find great answers to all traditional interview questions online without using artificial intelligence. AI just shortens the best answer discovery process. So, in one specific “problem-solving” competency area, I recommend avoiding standard interview questions of all types. Instead, ask the candidate to solve a real problem during their interview. This can be done by verbally outlining to the candidate a common problem they will likely face if they get the job. Then, ask them to “walk you through the steps” that they would take to discover the problem’s best solution. Consider them an excellent AI-free candidate if they provide a quality answer. Which includes not omitting any major steps and putting their steps in the appropriate order. You can learn more about this most impactful “problem-solving interview question” here.

——————————————

One final action that I do not recommend

• Consider training yourself to spot patterns in AI-produced answers – some in the field of AI detection argue that individuals can be trained to spot consistent patterns in AI-generated narratives. Those patterns might include the repeated use of “unusual words” like tone, structure, personalization, and community. Or the consistent absence of specific information like numbers, dates, names, and places. So, it’s important to note that I don’t recommend this “pattern discovery approach” because I find that it has a low ROI. If you would like to explore this pattern further. This site suggests looking for 9 identifier patterns in any narrative. While another site adds a few additional patterns to look for. Combined, these pattern areas include:

• Too Perfect to Be True.
• Unusually formal tone in text that’s meant to be conversational or casual.
• Overly complex sentence structures.
• Unusual or incorrect wording.
• Text that’s unnecessarily long and wordy.
• Lack of credible sources.
• Repetitive words or phrases.
• Redundant sentences.
• Lack of personal touch.
• Vague statements that don’t provide much information. 
• Weird Idioms and Slang.
• Playing It Safe with the content.
• Wrong Information is provided due to hallucinations.

—————————————————–

If you only do one thing – if your organization doesn’t publicly disclose its resistance to a candidate’s use of AI during their hiring process. Informally meet with your group’s new hires individually after they have become established in their jobs. In strict confidence, ask them directly if and how they used AI during your hiring process. Then, use this information to determine how widespread this practice is among your candidates.

Final Thoughts

Many hiring managers and recruiters have had great success using interviews. They can attribute that success to the fact that historically, most candidates have been completely genuine and mostly honest in the answers that they have provided. However, AI has literally changed everything in business. It would be naïve for those in recruiting to assume that somehow the hiring process would be exempt from this AI influence. So, in my view, there can be no hesitation. Because recruiters and hiring managers need to realize the playing field is no longer level. Today, so many candidates are using AI to write cover letters and resumes and generate interview answers. From now on, if you want to hire a candidate who will perform exactly the same on the job as they did during their interview. You had better quickly learn how to prevent and/or spot AI-generated fake interview answers.

Author’s Note

Please spread these ideas by sharing this article with your team/network and by posting it on your favorite media. 
Also, join the well over 20,000+ that follow or have connected with Dr. Sullivan’s community on LinkedIn. 
And, if you don’t already subscribe to Dr. Sullivan’s Aggressive Talent Management articles, you can do that here.