In the past, the only way to assess Azure cloud engineering skills in the hiring process was through multiple choice or whiteboard questions. But these question types don’t measure whether candidates can complete real cloud engineering tasks.

It’s time to change the way you hire Azure cloud engineers. 

Enter our latest feature: Azure hands-on questions. This addition to the HackerRank library lets you evaluate candidates’ real-world skills directly in the Azure portal, ensuring that you find the right fit for your team.

Why hands-on questions are important

Traditional interviews often rely on theoretical knowledge. But theoretical knowledge is exactly that—theoretical. It can fall short in predicting a candidate’s actual job performance—especially in cloud engineering roles. Hands-on questions bridge this gap by providing a practical assessment of real-world cloud skills.

Real-world cloud skills assessments

With our Azure hands-on questions, candidates can show off their abilities in a controlled environment. They won’t just answer multiple choice questions; they’ll roll up their sleeves to solve real challenges in the Azure portal. 

This hands-on approach ensures that you’re not just assessing theoretical knowledge but evaluating their capacity to apply what they know in a practical setting.

How it works

So, how do the Azure hands-on questions function?

1. Login to Azure portal: Candidates will log into a real Azure Portal using a temporary account provisioned by HackerRank. This provisioning happens automatically when the candidate takes the assessment. 
2. Complete tasks: During the assessment, candidates tackle a range of hands-on questions, like setting up a virtual network or configuring network security groups.
3. Automatic scoring: Once candidates finish their assessment, their results are automatically scored and logged in the test report. This streamlines the evaluation process while ensuring that you have clear, quantifiable results to guide your hiring decisions.

New library questions

Our library now includes a diverse range of Azure questions across varying levels of difficulty. A few examples of the real-world challenges candidates might face with our new test questions include:

• Setting up a virtual network: Candidates can showcase their ability to create and configure virtual networks that meet specific requirements.
• Configuring network security groups: This task assesses candidates’ understanding of cloud security and networking principles.
• Hosting a website using Azure web apps: This practical exercise evaluates both technical skills and understanding of web application hosting.

With these hands-on challenges, candidates will not only prove their skills, but also give you insights into their problem-solving approach and creativity.

Wrapping up

Incorporating Azure hands-on questions into your hiring process helps you better select candidates who are capable of tackling challenges in a real cloud environment. By evaluating practical skills, you can better predict job success, leading to more effective hiring decisions.

Say goodbye to those Azure multiple choice questions. Embrace the future of hiring with Azure hands-on questions and ensure you stock your team with the best cloud engineering talent. 

Request a demo today to see Azure questions in action.

The post Assess Real Cloud Skills in Real Azure Environments appeared first on HackerRank Blog.

At HackerRank, our mission is to change the world to value skills over pedigree. We believe every developer, regardless of background or location, deserves a fair, equal opportunity to showcase their abilities. Our commitment to maintaining the integrity of the entire hiring process ensures that every candidate is evaluated solely on their true skills. Nothing more, nothing less.

What does “Fair” mean at HackerRank?

For us, ‘Fair’ means ensuring that every candidate gets an unbiased chance to showcase their skills. Whether you’re coding live or solving complex algorithms, fairness is at the heart of every assessment. At HackerRank we embrace AI and encourage using it appropriately. In our assessments, AI tools are flagged if their use goes beyond what’s allowed, ensuring that the evaluation reflects genuine skills.

Preserving integrity and protecting fairness

The online nature of hiring assessments brings with it potential challenges, such as impersonation and unauthorized aid. But with HackerRank, we mitigate these risks through advanced tools and processes. Here’s how.

Monitoring suspicious environments

Physical surroundings are a common way candidates might try to gain an unfair edge, whether by using another device or having someone nearby offer help. HackerRank’s tools like Multiple monitor detection and Image proctoring keep an eye on the surroundings to ensure the integrity of the test.

• Multiple monitor detection: This feature spots if an external monitor is connected, helping us ensure the candidate stays focused on their work.
• Image proctoring: We capture images of the candidate and their environment during the test, giving hiring teams the context they need to flag suspicious activity.
• Image analysis: Through facial recognition and environmental checks, our algorithms detect any anomalies, like extra people or devices, and flag them for review.

Impersonation prevention

No one wants to hire the wrong person because someone else took the test for them. HackerRank ensures that the right person is taking the test by using Photo Identification and Facial Recognition throughout the assessment.

• Photo Identification: Before the test, we capture a photo of the candidate to establish their identity. This is checked periodically during the test to prevent impersonation.
• Image Proctoring: Capturing images at regular intervals helps detect any changes in appearance or unauthorized help during the assessment.

Fairness in the face of AI: Detecting AI usage

The rise of AI tools like ChatGPT has added a new dimension to assessment integrity. While AI can assist in coding, it can also undermine the fairness of tests if misused. HackerRank’s latest Plagiarism Detection ensures that candidates are evaluated on their true abilities without undue AI interference.

• AI-Powered plagiarism detection: Analyzes patterns in code submissions to detect unauthorized AI usage or copying. This keeps the playing field fair for everyone.
• Copy-Paste tracking: Detects when code is pasted from an external source, helping hiring teams assess if candidates are relying too heavily on pre-written solutions.
• Tab switch proctoring: Monitoring when candidates switch tabs helps us identify if they’re seeking external help or using unauthorized resources during the assessment.

Preventing test content leaks

Content leaks can give candidates an unfair advantage by exposing questions or solutions beforehand. HackerRank uses a variety of tools to combat this:

• Shuffled sections & questions: We randomize the order of questions for each test-taker, reducing the risk of content being shared.
• Watermarking: Each question carries a watermark tied to the candidate’s email, deterring them from sharing the test content.
• Hiding question labels: We hide the titles of questions to prevent candidates from easily searching for answers online.
• AI solvable question labels: Our library questions are labeled according to their potential solvability by AI assistants. Armed with this information, hiring teams can filter out questions that may be susceptible to AI-generated solutions.

A balanced approach to AI in Assessments

HackerRank doesn’t reject AI; instead, we embrace it with balance. If AI usage isn’t intended, our system flags it ensuring that assessments are genuine and reflective of a candidate’s real abilities. This approach is designed to ensure that the hiring process remains transparent, fair, and focused on actual skills.

Leveling the playing field for everyone

Fairness isn’t just about preventing shortcuts. It’s about making sure every candidate, regardless of where they’re from or what tools they have, gets the same opportunity to shine. By upholding the highest standards of integrity, we help companies find the right talent based on skill, not external advantages.

If you’re ready to see how HackerRank can help your team uphold the highest standards of fairness and integrity, reach out to us for a demo. Let’s build a more equitable future for technical hiring, together.

Frequently Asked Questions

Can your plagiarism detection system detect AI-generated code?
Yes, our system is designed to flag suspicious use of AI tools, ensuring that assessments remain genuine and reflective of the candidate’s skills.

Does your plagiarism detection automatically disqualify candidates?
No. Our system flags potential cases of plagiarism, but the final decision lies with the hiring team, who can review the flagged incidents in detail.

Who can I contact if I have more questions?
For more information on how we maintain integrity in assessments, feel free to reach out to your customer success manager or contact us at support@hackerrank.com.

The post Maintaining a Level Playing Field: HackerRank’s Commitment to Assessment Integrity appeared first on HackerRank Blog.

Being a SaaS platform admin isn’t easy. Defining the right level of user access can be challenging when you start factoring in numerous users and complex internal workflows. 

In addition, your platform may not give you enough customization to define what your users can and can’t do, which can lead to users being given a role with the wrong amount of access. 

Recognizing this need for flexibility and efficiency, we’re always working to make the lives of admins easier. That’s why we’re excited to unveil flexible user roles.

Gain Granular Control

Now, you can control granular capabilities for your HackerRank users and fine-tune access across 30 different entitlements. 

These entitlements define whether users can take specific actions, such as editing tests, creating interviews, sending test invites, and more. 

Adjust Access at the Right Level

Need to adjust settings for everyone in a certain role? Just adjust capabilities at the user role level. For example, if you toggle off “Edit Tests” at the developer role level, you can disable your developers’ ability to edit tests.

You can also adjust entitlements at the individual level. If you don’t want all of your developers to have the ability to edit tests, but you want a specific user to have that ability, you can toggle on the ability to edit tests for that individual user.

Opt in to Flexible User Roles Today

Want this granular control for yourself, simply contact our support team at support@hackerrank.com or your dedicated account manager to opt in. For more detailed information, refer to our comprehensive support article.

The post Introducing Flexible User Roles: Right Size the Access for Your Users appeared first on HackerRank Blog.

The new AI-powered solutions, SkillUp and Engage, help organizations mobilize developer talent and accelerate tech hiring.

HackerRank is excited to share the launch of two groundbreaking solutions, SkillUp and Engage. These new additions to our Developer Skills Platform deliver more impact to help our customers attract, hire, upskill and mobilize their technical talent.

SkillUp analyzes existing developer skill proficiency, compares the results with target role proficiency, and then creates a personalized learning plan to build skills. Engage leverages AI to assist companies in creating and conducting real-world hackathons that are specific to their industry to attract and nurture top developer talent. The two solutions leverage the latest advancements in AI and help companies become GenAI ready.

We unveiled these new solutions during HackerRank’s AI Day Virtual Summit. The event featured keynote speakers including:

• Josh Bersin, HR tech visionary 
• Vinod Khosla, Co-founder of Sun Microsystems and the founder of Khosla Ventures
• Beyang Liu, Sourcegraph Co-founder & CTO
• Vivek Ravisankar, HackerRank Co-founder & CEO

“AI is changing software development rapidly,” said Vivek Ravisankar, Co-founder and CEO of HackerRank, on the impact of SkillUp and Engage. “Copilots are now an integral part of how developers build software. This means it’s going to change the entire developer lifecycle – the way you learn to code, how you get hired & how you upskill. Our two new products are helping companies attract & upskill developers in an AI-first world.”

“Every company we talk with is focused on building AI skills, but the domain is moving very quickly,” said Josh Bersin, Global Industry Analyst. “As the company who pioneered skills assessment for hiring, I am very excited to see HackerRank launch a solution for AI skills development. I believe SkillUp and Engage could transform this market and help companies truly credential their AI skills at this critical moment in the technology industry.

“As a customer of HackerRank, CodePath is excited about the launch of SkillUp and Engage as part of AI Day,” said Tim Lee, Chief Learning Officer and Founder of CodePath. “Through our partnership with HackerRank, we have had the opportunity to train the skills of thousands of developers and validate the skills that are being truly learned by using HackerRank’s assessments. The launch of SkillUp and Engage allows organizations like CodePath to utilize HackerRank badging and certifications without having to build and maintain their own system.”

Introducing SkillUp: Building Developer Skills for Productivity & Growth

SkillUp helps companies mobilize their internal talent to become GenAI ready. SkillUp analyzes existing developer skill proficiency and compares that with target role proficiency that informs a personalized learning plan to build skills for current and future success.  With the support of an AI Tutor, SkillUp accelerates hands-on learning that helps developers grasp new skills and technologies and apply those skills to solving real-world problems. 

Organizations can now have a strong understanding of their developers’ skills and help devise strategies to close the skills gap. The product is designed to be developer-first to empower them to be the best versions of themselves. 

Introducing Engage: Showcasing Your Tech Brand to Accelerate Hiring

Developers want to know what kind of challenges a company is working on. Job descriptions do a poor job of it. 

Engage leverages AI to assist companies in creating and conducting real-world hackathons that are specific to their industry/company to attract and nurture top developer talent. By entering their event goals, themes, target audience, and brand tone, AI creates all the elements required for a successful hiring event. With a library of real-world challenges, developers who participate experience, first-hand, the type of challenges they would face every day at the hiring company. 

This innovative solution not only showcases a company’s tech talent brand but also builds meaningful candidate relationships that can accelerate hiring now and in the future.

Who Are We? 

HackerRank, the Developer Skills Company, pioneered and leads the market for developer skills with over 2500 customers and a community of over 24 million developers. Companies trust HackerRank to help them set up a skills strategy, showcase their tech brand to developers, implement a skills based hiring process and ultimately upskill and certify employee skills…all driven by AI.

The post HackerRank Launches Two New Solutions to Mobilize Developer Talent and Accelerate Tech Hiring appeared first on HackerRank Blog.

What’s going on?

Since ChatGPT came onto the scene in late 2022, test after test has proven vulnerable to the wiles of generative AI. The initial GPT-3.5 model was impressive enough, and the more advanced GPT-4 has shown an even greater proficiency for test-taking. Name a large, well-known test, and ChatGPT has probably passed it. In addition to bar exams, SATs, and AP exams, ChatGPT has also passed 9 out of 12 AWS certification exams and Google’s L3 engineer coding interview. 

At HackerRank, we’ve seen firsthand how AI can bypass MOSS code similarity, the industry standard for coding plagiarism detection. 

All of these sudden vulnerabilities can seem scary for those administering tests. How can you trust the answers you’re getting? If your tests rely heavily on multiple choice questions, which are uniquely vulnerable to large language models, how can you revise test content to be more AI resistant?

These developments are worrying for test-takers, as well. If you’re taking a test in good faith, how can you be sure you’re getting a fair shake? Interviewing is stressful enough without having to wonder if other candidates are seeking an AI-powered advantage. Developers deserve the peace of mind that they’re getting a fair shot to showcase their skills. 

What’s our stance?

At HackerRank, we’ve done extensive testing to understand how AI can disrupt assessments, and we’ve found that AI’s performance is intrinsically linked with question complexity. It handles simple questions easily and efficiently, finds questions of medium difficulty challenging, and struggles with complex problems. This pattern parallels most candidates’ performance. 

However, creating increasingly intricate questions to outwit AI isn’t a sustainable solution. Sure, it’s appealing at first, but it’s counterproductive for a few reasons. 

• First, this could potentially compromise the core value of online assessments, weakening the quality of talent evaluation. More complex questions don’t automatically translate into better signals into a candidate’s skills. They take longer to answer, which translates into either longer assessments, or fewer questions (and fewer signals to evaluate). 
• Second, it would certainly degrade the candidate experience by focusing on frustrating AI rather than on giving developers a chance to showcase their skills. Losing sight of the developer experience tends to diminish that experience, which could result in more candidates dropping out of the pipeline. 
• Third, it would set up a game of perpetual leapfrog as more advanced AI models solve more complex problems, and even more complex problems are created to trip up more advanced AI. 

Instead, our focus remains on upholding the integrity of the assessment process, and thereby ensuring that every candidate’s skills are evaluated fairly and reliably. 

Introducing our new AI solvability indicator

Upholding integrity means being realistic—and transparent. This means acknowledging that there are assessment questions that AI can solve. And it means alerting you when that is the case, so you can make informed decisions about the content of your assessments. 

That is why we are introducing an AI solvability indicator. 

This indicator operates on a combination of two criteria. 

1. Whether or not a question can be fully solved by AI.
2. Whether or not that solution is picked up by our AI-powered plagiarism detection. 

If a question is not solvable by AI, it does not get flagged. Likewise, if a question is solvable, but the answer triggers our plagiarism detection model, it does not get flagged. The question may be solvable, but plagiarism detection ensures that the integrity of the assessment is protected. 

If a question is solvable by AI and the solution evades plagiarism detection, it will get flagged as AI Solvable: Yes. Generally, these questions are simple enough that the answers don’t generate enough signals for plagiarism detection to be fully effective. 

Questions flagged as AI solvable will be removed from certified assessments, but may still appear in custom assessments, particularly if those assessments have not been updated in some time. 

If you’re browsing through questions, you can also select to hide all AI-solvable questions, just as you can hide all leaked questions. 

What else is HackerRank doing?

Beyond the transparency of the AI solvability indicator, we are building in measures to actively ensure assessment integrity. These include: 

• AI-powered plagiarism detection. Our industry-first, state-of-the-art plagiarism detection system analyzes dozens of signals to detect certain out-of-bounds behavior. With an incredible 93% accuracy rate, our system repeatedly detects ChatGPT-generated solutions, even when they’re typed in by hand, and even when they easily bypass standard detection methods. 
• Certified assessments. Let us handle assessment maintenance. Our certified assessments are out-of-the-box tests curated and maintained by HackerRank experts. We take on all the upkeep, including keeping content current and flagging and replacing any leaked or AI-solvable questions. 
• Expanded question types. We’re expanding question types with formats and structures that are more resistant to AI solutions, such as projects and code repositories. These have the added benefit of being extremely close to the real-world environments and challenges your candidates would face in their daily work, giving you a true-to-life evaluation of their skills. 

What can you do?

No matter where your company stands on AI, we believe it’s best to be transparent about its capabilities. Yes, AI can solve simpler technical assessment questions. We prefer you to know that so that you can take informed actions. 

So what can you do? Every company is coming at AI in their own way, so there’s no one right answer. What works for one organization may not work for another. But broadly speaking, here are some steps you should consider to protect the integrity of your assessments.

• Stay informed. Yes, some technical questions can be solved by AI. At HackerRank, we help ensure assessment integrity through our market leading plagiarism detection and through solvability indicators that give you the transparency you need to deliver fair assessments. 
• Replace solvable questions. When a question in one of your assessments is flagged as AI solvable, a simple course of action is to replace it with an unsolved question from our library. We also recommend looking at the type of question you’re asking, and what you’re hoping to learn from it. It may make sense to replace a solvable question with an entirely different question type.
• Embrace new question types. Newer question formats like projects and code repos are more resistant to AI, and their close resemblance to real-world scenarios gives you a truer-to-life evaluation of how a candidate would perform in their daily work. 
• Take advantage of certified assessments. Don’t want to deal with maintaining and updating assessments? Let us do it for you. With certified assessments, HackerRank experts handle all of the content curation and monitoring, including replacing any leaked or AI solvable questions.
• Leverage HackerRank professional services. Have special needs for your assessments? Engage our experts for monitoring and content creation customized to your specific business objectives. 

Ensure assessment fairness and your own peace of mind

Ensuring assessment integrity in a time of rapidly advancing AI can seem difficult. You can only dial up question complexity so far before it starts to degrade the assessment experience and even compromise the value of assessments in finding qualified talent. That’s why we’re focused on reinforcing key pillars of assessment integrity, including our industry-leading AI-powered plagiarism detection, certified assessments, and solvability indicators that give you the transparency and signals you need to make the best decisions about your assessments. 

Be sure to check out our plagiarism detection page to go into more detail about how HackerRank is ensuring assessment integrity. 

The post AI Can Pass (Some) Test Questions. Now What? appeared first on HackerRank Blog.

The world of work is undergoing a seismic shift. Employee expectations have changed, many traditional work models are obsolete, and the myth that workers are less productive outside of the office has been debunked. 

Add to the equation talent shortages and an even faster pace of innovation fueled by AI, and it’s a real challenge for even the most experienced people teams to keep up. 

Though the world of work has never been more uncertain, one thing remains evident: organizations that embrace change by putting their people first and focusing on skills can build a culture of trust, connection, and innovation that will help them come out on top.

That’s why we’re excited to announce an official partnership between HackerRank, the developer skills company and leader in technical assessments and interviews, and RADICL, a global employee experience advisory company.

Together, HackerRank and RADICL will help people teams connect with and support strong, technically skilled talent through a refined focus on individual and organizational values, purpose, whole-person experience and culture fit. 

Here’s How It Works

Through the partnership, customers gain access to a one-stop shop for all their talent development needs. HackerRank’s assessments provide valuable insights into the technical competencies of employees, while RADICL’s expertise helps companies design a decentralized, framework-based approach centered on a whole-person experience strategy. 

This collaboration will enable companies to enhance their skills strategy, bolster their talent development initiatives, and equip employees with the necessary skills to thrive in today’s fast-paced and dynamic business environment. 

“This partnership between RADICL and HackerRank will redefine the work experience journey, from candidate through employee,” said RADICL co-founder Dan Riley. “Though the world of work has never been more uncertain, one thing is clear — organizations who find the best talent, and consistently inspire change by putting their people first and building a culture of trust, connection and transparency will come out on top.”

“I am incredibly excited about our partnership with RADICL. Our shared passion for empowering organizations to attract, hire and retain talent through positive candidate and employee experiences will reshape the hiring and talent management space,” said Jesse Sims, Head of Strategic Alliances at HackerRank.

About RADICL

RADICL is an authority in people science and the whole person experience. RADICL enables organizations to move beyond traditional employee engagement, performance management and leadership development by using new people science, frameworks and data sources. They unlock what people are saying, thinking, feeling, and how they are connecting with an organization’s purpose.

About HackerRank

Trusted by over 40% of the global developer population, HackerRank is the leading developer skills company, helping businesses attract, evaluate, hire and upskill the best technical talent from around the world. Over 3,000 customers across industries, including over 25% of the Fortune 100, rely on HackerRank to raise their hiring bars. The HackerRank developer community has over 21 million members who trust HackerRank to advance and showcase their coding skills. 

The post HackerRank and RADICL Announce Partnership to Provide a Skills-Based, People-First Approach to Talent Strategy appeared first on HackerRank Blog.

Partnership between best-in-class coding assessment and best-in-class behavioral assessment ensures complete view of soft and technical skills

HackerRank, the developer skills company, and Harver, the industry-leading hiring solution helping organizations optimize their talent decisions, this week announced a working partnership between the two companies, designed to provide a full evaluation of job candidates’ hard and soft skills. This new collaborative effort enables HR professionals to seamlessly and simultaneously leverage assessment solutions from both companies to better predict who will perform better and stay longer within the business.

The need for strong, well-rounded talent has never been higher. With stark competition for the most in-demand roles across industries, talent acquisition teams need to ensure quality of hire while differentiating their candidate experience. Organizations want to know that their potential candidates have both the tech skills and behavioral traits to succeed in a chosen role. And since no two roles demand the exact same make-up of soft and hard skills, organizations require a solution in their HR tech stack that seamlessly evaluates across both skill sets for each role.

With HackerRank’s best-in-class coding assessment and Harver’s best-in-class behavioral assessment, companies get a complete view of their candidates. Harver’s fast and engaging assessments effectively predict candidate fit — and help eliminate bias in the hiring process. Meanwhile, HackerRank’s skills data and measurement tools allow hiring teams to confidently find, hire, and hold onto technical talent with the right skills and at the right level.

“We are thrilled to partner with HackerRank to offer our clients access to the most comprehensive technical skills assessments available in the market,” said Scott Landers, Chief Executive Officer at Harver. “By combining our innovative soft skills technology with HackerRank’s best-in-class hard skills technology, clients can now assess candidates in a more holistic way, ensuring they find the right candidate for their team.”

Harver and HackerRank are both dedicated to transforming talent acquisition and development processes; this partnership represents a significant step towards providing a quality, streamlined evaluation process, helping clients find the right candidates for their teams.

“This collaboration combines our shared vision of empowering organizations to find and hire top talent efficiently, while maintaining an unwavering focus on skills and potential,” said Jesse Sims, Head of Strategic Alliances at HackerRank. “With Harver’s cutting-edge behavioral assessments and HackerRank’s leading technical assessments, we are poised to redefine the future of talent acquisition.”

To learn more about this partnership, please check out this post on Harver’s blog.

About Harver

Harver is the industry-leading hiring solution helping organizations optimize their talent decisions. Rooted in over 35 years of rich data insights backed by I/O psychology and cognitive science, Harver delivers a suite of automated solutions that enables organizations to engage, hire, and develop the right talent in a fast and fundamentally less biased way. Having processed over 100 million candidates, Harver provides trusted, data-driven, and adaptable offerings, including assessments, video interviews, scheduling, and reference checking. Harver has helped more than 1,300 customers take the smart path to the right talent. 

About HackerRank

Trusted by over 40% of the global developer population, HackerRank is the leading developer skills company, helping businesses attract, evaluate, hire and upskill the best technical talent from around the world. Over 3,000 customers across industries, including over 25% of the Fortune 100, rely on HackerRank to raise their hiring bars. The HackerRank developer community has over 21 million members who trust HackerRank to advance and showcase their coding skills.

The post HackerRank and Harver Announce Partnership to Provide Holistic Job Candidate Evaluation appeared first on HackerRank Blog.

25% of technical assessments show signs of plagiarism. 

While it’s impossible for companies to fully prevent plagiarism—at least without massively degrading the candidate experience—plagiarism detection is critical to ensuring assessment integrity. It’s important that developers have a fair shot at showcasing their skills, and that hiring teams have confidence in the test results. 

And the standard plagiarism detection method used by, well, everyone, is MOSS code similarity.

MOSS Code Similarity

MOSS (Measure of Software Similarity) is a coding plagiarism detection system developed at Stanford University in the mid-1990s. It operates by analyzing the structural pattern of the code to identify similarity, even when identifiers or comments have been changed, or lines of code rearranged. MOSS is incredibly effective at finding similarities, not just direct matches, and that effectiveness has made it the de facto standard for plagiarism detection. 

That doesn’t mean MOSS is flawless, however. Finding similarity doesn’t necessarily translate to finding plagiarism, and MOSS has a reputation for throwing out false positives, particularly when faced with simpler coding challenges. In our own internal research, we’ve found false positive rates as high as 70%.

AI changes the game

While not perfect, MOSS has been a “good enough” standard for years. Until the advent of generative AI tools like ChatGPT. 

ChatGPT has proven effective at solving easy to medium difficult assessment questions. And with just a bit of prodding, it’s also effective at evading MOSS code similarity. Let’s see it in action:

Step 1: We asked ChatGPT to answer a question and it did so, returning a solution as well as a brief explanation of the rationale. 

Step 2: Next, we directly asked ChatGPT to help escape MOSS code similarity check, and it refused.

Step 3: However, with some creative prompting, ChatGPT will offer unique approaches. And the way that ChatGPT’s transformer-based model works, it generates distinct answers every time, giving it a huge advantage in bypassing code similarity detection. 

Here are three different prompts and three totally different approaches. Note that ChatGPT transforms many variable names from the initial solution to evade code similarity checks.

Step 4: The moment of truth! When we submitted the revised answer through plagiarism detection, it passed cleanly. 

What’s the implication? 

Basically, MOSS code similarity checks can be easily bypassed with ChatGPT. 

Time to panic?

If MOSS code similarity can be bypassed, does that mean that technical assessments can no longer be trusted?

It depends. 

On one hand, it’s easier for candidates to bypass the standard plagiarism check that the entire industry has relied upon. So, yes, there is a risk to assessment integrity.

On the other hand, plagiarism detection has always been a compromise between effectiveness and candidate experience. MOSS is not intrusive, but its high false positive rates render it less definitive than it could be. Ultimately, it’s not really detecting plagiarism. It’s detecting patterns in the code that could be plagiarism.

Move over, MOSS

What happens now?

Plagiarism detection gets rethought for the AI era. Expect companies to scramble for better versions of MOSS, more complex questions, different question types, and more to make up the difference. 

At HackerRank, we’ve taken a different approach. While we’re always improving our question library and assessment experience, we’ve completely rethought plagiarism detection. Rather than relying on any single point of analysis like MOSS Code Similarity, we built an AI model that looks at dozens of signals, including aspects of the candidate’s coding behavior. 

Our advanced new AI-powered plagiarism detection system boasts a massive reduction in false positives, and a 93% accuracy rate. In real-world conditions, our system repeatedly detects ChatGPT-generated solutions, even when those results are typed in manually, and even when they easily pass MOSS Code Similarity. 

What happens when the example shown above gets submitted through our new system? It gets flagged for suspicious activity. 

Clicking into that suspicious activity reveals that our model identified the plagiarism due to coding behaviors.

What’s more, hiring managers can replay the answer keystroke by keystroke to confirm the suspicious activity. 

There’s nothing even close to it on the market, and what’s more, it’s a learning model, which means it will only get more accurate over time.

Want to learn more about plagiarism detection in the AI era, MOSS Code Similarity vulnerability, and how you can ensure assessment integrity? Let’s chat!

The post ChatGPT Easily Fools Traditional Plagiarism Detection appeared first on HackerRank Blog.

Today, HackerRank is proud to announce a new partnership with Criteria, a market-leading SaaS hiring success platform. The partnership will allow HackerRank and Criteria customers to identify the best talent with a more comprehensive, data-driven view of their technical candidates by cross-promoting the companies’ assessments. 

HackerRank customers will now have the opportunity to obtain a more complete view of their technical candidates by offering Criteria’s comprehensive suite of game-based assessments, which include cognitive aptitude, customizable personality, and behavior assessments. By adding these types of assessments to the hiring process, HackerRank customers will now have a fast, easy, and reliable way to identify candidates who are likely to perform more effectively. 

“Our shared vision and commitment to empowering organizations to make data-driven, fair, and inclusive hiring decisions make this partnership a natural fit. Together, we are poised to redefine the hiring process, addressing the challenges faced by recruiters, technical hiring managers, and candidates alike,” said Jesse Sims, Head of Strategic Alliances at HackerRank.

In tandem, Criteria customers will now have access to HackerRank’s best-in-class coding assessments, which include a library of more than 3,100 pre-built coding questions that measure 170-plus skill areas for all levels. Commonly assessed job families include software engineers, web developers, QA and testing, cloud and DevOps, data science and ML, and cybersecurity. HackerRank’s assessments complement the Criteria assessments a hiring team already uses, resulting in a more scalable, valid, and efficient hiring process for technical roles.  

“The timing of this union is unique in that it fulfills an almost identical need for each company,” says Josh Millet, Founder and CEO of Criteria. “We were fielding inquiries from our client base and prospects about the ability to offer technical/coding skills assessments (which we do not offer) and HackerRank was receiving requests for non-technical types of assessments.”  

Currently, both Criteria and HackerRank assessments can be accessed through one of their mutual integration partners.  

This strategic partnership aims to revolutionize the way companies evaluate technical candidates by providing a comprehensive, data-driven view of their skills. By combining their respective assessments, Criteria and HackerRank empower organizations to identify the most qualified talent efficiently and effectively.

The partnership between Criteria and HackerRank marks a significant milestone in the realm of technical candidate evaluation. As companies strive to attract and retain top talent, this collaboration promises to streamline the hiring process and enable better-informed hiring decisions. With Criteria and HackerRank working together, businesses can ensure they are identifying the most qualified candidates who will excel in their technical roles and contribute to long-term success.

Want to learn more? Schedule a demo of HackerRank and Criteria in action. 

About HackerRank

Trusted by over 40% of the global developer population, HackerRank is the developer skills company helping businesses attract, evaluate, hire and upskill technical talent from around the world. Over 3,000 customers across industries, including over 25% of the Fortune 100, rely on HackerRank to raise their hiring bars. The HackerRank developer community has over 21 million members who use HackerRank to advance and showcase their coding skills. 

About Criteria

Criteria is a talent success company that empowers organizations to make objective, evidence-based talent decisions that reduce bias and drive positive outcomes. Their suite of rigorously validated assessments and decision-making tools enables companies to uncover the potential in every job candidate while providing an engaging experience for applicants. Backed by scientific methodologies, Criteria’s solutions deliver results, promote transparency, and prioritize a human-focused approach to hiring.

The post Streamlining Technical Hiring: Criteria and HackerRank Combine Forces for Better Candidate Evaluation appeared first on HackerRank Blog.

What you need to know

HackerRank has just launched its advanced plagiarism detection system, powered by AI. Designed to protect assessment integrity while ensuring developers have a fair and level playing field to showcase their skills, this system uses dozens of signals to detect suspicious behavior, including the use of external tools. 

The revolution will be prompted

AI is here, and it’s not going anywhere. 82% of developers have already experimented with some type of AI tool, and 55% are using AI assistants at work. AI is redefining what it means to be a developer, and in the future most code will be written with some kind of AI support.

AI opens up all kinds of exciting possibilities, but it can also muddy the waters of technical assessments. GPT-4 can not only pass AP exams and simpler coding challenges; it can also bypass MOSS code similarity, which has long been the industry standard for coding plagiarism detection. 

How can you ensure assessment integrity? If you’re relying on MOSS code similarity, you can’t. Not anymore. That means you can’t place full confidence in a candidate’s skills, and you can’t assure developers that they’re showcasing their skills on a level playing field. 

Fighting magic with magic

Basing any kind of detection system around AI usage would be a futile endeavor. AI is advancing so rapidly that keeping said detection systems ahead of it would be nearly impossible. 

Rather than relying on any single point of analysis like MOSS Code Similarity or AI usage, we took a different path and built an AI model that looks at dozens of signals, including aspects of the candidate’s coding behavior. This “defense in breadth” compensates for signals that may be bypassed on an individual basis and gives our new plagiarism detection system a more holistic way to detect shenanigans.

Think of it like a security system. A system that relies on a single factor, like a fingerprint scanner, is secure until that single factor can be bypassed. A multi-factor system that employs x-ray scanners, metal detectors, facial recognition, and gait analysis is far more challenging to sneak past.

Plagiarism detection, powered by AI

Our plagiarism detection system tracks dozens of signals across three categories—coding behavior features, attempt submission features, and question features—and analyzes them to calculate the likelihood of suspicious activity. 

After all of that crunching, our model predicts plagiarism suspicion as either High, Medium, or No. 

Importantly, we do not use any personal data such as gender, race, age, school, location, or experience in our analysis model. 

Currently, our advanced plagiarism detection system achieves an incredible 93% accuracy rate. In real-world conditions, our system repeatedly detects ChatGPT-generated solutions, even when those results are typed in manually, and even when they easily pass MOSS Code Similarity. 

What happens when a ChatGPT-generated answer gets submitted through our new system? It gets flagged for suspicious activity. 

Clicking into that suspicious activity reveals that our model identified the plagiarism due to coding behaviors.

What’s more, hiring managers can replay the answer keystroke by keystroke to confirm the suspicious activity. 

There’s nothing even close to it on the market, and what’s more, it’s a learning model, which means it will only get more accurate over time.

Ensure assessment integrity and candidate confidence

Our new AI-powered plagiarism detection is a groundbreaking innovation. It protects assessment integrity and ensures a fair playing field for developers to showcase their skills, and there’s nothing else out there that comes close.

As the AI revolution reshapes the industry, it’s vital to have reliable and efficient methods to detect and prevent plagiarism in online assessments. By analyzing various aspects of coding behavior and offering 93% detection accuracy, this system sets a new standard for maintaining transparency, fairness, and equity.

Want to go deeper and see our new plagiarism detection in action? We’d be happy to show you around.

The post HackerRank’s AI-Powered Plagiarism Detection Ensures Assessment Integrity in the ChatGPT Era appeared first on HackerRank Blog.