Hire your next 
S o f t w a r e   D e v e l o p e r
 

based on relevant skills

Get lists of ranked candidates that demonstrate skills you value in your new hire

Ranked Candidates

Ranked Candidates

Code Evaluation Process Documentation

How We Analyze and Score Code Submissions

This page explains, step by step, how our system evaluates code quality, calculates metrics, and generates actionable feedback for every submission.

1. Code Parsing & Language Detection

  • Detects the programming language of the submitted code.
  • Parses the code using language-specific parsers (Python, Java, TypeScript, C#) to build an abstract syntax tree (AST).

2. Static Analysis & Metric Extraction

  • Language-specific analyzers traverse the AST to extract raw metrics:
    • Number of classes, methods, and functions
    • Inheritance depth, coupling, cohesion
    • Messaging (method calls), composition, encapsulation, abstraction, polymorphism
    • Cyclomatic complexity (per function and total)
    • Recursion, loop depth, variable count
  • Complexity Metrics:
    • Cyclomatic Complexity: Counts control flow statements (if, for, while, switch, etc.) and logical operators in each function.
    • Function Size: Number of lines/statements per function.
    • Loop Depth & Recursion: Maximum nesting of loops and number of recursive functions.

3. QMOOD Quality Metrics Calculation

  • Raw metrics are normalized to a 0-1 scale, adjusted for project size.
  • QMOOD model formulas are applied to compute high-level quality attributes:
    • Reusability: Combines coupling, cohesion, messaging, and design size.
    • Flexibility: Based on encapsulation, coupling, composition, and polymorphism.
    • Understandability: Weighs encapsulation, cohesion, coupling, complexity, and polymorphism.
    • Functionality: Uses cohesion, polymorphism, messaging, design size, and hierarchies.
    • Extensibility: Considers abstraction, coupling, inheritance, and polymorphism.
    • Effectiveness: Averages abstraction, encapsulation, composition, inheritance, and polymorphism.
    • Modularity: Mixes encapsulation, cohesion, coupling, and messaging.
  • QMOOD Score: Weighted average of the above, scaled for project size (0-100).

4. Performance Analysis

  • Time Complexity: Estimated from loop and recursion structure.
  • Space Complexity: Inferred from variable usage and data structure allocation.

5. AI-Powered Review & Scoring

  • Metrics and code are sent to an AI model for further review and ranking.
  • Each metric (modularity, reusability, extensibility, etc.) is scored out of 7.5, plus completion (out of 20).
  • For each low score, the system provides actionable feedback, including file, line, code snippet, issue, fix, and severity.
  • Final Score: Total score is the sum of all metrics, multiplied by a completion factor, capped at 100.

6. Output & Feedback

  • Returns a detailed report with:
    • All metric scores
    • Deductions and improvement suggestions
    • Summary of strengths and weaknesses
    • Verification of requirements completion

Example: Metric Calculation

  • Cyclomatic Complexity: Each function starts at 1, +1 for each control flow branch (if, for, while, etc.).
  • Cohesion: Counts method groups within classes that share similar responsibilities.
  • Coupling: Counts dependencies between classes and external modules.
  • QMOOD Reusability: -0.25 * Coupling + 0.25 * Cohesion + 0.5 * Messaging + 0.5 * Design Size (normalized and scaled).

Full Process Flow

  1. User submits code.
  2. System detects language and parses code.
  3. Static analysis extracts raw metrics.
  4. QMOOD and complexity metrics are calculated.
  5. Metrics are normalized and combined into quality scores.
  6. (Optional) AI batch review provides additional scoring and feedback.
  7. User receives a detailed, actionable report.

Create Your Own Competition

Tailor and build your own competition to evaluate the best talent based on your unique requirements.

Unlock Pre-Created Competitions

Access a list of pre-created competitions that have been curated and refined to help you find the best talent quickly.

Zack Walton

CEO at Walton Systems

The Hireup team was a pleasure to work with and transformed our hiring process. We will definitely come back to them to identify and hire more top-tier candidates.

Fast and effective process

Top-tier candidates

Top Hire Recommendation

Top-Ranked Competitions

See how top talent has performed in previous ranked events. These are the best of the best!

Need help?
Ask any question here!