How to Review Open-Source Work Without Counting Stars

A popular repository can have thousands of stars.

That number may show that people are interested in the project. It does not explain who wrote a specific feature, solved a difficult problem or improved the code.

GitHub describes stars as a way for users to save repositories and show interest. Stars can also influence repository rankings. They were not designed to measure an individual developer’s ability.

For employers, the important question is not:

“How many stars does this repository have?”

It is:

“What did this candidate contribute, and what does that work show about their ability?”

A useful review focuses on evidence, relevance and context.

Why Repository Stars Can Be Misleading

Stars belong to the repository, not automatically to every contributor.

A project may become popular because of:

• Strong marketing
• An established company or creator
• Early entry into a growing market
• A large existing community
• Useful documentation
• Media coverage
• A popular technology trend

A candidate may have made one small contribution to a highly starred project. Another candidate may have completed important technical work in a project that few people know about.

The second contribution could be more relevant to the vacancy.

Repository popularity can provide background information, but it should not become a candidate score.

Start With the Role You Are Hiring For

Before reviewing a candidate’s public work, define what evidence matters for the position.

For example, when hiring a backend engineer, you may look for:

• API design
• Database decisions
• Error handling
• Testing
• Performance
• Security awareness
• System reliability

For a frontend engineer, the priorities may include:

• Component structure
• Accessibility
• State management
• Responsive design
• Performance
• User experience

A security engineer, DevOps engineer or machine-learning engineer will require different evidence.

Without clear criteria, reviewers may focus on whatever looks impressive rather than what is relevant.

1. Confirm the Candidate’s Actual Contribution

Do not assume the candidate created everything in a repository listed on their profile.

First, identify what they personally contributed.

Look for:

• Commits they authored
• Pull requests they opened
• Issues they investigated
• Code reviews they completed
• Documentation they wrote
• Tests they added
• Technical discussions they joined

Then check whether the work was merged, rejected, revised or left unfinished.

A contribution does not need to be merged to be useful evidence. An unmerged pull request may still show thoughtful problem-solving, clear communication or a strong technical approach.

Ask the candidate to explain their role instead of making assumptions based on the repository page.

2. Review the Problem Before Reviewing the Code

Code only makes sense when you understand the problem it was trying to solve.

Before reading the changes, review:

• The original issue
• The pull-request description
• The project’s contribution guidelines
• Relevant comments from maintainers
• The expected result
• Any technical limitations mentioned

GitHub recommends reading the pull-request summary and related issues before reviewing the changed files. This gives reviewers the context needed to understand the proposed solution.

A small code change may have solved a serious production problem. A large change may only have reorganised existing code.

Size does not equal value.

3. Review the Pull Request, Not Just the Final Code

The final code shows the result.

The pull request shows how the candidate reached it.

GitHub pull requests include the description, commits, checks, file changes, comments and reviews connected to a proposed change.

Review whether the candidate:

• Explained the purpose of the change
• Connected the work to an issue
• Kept the changes focused
• Added relevant tests
• Updated documentation
• Responded to questions
• Corrected problems identified during review
• Explained important trade-offs

This can reveal technical ability, ownership and communication more clearly than the final code alone.

4. Look at Code Quality in Context

There is no single coding style that proves a developer is strong.

Different projects use different languages, frameworks, standards and levels of complexity.

When reviewing code, consider whether it is:

• Understandable
• Relevant to the problem
• Consistent with the project
• Reasonably organised
• Tested where appropriate
• Clear about errors and unusual cases
• Easy for another developer to maintain

Avoid rejecting a candidate because their public code does not match your company’s exact internal style.

Focus on the quality of their decisions and whether they can explain them.

5. Check How They Responded to Feedback

Open-source work is collaborative.

A candidate’s response to review comments may show how they work with other developers.

Look for signs that they:

• Read feedback carefully
• Asked questions when something was unclear
• Updated the work when the feedback was useful
• Explained disagreements respectfully
• Confirmed when changes were completed
• Kept the discussion focused on the project

GitHub explains that pull-request reviews allow people to comment on changes, suggest improvements, approve work or request updates before code is merged. These conversations also create a record of feedback and decisions.

A candidate does not need to agree with every reviewer. A respectful technical disagreement can also demonstrate sound judgement.

6. Review Their Code Reviews

Candidates may show technical skill by reviewing other people’s work, not only by writing their own code.

Useful code reviews can demonstrate:

• Ability to identify bugs
• Understanding of security or performance risks
• Attention to maintainability
• Knowledge of project standards
• Clear technical communication
• Ability to suggest practical improvements

GitHub describes reviewing pull requests as an important form of collaboration that helps teams catch problems, share knowledge and decide whether changes are ready to merge.

This evidence may be particularly useful when assessing senior engineers, technical leads and engineering managers.

7. Include Issues and Technical Discussions

Some of the strongest evidence may appear before any code is written.

Issue discussions can show whether a candidate knows how to:

• Reproduce a problem
• Ask useful questions
• Separate symptoms from causes
• Share logs or examples
• Compare possible solutions
• Explain technical limitations
• Help other contributors move forward

A candidate who investigates a problem carefully may provide more value than someone who quickly submits a change without understanding the cause.

Review the quality of the thinking, not only the final output.

8. Do Not Ignore Documentation

Documentation is part of a functioning software project.

A candidate may contribute by improving:

• Installation instructions
• API documentation
• Troubleshooting guides
• Examples
• Release notes
• Contribution guidelines
• Project architecture explanations

Open Source Guides recognises documentation, translation and community support as valid forms of contribution.

Documentation work can demonstrate technical understanding, attention to detail and an ability to communicate with users and other developers.

This can be especially relevant for developer relations, support engineering, product engineering and technical leadership roles.

9. Look for Evidence of Testing

Check whether the candidate considered how the change would be verified.

Evidence may include:

• Unit tests
• Integration tests
• Manual testing notes
• Automated checks
• Examples of expected behaviour
• Steps for reproducing the original problem

GitHub pull requests can display automated tests, builds, code scanning and other checks connected to a proposed change.

Not every contribution requires a large test suite. The testing approach should be appropriate for the type and risk of the change.

10. Consider Project Difficulty and Constraints

A contribution should be judged within its environment.

Ask:

• Was the project large or difficult to understand?
• Did the candidate work with unfamiliar code?
• Were there strict performance or security requirements?
• Did the contribution affect many users?
• Was the project actively maintained?
• Did the candidate follow detailed contribution rules?
• Did the work require coordination with several maintainers?

A short change in a complex system may require more skill than a larger change in a simple personal project.

11. Ask the Candidate to Explain the Work

Public evidence should improve the interview, not replace it.

Choose one or two relevant contributions and ask the candidate:

• What problem were you solving?
• Which part of the work was yours?
• Why did you choose this approach?
• What alternatives did you consider?
• What feedback did you receive?
• How did you test the change?
• What would you do differently now?
• What did you learn from the project?

Strong candidates should be able to explain their contribution clearly, including its limitations.

This discussion can also help employers confirm genuine ownership of the work.

12. Do Not Penalise Candidates Without Public Work

Not every developer has an active public profile.

Many professionals work in private repositories. Others cannot share previous work because of employment agreements, client confidentiality or security requirements.

Some candidates may also have limited time for unpaid contributions outside their job.

GitHub’s contribution graph may not display all activity. Certain commits, fork activity and private work may be excluded or shown only as contribution counts.

A lack of public activity is not proof of weak technical ability.

Employers should allow candidates to provide other evidence, such as:

• A private portfolio walkthrough
• An anonymised technical example
• A structured interview
• A short and relevant work sample
• References from previous technical managers
• A discussion about past projects

Use a Consistent Review Process

Open-source work should be assessed using the same role-related criteria for every candidate.

A simple review can follow these steps:

1. Define the skills required for the vacancy.
2. Select one or two relevant contributions.
3. Confirm what the candidate personally completed.
4. Read the issue and pull-request context.
5. Review the code, tests and documentation.
6. Examine feedback and collaboration.
7. Ask the candidate to explain their decisions.
8. Record evidence using an agreed scorecard.
9. Combine the findings with other hiring evidence.

This prevents repository popularity, personal preferences or contribution volume from controlling the decision.

What Should Employers Avoid?

Avoid using open-source profiles to:

• Rank candidates by stars
• Count commits without reviewing them
• Require daily GitHub activity
• Assume all repository work belongs to one person
• Reject candidates who work privately
• Compare unrelated projects directly
• Treat one contribution as proof of every technical skill
• Replace the full interview process

The purpose is to find useful evidence—not create another automatic screening rule.

Better Hiring Looks Beyond Popularity

Stars can show that a repository has attracted interest.

They cannot show exactly how a candidate thinks, communicates or contributes.

To assess open-source work properly, employers need to review the problem, the candidate’s individual contribution, the technical decisions and the collaboration surrounding the work.

The strongest signal is not popularity.

It is relevant evidence that the candidate can solve problems, produce maintainable work and collaborate effectively.

How Nindar Can Help

Nindar Consulting helps technology, AI and Web3 companies identify and recruit specialist talent across global markets.

Through specialist recruitment, executive search, RPO and talent mapping, we help employers define technical requirements, assess relevant evidence and connect with suitable professionals.

Need help identifying developers with relevant technical experience? Contact Nindar to discuss your hiring requirements.

This article provides general hiring and business information only.