Skip to content

Set php and js coverage threshold to 0.1% - #5670

Merged
westonruter merged 1 commit into
developfrom
set-coverage-threshold-for-projects
Dec 13, 2020
Merged

westonruter merged 1 commit into
developfrom
set-coverage-threshold-for-projects

Conversation

@pierlon

@pierlon pierlon commented Dec 8, 2020 •

Copy link
Copy Markdown
Contributor

Summary

Bumps the coverage threshold for the php and js projects from 0% to 0.1%

Checklist

  • My pull request is addressing an open issue (please create one otherwise).
  • My code is tested and passes existing tests.
  • My code follows the Engineering Guidelines (updates are often made to the guidelines, check it out periodically).

@pierlon pierlon added the WS:Core Work stream for Plugin core label Dec 8, 2020
@pierlon pierlon self-assigned this Dec 8, 2020
@pierlon

pierlon commented Dec 8, 2020

Copy link
Copy Markdown
Contributor Author

Would love to get @schlessera's feedback on this to see if we would be shooting ourselves in the foot with this change.

@github-actions

github-actions Bot commented Dec 8, 2020

Copy link
Copy Markdown
Contributor

Plugin builds for dffc5c4 are ready 🛎️!

@schlessera

Copy link
Copy Markdown
Collaborator

@pierlon I might be missing something here: It looks from the diff like you're overriding the default threshold: 0.1% with language-specific settings of that very same threshold... ?

@pierlon

pierlon commented Dec 12, 2020 •

Copy link
Copy Markdown
Contributor Author

@schlessera the Codecov docs gives the impression that the default threshold is applied to the language-specific settings, but in practice that has turned out to be false. We've been having PR checks fail even when the threshold is below the expected 0.1%.

To verify this I forked the project and tested my hypothesis. As you can see in pierlon#3, at pierlon@424ddda the codecov php test failed when it decreased by 0.01%. When I set the threshold to 0.1% for php in pierlon@384e224, the coverage test passed.

@schlessera

Copy link
Copy Markdown
Collaborator

Alright. I do think that looks like a bug in CodeCov, it does not make full sense the way I read the configs.

@schlessera schlessera left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try and see how this goes. Right now, I think our code coverage is a bit flaky overall because it use pcov, which seems to have trouble flagging everything it should...

@westonruter
westonruter merged commit dbd7d60 into develop Dec 13, 2020
@westonruter
westonruter deleted the set-coverage-threshold-for-projects branch December 13, 2020 17:42
@westonruter westonruter added this to the v2.1 milestone Dec 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

WS:Core Work stream for Plugin core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants