Metrics
Introduction
The Agreement conformed from several Guarantees contain all the information needed to compute them. Each guarantee contains an objective, which can involve one or more metrics. The metric is the instrument used to fetch the required information and extract a value for the system to verify the fulfilment of the objective.
Collectors
The collectors are the component which receiving a metric as input, are able to connect to the different data sources to extract the information needed for it, and compute the final result.
Actually, there are 3 different collectors available. The metrics are modeled as a JSON object following the iAgree specification. Each collector has his own syntax and they are similar in order to be consistent but there are some differences in order to extend the features.
Event Collector
The Event Collector is capable of using different data sources as input and join the information in order to correlate different payloads composing more complex metrics. Actually it can fetch information from Github, Pivotal Tracker, Travis CI, CodeClimate and Heroku.
Patterns
The metrics available to use in the Event Collector follow 4 different patterns:
- Number of
[Event]
in[Tool]
every[Period]
by[TEAM|MEMBER]
[MAX|MIN|AVG|STD|NEWEST|LATEST]
[Property]
value of[Event]
in[Tool]
every[Period]
by team.[Frequency]
distribution of[Event]
in[Tool]
every[Period]
by team.- Percentage of
[Event1]
in[Tool1]
correlated with[Event2]
in[Tool2]
within[window]
every[Period]
by team.
Here are 4 different examples with their corresponding JSON format using the patterns above. Note that this is the DSL of the metric and the period is not displayed because it belongs to the guarantee:
- Number of
STARTED_STORIES
inPIVOTAL
everyDAY
byMEMBER
NUMBER_PT_STARTEDSTORIES{"metric": {"computing": "actual","element": "number","event": {"pivotal": {"activity": {"highlight": "finished"}}},"scope": {"$ref": "#/context/definitions/scopes/development"}}}
AVG COVERAGE
value ofCOVERAGE_REPORT
inCODECLIMATE
everyWEEK
by team.
VALUE_CC_COVERAGE_AVG{"metric": {"computing": "actual","element": {"value": {"parameter": "attributes.covered_percent","return": "avg","traceback": true}},"event": {"codeclimate": {"coverage": {}}},"scope": {"$ref": "#/context/definitions/scopes/development"}}
DAILY
distribution ofSUCCESSFUL_BUILDS
inTRAVIS
everyWEEK
by team
STDEV_TR_SUCCESSFULBUILDS_DAILY{"metric": {"computing": "string","element": {"stdev": {"period": "daily"}},"event": {"travis": {"builds_public": {"@type": "build","state": "passed"}}},"scope": {"$ref": "#/context/definitions/scopes/development"}}
- Percentage of
NEW_BRANCH
inGITHUB
correlated withSTART_STORY
inPIVOTAL_TRACKER
within1_HOUR
everyWEEK
by team.
PERCENTAGE_GH_NEWBRANCH_PT_STARTEDSTORIES{"metric": {"computing": "actual","element": {"percentage": {"related": {"github": {"events": {"type": "CreateEvent","payload": {"ref_type": "branch"}}},"window": 3600}}},"event": {"pivotal": {"activity": {"highlight": "started"}}},"scope": {"$ref": "#/context/definitions/scopes/development"}}
Pivotal Tracker Collector
The PT Collector connects with Pivotal Tracker and can produce more specific metrics using the different payloads.
Examples
There is a webpage with available examples for public usage.
GitHub Collector
Using GitHub API v4, it can access to the information from GitHub Projects Kanbans in order to keep track of the different cards and the changes.
Examples
There is a webpage with available examples for public usage.