Metrics python extraction
Metrics can be extracted both from the command-line interface and from the python API. You will find here instructions on how to use the API to customize your metrics extraction.
To extract metrics, you can choose to use the pipelines that are defined in the command-line or define all the parameters of the extraction yourself.
Use the existing pipelines
We first need to initialize the project and import the necessary functions. Here we use
the example project vandam-data.
We then initialize our 2 types of metrics LenaMetrics
and AclewMetrics with the desired parameters.
Here we choose to do a very simple LenaMetrics extraction using all the default values and the set named “its”. For AclewMetrics, we initialize the class to extract on the set “vtc” only between 8am and 5pm on periods of 6 hours, grouped by child_id values and adding the values of date_iso, child_dob and child_id to the resulting output.
>>> from ChildProject.projects import ChildProject
>>> from ChildProject.pipelines.metrics import LenaMetrics, AclewMetrics
>>> project = ChildProject('vandam-data')
>>> lmetrics = LenaMetrics(project,"its")
>>> ametrics = AclewMetrics(
... project,
... vtc='vtc',
... from_time='8:00:00',
... to_time='17:00:00',
... rec_cols='date_iso',
... child_cols='child_dob,child_id',
... period='6h',
... by='child_id',
... )
The ALICE set ('alice') was not found in the index.
The vcm set ('vcm') was not found in the index.
The programm warns us that the alice and vcm sets are not present which is expected given that the vandam-data corpus does not have vcm and alice annotations. So the output will not contain the metrics extracted from those.
We then launch the extraction for each pipeline. The function populates the .metrics attribute
and returns the resulting metrics. Here we save the resulting metrics in csv files with the
to_csv function
from pandas.
>>> lmetrics.extract()
recording_filename child_id duration_its ... voc_mal_ph voc_dur_chi_ph lp_dur
0 BN32_010007.mp3 1 50464512 ... 103.86705 178674.867598 NaN
[1 rows x 20 columns]
>>> lmetrics.metrics.to_csv('LenaMetrics.csv', index=False)
>>> ametrics.extract()
child_id period_start period_end ... avg_voc_dur_mal avg_voc_dur_och avg_voc_dur_chi
0 1 00:00:00 06:00:00 ... NaN NaN NaN
1 1 06:00:00 12:00:00 ... 1373.208247 935.654378 1159.420822
2 1 12:00:00 18:00:00 ... 1099.721472 808.012712 1011.550502
3 1 18:00:00 00:00:00 ... NaN NaN NaN
[4 rows x 18 columns]
>>> ametrics.metrics.to_csv('AclewMetrics.csv', index=False)
Define you own metrics
You can also create your own metrics by defining your python function calculating the output value. To do so, define a function taking as arguments:
annotations : pandas DataFrame, this is the actual segments of the converted set
duration : int, the represents the length that was annotated, use this value to calculate rates per hour for example
**kwargs : keyword arguments, this allows the user to give whatever arguments he likes to the metric function (such as ‘speaker’ for example). The value used will have to be given in the wanted metrics_list dataframe
and returning, in that order:
a default name for the metric to take, it will be used when no specific name was explicitly required by the user
the value of the metric, should be a number or np.nan (a distinction is made between 0 and np.nan as np.nan indicates that the value can not be calculated).
The function should check the presence of the required columns in the annotations and of the required keyword arguments.
To make this easier, use the function ChildProject.pipelines.metricsFunctions.metricFunction() as a decorator
to perform those checks as well as giving a default name based on the function’s name.
The decorator should be called along with the parameters :
args : a set of the names of the required keyword arguments
columns : a set of the names of the required columns in the annotations
emptyValue : the value to return when no annotations segments are found
name : the default name to use the designate this metric. If empty, uses the function name. Be aware that keyword
arguments found in the name will be replaced by their value (e.g. voc_speaker_ph with speaker='CHI' will return voc_chi_ph).
The only remaining task of the function is the calculation and return of the value.
Here we define a function that only requires the keyword argument ‘speaker’ and is calculated only based on the ‘speaker_type’ column. When no annotation is found, its value will be 0 and by default it will take the name ‘num_of_voc_speaker’ with <speaker> being replaced with the value of the ‘speaker’ keyword argument. The returned value is the number of lines belonging to the speaker_type (i.e. its number of vocalizations as an absolute value).
>>> from ChildProject.projects import ChildProject
>>> from ChildProject.pipelines.metricsFunctions import metricFunction
>>> import pandas as pd
>>> @metricFunction({'speaker'},{'speaker_type'}, 0, 'num_of_voc_speaker')
... def voc_speaker(annotations: pd.DataFrame, duration: int, **kwargs):
... return annotations[annotations["speaker_type"]== kwargs["speaker"]].shape[0]
...
We defined our custom metric, now we will create our list of wanted metrics. It must be a pandas DataFrame compatible with the metrics listing format. The callable function is used for both names of the default available metrics and callables functions that we defined ourselves. Here we only use the vtc set, we want to extract the number of vocalizations produced by the key child and the mother in absolute values (using our newly defined function) but also in values per hour (using the default metric <voc_speaker_ph>).
>>> input = pd.DataFrame([{
... 'set': 'vtc',
... 'callable': 'voc_speaker_ph',
... 'speaker': 'CHI',
... },{
... 'set': 'vtc',
... 'callable': 'voc_speaker_ph',
... 'speaker': 'FEM',
... },{
... 'set': 'vtc',
... 'callable': voc_speaker,
... 'speaker': 'CHI',
... }{
... 'set': 'vtc',
... 'callable': voc_speaker,
... 'speaker': 'FEM',
... }])
Last thing left to do is initialize our ChildProject.pipelines.metrics.Metrics with the correct
parameters and launch the extraction
>>> from ChildProject.pipelines.metrics import Metrics
>>> project = ChildProject('vandam-data')
>>> m = Metrics(
... project,
... metrics_list= input,
... from_time='8:00:00',
... to_time='17:00:00',
... rec_cols='date_iso',
... child_cols='child_dob,child_id',
... period='6h',
... by='child_id',
... )
>>> m.extract()
child_id period_start period_end ... voc_fem_ph num_of_voc_chi num_of_voc_fem
0 1 00:00:00 06:00:00 ... NaN NaN NaN
1 1 06:00:00 12:00:00 ... 244.5 1143.0 978.0
2 1 12:00:00 18:00:00 ... 253.4 1495.0 1267.0
3 1 18:00:00 00:00:00 ... NaN NaN NaN
[4 rows x 10 columns]
>>> m.metrics.to_csv('Metrics.csv', index=False)