Evaluation and Inspection Module¶
Here are contained the functions that related to the evaluation of the performance of the networks predictions, and also functions that relate to the inspection of data.
This module include a set of functions that are used to evaluate and inspect the time series in the dataset.
Author: Oliver Boom Github Alias: OliverJBoom
-
Foresight.eval_inspect.check_day_frequency(df, col_name='ds')[source]¶ Creates a bar chart showing the frequency of the days of the week.
Used to check that only business days are included in the dataset, and that there is a roughly equal distribution of entries across the week.
Parameters: - df (pd.DataFrame) – A DataFrame containing the time series to check
- col_name (string) – The name of the column of interest
-
Foresight.eval_inspect.check_length(universe_dict)[source]¶ Checks the name of all the DataFrames in the dictionary of time series.
Parameters: universe_dict (dict) – The dictionary of time series
-
Foresight.eval_inspect.df_std(df, col_name)[source]¶ Calculates standard deviation of a DataFrames column.
Parameters: - df (pd.DataFrame) – A DataFrame of time series
- col_name (string) – The column of interest
Returns: The standard deviation of the series
Return type: float
-
Foresight.eval_inspect.evaluate(y_true, y_pred, log_ret=False)[source]¶ Calculates the error metrics for between two arrays.
- The error metrics calculated are:
- Means Squared Error Mean Absolute Error Mean Directional Accuracy
For a log returns series the definition of mean directional accuracy changes. This is as for a log return series it is the signum values of the series that details which direction the series has moved. This is as a log return series is the first difference of the original series. For raw price. The signal needs to be differenced before the signum function is applied.
Parameters: - y_true (np.array) – The observed values
- y_pred (np.array) – The predicted values
- log_ret (bool) – Whether the series compared are log returns
Return error_metrics: The error metrics of the series
Return type: List
-
Foresight.eval_inspect.inverse_log_returns(original_prices, log_returns, lag=5, offset=0)[source]¶ Takes a DataFrame of predicted log returns and original prices and returns an array of predicted absolute prices
The offset parameter moves the series forwards or backwards to align the series with the DataFrame it might be appended to.
Parameters: - original_prices (pd.DataFrame) – A DataFrame of absolute prices
- log_returns (pd.DataFrame) – A DataFrame of log returns
- lag (int) – The lag in days between series
- offset (int) – Amount to offset the series forwards of backwards
Returns: The raw prices given by the log returns
Return type: pd.Series
-
Foresight.eval_inspect.mean_absolute_percentage_error(y_true, y_pred)[source]¶ Calculates the mean absolute percentage error between two arrays.
Parameters: - y_true (np.array) – The observed values
- y_pred (np.array) – The predicted values
Returns: The mean absolute percentage error of the series
Return type: float
-
Foresight.eval_inspect.mean_directional_accuracy(y_true, y_pred)[source]¶ Calculated the mean directional accuracy error metric between two series.
Parameters: - y_true (np.array) – The observed values
- y_pred (np.array) – The predicted values
Returns: The mean directional accuracy of the series
Return type: float
-
Foresight.eval_inspect.mean_directional_accuracy_log_ret(y_true, y_pred)[source]¶ Calculates the mean directional accuracy error metric between two series of log returns.
Parameters: - y_true (np.array) – The observed values
- y_pred (np.array) – The predicted values
Returns: The mean directional accuracy of the series
Return type: float