Can I specify my own Serach Metric

bhctuary

Mean Relative Error counts -50% same the same as +50%. I want to fix this so that -50% is equivalent to +100%.
(In relative error, the maximum % error above in unlimited, but the maximum percent error below is 100%).

Thanks

Bert

admin

Hello Bert,

This is an interesting modification to the Mean Relative Error metric, but unfortunately custom metrics are not supported at the moment. But it should be possible to add it in a future release similar to how "Custom Formula" is handled.

bhctuary

Thank you and I apologize for my typos.

RMSLE comes close but doesn't quite get there.

admin

Hi Bert,

v3.3 now has custom search metrics. The asymmetric relative error you described can be written as:

mean(abs(log(predicted/actual)))

This penalizes halving (-50%) and doubling (+100%) equally, since log(0.5) = -log(2). Requires positive values. Set it in the UI under "Custom metric" or via CLI with --custom-metric.