xgboost内置了足够丰富的目标函数(objective function)，正常来说是能够应付日常需求的，如果～万一～你有特殊需求，它也可以自定义目标函数，或者叫损失函数(loss function)
xgboost的documentation里有介绍如何自定义目标函数，https://xgboost.readthedocs.io/en/stable/tutorials/custom_metric_obj.html
Squared Log Error
按照文档中Squared Log Error (SLE)，记录一下这个过程
Squared Log Error:
$$\frac{1}{2}\left [ \log (\hat{y} + 1) – \log (y + 1) \right ]^2$$
其中$y$是实际值，$\hat{y}$是预测值,$\log$里面加1是避免出现$\log(0)$的情况。我们可以直接用损失函数来评估模型的好坏，或者用metric来评估，比如下面这个，文档里叫它Root Mean Squared Log Error(RMSLE)
$$\sqrt{\frac{1}{N}\left [ \log (\hat{y} + 1) – \log (y + 1) \right ]^2}$$
函数实现
可以把我们自己写的目标函数丢给xgboost.train方法，这样训练过程中目标函数和metric就会按照我们自定义的进行fit。看一下documentation中xgboost.train的部分：
正如你想象的那样，obj参数接收的是自定义的目标函数，feval是自定义的metric，目标函数(obj)接收两个参数：一个是predt, np.ndarray格式的，表示前面$i-1$轮迭代后输出的预测值$\hat{y}$，可以标记为:
$$F_{i-1}(x) = \sum_{i-1}f_i(x)$$
其中$f_i$是基学习器。另一个dtrain, xgb.DMatrix格式的，装了一些训练集的信息，features并不会传送过来，因为太大了，并且计算object也用不上。需要返回两个np.ndarray的变量，一个是gradient, 另一个是hessian，即一阶和二阶导数。如果你看过上篇的内容，可能会有疑问为什么要用二阶导数？这个在后面的文章再介绍吧～
Gradient:
$$\frac{\partial \text{obj}}{\partial \hat{y}} = \frac{\log(\hat{y} + 1 – \log(y+1)}{\hat{y} + 1}$$
Hessian:
$$\frac{\partial ^2\text{obj}}{\partial \hat{y}^2} = \frac{1 + \log (\hat{y} + 1) – \log(y+1)}{(\hat{y} + 1)^2}$$
用Python实现上述两个公式：
<span class="line"><span style="color: #F286C4">def</span><span style="color: #F6F6F4"> </span><span style="color: #62E884">gradient</span><span style="color: #F6F6F4">(</span><span style="color: #FFB86C; font-style: italic">y_pred</span><span style="color: #F6F6F4">, </span><span style="color: #FFB86C; font-style: italic">y_true</span><span style="color: #F6F6F4">):</span></span>
<span class="line"><span style="color: #F6F6F4"> numerator </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> np.log(y_pred </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4">) </span><span style="color: #F286C4">-</span><span style="color: #F6F6F4"> np.log(y_true </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4">)</span></span>
<span class="line"><span style="color: #F6F6F4"> denominator </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> y_pred </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #F286C4">return</span><span style="color: #F6F6F4"> numerator </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4"> denominator</span></span>
<span class="line"></span>
<span class="line"><span style="color: #F286C4">def</span><span style="color: #F6F6F4"> </span><span style="color: #62E884">hessian</span><span style="color: #F6F6F4">(</span><span style="color: #FFB86C; font-style: italic">y_pred</span><span style="color: #F6F6F4">, </span><span style="color: #FFB86C; font-style: italic">y_true</span><span style="color: #F6F6F4">):</span></span>
<span class="line"><span style="color: #F6F6F4"> numerator </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> np.log(y_true </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4">) </span><span style="color: #F286C4">-</span><span style="color: #F6F6F4"> np.log(y_pred </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4">)</span></span>
<span class="line"><span style="color: #F6F6F4"> denominator </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> np.power(y_pred </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4">, </span><span style="color: #BF9EEE">2</span><span style="color: #F6F6F4">)</span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #F286C4">return</span><span style="color: #F6F6F4"> numerator </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4"> denominator</span></span>按照API的要求实现obj函数：
<span class="line"><span style="color: #F286C4">def</span><span style="color: #F6F6F4"> </span><span style="color: #62E884">objective_function</span><span style="color: #F6F6F4">(</span><span style="color: #FFB86C; font-style: italic">pred</span><span style="color: #F6F6F4">, </span><span style="color: #FFB86C; font-style: italic">dtrain</span><span style="color: #F6F6F4">):</span></span>
<span class="line"><span style="color: #F6F6F4"> y_true </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> dtrain.get_label()</span></span>
<span class="line"><span style="color: #F6F6F4"> grad </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> gradient(pred, y_true)</span></span>
<span class="line"><span style="color: #F6F6F4"> hess </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> hessian(pred, y_true)</span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #F286C4">return</span><span style="color: #F6F6F4"> grad, hess</span></span>顺便按照要求把metric也实现了：
<span class="line"><span style="color: #F286C4">def</span><span style="color: #F6F6F4"> </span><span style="color: #62E884">evaluate_function</span><span style="color: #F6F6F4">(</span><span style="color: #FFB86C; font-style: italic">pred</span><span style="color: #F6F6F4">, </span><span style="color: #FFB86C; font-style: italic">dtrain</span><span style="color: #F6F6F4">):</span></span>
<span class="line"><span style="color: #F6F6F4"> y_true </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> dtrain.get_label()</span></span>
<span class="line"><span style="color: #F6F6F4"> n </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> </span><span style="color: #97E1F1">len</span><span style="color: #F6F6F4">(pred)</span></span>
<span class="line"><span style="color: #F6F6F4"> evaluate_name </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> </span><span style="color: #DEE492">'</span><span style="color: #E7EE98">rmsle</span><span style="color: #DEE492">'</span></span>
<span class="line"><span style="color: #F6F6F4"> evaluate_value </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> np.sqrt(np.mean(np.power(np.log(pred </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4">) </span><span style="color: #F286C4">-</span><span style="color: #F6F6F4"> np.log(y_true </span><span style="color: #F286C4">+</span><span style="color: #F6F6F4"> </span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4">), </span><span style="color: #BF9EEE">2</span><span style="color: #F6F6F4">)))</span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #F286C4">return</span><span style="color: #F6F6F4"> evaluate_name, evaluate_value</span></span>然后在train模型的时候像这样装填一下我们自定义的obj和feval
<span class="line"><span style="color: #F6F6F4">bst </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> xgb.train(param, </span></span>
<span class="line"><span style="color: #F6F6F4"> train_data, </span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #FFB86C; font-style: italic">num_boost_round</span><span style="color: #F286C4">=</span><span style="color: #BF9EEE">2</span><span style="color: #F6F6F4">, </span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #FFB86C; font-style: italic">evals</span><span style="color: #F286C4">=</span><span style="color: #F6F6F4">[(test_data, </span><span style="color: #DEE492">'</span><span style="color: #E7EE98">test</span><span style="color: #DEE492">'</span><span style="color: #F6F6F4">)], </span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #FFB86C; font-style: italic">verbose_eval</span><span style="color: #F286C4">=</span><span style="color: #BF9EEE">True</span><span style="color: #F6F6F4">, </span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #FFB86C; font-style: italic">obj</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> objective_function,</span></span>
<span class="line"><span style="color: #F6F6F4"> </span><span style="color: #FFB86C; font-style: italic">feval</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">=</span><span style="color: #F6F6F4"> evaluate_function</span></span>
<span class="line"><span style="color: #F6F6F4"> )</span></span>大概就是这些了，xgboost只是个引子，后面我们展开聊一下LightGBM自定义目标函数的事。