p1 = len(y[y == 1]) / len(y)
\n",
" entropy = -p1 * np.log2(p1) - (1 - p1) * np.log2(1 - p1)
\n",
" Entropy at root node: 1.0 | \n", "
if X[i][feature] == 1:
.\n",
" \n",
" node_entropy = compute_entropy(y_node)
left_entropy = compute_entropy(y_left)
right_entropy = compute_entropy(y_right)
\n",
" \n",
" w_left = len(X_left) / len(X_node)
w_right = len(X_right) / len(X_node)
\n",
" weighted_entropy = w_left * left_entropy + w_right * right_entropy
\n",
" information_gain = node_entropy - weighted_entropy
\n",
" info_gain = compute_information_gain(X, y, node_indices, feature)
\n",
" max_info_gain = info_gain
best_feature = feature
\n",
" Important Note: Please only do this when you've already passed the assignment to avoid problems with the autograder.\n", "
Here's a short demo of how to do the steps above: \n",
"
\n",
" \n",
"