As far as i can tell Colossus Slayer only works with bows if you haven't used a bonus action beforehand (which in this case is hunter's mark), i also run into the same issue if i hide before i fire a bow.
I found that if you apply a hunter's mark on a target and then shoot it the next round the 2 abilities do stack.