Algorithmic advice holds the potential to significantly enhance human decision-making, particularly in dynamic and complex tasks involving a trade-off between exploration and exploitation. We investigate the conditions under which people are willing to accept advice from algorithms in such tasks, focusing on the interplay between individuals’ and the advising algorithm’s exploration preferences. In an online experiment, we engineered reinforcement learning algorithms to favor either exploration or exploitation and observed participants’ decision-making processes, modeling them using a cognitive framework comparable to the algorithm. Interestingly, individuals showed a greater inclination to follow the advice of exploitative, consistent algorithms, possibly perceiving algorithmic consistency as a sign of competence. They did not exhibit a preference for algorithms with similar exploration tendencies to their own. Moreover, we found that participants’ exploration tendencies influenced the behavior of the algorithms, underscoring the importance of considering the mutually reinforcing co-behaviors of algorithms and humans.