Algorithmic advice has the potential to significantly improve human decision-making, especially in dynamic and complex tasks that require a balance between exploration and exploitation. This study examines conditions under which individuals are willing to accept advice from algorithms in such scenarios, focusing on the interaction between participants' exploration preferences and those of the advising algorithm. In an online experiment, we designed reinforcement learning algorithms to prioritize either exploration or exploitation and observed participants' decision-making behavior, modeled using a cognitive framework analogous to the algorithm. Contrary to expectations, participants did not show a preference for algorithms that matched their own exploration tendencies. In particular, participants were more likely to follow the advice of exploitative, consistent algorithms, possibly interpreting consistency as an indicator of competence. Although the participants benefited from the advice of the exploratory algorithm, their reluctance to follow it, regardless of whether the recommendation had been ignored previously or not, highlights a potential challenge in promoting effective collaboration between humans and algorithms. Explorative algorithms have the potential to promote behavioral diversification, but this effect is negated when humans disregard their advice. In such cases, algorithmic guidance can unintentionally decrease behavioral diversity by reinforcing established patterns.