This study examined the role of social media sentiment in identifying irrational herding behavior in the stock market. We selected the Chinese stock market, where majority are retail investors, as the object of analysis, and analyzed the sentiment of 227,353 microblog text messages through deep learning techniques and constructed the identification method of herd behavior. Results show that social media sentiment has a significant impact on irrational herding behavior in the stock market. The findings of the study complement the theory of investor behavior and can aid in investors' trading decisions and financial regulators’ policy recommendations.