End-to-end encryption (E2EE) is becoming a standard feature in many popular chat apps, but independent security assessments of these implementations are limited. In this paper we provide the first independent analysis of E2EE features in LINE, a messaging application popular in Asian markets, and identify a replay attack and an attack on a lack of forward secrecy. Based on our analysis and communications with LINE about the vulnerabilities we discuss challenges and new research directions to better bridge vendors, researchers, and end-users around security issues.