Health literacy is a well-studied concept that has a large literature base. Yet, little is known about health literacy and people diagnosed with mental illnesses, and even less health literacy research speaks to people diagnosed with mental illness. I used a framework and approach of lived experience of psychiatric diagnosis and psychiatric survivor research to conceptualize and inform the study. It explores health literacy with people diagnosed with mental and physical health diagnoses using a small quantitative survey, focus groups, and an individual interview. The findings suggest that health literacy is a dynamic and complex construct that interacts with the individual across the layers of an ecological model and over time. Relational, trust-based practice with a primary care provider is viewed as supporting health literacy. In addition, trauma-informed and culturally responsive care also can strengthen health literacy. In contrast, participants describe the emergency department as a place within the healthcare system where they commonly encounter barriers to health literacy. Other barriers to health literacy include coercion, lack of trust, lack of credibility, lack of informed consent, and iatrogenic experiences with the healthcare system. More research is needed to understand the complexity of health literacy and how it interacts with diverse people and structural elements of the healthcare system.