RNA granules are dynamic cellular foci that are widely spread in eukaryotic cells and play essential roles in cell growth and development, and immune and stress responses. Different types of granules can be distinguished, each with a specific function and playing a role in, for example, RNA transcription, modification, processing, decay, translation, and arrest. By means of communication and exchange of (shared) components, they form a large regulatory network in cells. Viruses have been reported to interact with one or more of these either cytoplasmic or nuclear granules, and act either proviral, to enable and support viral infection and facilitate viral movement, or antiviral, protecting or clearing hosts from viral infection. This review describes an overview and recent progress on cytoplasmic and nuclear RNA granules and their interplay with virus infection, first in animal systems and as a prelude to the status and current developments on plant viruses, which have been less well studied on this thus far.