The quantum modularity conjecture, first introduced by Don Zagier, is a general statement about a relation between quantum invariants of links and 3-manifolds at roots of unity related by a modular transformation. In this note we formulate a strong version of the conjecture for Witten--Reshetikhin--Turaev invariants of closed geometric, not necessarily hyperbolic, 3-manifolds. This version in particular involves a geometrically distinguished flat connection (a generalization of the standard hyperbolic flat connection to other Thurston geometries) and has a statement about the integrality of coefficients appearing in the modular transformation formula. We prove that the conjecture holds for Brieskorn homology spheres and some other examples. We also comment on how the conjecture relates to a formal realization of the quantum invariant at a general root of unity as a path integral in analytically continued Chern--Simons theory with a rational level.