Any quantum-confined electronic system coupled to the electromagnetic continuum is subject to radiative decay and renormalization of its energy levels. When inside a cavity, these quantities can be strongly modified with respect to their values in vacuum. In the planar circuit quantum electrodynamics architecture the radiative decay rate of a Josephson Junction qubit is strongly influenced by far off-resonant modes. A multimode calculation including all cavity modes however leads to divergences unless a cutoff is imposed. It has so far not been identified what the source of divergence is, or whether the divergence is a fundamental issue. I will show that unless gauge invariance is respected, any attempt at the calculation of circuit QED quantities is bound to diverge. I will then discuss an internally consistent theoretical and computational framework based on a Heisenberg-Langevin approach to the calculation of a finite spontaneous emission rate and the Lamb shift in an arbitrarily complex electromagnetic environment, that is free of cutoff.