As more data is collected in Prometheus and as a cluster gets busier, You may find that Prometheus stats in a CrashLoopBackoff state.


If you describe the pod you will see the pod has an OOMKilled error


Edit the Stateful set and increase the memory limit


Once saved delete the CrashLooping pod and check that the new limits have been applied.


It will take a while for Prometheus to reconcile the data in its store, Once this is completed the pod will become ready.