Question regarding training on "modern" models. I guess.

So, I realized I was sleeping a little bit on ZIT. I've started to train loras through Onetrainer using a preset that I found, can't remember right now from where. It had me download aaaaall of the models needed since the preset pointed to a huggingface directory for the models. Which is fine, I guess. However, I do not want to keep multiples of models that I might have on disk already for generation in ComfyUI. I mean, I have the base model, I have whatever encoder the model needs, etc. Then there's the transformers on top of that.