Take two: Aww look at th-AAAHHHH what the F is that (Custom script hacking the SDXL pipeline)

r/StableDiffusion
Generative AI NLP AI Research

I have a script that works like this: Grabs the tokenization of the prompt Also passes the prompt through spaCy, which parses it and tells which part of speech each word belongs to Matches each token with each word in the prompt (it isn't always a homomorphism) Randomly chooses a word from the prompt and indentifies the corresponding tokens Randomly chooses another word in the same part of speech Gets the uncontextualized embedding vector of both words Interpolates between the vectors Replaces the interpolated vector with the original Gives the new uncontextualized embedding to the encoder.