I have a ridiculous stockpile of colorants, and lately I'm only using micas, plus TD for white and activated charcoal for black. The micas can mix right into the batter with a tiny whisk. I dissolve my TD in water and keep it in a bottle. Also, I don't even mix to trace anymore, I get everthing mixed, divided out then bring it to trace after adding the colorants.
And as mentioned, the higher the amount of olive oil, the slower it'll come to trace. Sometimes you just know the soap is going to trace really fast, just have a "plan b" with your soap design, instead of an ITP swirl, do a layering "plop" method, and so forth.