This problem is actually really complicated, I tried to solve it in

this thread. That thread died and I continued later.

Basically I solved reduced versions of the problem. I first made it so that the set of edges, the set of centers and the set of corners only has one solution, even if you disassemble.

The edges can only have up to 6 removed, otherwise two edges will be identical.

Taus used GAP to verify the minimal range of solutions for corners.

The centers can only have 4 removed, otherwise you can rotate 4 unstickered centers around

Then you are allowed one extra swap from one of the three sets. A legal position can only have an even number of permutations, so this extra permutation will mess up the cube.

I removed one more sticker from the corners. There were only a few ways to do this on Taus's minimal corners such that one extra swap was possible.

As for the fixed color scheme, I myself forgot how I accomplished this. I made the sticker scheme a while ago. I do know that any two colors cannot be interchanged on the specific arrangement of corners I chose.