Another big minus point I had is that once I pulled each alphabet out from the plastic sheet, it seemed to lose the stickiness and refused to stick onto my acrylic block or the plastic sheet. I ended up rolling some glue tape on a piece of scratch paper and pasting the stamp onto the glue before getting that onto my acrylic block. I believe wiping a wet tissue over the back of the stamp will help in sticking onto the acrylic block too but I didn't have wet tissue on hand then.
That doesn't solve the problem of my stamps laying messily on the floor but it gave me an idea. I took sheets of paper, put some rows of glue tape, and pasted each individual alphabet stamp onto the tape in alphabetical order. To make sure I paste the stamp in the right orientation, I stamped each alphabet onto the paper first (which also acts as a label) and get the correct orientation of the stamp..
This made my life much easier as I can paste back each stamp very easily to the correct position and it's a breeze to search for an alphabet.