Over the past decade, numerous side-channel attacks on shared resources, such as the Last-Level Cache (LLC), have exposed security risks in the form of flush-based, conflict-based, and occupancy-based attacks, driving the development of secure cache designs. To defend against conflict-based attacks, which is one of the most effective classes of side-channel attacks, many modern designs randomize LLC set indexing to hinder eviction set construction. Various randomized cache designs have been proposed recently, offering distinct security guarantees. While these designs incorporate several microarchitectural modifications (we call them security knobs) over the conventional set-associative cache to ensure security, the individual impact of these microarchitectural modifications has never been evaluated. This leaves a gap in the understanding of randomized LLCs—the design space has not been explored completely and systematically.