Recently, deep learning has been widely used in forensics tools to detect and localize forgery images. However, its susceptibility to adversarial attacks highlights the need for the exploration of anti-forensics research. To achieve this, we introduce an innovative and query-efficient black-box anti-forensics framework tailored for the generation of adversarial forgery images. This framework is designed to simulate the query dynamics of online forensic services, utilizing a Markov Decision Process formulation within the paradigm of reinforcement learning. We further introduce a novel reward function, which evaluates the efficacy of attacks based on the disjunction between query results and attack targets. To improve the query efficiency of these attacks, an actor-critic algorithm is employed to maximize cumulative rewards. Empirical findings substantiate the efficacy of our proposed methodology. Specifically, it demonstrates pronounced adversarial effects on a range of prevailing image forgery detectors, while ensuring negligible visually perceptible distortions in the resultant anti-forensics images.