To address the high-dimensional, multi-peak, and nonlinear characteristics of magnetic field inversion in ship equivalent source modeling, as well as the substantial computational burden involved, this paper proposes a Full-Resident Matrix-based Enhanced Fireworks Algorithm (FRM-EFWA), aiming to enhance global search capability and significantly improve computational efficiency. In terms of the search mechanism, the algorithm integrates guided Gaussian mutation, tournament selection, and a sampling-without-replacement strategy into the Enhanced Fireworks Algorithm (EFWA) framework, achieving an adaptive balance between large-scale exploration and fine-grained exploitation, thereby effectively improving the ability to approach the global optimum. Ablation experiments confirm the actual contribution of each operator improvement to the overall performance enhancement. In terms of the computing architecture, drawing on Matrix Evolutionary Computation (MEC), all individuals in the population are mapped into a flat and compact unified high-dimensional tensor, with the entire iterative computation driven in a matrix-based parallel manner. By adopting branchless logic control, the entire algorithmic workflow is executed in a closed loop within GPU memory, with only initialization and final result output involving CPU interaction, thereby substantially increasing both search and computational efficiency. Magnetic dipole array inversion experiments for equivalent magnetic sources show that, compared with traditional CPU-scheduled GPU architectures, the computation efficiency of the proposed algorithm is improved by approximately 95 times, and by about 1.71 times compared with mainstream CPU-GPU cooperative architectures. For the 36-dimensional magnetic dipole array model, the average relative error of the reconstructed magnetic field at observation points is as low as 0.19%, with an average absolute error of 0.53 nT. In the anti-abnormal-data test, the algorithm can accurately locate faulty sampling points; in the anti-noise test, the reconstruction error remains at the noise-floor level. Magnetic field inversion of the electrical source model further confirms that, without the aid of dimensionality reduction strategies, the proposed method possesses the optimization capability to directly lock onto the global optimum within the wide-area search space. This paper provides an efficient algorithmic solution for the high-fidelity and rapid inversion of complex ship magnetic fields.