Although the theory of analog gravity suggests that we can simulate the space-time structure of black holes by using laboratory physical systems, it is difficult to find the analogs for rotating black holes in laboratory systems. In this work, we use a new field form for the optical vortex to study the analogous black hole structure close to the Bañados-Teitelboim-Zanelli (BTZ) black hole. We compare the similarities and differences between massless particles and sound waves by calculating their motions in space-time analogous to BTZ black holes and gravitational BTZ black holes. The effective potential energy values of massless particles and sound waves in both kinds of black hole spacetimes give the same forbidden-zone distributions of energy and angular momentum. The difference is that the classical forbidden area of the BTZ black hole will approach fixed energy values along the radial direction, while the classical forbidden area of the analogous BTZ black hole will be closed along the radial direction. Fortunately, near the event horizon and the ergosphere, the behaviors of massless particles and sound waves are almost the same. From this point of view, we can say that the analogous experimental system can simulate the BTZ black hole very well. In particular, the classically forbidden regions of particles with low energy and high angular momentum are wider in both types of black hole space-time.