Intermolecular vibrational states are calculated for Ne…HBr, Ne…HI, and HI(Ar)_n (n=1-6) complexes using potential energy surfaces constructed by accurate ab initio methods. Potentials of rare gas-hydrogen halide clusters exhibit two collinear minima, one corresponding to hydrogen lying between the heavy atoms, and the other to hydrogen facing away from the rare gas atom. The relative depths of the two minima are a result of a subtle balance between polarization and dispersion interactions. Moreover, due to a large quantum delocalization in the hydrogen bending (librational) motion the relevance of a particular stationary point on the potential energy surface is only limited. It is more appropriate to discuss the isomers in terms of vibrationally averaged structures. For Ne…HBr the potential minimum and the vibrationally averaged structure correspond to the same isomer with hydrogen between neon and bromide. However, for Ne…HI the global minimum corresponds to the Ne-IH collinear geometry, while the vibrationally averaged structure has hydrogen between the heavy atoms. In the case of HI(Ar)n we show that one can flip between the two isomers by adding argon atoms, which reconciles the seemingly contradictory experimental results obtained for the photodissociation of HI…Ar on one side, and of large HI(Ar)_n clusters on the other side.