Keep in mind that I’m solely basing myself on the API documentation here.
Anyway, I think you need more than toReferenceArray(). See, that gets you IReference[] but if you then look at the IReference API you’ll notice a few methods: getUrlAsModel() and getUrlAsShape().
That makes me believe that you’re going to need those before you can actually access the referenced model, and once you have the actual IModelElement then you should also be able to check its parent.