Using a rotation tool to carve internal patterns with offset

I use a rotating tool, when engraving the inner pattern engraves perfectly. When it goes back to do the outline it is off by a MM or 2 in one area and further in another area on the pattern. I don’t know if it is a setting issue or mechanical issues with my equipment.