Convert Excel and CSV files to vCards with much more options than Apple Contacts does offer.
It is not known why this behavior occurs, apparently it is a bug in macOS Server. Use the vCard conversion feature of the app instead.
UPDATE: Apple seems to have fixed this problem in macOS 10.13.x / macOS Server 5.5